Citizen Data Stores – The Gremlins in your data

Colleagues and industry peers are thrilled about leveraging ‘data’ for the opportunity that machine learning and artificial intelligence provide. And why not, there is a lot of potential and demonstrated work.

But, there is a different ‘data’ perspective that we see when engaging with clients in the region. While there is a compelling urge to leverage data in new ways. The urge quickly turns to agonising pain caused by issues in getting the right data at the right time. The easily diagnosed pain comes from the inability to unlock the value of data from monoliths (legacy). The hidden and more acute pain though is caused by failure to leverage data from first-generation citizen data stores – The Gremlins

For the uninitiated – Gremlins are peace-loving little creatures. If fed after midnight, a gremlin will turn into a wicked, troublemaking creature

The monolith ‘data’ problem has high visibility and IT support. Slow and painful but there are ways and means to manage. There will be improvements in due course as monoliths are eaten into by digital minions – microservices. This journey is difficult but clients have demonstrated intent and are allocating resources to build decomposed layers of functionality. Can this be done in weeks instead of months so that digital use-cases are first class citizens is a question that we love to work on. But that is for another day.

For today the attention is on the gremlins of the enterprise data world – Citizen Data Stores. The innocuous data stores created using spreadsheets and primitive DBs.

Applications built around these data stores were built in reaction to a business need by smart business specialists. Over time an ecosystem of implicit business processes has surrounded these data stores. These apps. were in some sense the first generation ‘systems of engagement’. They delivered application and business process management delivery ability without IT intervention. Fast forward. As digital use-cases take centre stage the data stores struggle and cannot be replaced creating a sense of evil will do anything to live.

Issues observed range from difficulty to integrate to cannot do web-scale. This translates into the data-stores being an uncontrolled debt. Reducing the ability of the business to deliver new experiences. The Gremlins have spawned and business now has a two-pronged problem. First, the citizen data store that intended to deliver systems of engagement cannot do so because digital use-cases are not supported. Second, the stores cause trouble because sharing implicit data and logic is difficult – to put it mildly.

On one hand, it is important to have citizen data stores since they enable experience delivery and innovation in the hands of the business. On other, it is important to have security, control and integration. The problem or pain point thus translate into :

  • How to unlock citizen data stores so that the enterprise initiatives and other applications can leverage valuable ‘data’?
  • How to build the next generation of citizen data stores that are integration ready from the get-go and can deliver digital use-cases at web-scale in a secure manner?
  • Ensure citizen data stores are not isolated but are instead enterprise functional blocks?
  • And more importantly how to do this in weeks, not months and with minimal IT involvement.


Non-cool but these are problem statements we see and are solving.

Images in this post are from wiki commons and are labelled for reuse. Credit is due to the owners and creators.

Leave a Reply

Your email address will not be published. Required fields are marked *