With the increasing volumes of data that can be cost effectively stored in the cloud, comes increasing responsibility. The current trend in the industry shows increasing regulatory requirements for; privacy, storing data, managing the lineage of data and securing the data sufficiently.
Whilst Data Governance is not a new concept it is increasingly more relevant in modern enterprises. Especially, since the more stringent requirements of the European, General Data Protection Regulation (GDPR) have taken effect. Companies have found themselves needing to review and update their processes and security frameworks.
However, others have turned a problem into an opportunity. As we are entering the third wave of business insights, data storage and analytics are moving into more de-centralised silos that are closer to production and transactional systems. This data is only loosely coupled and is left mostly in its original format allowing a reduction of complex and expensive ETL processes.
This has multiple advantages. Business insights can be surfaced much faster and directly integrated into the business process. The other main advantage is that instead of moving data into a centralised Data Warehouse it can be processed at the point of acquisition. This directly provides data driven benefits to customers and frontline business units, reducing the data cycle to seconds and minutes rather than hours and days.
The result is that businesses can create better services and improved sales opportunities. Many organisations have used this to disrupt established markets to the point that companies following traditional business models struggle to keep up.
Decentralisation of data responsibility into business domains requires a Governance Model that allows decentralised data management and centralised oversight.
Cornerstone of this model is a well-designed architecture that creates the foundation of how data is stored, what tools are available and how overall security is implemented and managed.
When we talk about decentralised data responsibility we do not mean that the data is necessarily stored in different locations. Cloud computing makes data increasingly location agnostic. What we mean is that instead of having a centralised BI team that owns all the data and business insights, individual business domains take responsibility of the data. This is achieved with Data Stewards assigned to the business domains.
Data Stewards are responsible for their data domain, maintaining the inventory of data types and elements. Updating a centralised data catalogue that not only contains all meta data for individual data domains, but that also pro-actively suggests connections between domains is a key to a successful governance framework.
Data Triage is becoming an important part of managing enterprise wide data connections. This is the responsibility of a Data Council, consisting of the Data Stewards from all domains, to determine and govern enterprise wide policies, standards and processes especially for data that goes across boundaries.
How does this help the organisation? It addresses a number of issues that companies face with the massive increase of data volumes in recent years. Firstly, they can gradually move away from expensive monolithic data warehouse solutions and achieve a best of breed architecture that keeps them current in the fast changing technology landscape of modern analytics platforms.
Secondly, they resolve many issues with domain oriented analytics solutions built by business units that currently sit in spreadsheets and MS Access tables. Whilst this might not remove these it helps to better integrate these into a wider framework or better yet, remove the need as a faster time to market can be achieved with Domain ownership of data.
Finally, it provides a more comprehensive data management governance structure, as the ownership of data is decentralised with a centralised Governance model that allows the building of well designed controls into the organisation without putting up too much red tape or creating delivery bottlenecks in a centralised team.
Let Fusion Professionals work with you to develop the most appropriate Data Governance framework and strategy for your organisation.
Achim Drescher is the Managing Consultant of the Big Data and Analytics Practice at Fusion Professionals.
With 30 years in the IT industry, he is an Expert in Enterprise Software and Data Architecture, Data Governance frameworks and modern analytics platforms for Big Data and Data Lakes.
留言