I have just been reading the results of the recent Teradata Data Analysis Index survey that was released of 152 senior IT executives from large Australia corporates. It further confirms that data has become an indispensable asset for most businesses with 88% of respondents using one or more analytic tools to manage and use their data.
Not surprising that the major obstacles continue to be the proliferation of silos of data sets, the lack of ability to show ROI and the buy-in/funding from executive management.
Today’s data environment is getting more and more complex. We have evangelists for big data, discovery platforms, agile labs and cloud services achieving great success in providing capability for quick insights, realisation to value and fail fast discovery processes. However, these are often delivered as point solutions that result in further data silos. This adds even more complexity to the data environment/landscape.
While the agile business driven discovery process ensures buy-in from business management, scaling and moving successful discovery/insights solutions to a reliable production environment remains a challenge for many.
This brings me to my favorite, albeit sometimes boring, subject – UNIFIED INFORMATION MANAGEMENT GOVERNANCE (IMG). This is about supporting the requirements of the business in using data for analytics – covering ALL DATA regardless of source, structure and location! So this includes creation, capture, deployment and access to data. This is the main domain for Chief Data Officers to ensure that business and IT planning processes are aligned and deliver a clear understanding of the consequences of data driven and analytical projects.
At Teradata, we adopt a Unified Data Architecture (UDA) view of this emerging data environment. Big data, discovery platforms, cloud, data marts and integrated data warehouses are part of an ecosystem that is about managing the different needs of data usage. Delivering the means of ensuring that the parts are in sync and platforms optimally used for the business purpose falls on a unified information management governance approach is the key.
Now this is a politically sensitive topic and often is poison to the “point solution” or ‘fast and furious” enthusiasts. But as we have observed in organisations that have successfully managed these multi-platform solutions across big data, integrated data warehouse and in combination with cloud services, adopting a unified data governance framework is essential.
In its most basic form, every new initiative needs to be assessed as to its data asset impact. When a new project is initiated to do data discovery work to identify new business opportunities most organisation have a clear understanding of the impact of the project during the discovery period (eg what parts of the ecosystem it will be using including cloud components).
What needs to be added is what will be the total impactwhen SUCCESSFUL! There are two parts to this assessment:
- the probability of success and
- the ecosystem requirements to productionise and then scale the solution.
The probability of success can be quickly assessed by whether the application or analytics being tried is new ground-breaking possibilities or something already proven and successfully implemented elsewhere (another country/company/industry). The later would have a higher probability of success and therefore requires more serious consideration of how to productionise and scale-up the solution before the success.
For example, people wanting to experiment on running an application on a new technology (Hadoop) instead of doing it in a traditional DW environment would need to consider the ecosystem impact (eg what data sources and data flows need to be modified and productionised) (audit, lineage and security considerations, backup, privacy, flexibility, etc), and if and how data will be accessed and shared.
This Unified Information Management Governance view has significant implications:
- Agile discovery solutions need to be part of the menu of available analytical solutions (Discovery platforms, agile data labs, cloud). It is no longer practical to offer data warehouse (integrated or not) as the only information management solution for analytics.
- The information management governance for discovery solutions need to be done together with the traditional data warehouses governance. This will facilitate an enterprise assessment of a project’s impact to the company’s total data assets.
- The need to build and maintain a central repository covering all meta data across the different environments.
- Having a clear view of the data impact of a project will help in managing expectation across the business and IT stakeholders.
Renato Manongdo is a Senior Financial Services Industry Consultant at Teradata ANZ and is also the practice lead for Business Value Assessment in Asia Pacific. Connect with Renato Manongdo on Linkedin.
Latest posts by Renato Manongdo (see all)
- Turbo Charge Enterprise Analytics with Big Data - February 22, 2017
- My Golf, Big Data and the Cloud - July 25, 2016
- What would you have done differently? - April 12, 2016
- A Race Towards Cloud: Commodisation of Systems and Analytics? - November 23, 2015
- Unifying Information Management – Why it Matters - July 13, 2015