I recently attended the Teradata Summit in Melbourne and was very intrigued by a presentation given by Venugopal Adooparambil (Venu) Director of BI Consulting – Big Data at UnitedHealth Group where he was talking about the experience of SuperValu in the US, and their journey in finding value in untapped sources of data and answering the CIO’s question of “What is this Hadoop?”. They struggled with the same things Australian organisations are wrestling with and that is figuring out if big data is something they should be interested in and also finding the value in it.
As clichéd as the term “You don’t know what you don’t know” might be, it succinctly captures the headache of uncovering the value in big data. There is a circular logic loop, when you know something is valuable you manage it better but you have to be managing it to know it is valuable…lather, rinse, repeat. You don’t ‘manage’ data with no previously exposed value and therefore its value is never uncovered.
The emergence of discovery technology as a means of exploring and prospecting in data is helping to make this process a little easier but you still get into the roadblock that you cannot justify the investment in the discovery process because you cannot demonstrate the value in it.
It was around August 2011 when Venu got the question “What is Hadoop?”. The team started looking into it, they started building a Hadoop cluster. Being business users, and SuperValu not being a technology company, they didn’t really have the skills to leverage Hadoop, so they decided to look at the competitors in this area. The usual suspects came up that included Teradata Aster. SuperValu put in place a process to get vendors to deliver against defined use cases.
What resonated with me about Venu’s talk was that the organisation embraced a mindset of innovation. That other often overused management word. SuperValu operate in a retail environment of “super thin margins” and have to continuously search for ways to improve the business. This provided the burning platform for discovery. Innovation was not pursued to discover value in data, rather embracing innovation led to looking for value in data. Having strong business cases and someone like Venu able to link the use of data to the business driver helped support the initiative.
Venu also touched on the business case and how they got to measure value.
You should start with the question to be answered, the use cases, and preferably by the person who will own using the outcome.
As obvious as this sounds in hindsight we regularly see the mindset of “store it and figure out its value later” or “build it and they will come”. I have not seen this work in practise, or certainly not in a predictable way. It is also difficult to justify this approach because it is all cost, faith and no hard value. Which means there is nobody taking accountability for it or staking their reputation on the project?
Another challenge for SuperValu was skills. Given the company’s location “on the icy Tundra of Minnesota” as Venu put it, attracting “Silicon Valley web tools types” was not likely. They needed tools that data analysts could use but that also did not limit their ability to capture future opportunities. They wanted a toolset that would allow them to start simple using the existing SQL skills they had but that would not become a constraint as they matured their data analysis capability.
SuperValu chose to test a number of vendor’s promises. Due to the limited time and resources approved by management the process applied significant pressure to the participants. There was an unexpected benefit in trying to deliver under tight deadlines, they got to see how vendors responded in tight situations. SuperValu considered this to be a good indicator of how the vendor would behave long term.
The process, despite all the challenges in getting it rolling, was very successful. SuperValu found a number of nuggets in the process. From this they elected to buy Teradata Aster because it helped them leverage existing skills, its ability to work with the broad data types and sources as well as the ability to leverage Hadoop. They had found value to justify expanding capability. They owned the use cases, they owned the business case and they owned the process, these are things that cannot be outsourced.
So my starting question stays out there, do you need to be innovative, or is it only innovative companies who find the value in big data?
Craig Rodger is a senior Pre-sales Consultant with Teradata ANZ focusing on advanced analytics. He has spent 20 years in the IT industry working on how to get value out of systems rather than getting things into them. Having been a member of a number of executive management teams in software, technology and consulting companies and helping build a number of technology business ventures he joined an advanced analytics vendor.