Big Data is Not Hadoop – Part 3

Thursday March 27th, 2014

Part 3 – If Hadoop is not Big Data, what is?

In Part 1 we let history show us that the world has changed with Big Data. We no longer need to carefully decide on a small portion of the data which will be kept and analysed; now we can keep all the data we have, add to it data we don’t have, store it for ever and analyse it to gain business advantage.

In Part 2 we saw that you need three ingredients for success in Big Data:

  • A Hadoop-like data storage – allowing storage of large amounts of data, doing it cheaply, with no schema, no tuning and a guaranteed retrieval method.
  • An Exploration Platform – where ideas can be tried quickly.
  • A Data Warehouse – where the definitive enterprise data is stored and accessed.

More than that, you want all three to be integrated.  You will need to write queries that combine all three sources, otherwise you need complex load-unload mechanisms between them, combined with data translations and mappings.

 And this is why the title of this blog is “Hadoop is not Big Data”.  Because Hadoop gives you just one piece of the puzzle: the data storage & retrieval.  Your Big Data architecture must have the other two pieces.   

So Hadoop can be a component of your Big Data enterprise architecture, but it can’t be your Big Data enterprise architecture.

So, how do you combine all three components into a coherent architecture?  The answer can come from one of two places: either the Open Source community will deliver Hadoop integration to every possible database environment (Hadoop has some initial signs of SQL integration), or (and this is more likely) the main vendors will provide seamless Hadoop integration.  This is already happening.

Here’s a link to a coherent approach to Big Data Enterprise Architecture that includes all three components:  A Unified Data Architecture.

Ben Bor is a Senior Solutions Architect at Teradata ANZ, specialist in maximising the value of enterprise data. He gained international experience on projects in Europe, America, Asia and Australia. Ben has over 30 years’ experience in the IT industry. Prior to joining Teradata, Ben worked for international consultancies for about 15 years and for international banks before that. Connect with Ben Bor via Linkedin.

The following two tabs change content below.
avatar

Ben Bor

Senior Solutions Architect at Teradata
Ben Bor is a Senior Solutions Architect at Teradata ANZ, specialist in maximising the value of enterprise data. He gained international experience on projects in Europe, America, Asia and Australia. Ben can count some of the largest international conglomerates amongst his clients, including the UK tax office, Shell, Exxon, Credit Suisse, QBE, Woolworths, Westpac and others. Ben is an international presenter on Information Management (IM) topics, having presented in Europe, Asia, USA, Canada, NZ and Australia on IM topics ranging from performance through data warehousing and data quality to Social Media analysis and Big Data. Ben has over 30 years’ experience in the IT industry (he wrote his first program in 1969, using punched cards). Prior to joining Teradata, Ben worked for international consultancies for about 15 years (including CapGemini, HP and Logica) and for international banks before that
Category: Ben Bor Tags: , , , ,
avatar

About Ben Bor

Ben Bor is a Senior Solutions Architect at Teradata ANZ, specialist in maximising the value of enterprise data. He gained international experience on projects in Europe, America, Asia and Australia. Ben can count some of the largest international conglomerates amongst his clients, including the UK tax office, Shell, Exxon, Credit Suisse, QBE, Woolworths, Westpac and others. Ben is an international presenter on Information Management (IM) topics, having presented in Europe, Asia, USA, Canada, NZ and Australia on IM topics ranging from performance through data warehousing and data quality to Social Media analysis and Big Data. Ben has over 30 years’ experience in the IT industry (he wrote his first program in 1969, using punched cards). Prior to joining Teradata, Ben worked for international consultancies for about 15 years (including CapGemini, HP and Logica) and for international banks before that

Leave a Reply

Your email address will not be published. Required fields are marked *


*