Following my blog series on “Big Data is Not Hadoop” it seems reasonable to discuss WHY people think that Hadoop is Big Data.
I will use the humble Mobile Phone to illustrate.
Mobile phones had a humble beginning; to (mis) quote John 1:1 :” In the beginning was the Word, and the Word was ‘Hello…’ ”. In other words, mobile phones, at the beginning, had exactly two advantages:
- You could use them to talk to other people.
- They were mobile: lightweight, requiring no accessories, you could carry one in your pocket.
Since they used LCD display, the power consumption was minimal, so you didn’t need to carry a charger with you.
And then we started piling new features. A big hit was a built-in camera.
Manufacturers persuaded consumers that the number of pixels is the important factor, therefore a camera in your mobile phone is good enough – you don’t really need a camera
This is absolutely wrong !!
Far more important than the number of pixels is the quality of the lens system. Look at the following picture and try to imagine taking it with a mobile phone.
To cut a short story shorter, by trying to get your mobile phone to function over and above its ‘comfort zone’ you get a lot of benefits, but three BIG side effects:
- Your mobile phone is less mobile; you need a bag and a charger.
- Your mobile phone is no longer a phone; talking takes probably 10% of the usage time.
- Your pictures are of low quality (good enough for a selfie, but not good enough for a decent-quality picture).
What does this have to do with Hadoop?
Hadoop is great. It is a fantastic (and very cost-effective) solution to a large set of business problems.
“A simplistic approach is to call the total set of these problems “Big Data”.”
However, some organisations are trying to do too much with Hadoop: this is the equivalent of taking ALL your pictures with your mobile phone. They want to implement every system and every data warehouse on Hadoop.
Like the mobile phone camera, this will be good enough for some. But don’t be mistaken: Hadoop is not the all-in-all answer to all your problems. A transaction-processing system is still better on existing technology. A Data Warehouse is still better (and when cost-of-ownership is taken into account, also cheaper) with existing, not-only-Hadoop, technology.
And this is where architecture enters the picture. An active Enterprise Information Architecture will pave the way for your organisation to select the best set of implementation choices for your business problems. In the modern era, this is likely to include TP systems, data warehouses, discovery platforms and a Data Lake.
Ben Bor is a Senior Solutions Architect at Teradata ANZ, specialist in maximising the value of enterprise data. He gained international experience on projects in Europe, America, Asia and Australia. Ben has over 30 years’ experience in the IT industry. Prior to joining Teradata, Ben worked for international consultancies for about 15 years and for international banks before that. Connect with Ben Bor via Linkedin.