The Lessons of Big Data During 2013

Tuesday December 10th, 2013

As 2013 quickly draws to a close, I thought I’d look back and pick three things I’ve heard/read/discussed about Big Data, in no particular order…

Data Science for the Masses

The interest in Big Data has also generated an interest in data science. There are considerable efforts underway to broaden the talent pool of data scientists both from an academic perspective in terms of producing the data scientists of the future and from a business perspective in terms of increased use of analytical tools such as R and Mathematica.

The availability of a $25 computer (Raspberry Pi) which now comes loaded with a free copy of Mathematica software and a new programming language encourages millions of people around the world to develop their programming and mathematics skills without an expensive barrier to entry.

This should all help to democratise data science.

Hadoop and SQL

Listening and speaking to some customers over the last 12 months I think it’s inevitable that Hadoop will start to offer a serious SQL interface providing reasonable performance and the ability to join data. Apache Hive is the de facto standard for SQL-in-Hadoop but this has many well documented shortcomings. The Stinger Initiative is an effort to drive the future of Apache Hive, delivering 100x performance improvements with familiar SQL semantics. This might then drive the wider business adoption of Hadoop and the use of SQL on Hadoop. Of course, you don’t have to wait for Stinger or another initiative to efficiently query data in Hadoop, you can always use Teradata SQL-H, but that’s for another blog.

Big Data in the Omnichannel

The customers I speak to that use data to drive their decisions are more and more looking at customer behavior data to better understand their most valuable customers. Banks are looking at web browsing and mobile banking app data and looking at combining this with geographic data and proximity to a local branch to provide a better customer experience.

The more (big) data the organisation has about its customers the better it can tailor personalised interactions. This tailoring can also include prioritising high-value customers.

Steven Lawton is a Senior Solution Architect for Teradata based in Melbourne where is responsible for technology & architecture within Teradata across Australia and New Zealand.

Leave a Reply

Your email address will not be published. Required fields are marked *