Smells like Big Data

Some people just call it a fridge. And some people regard it as a special sanctuary within in their private castle where the most cherished consumable goodies rest. What most people may not realize is that we are talking about a vault holding an untapped data treasure. Most people outside our line of business, that is. I have written "vault" because it is not so easy to get inside. For some reason, most people tend to keep their fridges shut in the presence of "strangers", keeping their supplies and diet as much a secret as Grandma's recipe for that special birthday cake. While it's obvious that supermarkets, for example, know exactly what groceries their customers buy, we have come to accept that even they can't tell how customers prepare their food (and who is a good cook).

And then you realize that there is an alternative data source readily at hand. Where do you turn to when you need a recipe really fast? You search online sites like www.allrecipes.co.uk where users share instructions for their favourite meals. They are creating vast collections – vast enough to subject them to analyses yielding significant results. Scientists at the University of Michigan were the first to think of it and mined the site. They analysed preferences of cooking methods depending on, for example, the geographic region from which the recipes originate, or on ingredients that tend to be essential or can be dropped. The latter bit of information, then, would be useful for supermarkets – slashing a slowly-selling essential from their product range could result in greater revenue losses than expected.

Isn't this amazing? This is what the big data buzz is all about. New methods to treat semi-structured (and other types of) data, easily available online, have opened up new sources of knowledge, producing previously unconceivable insights. Teradata's Todd Walter is quoted with a similar argument in Joe McKendrick's report Big Data: we have the technology, but not the vision: "Right now, when it comes to Big Data, more vision is needed, as organizations start to learn what data they have." While organizations are starting to adjust their knowledge management, there seem to be some fruitful ideas that are just waiting to occur. After all, we all turn to the fridge sooner or later.

Paul Winsor