Big Data: Myth vs. Reality

By | January 9, 2014

There is a lot of hype around Big Data in the Consumer Goods industry, but what is really happening — or, better yet, what is really even possible — behind the buzz?

Notwithstanding the hype, the buzz around insights and analytics has created a lot of confusion, particularly in the Consumer Goods industry where companies have been mining data for years. So, is Big Data really new or is it just about incorporating data from social media and digital marketing into existing infrastructures? Let’s debunk a myth.

The Myth:  Big Data is an “IT” problem, not a “business” opportunity

What’s most important to consider when you think about the Big Data concept is both the types of new data sets and new data sources that are now available for Consumer Goods and Retail enterprises. These are driving new analytics and technologies which in turn are driving the business value of Big Data across an organization to many users.

Big Data involves multi-structured data sources. In the past, people would use only one analytical tool: statisticians would use a statistical tool and analysts would use a business intelligence tool. Big Data requires an architecture comprised of many different technologies including traditional data warehousing, a data discovery platform, and a layer that performs multi-structured data refining enabled by Hadoop.

The entire organization can benefit from Big Data analytics. The statistics from McKinsey indicate that use of Big Data has the potential of improving global productivity by 1%, which is a huge amount when you’re talking about a global business.

In the CG space, Big Data enables Retailer collaboration efforts. Big Data allows CG manufacturers to merge consumer shopping behavior and social network insights with loyalty and transaction data to really understand what drives path to purchase across their different customer segments. This insight can lead to increased engagement, thus driving more sales and loyalty opportunities for both the Retailer and the CG firm.

The prospect of data going stale within three months would seem to fly in the face of the benefits of keeping data for years on end that have been extolled by Retail industry leaders, but it actually speaks to the high-velocity aspects of Big Data. In a number of cases, the true benefit of Big Data-generated insights comes from their usage in real time.

For example, sending a promotion to a customer’s mobile phone that has been triggered by her scanning a product’s QR code in a store can be a relatively simple automated process that doesn’t require extensive analysis. However, let’s say the promotion is not only addressed to the shopper but is specifically designed for her, based on factors including previous purchases, online and mobile searches and even her potential to be a high-value shopper, based on her demographic similarities to other shoppers. Such highly personalized promotions would require bringing together data from customer loyalty and digital analytics solutions, shaped by sophisticated predictive analytics applications, and all tied into in-store and mobile promotional tools for delivery while the shopper is at the point of decision. This means a significant investment in not only analytics but good data management, but it could also produce conversion rates that would dwarf those of broader promotional efforts.

This type of Retailer promotion ties directly to the CG partnership as many product promotions are funded by trade fund dollars provided by the CG firm to drive transactions/product movement, trial/acquisition, get traffic into store, and increase overall sales.

Another example where speed is of the essence is in the area of store-based fulfillment, which is becoming increasingly popular as Retailers realize the value of their existing store network in getting products into customers’ hands. However, operating such a fulfillment network requires Retailers to establish business rules that analyze all the costs involved in fulfillment and have a direct relationship with CG suppliers to fulfill specific orders and ensure on-time, in-full replenishment.

In some cases, the shortest geographic distance between two points might not be the most cost-effective route. Drop-shipping an item from a CG manufacturer or distribution center, even one located a few states away, could conceivably be better for the Retailer’s bottom line than shipping it from a store located in the same city as the customer – particularly if the store risks an out-of-stock situation if the item is taken off the shelf or out of the backroom.

Determining how these multiple factors interact depends on real-time inventory information, variability in shipping costs and even weather conditions that might slow down a delivery. The fulfillment decision, however, needs to be delivered immediately and automatically, either directly to the customer or to a call center or store associate.

“Transforming raw data into timely insight is at the core of a good BI strategy, and doing it quickly even with high volumes of data is the mark of a good Big Data initiative,” noted Aberdeen Group’s Rowe. CG’ers that can master not just the Velocity but the Volume and Variety that define Big Data will be well on their way to unlocking its tremendous – and to a great extent still untapped – Value.

Name:  Justin Honaman
Partner, Consumer Goods / Retail National Practice Leader
Teradata Corporation
@jhonaman / @TeradataCPG

For the full article, see Consumer Goods Technology ( and @CGTMagazine) (

Download PDF:  CGT StraightTalk Teradata – Honaman

2 thoughts on “Big Data: Myth vs. Reality

  1. avatarNeal Downs

    I tried to read the full article and the links do not work properly. (

    1. avatarJustin Honaman

      I updated the links and also included the pdf in the post. THANKS!


Leave a Reply

Your email address will not be published. Required fields are marked *