Teradata Blog Feed

Some Thoughts on Measuring Consumer Marketing’s Affect on the Path to Purchase

David Placke — 1/25/2012 3:05:08 PM

For Consumer Goods companies it’s getting harder to argue with the value of knowing consumers and shoppers at an individual level. It’s simply best practice for any industry, but a relatively new idea in Consumer Goods where the historical emphasis has been brand marketing and trade promotions. Both of these very significant investments have been relied upon to drive sales absent direct relationships or detailed knowledge of individual consumers. That has to change, as noted in posts like this one.

The challenging part of the discussion begins with how difficult this visibility is to achieve given the lack of an existing consumer or shopper database that has a real person’s name at the center. Few Consumer Goods companies have complete visibility into the data generated across every agency, promotion, campaign, channel or brand. It’s a data integration and agency-process issue Teradata has worked hard to solve for some of the largest Consumer Goods manufacturers. The lens then focuses quickly on what consumer actions can be captured, measured, and improved upon to increase sales.

One idea to explore is mapping all consumer interactions to a “state-based” path to purchase which can be tested, validated then optimized. Consider all the ways both offline and digital marketing efforts have targeted consumers over the past several years and how these relate to different markers along the path.

Offline to online interactions out of store (television advertising, newspapers or magazines, presumably at or near home, featuring a call to action to register, scan or text).
Desktop web experience – register or log-in (presumed to be at or near home).
Email read on a desktop client (presumed to be at or near home).
Email read on a mobile device (“on the go”).
SMS text message interaction (“on the go”).
In-store interactions (QR code scan, unique SMS shortcode, unique email or mobile web, mobile application interaction data).

If you start with the assumption that certain interactions are inherently performed at-home, “on the go” or in the store environment you can begin to paint a picture of how digital marketing efforts affect the path to purchase. You start by asking relatively basic questions, such as:

How many consumers have you interacted with in these ways?
What consumers have interacted with you across these ways? A segmentation or profile emerges.
How have consumers interacted with you across these ways – the data provided and captured, the sequence and timing. Does an observable path emerge?
What patterns exist among the interactions that are especially insightful – in terms of moving a consumer through a hypothetical “buy pipeline” or along the path? What campaigns, incentives or creative ideas were responsible?

And that’s just the beginning. All interactions can be related to geographies, markets and individual retail locations to begin gauging a more direct impact consumer marketing has on sales than ever before possible.

How else might Consumer Goods marketers leverage consumer data to understand and shape the path to purchase in their favor?

Gib Bassett

An Agent for Change

David Placke — 1/24/2012 11:22:41 AM

Travel solutions provider Sabre stays ahead of the competition by utilizing an executive dashboard that delivers accurate and timely operational information. As described in the article “Sabre Streamlines Decision Making,” the dashboard’s information is updated as frequently as every 15 minutes allowing top executives at Sabre’s customer companies to view and analyze key bookings, sales, revenue and flight informa¬tion that defines an airline’s performance.

Sabre, which has been working to transform the travel industry for more than 50 years, has enhancements planned for its dashboard and will continue to be an agent for progress in an ever-changing industry.

Jennifer Niemela
Executive Editor
Teradata Magazine

On top of the world

Darryl McDonald — 1/17/2012 5:13:07 PM

As I was preparing for this week’s kickoff with Sales, I thought a lot about the key messages that would help them tell the Teradata story. What occurred to me is that succeeding in business is like climbing a mountain. The view from the top is awe inspiring—but it can’t hold a candle to the satisfaction of conquering the challenges that could have meant failure.

The way up (and down!) can be fraught with disaster. So beforehand, you do your research, train your body and mind, and gather all the resources available to help you get there safely. Once underway, it’s really about decision making—making the best decision possible under any circumstance, at any particular moment.

Just like business, right?

I am a firm believer that success and more fundamentally, survival, depend on the advantage you achieve in the marketplace. When I boil it down, the first step up that mountain is to integrate everything in one place. Then it’s about what I can learn; the real insight I can get out of my information. And finally as a leader of the organization I act on that knowledge in ways that make us more successful and create market advantage. It’s really a simple proposition: Integration. Insight. Advantage.

For Teradata, it simply means helping our customers make the best decisions possible for their business so they successfully reach the top of every mountain they choose to climb. And that’s exactly where our efforts will be focused throughout 2012.

Darryl

Work Smarter, Not Harder

David Placke — 1/16/2012 3:03:40 PM

Database administrators (DBAs) use technology, including a database, to get answers to challenging business questions. But sometimes, setting up workloads can be time-intensive, trial-and-error. Now, the process becomes faster and easier with Teradata Workload Analyzer.

Teradata Technical Lead Birendra Kumar Sahu says in his latest article, “Drill Down,” that Workload Analyzer enables higher throughput and more satisfied business users. That’s because it helps DBAs fully understand normal workloads, peak surges and unusual workloads in multiple time periods. The solution allows users to perform a deep analysis on workloads using various parameters. And if they don’t like the results, they can select new parameters.

By helping establish and refine workloads, Teradata Workload Analyzer enables DBAs to work efficiently. The solution also reduces the time and effort spent on unsuccessful attempts to set up the workloads. This helps optimize the data warehouse and gives businesses the information they need to make good decisions.

Brett Martin
Associate Editor
Teradata Magazine

Brand Management in the Age of the Connected Consumer

David Placke — 1/18/2012 9:00:01 AM

It’s not surprising that Procter & Gamble pioneered the discipline of Brand Management, but what you may not know is the reason. In the 1930s, as P&G entered single categories with multiple products it became apparent that for each to succeed a new model was needed to focus resources on supporting individual brands; absent this shift, products without clear differentiation competed for both internal marketing resources and customers.

Since then, times have changed, and so has P&G. In a recent interview CEO Robert McDonald described how the company was embracing all things digital in order to remain a dominant force.

“…it’s now possible to have a one-on-one relationship with every consumer in the world. The more intimate the relationship, the more indispensable it becomes. We want to be the company that creates those indispensable relationships with our brands, and digital technology enables this.”

Traditional one-way brand messages broadcast to a homogenous audience are losing effectiveness at a time of unprecedented consumer adoption of digital channels -- from the web and email, to mobile and social media. The “consumer connectedness” this environment yields means consumers are as much in control of brand meaning and value as the company itself.

Brand management isn’t going anywhere and remains a critical function in Consumer Goods, but what is interesting about digital is the opportunity it presents across brands. Digital paves the way for direct consumer relationships that in the past were the domain of retailers, yet few Consumer Goods brands have realized significant benefits. Reasons vary, but for most, digital is an extension of the brand marketing model, where efforts happen within silos and are outsourced to agencies.

As opposed to Brand Management of individual products or product lines, Consumer Management seeks to capture deep consumer understanding within and across brands. Consider that consumers who research, browse and buy via multiple channels have been shown to be the most valuable – as in those who spend the greatest, most frequently and often refer others or are looked upon as credible influencers.

Even Google recognizes this, coining the term “zero moment of truth,” a play on words with a P&G originated concept that seeks to influence shoppers at different stages leading to selection at the shelf. Google posits there are no such moments; that product selection decisions are fully made outside the influence of the store by digitally connected consumers.

This “omni-channel” consumer is becoming the norm as smartphones replace less sophisticated feature phones and adoption of digital channels like email and social media proliferate. These individuals represent a more highly engaged audience that invests mindshare in brand relationships. They are the target of any b-to-c business, especially retailers.

Now consider some of the ways brands engage with consumers digitally, and the data these efforts generate.

Branded smartphone applications, either utilities such as shopping helpers or promotional in nature, are becoming commonplace. Often to activate key features of these applications, consumers must register, self-identify and elect to receive push or geo-aware notifications.
Opt in channels like email and SMS text messaging are commonplace in a lot of brand marketing, as either a promotional entry mechanism or communication channel. Permissions typically carry with them consumer demographic, attitudinal and preferences data.
Brand websites typically feature user registration, presenting the opportunity to customize the user experience based on self-identified interests, and enable sharing of content via other channels such as email or social media. Data is captured in both cases, to say nothing of the data generated by web activity itself.
Brand presence in social media is now common, claiming thousands of fans or followers, each of whom in turn have unique characteristics captured as data and often made available to the brand. Their influence or sentiment can be gauged, recorded and levered as part of many marketing efforts, from product trial and launch to promotions intended to increase a brand’s following via network sharing.

It’s no small challenge to isolate individual consumers across every brand marketing interaction, regardless of agency, product, campaign, channel or type of data (many times unstructured), yet this is precisely what Teradata has mastered. By enabling integration on this level, Consumer Insights becomes a living asset for the entire Consumer Goods organization, to be leveraged within and across brands -- even by the very agencies sought out for their creative, media buying and execution expertise. Consider the possibilities enabled via:

Consumer profiles fed with data from every brand’s efforts such that brand teams have access to insights that yield more relevant interactions based on preferences, permissions and interaction history.
Benefits beyond marketing, to include the sales organization and research and development. Consumer segmentation by geography, down to the store level via inference or self identification, can inform trade promotion messages, tactics and investment strategies based on highly engaged (and valuable) consumers. Such inference can also tie digital efforts to how they affect the path to purchase. The same insights represent a resource to help R&D innovate faster and ultimately yield more successful new product launches.
The ability to stage interactions across brands to avoid the same intra-company competition Brand Management was created to resolve. Consumers have only so much mindshare to give in today’s marketing-message saturated environment.
Never-before possible cross-brand marketing opportunities to take advantage of analytics designed to surface like-segments, response propensities and optimal marketing mixes.

Marketing organizations unaccustomed to this sort of thinking should heed advice from experts such as Forrester, who’s research shows integrated data key to unlocking the value of digital marketing:

“Forrester expects a convergence of IT and marketing budgets as the marketing budget becomes increasingly digital; and digital marketing campaigns, with their outcomes, become intertwined with customer data systems and other back-office systems.”

“The new marketing systems will need integration to be effective. Emerging marketing technologies successfully use multiple classes of systems, from the Web to CRM to order processing to ERP, as data sources. Without integration, data from one system has to be moved to another via clunky batch interfaces, or worse (e.g., “sneakernets”), which slows down marketing analytics, decision-making, and action taking, if the newer marketing technologies are going to work properly.

“Customer behavior analysis is predicated upon good database management, business analytics, and reporting — technology and skills that IT has long provided to business users across the enterprise.”

Similar to how a retailer views its customers according to the products in a market basket to optimize that transaction, Consumer Goods makers should seek insights into consumers according to the brands they engage with to inform intelligent path to purchase strategies. Doing so offers many benefits that set the stage for even greater value by integrating the consumer marketing environment with retail sales and production data.

Gib Bassett

Gold Standard

David Placke — 1/9/2012 3:51:17 PM

Anyone who’s been through a corporate merger knows how challenging it can be from a cultural and logistical standpoint. The other side of the coin is that it can be a win all the way around.

Case in point: When Austrian bank BAWAG PSK was formed in 2005 through the merger of two traditional Austrian institutions, it gained an advantage from its combined product and customer portfolio. However, the bank could not capitalize on this opportunity with its inefficient, manual marketing campaign system.

To overcome this obstacle, BAWAG implemented a new CRM system that combined the expertise of Teradata customer data integration, analytical processing capacity and predictive analytics with Aprimo Relationship Manager. As reported in the Teradata Magazine case study “Gold Standard,” the results have been striking: a six-fold increase in campaign success rates, and contact rates for sales leads have increased by more than 30% over 18 months.

And that’s just the beginning. BAWAG now has a robust analytic and campaign management solution that they can bank on well into the future.

Jennifer Niemela
Executive Editor
Teradata Magazine

Is CPG Losing a Zero Sum Game for Consumer Attention?

David Placke — 1/9/2012 5:11:48 PM

Consumers have limited bandwidth, mindshare and attention-span when it comes to receptivity to marketing messages. In the CPG world, whether it’s brand or shopper marketing, these investments compete against all other b-to-c businesses trying to connect with consumers and influence the path to purchase.

You could say then, that consumer attention is a zero sum game; that is, consumers possess a fairly fixed amount of attention and therefore some marketers succeed at others’ expense. While a push/pull effect can sway mind- and wallet-share over time, it’s those marketers able to create the most personalized, relevant and timely connections who will win more times than not. The highest performing examples are based on an integrated view of consumer data inclusive of channel preferences and permissions.

For Consumer Goods makers unaccustomed to this brand of marketing, the situation should be viewed as a shot across the bow with respect to sales and profitability. For some, it already has, with published research showing growing interest in direct to consumer efforts, such as e-commerce and social media marketing.

Other industries, such as retail and financial services, have been doing this for years. The mobile channel in particular is a key battleground for consumer attention; the one channel that happens to be Consumer Goods’ best opportunity to connect with consumers. Consider this Forrester report about marketing technology adoption (across industries), which finds mobile top of mind among marketers:

“Mobile takes top billing…we see this reflected in marketers’ technology investment plans. CI professionals want to reach consumers in their preferred medium, and 30% of respondents plan to invest in mobile technologies.”

So the battle for consumer attention is only intensifying, with the mobile channel in the crosshairs. You can be certain that the marketers who base their efforts on the best integrated data will be on the winning side of the zero sum attention game. CPGs lacking detailed and actionable insights into their consumers must confront this challenge sooner than later. Per that Forrester report:

“CI professionals prefer retaining technologies that depend heavily on customer data while outsourcing technologies where they need to move quickly or lack the skills needed to support them.”

This all is sage advice that should be taken to heart now, but consider that the report was published in February of 2011! Almost a year ago. Time is not CPG’s friend.

A more recent analyst report, by Gartner and titled, “Predicts 2012: Demand Sensing Will Be Key to Success and Growth in Consumer Goods Manufacturing,” includes among its key findings:

“The battle for consumers and shoppers in consumer goods will be about personalization and context-aware promotions.”

Continuing to attempt such efforts based on silos of brands, data and business processes is a losing proposition for CPG when marketers armed with better insights are targeting the same, mindshare-constrained consumers. Among the projects on the table at all CPGs for 2012 should be one aimed at developing an actionable, integrated view of consumer data that becomes an asset for the entire enterprise.

Gib Bassett

Avoid Unpleasant Surprises

David Placke — 1/3/2012 10:34:35 AM

Most of us know stories of inventions that came about by mistake, like Post-It Notes, the Slinky and even chocolate chip cookies (sold to Nestle for a lifetime supply of chocolate chips). But when it comes to data warehouse environments, unexpected outcomes can cripple a business.

To avoid potential problems with hardware and software upgrades, Teradata offers a Pre-Upgrade Assessment service that identifies and alleviates risks to the Teradata Database system. The article “Smooth Road to System Upgrades” emphasizes that while upgrades are part of a healthy and vital system—and even increase its use and value—problems can arise that cause performance and workload management problems.

With the Teradata Pre-Upgrade Assessment service, companies can eliminate problems before they occur, making sure organizations get the results they want. And unless you’re in the invention business, getting the desired outcome can save time, money and hassles.

Brett Martin

Associate Editor

Teradata Magazine

Welcome to the New Year and NRF

Darryl McDonald — 1/3/2012 11:34:55 AM

Welcome to the New Year and the ramp-up to the National Retail Federation show. The holidays are over now but as I think about this event I remember my days in retail when, as a sales VP, I would have sleepless nights over black Friday and wonder how the numbers would shape up when all was said and done.

NRF is still important to me—and Teradata. This year, Teradata will show you how to build an analytical eco-system that supports all areas of your business. Adding Aster Data and Aprimo to our portfolio, along with the strength of our partners, means we have never been in a better position to support cross- channel and big data analytics that enable retailers to garner more customer insight and leverage new opportunities from web, mobile, and social media.

Integrated Marketing Management is a burgeoning—and important—discipline in which Teradata and Aprimo thrive as demonstrated by placement in the Leaders quadrant of the latest Gartner, Inc. Integrated Marketing Management (IMM) Magic Quadrant report. In her Big Ideas session, Lisa Arthur, CMO, Aprimo will talk about IMM and the retail marketing revolution that’s heightening the pressure on organizations to understand consumer behavior and the vehicles that drive their path to purchase.

Teradata Aster will be present as well. So if you have ever been interested in understanding how much Twitter influence you have, for example, we can show you that and much more.

We look forward to NRF and hope you’ll visit us. I’ll be thinking about you and remembering the “good old days.”

Darryl

Magic of free market economy on ‘Big’ data focus

David Placke — 12/19/2011 2:27:19 PM

Looking back in the history of telecommunications, the main characteristic of the telephone in the 1960s was that it was predominantly fixed-line and solely used for voice communication under monopolistic regime. Back in those days, ‘customers’ were mere ‘subscriber numbers’! Phone bills were generated once a quarter from aggregated meters that resembled water meters found in households today. However, the meters were located in the telephone exchange building, one for each fixed-line telephone, in a meter rack.

With aggregated metered data, there was no way to identify how many calls, to what destinations and at what times the calls were made. The service provider could neither prove nor disprove usage that often resulted in frequent disputes, but that did not matter in a monopolistic regime! ‘Customers’ were at the mercy of the service provider; so could not complain too much for fear of losing their line! In fact, my parents in India had to wait 20 years to get a phone line connected!

Deregulation of the telecommunications in subsequent decades heralded the death knell for the meter racks. The new born service providers were scrambling to win customers by trying to find ways to differentiate their services that appealed to their prospects / customers. They resorted to Call Details Records (CDR). The CDRs captured details of every call that a customer made; which became critically important as they provide usage behaviour patterns of their customers which can be used for customer analytics. The CDRs paved way for xDRs, which represents a variety of detail records denoted by substituting ‘x’ in xDR, to more holistically capture all types of detailed transactions of the customer that went into their billing statement (see figure below).

Thus, the era of ‘Big data’** was born for the telecom service providers, with an explosion in the volume of records that they had to manage - from 4 records per subscriber per year to a staggering 36,000 records per subscriber! Teradata has been a natural choice for tier-1 telecom service providers due to Teradata’s leadership in multi-terabytes scalability and performance of the enterprise data warehouse.

Energy and utilities companies are where Telecoms were in the 1960s. But that is starting to change with deregulation of the industry and a wholesale market focus. The price of the highest bid in the wholesale market required to meet demand dictates the 30 minute clearing price. The spot price generally exhibits extreme volatility across different regions based on supply and demand balance driven by unpredictable weather patterns and other factors. This was seen in the state of Victoria in Australia, where spot prices fell to a low of -$100 per megawatt hour (MWh) and reaching the maximum $10,000 per MWh in more recent years. Energy retailers have recognised that quarterly meter readings do not provide the ability to forecast usage and demand when spot prices change every 30 minutes, potentially leading to an unprofitable outcome. Hence, retailers are resorting to Interval Meters that can capture usage every half-hour or less. Armed with granular level usage data, together with corresponding weather data, energy retailers will be able develop accurate predictive modelling that will enable them to be competitive and profitable in a free market.

In a free market, where concerns for cost of customer acquisition, retention and service provision as well as revenue growth become top agenda for an organisation, every bit of information it can get about their customers become business critical. For the telcos the source of ‘Big’ data some 30 years+ has been xDRs. For the banks it has been the ATMs and credit card transactions. Retailers have long relied on point of sale transactions. For the energy retailers the source of ‘Big’ data today is the Interval Meters. For Googles and Amazons of the world their ‘Big’ data has been the web click streams.

What is the next ‘Big’ data for the telcos? Undoubtedly, it will be from the same xDRs containing network signalling data that service providers have long ignored / discarded. It will also be from the adjacent markets that are beginning to erode their growth and sustainability in their own turf. NOW is the time to start adopting Big Data in all earnestness!

**Big Data is defined as those types of data that exceed the reach of commonly used hardware environments and software tools to capture, manage and process it within a tolerable elapsed time for its users. The “bigness” in Big Data is a relative measure and a constantly moving target for most industry / organisations depending on where they start. Where they start depends on market and competitive pressure. Readiness to adopt ‘Big’ data depends on compelling market drivers.

Sundara Raman

This is our last blog of the year. We wish you season’s greetings and all the best for coming year. We will commence blogging again in mid-January 2012.

Strength in Numbers

David Placke — 12/19/2011 2:05:08 PM

As management guru Ken Blanchard says, “None of us is as smart as all of us.” How true that is in a digitized world, where enterprises are under increasing pressure to make use of the tsunami of data washing over them every day. Not only do companies need to incorporate diverse types of data in their analytics, but they must also bring in varied perspectives from all across the enterprise to make decisions based on that data.

Smart decision making? Or too many cooks in the kitchen? The answer depends on the enterprise’s ability to incorporate collaborative decision making, writes Joe McKendrick in the article “True Teamwork.” Thanks to integrated data and tools like social networking used to gather that data, it’s possible to break down barriers between divisions, business units, vendors and people. That makes it easier to make decisions and act on those decisions.

After all, working collaboratively is not just about being efficient, reminds Harvard Business School professor Ranjay Gulati in the article. It’s about being effective as well. And that often takes a team.

Jennifer Niemela
Executive Editor
Teradata Magazine

Companies that don’t integrate Social Data fail in the age of the customer, says Forrester

David Placke — 12/19/2011 5:50:19 PM

Given the prominence of “all things social” in 2011, it’s an appropriate topic for this last post of the year. As 2012 begins, few executives will argue with the need to integrate social media better with their business. The questions will arise, however, regarding the best way forward. It can be confusing.

Possible answers will come from many sources and occasionally conflict. Social media is being used to monitor conversations to avoid public relations disasters, provide customer service often for the same reason, and engage consumers in promotions and other marketing programs.

If you haven’t been paying attention you might have missed the socialization of e-commerce, as in selling via online stores hosted within Facebook (f-commerce), or the fact consumers are using social login for buying on retail websites. Adding to the mix is e-commerce in general, especially as a trend among CPG companies looking to grow sales amid a down economy through as many channels as possible.

In some cases, one tactic is a means to an end, as in “Brands…embracing ecommerce to gain access to shopping behavior data typically retained by the retailer,” according to this recent eMarketer article. Such is one goal of the Demand Signal Repository or DSR (among others), not typically associated with e-commerce but rather the sales organization.

Most businesses simply want to remain relevant in this newly social environment defined by Connected Consumers, but that will be increasingly difficult when social actions happen in disconnected and disparate ways.

It’s against this backdrop that I thought it worthwhile revisiting an October report from Forrester Research titled, “The Road Map To Integrating Social And Customer Data.” The subject of this report is sure to be a topic of discussion in numerous businesses throughout 2012, especially Consumer Goods Manufacturers. The report’s executive summary states:

“Too many companies remain trapped by merely monitoring or passively collecting social media. Few actually reach Social Intelligence: driving their marketing and business strategy using the data that social media creates. Today, companies underutilize social data and often leave it sitting in its own silo. To overcome this problem, Customer Intelligence (CI) professionals must start looking at social media as a new source of customer information that integrates with their existing data. But this is far easier said than done.”

Forrester prescribes a crawl, walk, run and fly approach to achieving this state, that makes little sense to attack in a piecemeal fashion using point products and outsourced services. As pointed out in this recent CMO.com article titled, “Why Traditional Marketing Isn't Clicking With Consumers”:

“…there is no third-party cookie or panel data that can possibly compare with a brand’s own unique customer insight, including customer transactional and service history, their preferences, their interests and attitudes, and even past receptivity to brand content. For many brands, this insight is multichannel and found in their marketing databases.”

Unfortunately few Consumer Goods makers possess internal marketing or consumer databases, let alone actual DSRs with retail transaction data. Instead, the data resides outside the company, in the care of dozens of agencies and marketing services companies. Teradata’s approach involves a single platform to support the Consumer Goods maker’s journey from social media listening through the integration of this data with consumer, transaction and other sources to inform multiple business centers. It is this maturity curve which Forrester describes.

To “fly” in Forrester speak involves “Integrating social and customer data in the customer database through social listening, appending existing records, or collecting customers’ social information through opt-ins.” Of the three ways to collect consumer social data, it is an opt-in or pull approach which Forrester describes as the one with the greatest accuracy and fewest privacy concerns.

Teradata works with leading consumer Goods companies in this very way, providing the marketing campaign platform to opt-in and nurture consumer relationships across channels as well as the data integration platform for resolving the diverse and rapidly growing sources of data to fuel these interactions more intelligently.

The role of analytics cannot be discounted either. As businesses become defined by their mastery of the social universe, analytics will play a central role in helping understand the relevance and value of social data across the enterprise. One such application, marketing attribution as supported by Teradata Aster Data, is described in this recent Forbes article.

Forrester says “Companies that don’t integrate Social Data fail in the age of the customer,” but that time is now. Don’t let social data overtake your business before it’s too late. 2012 is sure to be the year of the socialization of business.

Gib Bassett

Awards 2011: Winners, Leaders and Innovators Challenge their Data

Darryl McDonald — 12/19/2011 5:22:18 PM

Teradata, Aprimo and Aster customers are the best in the world – and we’re not the only ones who think so. Whether it’s best practices that provide models for others to aspire to or whether it’s innovative solutions that break new ground, solve problems and bring extraordinary benefits and financial results, our customers win the respected award programs.
Darryl

Just take a look at the 2011 experience:

Nationwide’s enterprise Customer Information Management Initiative (CIM) initiative offers a differentiated and exceptional customer experience, again demonstrating Nationwide’s industry leadership in both customer management and innovative use of technology. |More

Singapore's Land Transport Authority (LTA) won the 2011 CIO Asia Award and was named to the CIO Asia 100. LTA won for its PLANET (Planning for Land Transport Network) project, a data warehousing initiative designed to carry out advanced analytics about traffic and travel patterns in Singapore whether by car, bus or train. |More

Kodak won a DemandGen Report award. Winning organizations are recognized for their accomplishments in driving revenue by upgrading lead generation and lead management programs and leveraging marketing automation technology. |More

Oklahoma Tax Commission was one of the top 20 finalists in the Excellence.Gov Awards for 2011 for its Tax Compliance Revolution Project with a Teradata foundation. |More

The LTA won the second annual Gartner BI Excellence Award for Asia Pacific. ROI for the project includes cost savings of 19% per year and productivity savings of 13% per year. |More

Aprimo customer, International Speedway Corporation (ISC), the world leader in motorsports entertainment, received the Gold for Integrated Marketing in the Gartner/1to1 CRM Excellence Awards. |More

“Top innovators are changing the way business manages information,” said Information Management Magazine in its Nov/Dec 2011 edition announcing winners of its 2011 Innovation Solution Awards. “This award was created to recognize solutions that are groundbreaking and provide quantifiable business value.” The State of Michigan and Teradata won the BI category.|More

In the 2011 InformationWeek 500 Rankings – “The Very Best Business Technology Innovators,” customer Centene ranked #22.

The National Association of State CIO’s (NASCIO) honored the State of Michigan with 5 of its 2011 Recognition Awards, including one with Teradata – the award for Data, Information and Knowledge Management.

|More

Lt Col Walter Nichols of USTRANSCOM was a finalist for TechAmerica Foundation Award for Program Manager of the Year.
|More

Customers McCain Foods and State of Michigan are winners with Teradata in the Ventana Research 2011 Leadership Awards. McCain Foods CIO Roman Coba has won the CIO category. The State of Michigan Department of Technology, Management & Budget won the BI category. Congratulations to customer Westpac Banking, submitted by Teradata partner Informatica and named the overall IT Leader for 2011. |More

Customer MLC & NAB Wealth Won a Bronze in the 2011 NCDM Database Excellence Awards BtoB Marketing Category - Entering an awards program for the first time and coming up a winner, MLC is the wealth management division of long-term Teradata customer National Australia Bank (NAB). The prestigious NCDM Database Excellence Awards honor organizations that demonstrate ingenuity and creativity in leveraging their customer data.

Using Social Analytics to Get You to the Top

Darryl McDonald — 12/16/2011 10:17:50 AM

Say "social media" today and it doesn't elicit the same response it did several years ago when many organizations wondered if it was a viable business strategy worth time and energy. Now, we all get it. Yes, there's value in social media engagement. But what's new and innovative today? Will social software and analytics give companies that extra edge? Do the improvements in social networking software constitute that new innovative "thing" to help businesses leapfrog over the competition and to the top of the leader board?

Mzinga, a leading provider of social solutions, definitely believes that they have the secret sauce to help organizations increase revenue and lower costs. Making this happen for their customers requires innovative technology that helps Mzinga manage their big data volumes so user metrics are delivered in a timely manner. A company that can analyze and uncover patterns within social networks in minutes versus days already has an advantage over the competition. They are that much closer to better understanding their customers and creating more value, more loyalty.

I am proud to say that the Teradata Aster MapReduce Appliance is part of Mzinga's analytical infrastructure. Aster technology is helping Mzinga's customers understand the impact of social media with pinpoint accuracy. They have a complete picture so they can analyze and relate the data to actual customer behavior.

What Mzinga represents is, indeed, a giant step in the direction of giving companies competitive advantage. Managing customer relationships at that level creates real business value. Companies that strive for data equity by using cutting edge analytic technology will be the industry leaders.

How will you socialize your data and win your race?

Darryl

Was this a sales pitch?

David Placke — 12/12/2011 10:32:49 AM

I had the opportunity to attend (and jointly present at) the Teradata Partners event in San Diego in October and was blown away. This event is the main global Teradata User Group and the emphasis is definitely on the ‘User Group’!

Having spent most of my career working in banks I have attended a few vendor events in my time and have been disappointed (and sometimes annoyed) when vendors try the hard selling approach.

The Teradata Partners event was completely different; it is organised and run by Teradata customers and there’s a couple of details which I believe make it unique:

The event was not introduced by Teradata; instead the President of the Partners Steering Committee - Erin Redshaw from Loblaw (one of the largest Canadian supermarkets) did an excellent job of facilitating the event.
Most sessions were led by customers (or in my case jointly presented by a customer & Teradata), with the emphasis on knowledge sharing. I was really impressed by the candour of the presenters, openly sharing their challenges, thought processes and results – hoping that their peers learn from their experiences.
Evaluation forms were circulated for each session and one question stood out: ‘Was this a sales pitch?’ This was really impressive - sales pitches (either from Teradata or from customers) were outlawed; I can only imagine presenters scoring badly on this question would not be invited to speak at future events!
Another important aspect of the event was the opportunity for customers to interact directly with their global peers on a 1:1 basis. Teradata actively encourages this type of interaction and is happy to facilitate introductions across their global client base.

Next year the event will be held in Washington D.C., I definitely recommend you all attend.

Mark Hunter

New Analytic Approach to Data Overload

David Placke — 12/12/2011 12:53:57 PM

We’ve all seen the stats: The world’s data is doubling every two years and will reach 1.8 zettabytes by the end of this year. The question is, how can a business rapidly analyze the growing mounds of information—and do it in an efficient manner?

One solution is an analytic platform, uniquely designed to deliver “big-data analytics” by combining data and analytic processing in a single system. As Mayank Bawa, co-founder of Aster Data, explains in “New Class of Analytics,” a platform with an embedded analytics processing engine, built on MapReduce, can identify trends and garner insights from massive amounts of data. It can even analyze new types of data from non-traditional sources, such as mobile devices and texts, which pose challenges for traditional analytical tools and architectures.

Bawa further points out that the Teradata Aster MapReduce Platform has a unique set of capabilities to analyze multi-structured data. The information gleaned from the analysis can be integrated in a data warehouse to be shared with other users giving organizations the information—and advantage—they need to stay competitive.

Brett Martin
Associate Editor
Teradata Magazine

Passing the Baton at Teradata University Network

David Placke — 12/9/2011 12:14:51 PM

Teradata University Network Executive Director Michael Goul is passing the leadership baton, as he moves on after three years of service in building the Teradata University Network to a significant point in its existence. We appreciate Mike’s contributions. Begun as a free teaching portal where Teradata and its business partners provided software, real-world business conditions, and very large data sets to academics and students, Teradata University Network is now a valued learning resource in more than 1,600 universities in 90 countries. More important than its great reach is the expansion of its mission to include a greater role in ensuring that businesses have access to data analytics workers with the necessary knowledge and skills to be effective.

It is through Mike’s efforts, and those of his predecessors Barb Wixom and founding director Hugh Watson, that Teradata University Network has come to this place. These people are the leaders who have been at the forefront for many years, working to train university students to be break-through thinkers and doers who use data to create business value. Mike will continue to serve on the Teradata University Network advisory board as a Fellow and continue his work to incorporate business intelligence into core IT and business curricula.

Join me in extending a warm welcome to the new Executive Director, Paul Gray. Paul is well respected as one of the founders and creators of decision support system database curriculum and is a distinguished professor emeritus of Claremont Graduate University. You might want to take a minute to check out the Paul Gray PC Museum, home to many of the industry’s first computers. As a senior advisory board member for several years, Paul is well acquainted with the mission of Teradata University Network. As the number of data-driven businesses proliferate and the sources of data and the tools to analyze data grow, we’re fortunate to have Paul at the helm of Teradata University Network as it continues to foster the business acumen and technical expertise that companies need to integrate, investigate and implement data analytics.

Darryl

Wave of Change

David Placke — 12/6/2011 12:28:41 PM

The more information you have about a subject, the better decision you’ll make, right? Well, sometimes. The fact is, having lots and lots of information is a double-edged sword. On one hand, the more evidence you can draw from, the more informed your decision could be. On the other hand, some of that information could be incorrect (ever do a Google search?), and sorting through and organizing the information is a task unto itself, before you even start to make a decision.

Magnify this phenomenon to the nth degree, and you’ve got the big data conundrum. In his Teradata Magazine article “Sea Change,” Steve Jones of Capgemini argues that companies ignore the coming big data wave at their peril. The growth in volume and diversity of data coming into any data warehouse makes it easy for unwary companies to make bad business decisions based on poor information. Jones outlines the steps necessary to begin harnessing the big data explosion.

After all, the overall exploitation of big data is worth big money—trillions of dollars— according to the McKinsey Global Institute. Who’s willing to let that pass by?

Jennifer Niemela
Executive Editor
Teradata Magazine

Indispensable Consumer Relationships at the Heart of Consumer Goods Strategy

David Placke — 12/9/2011 4:39:57 PM

You don’t maintain a leadership position without continually out-innovating your competition. For Procter & Gamble, this means embracing all things digital across its business. From marketing to retail relationships and production, P&G is leveraging the latest technologies to advance its purpose to “touch and improve lives,” according to CEO Robert McDonald in this November McKinsey Quarterly interview (registration required).

The interview begins with a bold statement by Mr. McDonald that should have all Consumer Goods makers evaluating their approach to brand management, marketing and retail collaboration:

“With digital technology, it’s now possible to have a one-on-one relationship with every consumer in the world. The more intimate the relationship, the more indispensable it becomes. We want to be the company that creates those indispensable relationships with our brands, and digital technology enables this.”

One of the ways P&G does this today is via something called “Consumer Pulse,” which applies analytics to data generated in the blogosphere, social media and other sources to uncover the real time conversations which can impact sales and brand value. For perhaps the most recent example of where this capability applies, consider this past week’s news around Coca Cola’s holiday packaging reversal.

At a time of unprecedented consumer participation in social media and adoption of mobile technology, consumer relationship development should be a key initiative for all Consumer Goods makers. Not only because of the opportunity to reach consumers, but the fact consumers are today targeted by all businesses, in all industries. Only those wise enough to create, nurture and develop direct relationships with these Connected Consumers are certain to be heard above the noise.

We’re seeing fewer Consumer Goods marketing teams delegate digital efforts to third parties for many reasons, but all which relate back to more effective marketing, relevant communication, and results with a focus on demonstrating return on investment. Even fractions of a percent in sales can be affected by these proven methods, more than paying for the investment.

The quite literally “federated” approach to consumer marketing that has yielded multiple agency relationships, dozens if not hundreds of websites, mobile applications and opt-in email/SMS text message lists, places data, insights and ultimate value beyond the grasp of most Consumer Goods marketers. Only by integrating and reconciling these sources, and melding them with increasingly available retail purchase data and production/forecasting, can the best and most competitive results be attained.

Teradata is an ideal platform for such a solution, that both speeds time to integration while also offering a best in class platform for consumer relationship creation and development (called Aprimo Relationship Manager).

With P&G setting the pace for the entire Consumer Products industry, other manufacturers should consider embracing consumer relationships in the same way they do other facets of their business.

Gib Bassett

Near Field Communication, but far reaching impact with analytics

David Placke — 12/6/2011 10:48:41 AM

"There are some things money can't buy. For everything else, there's MasterCard"; “Don't leave home without it” - These may have been catch-phrase slogans of yester-years by Master Card® and American Express®! These days, the slogan seems to be changing to, “Leave everything else behind, but don’t leave home without your smartphone. There is nothing that your smartphone can’t buy…. ”!. In other words, the mobile device is becoming a multi-function device, the idea of which I was trying to promote in a conference about 10 years ago at the height of the hype cycle associated with 3G network launch, lack-lustre mobile devices and WAP (Wireless Access Protocol), none of which stood up to its promises in terms of speed, ease of use, affordability etc.

Fast forward 10 years; with the ubiquitous availability of smartphones and broadband access combined with affordable prices and multitude of easy to use applications, the time now seems ‘right’! The CEO of Vodafone UK reckons people will now start leaving their wallet at home as well as their keys when these are integrated into their mobile phone as Near Field Communication (NFC) enabled digital access card.

NFC is a standards-based connectivity technology that harmonises today's diverse contactless technologies to make life easier and more convenient for consumers around the world by making it simpler to make transactions, exchange digital content, and connect electronic devices with a touch. It is believed that NFC-enabled mobile wallet will eliminate the need to carry cash, credit and debit cards, reward cards, coupons, tickets and transit passes, fundamentally changing how you shop, pay and save. All with your phone! What if you lose your phone? A Day in the Life covers such a scenario!

There is no shortage of recent NFC trials around the world. Mobile operators, banks and card franchises all want a piece of the action and customers’ wallet share! ISIS in the USA, mobile wallet in Europe, Sony Felica and DoCoMo mobile wallet in Japan, Commbank Kaching in Australia and Visa payWave application for Vodafone NZ and Bank of New Zealand are all attempts at vying for the mobile commerce transaction!

Mobile commerce is not new for mobile operators. After the launch of 3G networks several years ago, customers have been able to pay for micro transactions such as charges for ring tones, screen savers, video downloads etc. through their monthly phone bills. What is new with NFC is that it enables the online and brick-and-mortar worlds to be seamlessly integrated into a common eco system. By bringing together banks, merchants (i.e. retailers, food stores, gas stations and others), advertisers, other marketing partners and consumers, the mobile commerce network is expected to provide an enhanced real world shopping experience that is more accessible, convenient and personal.

NFC allows mobile operators, banks, retailers and advertisers to benefit from a fresh and steady stream of consumer information. By tying existing loyalty programs to NFC smartphones, retailers, wholesalers and vendors will be able to gather customer data for:

demographic analysis, evaluating sales and store trends
judging product preferences, and powering other analytical tasks
operational decision making enabled by active data warehousing, such as detecting stolen products and generating automatic security and stock replenishment alerts
allowing retailers to use customer-generated data to create precisely targeted marketing programs based on gender, age, location and other criteria, as well as to instantly adjust prices and
other purchase terms based on place, date, time and other factors
customer analytics that utilises detailed data on customers’ use of transportation services and commercial facilities.

To keep business cycle moving effectively and positively in the NFC eco system, it is important to create value for both customers and businesses. Customers should be able access and review how many loyalty points they have earned and the value that remains on NFC-enabled mobile wallet. Usage data can be collected from ticket gates at the cinemas, transaction in retail outlets, machines installed in transportation networks and electronic money terminals. At the core of customer analytics is the Enterprise Data Warehouse (EDW) which integrates data across several dimensions and provides actionable information and knowledge-based decision making capability. The history and details held in the EDW can even be used to track the location where you lost your NFC-enabled mobile wallet and possibly send you an alert to your TV screen when it has been found! Imagine that!

Sundara Raman

Setting a New Standard

David Placke — 11/29/2011 4:02:22 PM

Teradata has just raised the bar—again—with Teradata Columnar, a new physical database design option in Teradata 14.0.

Teradata Columnar doesn’t just tweak an existing system—it takes a totally new approach. As Teradata’s Alan Greenspan explains in “Pillar of Performance,” the database allows tables to be partitioned by column or row, or both. This hybrid design improves query performance by eliminating the need to access all columns in a row or all rows in a column. In layman’s terms, it lets users break a data set into small pieces and search only what’s needed to complete a query.

This latest release also makes significant advancements in compression. Teradata Columnar offers six algorithms to automatically compress data as it’s loaded, which reduces the storage footprint for the columnar tables. With this new offering, Teradata has reset the industry’s database standard once more.

Brett Martin
Associate Editor
Teradata Magazine

Why Consumer (and Shopper) Insights Should Trump Facebook

Cesar Perez — 12/16/2011 2:34:42 PM

Facebook is more than the “marketing de jour” for many businesses; it has become almost the sole focus of online marketing activity. Especially for Consumer Goods makers and their marketing teams, the need to “follow the consumer” has led to the prioritization of establishing and growing a presence on Facebook -- almost to the exclusion of other channels and in some cases, logic. Consider:

- As reported in this recent Ad Age article and mentioned in this blog post, Facebook fan page views are stealing eyeballs from brand websites. Surely this is an unintentional side effect of the rush to implement a Facebook presence without consideration for a consumer’s needs throughout the path to purchase.

- Facebook attracts existing customers (“fans”), as pointed out in the above Ad Age article; it’s more like a loyalty platform than acquisition tool. Despite this, brands are the most aggressive buyers of ad inventory on other sites intended to drive visits to Facebook. Online ads, like search, are more commonly used for acquisition.

- Few topics are as hot as “f-commerce” or the sale of products to consumers via Facebook. Yet, as point out in this recent article, direct sales may not make sense for every Consumer Goods maker, and again raises a red flag around these efforts cannibalizing visits to a brand’s website. It also places Consumer Goods makers a step removed from the consumer relationship, not unlike their current dependency on retail channels. Examples like this Heinz Facebook product launch illustrate how control should be a material consideration in any f-commerce strategy.

Procter & Gamble is among the most progressive marketers and although it sells products via a dedicated e-commerce site, this article points out the CPG leader is “learning that more and more people are spending time online to interact with the brands, even if they buy offline.” Stated another way, it’s wiser to first understand consumer behavior and the path to purchase before moving to sell direct to consumers or placing all bets on a Facebook presence.

At least one respected analyst I’ve spoken with agrees, predicting the final winners will be those that focus on relationship creation based on consumer understanding. That takes insights, based on data and analytics, all of which are increasingly complex, multi-source and come in a variety of forms, both structured, unstructured, and multi-structured.

Gib Bassett

Double Dutch

David Placke — 11/28/2011 10:27:03 AM

In 2004, Teradata withdrew from the public TPC-H data warehousing benchmark. At the time we took a certain amount of heat for our decision - with various analysts claiming that our withdrawal was part of a strategy intended to deliberately obscure our pricing model; and various competitors claiming that it amounted to an admission that we had lost our leadership position in the marketplace to the “me-too MPP-lite” products that they had brought to market.

In fact, our reasoning was very simple - and far less calculating. It was based on the old saw that whilst benchmarks may not lie, this has never been known to stop liars from using benchmarks. Well-constructed and well-executed benchmarks – those that represent real-world requirements and operating constraints - can certainly help organizations to understand the relative strengths and weaknesses of competing analytic database products. But many benchmarks are poorly designed and poorly executed. Benchmarks that fail even to reflect the complexity of an organization’s current analytical requirements - let alone the anticipated future complexity arising from the requirement to store ever greater volumes of detailed, integrated data and to expose them to an ever greater number of users for analysis – are at best a waste of time-and-effort, and at worst seriously misleading. The TPC-H benchmark – consisting as it does mostly of trivial queries run one-at-a-time against an 8-table physical data model - was light years away from the reality of real-world data warehouse implementations even seven years ago. It had failed to advance with evolving requirements and had become an irrelevance.

The intervening years have seen data warehouses continue to grow in size (by one recent estimate, by 40% per annum); the continued rise in the importance of advanced analytical workloads, requiring sophisticated in-database processing; and the widespread deployment of tactical workloads alongside traditional decision-support workloads. The TPC-H specification now looks like an historical curiosity; an artefact of earlier, more carefree times, when the industry was young, requirements were simpler - and I still had my own hair.

So imagine my surprise when a vendor whose analytic DBMS technology still cannot run even the obsolete TPC-H benchmark unless some of the queries are first modified claimed in a recent interview that their analytic DBMS is twice as fast as Teradata, at half the cost. The vendor in question is EMC; the product for which these extravagant claims are made is the Greenplum analytic DBMS; and the claimant is EMC / Greenplum’s VP Central and Eastern Europe, Uwe Weimar, in an interview with Hans Lamboo for Database Magazine, issue 7, published November 8th, 2011.

I thoroughly recommend this article to all of you. It made me laugh out loud - and it wasn’t even the predictable inaccuracies of Google Translate’s interpretation of the original Dutch text that were the most unintentionally hilarious parts of the whole exercise.

For those of you without the patience to engage with the full article, I will summarize Weimar’s key claims, as they are reported in the article –

Greenplum has “maximum scalability”, arising from its “unique shared nothing massively parallel [processing] technology”. As some readers will doubtless be aware, Teradata pioneered the use of shared nothing massively parallel processing (MPP) architectures for data processing in… 1979.
“Greenplum has the fastest database” because it is based on open source PostgreSQL. No other explanation is offered by Weimar for this claim - which seems debatable at the very least, given that the Greenplum optimizer cannot even process correlated sub-queries (hence that little difficulty with running the antiquated TPC-H benchmark in its unmodified form). And isn’t this a little bit like me claiming that my car is faster than yours “because it is red”, or that my wife is smarter than you “because she has brown eyes”? (Actually, my wife doesn’t need to be cleverer than you; it’s enough for her that she’s smarter than I am.)
“Teradata comes as an appliance” – well, yes, mostly it does, although there is also a software-only version of the product. What Weimar conveniently fails to mention is that the Teradata model – a vertically integrated appliance, consisting of hardware, operating system and DBMS, integrated and optimized with one another – is now completely dominant, with Oracle (Exadata) and IBM (DB2 and Netezza) all going to market this way. Even Greenplum, of course, has an appliance – the EMC Greenplum DCA machine – that it leads with in the vast majority of its sales motions.
We also learn that Teradata sells a “beautiful NCR appliance”. Whilst we’re flattered by the compliment, Teradata has been an independent and publically traded company since we spun off from NCR in late 2007. Nothing in any Teradata system shipped since that time is badged or sourced from NCR.
“Teradata provides half as good performance at double the price” – leaving aside the lack of any evidence or corroboration whatsoever for this bold and implausibly generic statement, Weimar does rather assume that you don’t want to run a query that contains something as basic as a correlated sub-query. In which case, of course, you can’t run on Greenplum at all - quickly or otherwise - as this basic functionality simply isn’t supported. And given that Teradata offers five different systems with radically different price / performance characteristics, with which member of the Teradata platform family is this comparison supposed to be made, anyway?

At least as striking as Weimar’s claims is his failure to explain that whilst the Greenplum DBMS might be based on PostgreSQL, it also includes a proprietary code base and is emphatically not open source.

Now it’s time for Bart Sjerps (Advisory Technology Consultant (EMEA) for EMC / Greenplum) to join the interview. Sjerps tells us that Greenplum is “strategic” to EMC, because EMC’s headcount has grown from 40,000 to 56,000 in the past year. If, like me, you are excessively pedantic, analytic and rarely get invited to dinner parties, you may be concerned at this point that these statistics conflate total EMC headcount with Greenplum headcount, perhaps to lead us to believe that Greenplum is more important to EMC than actually it is. (When did we all become so cynical? And do you have plans for this weekend? We could all get together and be sarcastic.)

Next Sjerps turns his attention to the buzzword du jour – in-memory computing – and treats us to a survey of sister company VMWare’s Campfire in-memory technology. Reading the interview, you could be forgiven for thinking that this technology is integrated with the Greenplum technology – except that in the next breath, Sjerps appears to suggest that in fact this technology should be deployed alongside Greenplum and will require an administrator to manually assign “hot” data to the in-memory system. Since the temperature of data changes rapidly and unpredictably, this approach is basically irrelevant for large-scale data warehousing – and the reason that our own Teradata Virtual Storage product was designed with the explicit goal of automating the migration of hot and cold data between higher- and lower-performance storage devices.

At this point in the interview, Sjerps appears to claim that the Teradata RDBMS only runs on specialized hardware. For the record: it doesn’t. Teradata systems use x86 processors and are built largely from Intel Standard High Volume (SHV) server technology and always have been. We once flirted with Itanium – didn’t everyone? - and we use hardware compression technology alongside the x86 processors in the latest version of our Data Warehouse Appliance (one member of the aforementioned family of products). But that’s about as “out there” and “non-standard” as we get with hardware because, at heart, we’re software guys. The “proprietary ASICs” providing core functionality that Sjerps refers to are found in the Netezza Performance Server, not in any Teradata system.

Sjerps does make another interesting observation, although he doesn’t say it in so many words: Intel is not delivering us faster CPUs any more, it’s delivering us more processing cores on each CPU socket. In Sjerps own words: "Intel CPU performance doubles every two years. Greenplum does not have to do anything”.

As I noted in my last post about Exadata, multi-core CPUs represent a fundamental architectural change. On the plus side, they mean that Moore’s law can be sustained where the Laws of Physics would otherwise have got in the way. But there is a catch. Or more precisely, there are two catches where analytical workloads executing against large volumes of detailed data are concerned.

The first catch is that the doubling in performance is only realized for explicitly parallel, multi-threaded software. Clock speeds are actually slower than they were a couple of hardware generations ago, so serial processes run slower than they used to. As I explained in the Exadata post, many years ago - and long before virtualization was trendy - much smarter Engineers than me (OK, actual Engineers) took the decision to virtualize the CPU in the Teradata DBMS, so that multi-threaded parallelism is built-in to the Teradata software, not bolted-on after the fact.

The second is that unless the I/O sub-system also gets twice as fast every 2 years, those super-fast-shiny-processors are left there, open-mouthed, waiting for data to process. (This realization, incidentally, is what motivated us to virtualize the storage sub-system in Teradata, so that we can now support a tiered hierarchy of storage; with frequently accessed data on super-fast-but-rather-expensive SSD storage; less frequently accessed data on rather-more-economical-but-much-slower magnetic storage; and automated, software-driven migration of the data between the two tiers).

What is Greenplum’s strategy for ensuring that its DBMS software can scale with Intel’s multi-core CPU architecture, and how will Greenplum exploit emerging storage technologies to make sure that data gets to those processors without them stalling? Alas in this article at least, Sjerps doesn’t say.

As an aside, ask yourself whether kicking back on the beach and relying on Intel to double data warehouse performance for you every two years is really something to shout about. Assume that those commentators claiming average data warehouse growth rates of 40% per annum are correct. That means that a data warehouse that is 10 TB in size at the end of this year has a fighting chance of being 19.6 TB in size at the end of 2013 (since we’re all friends, let’s call it 20 TB in round numbers). If Greenplum relies on improved hardware to double performance – meaning, incidentally, that customers will have to throw away the system that they were sold in 2011 and buy an entirely new one at the end of 2013, since, unlike Teradata, Greenplum has no proven references for hardware co-existence – then, all other things being equal, the performance-to-data ratio of the new system will be exactly the same as the performance-to-data ratio of the old one that it replaces. In this scenario, you will literally be paying Greenplum every two years - just to stand still. You (just about) get the help you need to deal with the requirement to store ever greater volumes of detailed, integrated data – so long as you don’t mind forking out for a brand new system with twice as much storage as the last one - but you’re on your own when it comes to the requirement to expose them to ever greater numbers of users for analysis so that you can actually derive some value from them.

Whist it is easy to see the attraction of this business model to Weimar and Sjerps – those beach holidays don’t pay for themselves! – it may prove less appealing to the guy or girl asked to keep writing the cheques every two years. In reality, explicitly parallel software that can scale with the latest-and-greatest hardware is the absolute bare minimum that you should demand of your analytic DBMS technology vendor; and if you’re going to out-run those exploding data volumes, said vendor’s technology had better do very much better than this by making the analytic DBMS software smarter every year, year-after-year.

In plain English: it simply isn’t enough to scale with increasingly powerful hardware, we need our RDBMS software to use the available hardware resources ever more efficiently.

Empty vassals?
The analytic database market is growing rapidly. Technology - smartphones, sensor networks, the Internet and Social Media - continue to make it possible to create more and more detailed data; and organizations continue to collect and integrate these data so that they can exploit them for business advantage.

Fast-growing markets can support many products; even less mature, less robust and less functional products can find a profitable niche that, if they are smart, their owners can then exploit to generate the revenues that they need to re-invest in making their products more mature, more robust, more functional.

Precisely because a fast-growing market can support many products, a vendor who is confident that his or her analytic DBMS product meets a genuine market need and is differentiated from the competition shouldn’t have to spend much time trashing the competition to win new business. After all, if Greenplum is as important to EMC as Weimar and Sjerps claim it is, EMC will presumably continue to invest in it (Who knows? One day it may even support correlated sub-queries!) And if the product is already as performant as they claim it is, then they can enter into demanding, well-designed benchmarks, secure in the knowledge that their product will do the talking for them. All of which begs two very obvious and very interesting questions which, in the spirit of not trashing the competition, I will let you contemplate for yourself.

I will leave you with a few parting thoughts.

Later in the same article, Weimar is quoted as asserting that Teradata customers don’t buy their systems, but rather are forced to lease them. That is simply untrue - and given that unit list pricing for Teradata systems has been available at www.teradata.com for the last several years or more, it’s difficult to accept that Weimar didn’t know this claim to be incorrect when he made it. Then again, he apparently still thinks that Teradata is part of NCR, that the Teradata RDBMS runs only on proprietary chip-sets - and that Teradata has only one product line. So perhaps it really was an honest mistake. Sun Tzu once said, “If you know yourself and you know your opponent, then you need not fear the result of a thousand battles”. For the sake of his stakeholders, let’s hope that Weimar knows himself rather better than he appears to know his competitors.

In concluding the interview, Weimar is quoted as claiming, “Our main target market is the replacement of Teradata”. If you ever meet Weimar face-to-face, I encourage you to ask him if he really means what he says. If he answers “yes”, look him in the eyes, ask him to speak slowly and make him tell you exactly how many customers – meaning organizations that have paid money for product, not organizations that have taken delivery of an evaluation machine - have migrated from Teradata to Greenplum. Because we count four since Greenplum was founded. Not in the Central and Eastern European markets that you might assume Weimar is most concerned with, but globally. And even that figure requires a generous interpretation of the word “replaced” – and ignores the fact that we, in turn, have replaced Greenplum at between two or three customers (whether it is two or three again hinges on your definition of “replaced”).

In case those data make it sound like this is a two horse race that may come down to a photo finish, then you should know that in 2011 we have competed directly with Greenplum - thus far and by our own count - in fourteen competitions where the result is in. And that we have defeated them in thirteen of those contests.

All of which is rather hard to reconcile with Weimar’s fighting talk. If Greenplum really is twice as fast, at half the cost, why do thirteen-out-of-fourteen customers who have evaluated both solutions chose Teradata?

Martin Willcox

Rise of the machines

David Placke — 11/27/2011 11:08:15 PM

It seems that we have been talking about machine-to-machine (M2M) communications for quite a while, but I have been yet to see the rubber hit the road. Simply it requires two things:

Interfaces and communication protocols to enable machine to machine communications and
Communication networks that can support this type of communications.

While both have been in place for some time, it had not been considered cost effective. What has been interesting is that the communication providers, notably Telstra, has started to communicate to the investor markets that M2M is a big part of their future and that the average revenue per user (ARPU) would typically be < $2.

As the advancement of M2M moves forward, the backend plumbing is going to have to be looked at, this considerable step increase in data volumes will be similar to what the Utilities companies are experiencing today as they roll out the smart meters (4 reads a year to a read every 15 minutes).

The exciting feature of all of this new data is that it will provide new insights into how machines serve their masters, us. The scary feature of all of this new data is that “The Terminator” might become the end point as we enable machines to do more for us, or am I just a tragic science fiction reader?

What can you envision the future of M2M will look like?

Daniel Tehan

The Next Great Discovery

rita zimmerman — 11/22/2011 4:33:02 PM

Human history is rich with curious and enterprising people discovering new ways to make life—and business--better. One famous example is aspirin: Originally developed simply as a pain reliever, it was found to have blood-thinning properties that now make it a life-saving tool for preventing heart attacks. Innovative companies can similarly find new, profitable ways to use business intelligence (BI) by exploiting previously untapped opportunities.

In the latest issue of Teradata Magazine, the cover story, “Challenge Your Data,” examines four key areas that hold promise in helping businesses push data harder to move faster and innovate:

Time awareness features to easily enable historical analysis
The ability to combine geospatial and business data for enriched location analysis
Processing platforms that specialize in finding hidden gems in large, diverse data sets
Storage management systems that automatically move data over time based on how frequently it’s used

The possibilities that lie hidden in data are as endless as the imaginations of the people using it. Like the doctors who discovered aspirin’s life-saving properties, companies have the formula, they just need to find new ways to use it to greater advantage.

Jennifer Niemela
Executive Editor
Teradata Magazine

Integrated data among top challenges P&G, Kraft and others face in tapping Gartner’s “The Power of Me”

David Placke — 11/18/2011 10:35:06 AM

At first glance, the title of a Gartner research paper released a few days ago can be misleading. Titled, “How Manufacturers Can Harness the Power of Tech-Savvy Consumers,” by Analyst Don Scheibenreif, the label “tech-savvy consumer” suggests a focus on a segment of the larger population. Fact is, these consumers increasingly represent the entire addressable market, a point raised later in the paper:

“Recent economic conditions have changed shopping behaviors of almost all consumers, regardless of income bracket. Consumers are doing more research, using more incentives (both online and coupons) and expecting to find the “daily deal” — some kind of savings program at the market.”

No matter your view of Connected Consumer pervasiveness, you cannot ignore the impact their behavior can have on sales, loyalty and brand reputation, as pointed out in my last post. Don’t just take my word for it though. According to Gartner:

“Manufacturers clearly have the opportunity to have an unprecedented connection to the consumer and shopper never seen before, and the ones that do get connected will enjoy significant advantages over those that don't.”

This advantage is summed up by Gartner as “The Power of Me,” or ““…the personal power that each consumer has in terms of shopping for and buying products,” due to their adoption of mobile, social media and online communities. The paper describes how several top Consumer Goods companies are doing it right, or are moving in the right direction, including P&G, Kraft and Pepsi.

Manufacturers possess the “building blocks” to leverage consumer connectedness for competitive advantage according to Gartner, but “…many companies are not reaping the full value of their marketing technology investments due to a lack of integration of the different technologies and organizational silos...”

Although agencies are not mentioned, they are behind many of the “building blocks” Gartner cite which offer Consumer Goods makers the opportunity to “to engage consumers in a two-way dialogue that drives revenue and engagement.” These “blocks” include mobile and social channels, and the programs agencies execute within them – such as promotions, campaigns and coupons, to smartphone applications and mobile websites.

For both single and multi-brands, many of these activities occur outside the four walls of the business and have limited lives. This places much of the resulting data and insights beyond the grasp of the manufacturer’s institutional memory -- absolutely critical if Consumer Goods companies hope to compete for what is sure to become a land grab for Connected Consumer attention and mindshare. Where consumer, customer (retail) and operational data silos and data come together to inform Connected Consumer strategy and execution is where Consumer Goods companies should focus their attention.

Gib Bassett

The “intelligent car” may be just around the corner

David Placke — 11/17/2011 10:10:28 AM

Earlier this year, I noted on this blog that automakers were focussing more and more on their vehicle’s software rather than hardware, to the point where they are actually starting to rethink their business models. The idea is that vehicles might evolve into “app players on wheels”, comparable to the rise of the smartphone and its implications for hardware vendors.

Recent news indicates that this trend is gaining momentum. Toyota and Microsoft have agreed on a joint-venture to build a cloud-based platform that connects cars, homes and electrical smart grids. The plan is to support mobility apps that can encompass communications, GPS, power management and safety – a scope of functionality that goes way beyond cars. GigaOM gives the following example: “Car owners could start their vehicles and air conditioning systems remotely through a smartphone and could also turn on their home heating system from their car using the same platform.” According to GigaOM, the joint platform will be launched for US and Japanese electric and hybrid cars as early as 2012. As for the integration of web-based infotainment systems into vehicles, this has already been realized by vendors such as Daimler. In other words, the intelligent car may be just around the corner, literally.

At the recent Teradata Universe conference, Daniel Rodriguez Sierra of Telefonica said that data analytics has become as crucial to the company’s success as its physical network. If the automotive industry really does evolve as the telecommunications industry has, we can expect its representatives to draw similar conclusions in the coming years. Integrated data will be the new oil? Quite so…

Hermann Wimmer

Benefit From Hot and Cold Data

David Placke — 11/14/2011 5:36:23 PM

Russell Amer, co-founder of Ward Analytics, says managing data by its frequency of use, or “temperature,” enables a more effective use of a data warehouse while providing business and technical benefits.

In “Why Multi-Temperature Data Matters,” Amer offers an easy-to-understand example of how the five temperatures of data affect a retail campaign. The data starts out “white hot” when the campaign is launched and the data is repeatedly queried, then gradually cools to “dormant” after the campaign ends.

Organizations need a solution that can migrate data, based on how often it’s accessed.
Unique to the data warehouse industry, Teradata Virtual Storage provides intelligent hybrid storage man¬agement that automatically places and migrates data based on its temperature. This allows an enterprise to reap the benefits of managing multi-temperature data without user or adminis¬trator intervention. As data use frequency changes over time, it is moved automati¬cally to the appropriate storage location.

Brett Martin
Associate Editor
Teradata Magazine

MapReduce == Easy && Open Source != Accessible

David Placke — 11/14/2011 4:31:57 PM

I’m not a Data Scientist so I can generalise with some sweeping statements, still look myself in the mirror and sleep at night. I do think though Open Source technologies in the Big Data space are being given too easy a ride. The assumption that open source will mean low cost is too easy and in many cases, neglects to look at the whole picture. Big Data needs to be more accessible.

The big data opportunities are surely vast and there is a huge buzz in the market at the moment. It’s still the case though that many business use cases that are espoused are actually commonly done in leading structured environments. So where is the incremental value? How can this be realised? The clamour to be a data scientist is unrelenting but the hype can sometimes cloud the opportunity and I propose that we are still too often missing the necessary alignment between the ‘science of data and the art of business’.

The reason that Data Scientists currently have such mystique is that many implementations of the MapReduce framework are reliant on the ability to not only understand complex analytics against data that is presented in varying structures but also able to code systems in programming languages such as C++, Java, Perl, Python and Ruby.

That’s a challenge and when you have to delve so deep, we are back to the issue on actually deriving and most critically articulating business value and outcomes that I referred to in Get to the Point and that can cause missed opportunities and a failure to realise business benefit.

Aster Data’s SQL-MapReduce capability makes MapReduce simpler and accessible to less tech-savvy but equally statistically-gifted business analysts. This enables users who understand the business questions access to this rich source of insight and then the ability to turn this insight into business benefit.

So is it possible for MapReduce to be made easy? According to this great Forbes article, absolutely! I’ve always loved seeing statistics presented in interesting ways and the mining of Big Data can be fascinating. I was hooked by the presentation given by LinkedIn at the Teradata Partners Conference in San Diego in October and how the new ‘Skills’ product was developed. What was key though was the way the story was told and how it was presented. Giving access to a wider group of users can only increase the likelihood of stronger analytical communities driving greater value and innovation.

Alec Gardner

Why Integrated Data is at the heart of every challenge and opportunity facing Consumer Goods Organizations – And what to do about it

David Placke — 11/9/2011 10:49:47 PM

Recently, a several news items came across my Twitter feed that collectively illustrate how critical integrated data is to the viability of any Consumer Goods business. It’s not too far-fetched to say that those who fail to act, and act soon, will be relegated to commodity producers sooner than they realize, with the “brand” identity fully owned by retail channels. Consider the following:

- 3M is taking the issue seriously, as highlighted in Forbes: “A recent report from Comscore showed that two-thirds of shoppers begin their process online, and the most frequent starting point is the retail site itself. And these shoppers are not simply looking at products—they are engaging, connecting and asking questions. They want to understand what their friends liked, what other options are available, and what people like them ultimately bought.” Moreover, “…you can’t afford to ignore this shift in consumer behavior. Because the battle for the hearts and minds of shoppers is being won and lost deep in retail sites every day, and the brands who get it right will be the ones that thrive.” Implementing the means to capture and act on consumer insights is imperative at a time when the Zero Moment of Truth becomes more real for products in all categories, every day, even those sold in grocery channels.

- As Bank of America recently found, consumer power is real, and is being powered by social media and the connections consumers make among themselves and with the brands they care about. This Chicago Tribune piece highlights BofA and other recent corporate U-turns (e.g. Netflix) within the context of the need for brands to embrace the voice of consumer. Interestingly, Teradata partner-specialist in this area, Attensity, is quoted.

- A blog post summarizing the author’s experience at the Shopper Marketing Expo describes how traditional shopper marketing, conducted within a silo using old-school tactics, must adapt: “End caps, BOGOs, fun displays, print circulars and rebates have long been the go-tos in shopper marketing to drive in-store sales. But as people are not reading digital circulars as often, and there are more options for shopping (grocery delivery, ecommerce, specialty stores, etc.), sometimes consumers need a reason or incentive to go to the store in the first place.” However, “there is still some ambiguity and confusion in the industry on how effective digital can be in driving in-store sales.” Connecting the “data dots” across consumer marketing, in-store shopper marketing, and retail sales transactions is the answer.

- Many in Consumer Goods are loathe to discuss the implications of regulation, in light of impending rules as a result of the U.S. Government’s Food Safety Modernization Act. Viewing this solely within the context of a “risk management” issue misses the larger picture of globally connected and influential consumers. I have seen few examples like that cited in Helsingin Sanomat, the biggest daily subscription-based newspaper in Scandinavia: “Kesko Food’s subsidiary Kespro, a leading wholesaler in Finland, managed to find out within a very short space of time a great many of those shoppers who had bought from them the jars of tainted organic lives that were possibly life-threatening. The names of customers were picked from the K-Plussa programme, the loyalty cards of which are used by more than 3.6 million Finns. The same trick could also be done for other retailers’ customer registers. For example, purchases made with the S loyalty card could be traced back six years.” Brands that opt in consumers to direct relationships, including channel communication permissions, possess a powerful tool to help ensure the health and safety of their consumers in cases just like this.

With “Customer Data, Social Media Top Marketing Priorities for CMOs Worldwide,” you would expect to see better use of social channels than what an astute blogger recently pointed out with regard to Consumer Goods Twitter use. Social media is the very embodiment of the “shiny new object” all industries are focused on, often missing the fact that social is one or any of the following: a community, a data source, a channel, a customer service function or marketing program. With so much at stake with connected consumers, Consumer Goods companies cannot focus on just one aspect of social media to the exclusion of others.

Top Consumer Goods brands are taking a step back and looking at the bigger picture, inclusive of data from all the sources described in the above examples. Their objective: to develop a detailed view of consumers, actionable in multiple ways, from marketing, retail collaboration, and research and development, through operations and production. The data integration challenge this poses cannot be taken lightly, but absolutely must be addressed. Consumer power is not stagnant -- it is accelerating -- and competition for consumer attention and loyalty is a target of competitors and channels alike.

Gib Bassett

Investing in Customer Engagement

David Placke — 11/9/2011 12:01:35 PM

For Malaysian bank Maybank, being innovative and having a leadership position in the market isn’t enough. The bank shifted its focus from selling products to building service-oriented customer relationships through improved marketing techniques.

Maybank employs customer relationship management software from Teradata with impressive results: Lead generation time has dropped dramatically—from weeks to hours. New campaigns take only days to launch, rather than the weeks it took before the CRM initiative. Marketing campaigns are now 10 times more successful. And most impressive of all: Maybank gained $961 million in new sales from the project in its first two years.

All this thanks to the 360-degree view of the customer the CRM software gives the bank and its branches. Read “On The Money” for more on this success story.

Jennifer Niemela
Executive Editor
Teradata Magazine

Protecting the Warehouse

wsol user — 11/2/2011 2:20:39 PM

Cyber attacks are becoming more commonplace, according to a study released in August by the Ponemom Institute. The 50 organizations in the study collectively experienced 72 successful cyber attacks per week, up 44% from last year, at a median cost of $5.9 million per year and as high as $36.5 million annually.

As pointed out in a new Teradata Magazine article “Tokenization on the Node,” protecting data is the ultimate cat-and-mouse game—organizations invest in the latest security system, and cyber criminals try to figure out new ways to get around it. Tokenization, an innovative technology that helps protect the integrated data warehouse, is a solid line of defense against cyber crime.

Instead of using traditional encryptions, which rely on a potentially vulnerable algorithm, tokenization is random. It also replaces sensitive data with alias values called “tokens,” so even if the system is hacked, the tokens are meaningless without the protected lookup tables that reside on the tokenization server. It may not be the silver bullet for every data security situation, but it’s a good place to start.

Brett Martin
Associate Editor
Teradata Magazine

Lessons of the Future

David Placke — 10/24/2011 5:28:31 PM

If there’s one thing you can count on, it’s that cutting-edge technology will evolve at breakneck speeds in the next decade. Data warehousing is no exception. Between now and the year 2020, a relentless push forward will create huge strides in data warehousing and business intel¬ligence (BI).

As the Teradata Magazine article “A Forward Look at the Past” explores, one of the most marked changes we can expect is the inclusion of consumer-sourced data and tools. Social network information, voice data from customer service interactions, instant message feedback and more will be common generators of material for decision mak¬ing. The cool part is, many of the coming innovations will improve the lives of individuals, such as helping people manage their health through personalized analytic programs that measure their cholesterol and exercise levels, and helping them make healthy food purchases. Through innovations like this one, data warehousing is helping make the future look pretty bright.

Jennifer Niemela
Executive Editor
Teradata Magazine

Aussies go bananas at Partners!

David Placke — 11/14/2011 4:49:36 PM

I love Bananas. I’m sure ‘banana’ was one of my first words as a baby and it was certainly one of my 4 year old daughter’s as well. My 2 year old son only started saying it last month after our trip to the UK, where they are 69p per kilo – about $1 as opposed to around $15 that they are in Sydney since cyclone Yasi hit North Queensland in February. My boy doesn’t get many bananas so it wasn’t a word he’d needed to learn until recently!

Some of my colleagues, a number of our local customers and I were at Teradata Partners to hear the latest and greatest stories of how organisations are using analytics for competitive advantage. Just ahead of the Monday morning break the convention centre staff were readying the refreshments and one member of staff was pushing a trolley laden with bananas. We couldn’t help ourselves and set upon him before he could begin to unload them onto the tables. He looked rather bemused until I explained that his trolley load was serious currency where we came from and that we had all been on banana rations for most of the year. A quick snap survey of a selection of other Australians revealed an average consumption of 2 bananas per day throughout the week (with many initially bingeing early in the week).

Having (just a little) more information about his customers meant he had a clearer understanding of how to provide them with better service (or more bananas, to be precise!).

Knowing your customer intimately was a theme of many presentations from Teradata clients at PARTNERS. My favourite was the presentation from Station Casinos that cited and gave evidence of a “cultural change driven by analytics”. Station Casinos has earned industry accolades for its success in the value derived from Data Analytics and Operational Excellence, namely the PMI Southern Nevada Chapter Project Manager of the Year award and 2011 Aprimo Excellence (APEX) award. Anyone that has seen me present publicly knows how I love the ‘Luck Ambassador’ stories in leading Casinos where player analytics coupled with real time monitoring of play enables real time marketing and significant increases in customer satisfaction and loyalty. All processed and executed through an Active Data Warehouse environment.

Marc Oppenheimer, Corporate VP of Relationship Marketing talked about the revolutionary journey Station has been on. How going from 14 customer segments that were based only on volumes of slot play to more than 200 based on detailed analysis completely changed the definition of a top customer. Previously no consideration was made on whether the customer was playing a machine with a yield of 15% or <1% and it was almost impossible to view a guest’s activity and spend across the whole hotel.

A list was generated of some 200 guests whose combined play was costing the company a million dollars. Station had processes in place to target key customers with offers and further analysis showed that these customers who had previously been considered key had as a result received a further 4 million in offers from programs based upon the 14 basic segments.

This has been addressed and Station have focused their activity to the concept of “Marketing to one” with significant success, increasing profitability of slot machines, reducing budgets for offers and reducing error rates to less than 1% - all this in a down economy.

It’s possible to buck the trends by being smarter, by harnessing the power of deep analytics and understanding your audience. It was fortune that enabled the Aussies to satisfy their banana hunger but Analytics that enabled Station to change the culture of Relationship Marketing.

Just an FYI for those of you reading this from foreign shores and concerned about our potassium deficiency down under, we have finally turned the corner and will be enjoying bananas at a reasonable price again soon!

Alec Gardner

In Bringing Gold to Customers, Teradata Earns a Platinum

David Placke — 10/19/2011 10:59:06 AM

Earlier this month Teradata received the highest honor, the Platinum Award, from Gaming & Leisure magazine as part of the G&L Gaming & Hospitality Awards for 2011. We appreciate this recognition from the industry for our efforts to help gaming companies mine the gold that is in their data.

At one time, companies viewed data as a cost – a by-product of their activities that then had to be stored and managed. Today, data analytics is firmly established as the silver lining for companies seeking competitive advantage. Consider how these gaming industry leaders have gained from bringing together information from across their enterprises:

Anschutz Entertainment Group is managing venue performance and effective cross-selling.
Harrah’s Entertainment is ensuring its VIP hosts are interacting with customers who have the greatest revenue potential.
Isle of Capri Casinos is anticipating guests’ needs and exceeding their expectations.
Station Casinos is designing, developing, and deploying highly personalized marketing and promotional campaigns.

Judges for the G&L Gaming & Hospitality Awards were an august group – the industry thought leaders who collectively represent the vast majority of all gaming technology spend in North America. We look forward to continuing our partnership and driving innovation and profitability with our gaming customers – and all of our customers – for years to come.

Darryl

The emerging role of the data scientist

rita zimmerman — 10/18/2011 5:20:54 PM

In 1848, gold was found in California. The news drew hundred thousands of immigrants from all corners of the continent into the area. Miniscule settlements like San Francisco became fast-growing cities, and California quickly developed into the 31st state of the USA. These days, we say that big data is a largely untapped gold stream – so where is the gold rush?

Oh, it’s coming… At the 2011 Teradata PARTNERS conference, Darryl McDonald estimated that about 1.5 million data-related jobs will be created over the next few years. It’s just that mining data is more demanding than panning the mud on the riverside. Indeed, there is a new profile of knowledge workers emerging here, as Stephen Brobst pointed out. IT people usually don’t look much further than the actual programming, based on given specifications, whereas business analysts expect their tools to answer business questions, but will not necessarily think up new ones. This is left to a breed of creative experts that are called data scientists. “A data scientist is neither of these. They are not interested in answering business questions but are more concerned with finding out what the question should be. It’s much more exploratory,” Stephen told media representatives. Data scientists search for patterns and relationships within the data that could yield insights relevant for the business. “They are statisticians and mathematicians looking to get answers from the data.”

As it turns out then, the coming gold rush will be a rush for talent. We already mentioned the prognosis that there will be a massive shortage of deep analytical skills in the United States by the year 2018. Indeed, the study urges companies to recruit capable personnel aggressively, partly because those experts hired early-on will be the ones who shape the big data analytics teams that are being built up now. It also makes clear that a set of highly-skilled data scientist won’t be enough to integrate big data analytics into business processes: Business leaders in an organization will have to develop a baseline understanding of analytical techniques in order to become effective users of these types of analyses. For data scientists will be explorers, not translators. A data scientist would be prone to refine his analysis more and more, possibly close to the point that human decision making will be replaced by automated algorithms. For example, he could identify dubious tax declarations and flag them for closer inspection. But still the taxman who will examine the case needs to understand why it was flagged in the first place to have a workable starting point for his investigation. After all, what would all the gold be worth if you had no concept of currency?

The Power of R

rita zimmerman — 10/17/2011 6:34:58 PM

There was a lot of excitement at the Teradata PARTNERS Conference & Expo earlier this month about new technologies that can be used in conjunction with a data warehouse to produce insights that can be shared across the enterprise.

"R" is one of those technologies that provide a powerful solution to implement analytic methods to resolve business problems such as churn or credit risk analysis.

As Karl Krycha explains in his article "Plug in With Muscle," the functionality of R combined with a Teradata Database offers a rich environment for analytics. An open-source program, R lets users explore and experiment with analytic techniques to determine the best statistical analysis to apply to their business applications.

Krycha notes that Teradata offers a downloadable add-on package for R that allows users to connect to the Teradata Database and use the more than 45 analytic functions available. And since R and the add-on are free, users can reap the benefits without incurring software licensing costs.

Brett Martin
Associate Editor
Teradata Magazine

Is Big Data turning Information Engineering upside down?

Jamie Flynn — 10/14/2011 6:13:46 PM

According to a survey of senior managers, data seems to be spiralling out of control and is having a major impact on organisational decision making. Is Big Data making it more difficult or is it a blessing in disguise?

Whichever way you look at it, the Big Data movement is getting bigger and bigger and is giving raise to runaway growth of the NoSQL community of software developers who work with semi-structured or unstructured data and non-relational data stores that fall into the following categories. (One way or another they employ Map/Reduce parallel programming framework to process and query the data.)

Multidimensional map store - Each record maps a row name, a column name, and a time stamp to a value. (e.g. Google’s BigTable, HBASE and Cassandra).
Key-value store - Each record consists of a key or unique identifier, mapped to one or more values. Redis falls into this category.
Graph store - Each record consists of elements that together form a graph. Graphs depict relationships. E.g. network of transport routes that show shortest path to destination. Neo4j and InfoGrid are prominent in graph store.
Document store - Each record consists of a document such as, Invoice in XML format. CouchDB and MongoDB come up frequently when discussing Document store.

One of the hallmarks of the NoSQL movement is that they all have non-relational distributed data stores that are designed for fault tolerance and based on the principles of BASE (Basically Available, Soft State, Eventually Consistent) and CAP (Consistency, Availability, Partition Tolerance)instead of the traditional ACID (Atomicity, Consistency, Isolation, Durability) properties of the relational database systems (RDBMS).

Another defining characteristic of the NoSQL movement is schema-less or flexible schema architecture. Above all, many of them lack the concept of ‘Transaction’ and data integrity! Also, the methodology for software design seems to be a bottom-up approach.

For me, who advanced my profession in the traditional structured approach to business system development using the principles of Information Engineering (IE), all this is quite unsettling! What is more, the data scientists seem to wield too much power in deciding the ‘schema’ on the fly as well as the business process and software functions all at the same time when writing a piece of code! Don’t even bother mentioning Master Data Management (MDM)!

Although the NoSQL community is divided in their opinion on the virtue of data modelling for NoSQL, they do provide some comfort by emphasising that NoSQL means ‘Not Only SQL’ and does not imply no more SQL! All that is well and good! But, how does the Big Data / NoSQL fit into RDBMS-world, the Entity-Relationship data modelling and development methodology? What is the implication for business planning and information system design?

Let’s take a brief look at the traditional information design approach and review where the Big Data / NoSQL may fit. Information engineering (IE) serves many purposes, including organisation planning, business process re-engineering, application development, information systems planning and systems re-engineering.
As a top-down methodology, IE uses facilitated modelling sessions with senior business managers that help to review the strategic business plans and to develop a strategic enterprise data, function and process models independent of each other but nevertheless in an integrated fashion. It results in reusable processes for rapid delivery into production as integrated databases and reusable systems.

A key aspect of information strategy planning is the design of a logical data model that represents data and their relationship to the business. It is based on entity-relationship mapping that conforms to the principles of relational model and normal form which helps to reduce redundancy and ensures integrity of the database. The logical data model serves as a blueprint for the enterprise data warehouse (EDW) which is implemented in a RDBMS that collects the enterprise data and makes it usable (timely, easy to manage, available, etc). Then the business can gain value from the data time and time again by asking many different business questions. Business managers typically supplement with data from external sources, such as market surveys, and integrate them into the data in the EDW. This allows them to enhance and enrich the data that is already familiar to the enterprise.

With the proliferation of online data and large-scale adoption of social media, business managers are beginning to find a wealth of data in market - sentiments about their products and competitors in the web servers and log files. Big Data / NoSQL can enable agile analytical capabilities on semi-structured data not seen in traditional information engineering which is great news! The managers no longer need to fully rely on the traditional random surveys. Instead, they now have the ability to use Big Data / NoSQL tools to more accurately explore and analyse such real-time market data, then integrate and operationalise them in the EDW while at the same time relying on the stability, maturity, scalability and availability of the RDBMS.

What will information engineering look like in the new age of information management? It will likely leverage the best of both worlds: RDBMS and NoSQL, as shown in the concept diagram below. Interestingly, Microsoft Researchers Erik Meijer and Gavin Bierman have proposed a model for co-existence of SQL and NoSQL which they call ‘Co-SQL’ that is based on Category Theory - which I am still learning! Erik Meijer and Gavin Bierman argue that Big Data / NoSQL is now in the same state of disarray as where relational movement was 30+ years before E.F. Codd was instrumental in standardising it by means of his relational theory. It is interesting times for business intelligence and analytics!

Sundara Raman

PARTNERS 2011 What a great success!

David Placke — 10/14/2011 2:29:41 PM

What a great success!

The 2011 Teradata PARTNERS User Group Conference & Expo is now history and by any standard it was a great success. I want to thank our customers, partners, associates and all of our speakers for their valued contributions.

A vast army of analytic experts are returning to their organizations better able to put their data to work for real value. By doing so, they create data equity – equity that has measurable and quantifiable value to their companies. They are now empowered to turn the dead data lying around their organization into living data - making it useful and relevant to everyone.

The media and key BI influencers were very busy covering our activities and I’d like to share a few of my favorite statements.

In a Wayne Gretzky analogy, a major analyst firm said, “Teradata keeps moving the puck forward, while the competition skates to where the puck used to be.”
“I had the best time at the Teradata User conference. What a wonderful, sane, smart group of people,” said a journalist from SiliconAngle.
“With these latest moves, Teradata continues to keep some distance between itself and the rest of the data warehousing pack in terms of advanced features and the size and influence of its customer base,” stated InformationWeek
“Among the hottest tech trends is the field of data analytics, where Teradata has long been a leader. With the explosion of technology has come a mass influx of data to be collected and analyzed. Teradata helps companies manage and mine that data for making better business decisions,” reported Investor’s Business Daily.

Conference attendance was up from previous years, as attendees enjoyed access to hundreds of business and technical sessions led by industry leaders, plus in-depth workshops, industry-specific presentations, and exposure to the latest products and services in the expo hall. Networking opportunities abounded, with thousands of professionals active in data warehousing and associated technologies.

Thanks to our dedicated group of customers that help guide the conference content, and thanks to all who joined us in San Diego you for making this one of our best ever conferences. My challenge to each of you is take the knowledge gained from PARTNERS and put it to work to help your organization leverage all of your data for competitive advantage. Then come to DC next year and share your story of analytic success!

Darryl

Consumer Intelligence at the Partners Conference

David Placke — 10/13/2011 9:45:43 AM

Teradata CTO Stephen Brobst and his MicroStrategy counterpart, Jeff Bedell, held a panel at the Teradata Partners conference 2011 last week, debating the implications of consumer intelligence – the equivalent of business intelligence at the hands of the customer. This is partly the result of an emerging breed of consumers that is up to using data to make better personal decisions, partly linked to the ever-faster spreading of mobile devices. Stephen cited a Gartner study saying that a third of BI will be used on mobile devices in 2013 – and said that he expected this number to be twice as high, two thirds. There is little reason why the development of consumer intelligence should lag far behind. So it is time to ask, “what are the criteria upon which our customers will judge us, and which data should we provide them to support their decision?” Stephen and Jeff gave two examples: first, a maternity ward could be actively avoided by mothers if its performance is not beyond doubt. This could actually lead to a closure, brought about by market forces. Second, airlines compete both over price and service. Information on the price is easily available but, in the future, so might be the frequency of delays on a certain route. In fact, Jeff said that he checked this before he came to San Diego, finding that the cheapest airline had a track record of one in four flights delayed. Those criteria can be based on big data as much as on traditional data but in both cases the objective must be the same: turning them into new competitive advantages by sharing them with potential customers.

Taking Right Action

David Placke — 10/12/2011 11:29:58 AM

There’s a saying about decision-making that goes like this: Three frogs are sitting on a log and one of them decides to jump off. How many are left? Three. The frog only decided to jump, he didn’t actually do it. How apropos is this saying to a business? When faced with information that requires decision-making at ever-faster rates, sometimes inaction seems the best course.

But predictive analytics provides the perfect opportunity to practice decision management, as James Taylor writes in his article “From Insights To Action.” With automated systems such as those used in ATMs and websites proliferating, businesses can’t afford to hesitate, much less opt for inaction. While some small decisions—so-called “micro-decisions”—seem insignificant in the grand scheme of things, they have a cumulative effect on businesses. That’s why it’s important to practice the three key phases of decision management, as described by Taylor in his article.

The rewards for making the leap are limitless—and predictive analytics is the next wave of business innovation. You can’t afford to hesitate.

Jennifer Niemela
Executive Editor
Teradata Magazine

Innovation’s Loss

Darryl McDonald — 10/7/2011 5:26:23 PM

The closing ceremony at the 2011 Teradata PARTNERS User Conference celebrates the customers and teams that make us the great company we are. But this year it was different. At one point, small murmurs began. Following the event on Twitter – on my iPhone – I saw the reason for the change in the audience’s attention.

We lost an innovator, pioneer and legend in business - Steve Jobs, gone at the age of 56.

Now, whether you are an Apple zealot or skeptic, no one can deny the impact of Steve Jobs. I considered Jobs to be a brilliant man. He had elegance and style. He took risks and changed the world. Jobs once said, “I want to put a ding in the universe.”

Mission accomplished.

Thank you, Steve Jobs, for your inspiration, passion, and relentless vision to redefine the digital age.

The Teradata team sends condolences to his family, the Apple team, and his friends.

Darryl

Ralston: Adversity presents opportunity

David Placke — 10/7/2011 4:01:20 PM

As we all try to do more with less, the challenges present the best opportunities, said Aron Ralston in his keynote speech at PARTNERS. “Adversity shows us what’s important to us, what we’re capable of,” Ralston said.

Ralston, whose story of amputating his own right arm inspired the film “127 Hours,” addressed attendees at PARTNERS User Group & Expo for the final keynote speech Oct. 5 in San Diego. Ralston humorously connected his experience as an amputee with the plight of all businesses these days: “As a guy who’s missing his right arm, I know about doing more with less.”

Ralston recounted his experience of going for a 15-mile hike in Utah in April 2003 alone, having told no one where he was. After climbing down into a three-foot-wide slot canyon, he was pinned by a half-ton boulder for nearly a week. Ralston narrowly escaped death by amputating his own right forearm. Ralston wrote a book about his experience, “Between A Rock and a Hard Place,” which inspired the movie “127 Hours.”

The audience at the closing keynote speech was riveted by Ralston’s recounting of the experience, in which he made several metaphors to using life’s most daunting challenges to your advantage. As an example, Ralston used the boulder to break his arm as he worked to amputate it. “The boulder set me free,” Ralston said, ironically. “May your boulders be your blessings, too.”

Jennifer Niemela
Executive Editor
Teradata Magazine

Bikes and Blankets for Local Children

David Placke — 10/7/2011 3:36:12 PM

Volunteers helped Teradata Cares assemble approximately 110 bikes and 35 blankets at PARTNERS Conference & Expo in San Diego. The bikes and blankets, purchased by Teradata, will be donated to local children.

“Through Teradata Cares, we try to give back to communities around the world. With PARTNERS, we have an opportunity to give back to the city that is hosting us,” Susan Baxley, Teradata director of community relations, told Teradata Magazine.

Baxley expected between 200 and 300 volunteers to participate in assembling the bikes and blankets on Tuesday afternoon. The Teradata Cares area in the convention center was bustling with enthusiastic employees and customers who wanted to show their support and help local kids.

This is the fourth annual Bike Brigade at PARTNERS. The bikes will be given to children through the help of the Salvation Army. The blankets will be given to the Salvation Army and also to The Project Linus Mission, which distributes them to children in need. The group has donated more than 3 million blankets since 1995.

Teradata Cares is active in 45 cities in 14 countries. It is dedicated to building better communities and improving peoples’ quality of life.

Brett Martin
Associate Editor
Teradata Magazine

Big Announcements at PARTNERS!

David Placke — 10/7/2011 3:32:49 PM

The Teradata PARTNERS User Group Conference & Expo is just getting started in San Diego, and already there are big announcements that have everyone talking.

Generating the most buzz is Teradata Database 14, a breakthrough that provides enhanced workload management, optimized multi-temperature data processing, increased temporal capabilities, better security and more application portability.

Scott Gnau, president of Teradata Labs, called it “the industry’s most intelligent high-performance database for analytics. Period.”

The new release of the database will be available this December.

Included in Teradata Database 14 will be the unique option Teradata Columnar. Teradata Columnar is a database technology that fully integrates columnar and row-based tables, offers new levels of flexibility, performance and compression. Its advanced architecture delivers the most comprehensive, ready-to-use environment on the market.

Unlike relational database management systems that only store data in rows, a columnar database stores data in columns. This new columnar capability lets users mix and match columnar and row-based physical storage when it best suits an application. With Teradata Columnar, applications get access to both row- and column-structured data for enhanced flexibility and performance.

For more information on these and other news events, visit the 2011 PARTNERS Media room. And be sure to follow Teradata Magazine on Twitter—the magazine’s editors will be Tweeting live throughout the conference.

Brett Martin
Associate Editor
Teradata Magazine

Driving Faster Time to Solution

David Placke — 10/7/2011 3:26:39 PM

A new opportunity to reach the consumer directly

David Placke — 10/7/2011 3:23:50 PM

Digital Marketing Insights from PARTNERS 2011

David Placke — 10/7/2011 3:18:56 PM

Consumer Power & the Path to Purchase

David Placke — 10/13/2011 10:37:43 AM