Monthly Archives: May 2017

Teradata Aster Analytics on Azure: Available Now!

May 31, 2017

Teradata® Aster® Analytics is a multi-genre advanced analytics solution that provides powerful, high-impact insights on all data types of any volume.   Multi-genre capabilities refer to the seamless application of different analytics techniques to address any use case within a single solution.  Aster Analytics does precisely this by allowing users to execute a wide variety of advanced techniques (e.g., Path, Graph, Text, Machine Learning, Visualizations, and R packages) to ingest,  prep,  and analyze data in addition to visualizing and operationalizing insights all within Aster Analytics on Azure  at speed and scale. Aster Analytics enables organizations to attain unmatched competitive advantage and drive pervasive adoption of advanced analytics by every user based on their skills and preferences.

Teradata Aster Analytics on Azure is offered on a variety of scalable Virtual Machines (VMs) in supported Microsoft Azure regions. This new deployment option gives businesses the flexibility to deploy the same analytic software across a hybrid cloud environment for analytic model portability.  For example, businesses can experiment and build models in a sandbox in the cloud then port and operationalize their model on an appliance, commodity hardware, a Hadoop cluster or cloud production environment.  Options are endless.

By incorporating Microsoft Azure as a strategic public cloud deployment option for Aster Analytics, we make it easier for companies of all sizes to realize the value of data across their enterprise with best-in-class multi-genre advanced analytics. Subscribers can be up and running with an Aster Analytics in about an hour.

Aster Analytics on Azure is both similar to, and different from, what we currently offer via other deployment options.


  • Same Aster Analytics software as available in our on-premises solutions
  • Same Teradata database and ecosystem software are also available on Microsoft Azure to create the same unified data architecture in the cloud
  • Same Teradata Consulting and Management Services as available across all Teradata offerings


  • Initial lower worker limit than what currently exists in other deployment options
  • Initial hourly pay-as-you-go pricing without an annual subscription option
  • Initial narrow set of Virtual Machines (VMs) available for deployment – but this will be expanded in coming quarters

Many companies – especially in retail – have already invested in, and aligned themselves with, Microsoft software and Azure cloud services. With the addition of Aster Analytics, retailers can better understand all aspects of their customer’s journey in high-definition.

Any organization of any size can use and benefit because the offer combines the power of Teradata with the convenience and performance of Microsoft Azure. New-to-Teradata customers will find it just as compelling as existing Teradata customers.

To summarize: Teradata Aster Analytics on Azure is available NOW for global deployment via the Microsoft Azure Marketplace. See for yourself. Explore how Teradata’s innovative advanced analytics can help you uncover new data and insights.

Please feel free to let me know if you have any questions about Aster Analytics in the cloud.

Arlene Zaima

Aster Analytics Product Marketing


Mind the Gap: Cloud as a Temporary Fix

May 30, 2017


Businesses working towards entire data centre rebuilds are increasingly considering cloud to ‘fill the gap’ as demands on business-critical applications creep up. Cloud solutions of this type are allowing organisations to extend legacy infrastructures until they can be re-built, upgraded, or until they take the decision to deploy more cloud computing services.

Let’s explore how cloud is bridging the gap as a temporary fix:

Avoiding a ‘floor sweep’ of the data centre

A ‘floor sweep’ of the data centre is when a company faces major capital investments in the replacement of data centre hardware: cloud can be used to mind the gap by avoiding this. Additionally, when faced with limited IT capacity, cloud can be used to support more traditional IT systems to deliver the flexibility that is required in terms of capacity. This is also known as ‘cloud bursting’, and lets companies augment their on-premise systems with short term additional capacity in the cloud.

Example: A national satellite entertainment service provider had aging hardware systems, some of which were nearing their limits or facing reduced support from vendors. The company had a mature IT group that operated its own data centre. In addition to maintaining a large corporate data warehouse, its production applications ranged across all the corporate functions, including managing a large service fleet to install and repair devices among their customer base.

The provider knew it faced major capital investments in hardware upgrades, or a “floor sweep” of its data centre. In facing a limitation in IT capacity, the company put the cloud to the test to see if it could be used to supplement traditional IT systems and provide the flexibility and capacity required.

The organisation found that the cloud was an ideal solution to the demonstrated limitation in IT capacity that it was experiencing, particularly around its needs for development and testing. In its attempt to avoid a floor sweep, the business began to get to know cloud architectures better, and could evaluate the whether the transition of its production systems to the cloud would be an efficient way to fill the gap.

As a result, the company decided to supplement its traditional IT systems with a cloud based development system, minimising a floor sweep across its entire data centre.

This is one story of many that demonstrates the cloud’s ability to bridge gaps previously left open by large scale IT transformation programmes. Therefore, when it comes to considering a full floor sweep, it’s hardly surprising that a multitude of companies are choosing to use the cloud to both plug gaps and capitalise on market opportunities in a cost-effective way.

Bridging the data centre deployment gap

For its traditional big-box data centre, the hardware purchase and installation of an equivalent system would take nearly a year to become production ready. Numerous companies facing these extended timescales have begun to consider the benefits of cloud solutions as a ‘temporary bridge’ to a future hardware purchase.

The right cloud solution can provide a company with the ability to extend the life of a critical application, enabling a medium-term bridge to continue operations and allowing the option to bring the application on-premises at a future time. This can all happen while a major data centre is being re-built, or in place of a large-scale hardware purchase.

Minding the staff gap

Even if an organisation has significant IT infrastructure on-premises, it may choose to adopt a cloud approach to deliver the desired flexibility and speed required for new initiatives. The right solution means a business will be able to avoid any impacts on the existing production environments and IT staffing: it is relatively straightforward to add and subtract cloud resources without diverting significant resources away from the day to day operations activity for the on-premise systems.

Adopting a multi-cloud structure to fill the gap

In order to deal with increasing demands on its current IT system and challenges associated with mixed workload environments, a business might look to a solution with a multi-cloud structure, with different workloads allocated to different cloud providers and technologies.  For example, database management could be hosted on a private cloud behind the business’ firewall with report generation workload being run on a public cloud and sized based on reporting and processing requirements. The reporting systems could be augmented at busy month end periods with capacity reduced during alternative times of the month.  This can ensure a company can support its IT operations economically with a small number of employees.

Ultimately, it’s easy to see the appeal in terms of businesses adopting cloud solutions as ‘temporary bridges’ to a future hardware purchase. In contrast to hardware installations, cloud solutions can be production ready within a matter of weeks, with the additional benefit of costing the fraction of a full re-build. Of the many companies that have successfully chosen the cloud to mind all manner of gaps, it doesn’t come as a surprise to learn that many find their stop gap cloud solutions rise to meet end-to-end business challenges, becoming the go-to solutions for many organisations.

Mike WhelanMike Whelan – Head of Product Management, International

Mike is responsible for Product Management for Teradata’s International region driving global innovations. He serves as the linkage point between the International management, International Field organization, the corporate product management and corporate product marketing.

Mike has held a number of technical roles spanning both pre-sales and post-sales activity. Mike’s background in open distributed systems and network systems led him into Enterprise Architecture and Systems Design.

Since the early 1990s Mike has worked with Teradata systems and has been involved with many large organisations across a number of industry sectors. Mike recently led the Teradata International Big Data Technology COE so has experience of Aster, Hadoop and the Teradata Unified Data Architecture. Mike has a BSc in Computing & Data Processing from Napier University in Edinburgh



Is Automation a Risk to our Job Future?

May 25, 2017

In May 2016, Foxconn reduced its employee strength from 110,000 to 50,000 in one of its factories in China, replacing a majority of its human workforce with robots. It was a shocking development and the reverberations were heard across the world as it reinforced scary predictions by futurists, academics, tech titans and economists that automation will take over up to 50% of all jobs in next five years. Three months later, India’s largest textile player, Raymond, announced plans to reduce its 30,000-strong workforce, replacing one third of its workforce in India by robots. While a lot had been said about automation and use of robotics in India, this was perhaps the first time that a tremor was felt in India. In America, a person of Indian origin has created a machine to make the ‘perfect’ salads. The $30,000 (USD) machine is the size of a refrigerator and will replace humans in the kitchen, making any salad in 10 seconds. For now, a human waiter will still be required to serve it though.

Before we continue, let me define “robots” as technologies, such as machine learning algorithms running on purpose-built computer platforms that have been trained to perform tasks that currently require humans to perform. These technologies may or may not be physical in form. First introduced in the manufacturing sector, automation has proliferated into other segments of the economy. This has been due to availability of huge amounts of data combining with neural and other technologies to enable artificial learning that can be programmed in machines or via software to perform specific and increasingly complex tasks.

The prognosis is not good. Oxford University researchers have estimated that 47% of U.S. jobs could be automated within the next two decades. Thirty-eight percent of jobs in the U.S. are at high risk of being replaced by robots and artificial intelligence over the next 15 years, according to a new report by PwC. Meanwhile, only 30% of jobs in the U.K. are similarly endangered. The same level of risk applies to only 21% of positions in Japan.

The World Bank predicts 69% of jobs being threatened by automation in India. Given that policy makers are already challenged with creating enough jobs for a demographically young India, this is not good news. As per labour ministry data, around 1 million people enter the workforce in India every month. By 2050, at least 280 million more Indians will enter the job market in India.

Automation is not the answer. Or is it?

Technological progress is inevitable. And automation is not a new phenomenon. As part of a globalised and tech-enabled economy, India cannot ignore this phenomenon. What it can do however is learn to adapt to it so that we maximise its benefits for greater rather than lesser good. Machine learning will automate many, if not most, low-level cognitive tasks in the near future. But that doesn’t mean that jobs at the lower end of the spectrum will disappear.

Innovation is one approach. Not just innovation of ideas and technologies but also of thinking. The idea is that our workforce not waste their energy and time on tasks that can be done by machines. Instead they need to look at newer possibilities which will require highly cognitive, creative and critical thinking that is beyond machines. We should therefore focus on the labour force developing new skills that can propel them to the top of the professional food chain. Big data, data analytics, data scientists, machine learning, and IoT are new-age professions that have created a strong and relevant niche for themselves in a short period. Not only has it opened new avenues for developing workforce skills, its demand across a wide array of applications for business practices is consistently growing. Also, we need to come up with newer ways on how to adopt these technologies or how to develop or implement them further. Data scientists have emerged as one of the most sought after workforce by industries across sectors. India is seeing a 32.2% demand with people having such qualifications over and above degrees in IT or business administration or even doctorates per reports.  Medicine is another area that will always require a steady stream of manpower and will be difficult to replace with machines.

At the lower end of the spectrum, a booming digital economy has led to the emergence of new tech based business in sectors such as e-commerce, logistics, fin-tech, ed-tech and agri-services. These will continue providing an opportunity to a young workforce across India. For instance, India’s $17-billion e-commerce industry for instance holds the potential to add 12 million new jobs in India over the next decade. Given the focus on financial inclusion enabled in large part by broadband penetration, India will see the revitalisation of profiles that many considered to be obsolete. With the introduction of new payment banks and a new postal service, a majority of 4,60,000 postal workforce will have a new role to play across 1,50,000 post offices, not just delivering couriers but also offering a host of financial and non-financial services. As technology, financial access and digital connectivity catches up with rural India, expect new opportunities across the value-chain to emerge and flourish.

The debate can continue over the rise of automation and the possible impact on our workforce but the fact is that technology and automation should be enablers and not inhibitors. Therefore, the approach should be on re-skilling ourselves to adapt to this new ecosystem and become adept at leveraging man-machine partnerships.

Sunil JoseSunil Jose – Managing Director,Teradata India

Sunil joined Teradata in June 2014, bringing with him more than 25 years of technology industry leadership experience that encompasses enterprise software and hardware knowledge, general management, sales & marketing , strategy development and executive management experience.

At Teradata, Sunil is responsible for providing leadership and strategic direction to the company’s India business, as well as overseeing and providing guidance to sales, professional services, support, alliances and marketing initiatives. His focus is on strategy, innovation, people management and customer experience.

Sunil is best described as a business leader, an industry veteran, a technologist. He is a results-oriented, decisive executive with proven success in driving key initiatives, identifying new markets and transforming businesses to become more profitable in the Asia-Pacific & Middle East marketplace.

Predicting the Path of Predictive Analytics

May 24, 2017

RS2964_shutterstock_275491472Every day, businesses are predicting the future. Huge sums of money are being spent on collecting ‘historical’ data, but far from being redundant, there is a way to predict what’s coming using this data in combination with predictive analytics. This technique is booming in the world of business, to such an extent that by 2020, Gartner predicts that predictive and prescriptive analytics will draw in 40% of enterprises’ net new investment in analytics and business intelligence (BI). It’s developing rapidly and marketers, analysts, and business owners need to be prepared for what’s on the horizon.

Predicting the path of predictive capabilities may seem counterintuitive but a key challenge facing today’s businesses is understanding how to decipher and utilise the variety of predictive techniques available, how to weave predictive outputs into business processes and the anticipated affect these advancements are likely to have on the automation of intelligence across the organisation. Here’s six ways we think predictive analytics will have an impact:

  1. Greater control & customisation

Predictive analytics, by its nature, provides users with a way to customise and personalise services and products. Organisations will have far greater control with the ability to anticipate needs as well as deliver tailored experiences to audiences. The days of marketing to broad segments of similar customers are long gone and businesses are enabled to have sophisticated campaign execution that creates a one-to-one dialogue with consumers in real-time on any channel of their choice.

  1. Improved diversification

Currently, a large amount of predictive analytics activity is focused on how customer behaviour today will affect the way in which they spend in the future. That being said, many additional applications are being developed and we will see huge diversification in this space in the coming years. This will include the application of predictive analytics to assist businesses above and beyond consumer revenue generation to all organisational areas. For example, in HR this could impact tasks such as recruitment to find suitable candidates, or tracking patterns in employee well-being for productivity. In addition, we will see this extend into smaller, more niche industries.

  1. Internet of Things integration

From mobile phones and automated homes to traditional manufacturing processes, sensors are continuing to flood the marketplace. Sensors and devices can collect huge amounts of data related to environmental conditions, consumer interactions, products on a supply chain, and more. The way this data is interpreted to meet customer needs, carry out preventative maintenance, optimise processes and so on, is a key challenge for organisations searching for success in the marketplace of tomorrow.

  1. Supply and demand

Affordability and availability of a product or service is directly proportional to the amount of people embracing it (as any economist will tell you). At the moment, predictive analytics is relatively accessible, however as the organisational benefits are more clearly understood, the speed of predictive analytics adoption will expand from isolated pockets to enterprise wide business deployment. Consequently, relying on predictive analytics will become commonplace for the leaders of all companies, no matter the size.

  1. Data visualisation

Data, in its raw form, is difficult to interpret and monitor, even for the most experienced data analysts. This explains why data visualisation is such a growing trend; predictive analytics will soon take huge leaps forward in terms of presenting data in an increasingly visual way, helping users gain intuitive takeaways, as well as better communicate their conclusions with improved ease.

  1. Understanding the full picture

Predictive analytics often focuses on the ‘big picture’ insights, patterns and high-level takeaways, which is important for organisational consumption, however as the ability to carry out predictive analytics at scale develops, companies will be able to delve deeper when it comes to predicting the micro behaviours of customers.

What does the future look like?

From data ingestion through to delivery, the entire business analytics framework is going to develop to become better equipped to create real-time insights at the point of work, provide automation for every day, mundane activities, and advise business users of actions. This is all positive stuff. Despite this, to be able to maintain a competitive edge, companies should be looking toward progressing predictive analytic capabilities, as this is the future for data driven organisations. Those that don’t invest will only fall towards the back of the pack.

Yasmeen AhmadYasmeen Ahmad – Business Analytics Leader and Partner Practice at Teradata

Yasmeen is a strategic business leader in the area of data and analytics consulting, named as one of the top 50 leaders and influencers for driving commercial value from data in 2017 by Information Age.

Leading the Business Analytic Consulting Practice at Teradata, Yasmeen is focused on working with global clients across industries to determine how data driven decisioning can be embedded into strategic initiatives. This includes helping organisations create actionable insights to drive business outcomes that lead to benefits valued in the multi-millions.

Yasmeen is responsible for leading more than 60 consultants across Central Europe, UK&I and Russia in delivering analytic services and solutions for competitive advantage through the use of new or untapped sources of data, alongside advanced analytical and data science techniques.

Yasmeen also holds a PhD in Data Management, Mining and Visualization, carried out at the Wellcome Trust Centre for Gene Regulation & Expression. Her work is published in several international journals and was recognised by the Sir Tim Hunt Prize for Cell Biology. Yasmeen has written regularly for Forbes and is a speaker at international conferences and events.


The Future of Health and Human Services Data Modeling (Part 2)

May 23, 2017

RS3607_shutterstock_577112506This is the second half of my thoughts on the question “When will the IT industry provide reusable clinical and administrative data warehouses for the Medicaid enterprise?”  The first posting discussed the current models and systems, and enumerated some barriers to creating a common data model.  In this post I consider how to overcome the barriers to reach the goal of a comprehensive data model for analytics and administration for the HHS enterprise.

Overcoming barriers

How might these barriers be overcome?  How might a common data model be created and adopted?

The profit motivation of commercial business combined with the altruistic drive for health care solutions shown by standards organizations is the most powerful and likely path to a successful HHS data model standard.  One example of this success is the Consolidated Clinical Document Architecture (C-CDA) developed by HL7 and adopted by CMS for Meaningful Use regulations.

Creating a common model is challenging in any industry, but has been accomplished by industry groups for accounting and government billing.  Industry and vocational groups provide the most promising avenue for development of enduring standard data models.  Business focused non-profit standards organizations coupled with government target-setting has been the most successful approach in the finance industry, and is the best path to a standard model for HHS.

However, healthcare industry standards organizations dealing with IT have less history and weaker accreditation associations than some other industries, e.g. accounting and finance. To the large healthcare IT organizations, and to the medical profession itself, there is still significant financial dis-incentives to promote interoperability.   This leads to lackluster results in health and human services data model standards.  Two promising standards come from Health Level 7 (HL7) and Observational Medical Outcomes Partnership (OMOP).

These models are designed for a specific purpose to meet a specific need.  Consequently they have limited scope, and are for the most part, very generalized.  These models are hindered by their development process… development by committee, and also by the need for a solution that attempts to satisfy everyone at the cost of specificity.  The result is a “one size fits all” data model containing compromises, including some designs using academic/institutional logic rather than direct business requirements.

Effects of establishing a common data model

If a common data model is established, would the healthcare IT industry respond?  Yes, the industry would attempt to deliver “best of breed” data sourcing and analytic solutions based on the data model standard.  The financial incentive of high dollar sales and the potential future subscription revenues will motivate developers to meet the need.  However, the market forces hindering this development are the limited number of customers combined with the high cost of development.  With state and Federal governments being the primary customers, the opportunity for volume sales does not exist and the business requirements for each state procurement will remain unique, thus making the market even more challenging, particularly for smaller businesses.  Procurement processes are lengthy and prohibitive to all but very well-funded sales organizations, which stifles the creativity and the competition that could otherwise be expected when data models become standardized.  There might be easier ways for a developer to turn their capital and time into a return on investment.

What Teradata adds

The Teradata Healthcare Data Model (HCDM) is designed for the entire scope of an organization providing health and human services.  It is based on the known needs of many healthcare customers over years of development.  The customer requirements are augmented with information from successful industry standards.  The customer needs provide concrete requirements for known current needs.  The industry standards provide content that informs current implementation, and enables flexibility for potential or unplanned implementation.

In addition to the HCDM, Teradata provides advanced analytics for both structured and unstructured information with their Unified Data Architecture (UDA).  The UDA has been evaluated and praised by industry thought leaders like Gartner and Forester.

The combination of structured and unstructured data, analytical capability, and a powerful database can result in unprecedented insight into population health, healthcare practice, and business action.

00-Lee Arnett - blog bio 16 clearLee Arnett specializes in data warehouse architecture and implementation. For the last 5 years he has been the Product Manager for the Teradata Healthcare data model. Previously he implemented Teradata products, involving project management, development, and implementation. Most of that implementation involved building and loading third-normal-form and star-schema data warehouse models, with a focus in financial and insurance systems. Lee also worked for Kaiser Permanente, a Teradata customer.


Stuck in a Marketing Rut? Key Questions to Ask Yourself

May 19, 2017


There’s one marketing problem that plagues us all — the abandoned shopping cart. Is the best approach to closing a sale to send out an email offering a 30 percent discount to that customer or is there a better, savvier option? Perhaps engaging on another channel altogether?

If your company is still taking a line-item approach to its data management, the answer can be hard to quantify. The good news is that you are not alone in the marketing data confusion.

A 2017 survey by the CMO Council reveals that while 42 percent of marketers’ top priority this year is turning standalone campaigns into comprehensive customer experiences, only 5 percent feel their technology investments are up to the challenge.

At a recent event co-hosted by Teradata and the University of Pennsylvania Wharton Customer Analytics Initiative, speakers parsed through the disparate data points that drive enterprise marketers and their digital channel-focused counterparts. The answer, they found is bringing those disparate activities together in a customer hub.

Instead of staying the course, marketing professionals can reinvigorate their efforts by calibrating enterprise data with embedded digital channels, honing in on customer interaction throughout the entire buying cycle. By taking a data-driven approach, marketers can deliver a compelling customer experience, but to deliver on this goal, marketers need to ask themselves a few key questions.

Are You Still Tagging Your Website to Track Data?

In 2012, Google implemented website tagging — a way of pinning metadata from each page of a site to search engine results — and the practice is still going strong five years later. It has allowed companies to grow the approach into a full-on web analytics strategy. But, despite its merits, it’s not enough to inform customer-journey marketing.

Website data doesn’t provide the whole picture of how a buyer is behaving. Let’s get back to that shopping cart question. What if that particular buyer uses a shopping cart to pre-shop before buying in-person? What if they typically abandon their cart just to come back two weeks later when it’s payday? Website tagging (and tracking) alone will never give you insight into these more-profitable options, but a holistic view of customer-level data can.

Companies need to get their data out of silos, so marketers can analyze the entire buying picture.

The other consideration is marketing clouds. Marketing clouds are supposed to give you a more integrated way to deliver your marketing messages to your buyers. In reality, these big databases are really just repurposed email service providers in disguise, with a few extra digital channels thrown in. You can think of these marketing clouds like factories, where the focus is on output and productivity. Sounds a lot like the abandoned shopping cart again, right?

Is a Customer Journey Hub the Solution?

As part of the University of Pennsylvania Wharton and Teradata event, the solution that arose is the idea of a customer journey hub. This hub should talk with email, mobile, web, social, paid media, your CRM, DMP/DSP and more. Through this hub, you can deliver specific messages instead of campaigns.

The messages from a central hub fall into three categories — planned messaging, real-time interactions and journey detection. This last item supports customer needs proactively via automated opportunities found using your data. For your customer, this type of interaction will feel less like they are getting processed through that factory and more like you are the air traffic controller allowing them to land a plane.

While each enterprise will have varied data and analytics issues, to connect with the modern buyer, all of the information should come together to help personalize their journey. If your marketing methods are steeped in some five-year-old best practices and unnecessary silos, then it’s valuable to rethink your data, analytics and interaction strategies.

How are other marketers doing this? You can read real world stories here.

Portrets Teradata_13Mark Swenson is a Principal Consultant for Customer Journey Solutions at Think Big Analytics, a Teradata Company.

In his 18 years at Teradata Mark has helped clients around the globe drive successful outcomes in digital marketing, optimization, marketing automation and customer analytics.

Improve your marketing through AI-influenced analytics

May 18, 2017

analyticsFor businesses to survive today, they have to embrace new trends quickly.

We’re no longer living in the world where new developments, like airplane travel or television, take decades to reach market saturation. Today’s technology adoption cycle is blink-and-you’ll-miss-it fast, even compared to the advent of computers, mobile devices and the internet, which transformed their respective markets in just a few decades.

We are now living in the age of exponential connectivity. It’s hard to keep up with all the new devices out there that can provide connected data and, therefore, a real-time look at how buyers are making their decisions. Expectations are running high on how marketers can take advantage of this plethora of data streams. With Internet of things devices projected to exceed the 50 billion mark by 2020, do marketers have the right business solutions to keep up?

This is a question I explored during a recent in-house event held in conjunction with the University of Pennsylvania Wharton Customer Analytics Initiative. And the answer is, yes, they do, but they must integrate artificial intelligence into existing analytics.

Analytics in the Enterprise

Before hitting fast-forward on their data efforts, companies need to assess the state of their current analytics.

A guy who knows a thing or two about technology transformation, Jeff Bezos, once said, “if you are fighting analytics, you are fighting the future.” But when you go to describe the state of analytics at most enterprises, the words that come to mind may more likely describe hindrances instead of enhancements.

During the event, I touched on these five — static, reactive, siloed, opaque and rules-based. As data grows, different business units have different data — or different definitions of the same data — making it hard to move, share or analyze. But the data is still there, so expectations of how a company will perform based on its analysis are pegged high.

How can a company measure incremental sales in this disparate data landscape? Getting it all right in real time, to make a real impact with the buyer, is possible by using artificial intelligence as a business solution.

AI for Analytics

From housing it to gaining insights from it, AI can help businesses deliver on the promise of their data. Gartner predicts that by 2019, deep learning — AI’s self-improving grandchild — will provide best-in-class performance for demand, fraud and failure prediction. Enterprise needs to prepare for this revolution that is two short years away.

Big companies are already making significant technology investments in AI. Google, for example, used its DeepMind AI to cut the cost of its hardware energy consumption by 15 percent. By leveraging neural networks, Google determined the most efficient way to control for 120 variables in its data centers, analyzed through a series of sensors.

This same concept could be applied on the analytics side. Currently around 80 percent of time spent with data is used up simply manipulating the data. But if AI were leveraged for the analytics itself, companies could spin up the next generation of data analysis — self-service analytics. Instead of feeling deflated by these siloed business intelligence tools, self-service analytics will enable more business users to get marketing information from their data, exactly when they need it. This move will allow enterprise to focus on their buyer in real time and deliver on their business’ value.

Bring it In

Users are demanding analytic innovation. Instead of letting it happen outside your company, bring it internal, so your marketers can meet the high demands placed on them. Adopting AI creates a flexible strategy that can address both accelerating technology and the increasingly personalized needs of your buyers.

How does AI help deliver a better customer experience? Learn more here.


Mo Patel – Practice Director, AI & Machine Learning, Think Big Analytics

Mo Patel, based in Austin Texas, is a practicing Data Scientist at Think Big Analytics, A Teradata Company. In his role as the Practice Director, Mo is focused on building the Artificial Intelligence & Deep Learning consulting practice via mentoring and advising clients and providing guidance on ongoing Deep Learning projects.  A continuous learner, Mo conducts research on applications of Deep Learning, Reinforcement Learning, and Graph Analytics towards solving existing and novel business problems. Mo brings a diversity of academic and hands on expertise connecting business and technology with Masters in Business Administration and experience as a former Management Consultant while also having worked as a Software Engineer with Masters in Computer Science and Bachelors in Mathematics.

Discovery, Truth and Utility: Defining ‘Data Science’

May 16, 2017

RS2980_shutterstock_304372466Gregory Piatetsky-Shapiro knows a thing or two about extracting insight from data. He co-founded the first Knowledge Discovery and Data Mining workshop in 1989 that we briefly discussed in the second installment of this series of blogs. And he has been practicing and instructing pretty much continuously since then.

But what is it, exactly, that he has been practicing? Even Piatetsky-Shapiro might struggle to give you a consistent answer to that question, as this quote of his from 2012 hints:

Although the buzzwords describing the field have changed – from ‘knowledge discovery’ to ‘data mining’ to ‘predictive analytics’, and now to ‘data science’, the essence has remained the same – discovery of what is true and useful in mountains of data.

We like this quote a lot. Firstly, because it speaks to the fact that historically we have used at least four different terms – knowledge discovery, data mining, predictive analytics and data science – to describe substantially the same thing. The tools, techniques and technologies that we use continue to evolve, but our objective is basically the same.

And the second reason that we like this quote so much is because it contains three words that we think are key to understanding the analytic process.

Discovery. True. And Useful.

Let’s take each of these in turn.

Analytics is fundamentally about discovery. It’s about revealing patterns in data that we didn’t know existed – and extrapolating from them to try and know things that we otherwise wouldn’t know.

In fact, the analytic discovery process has more in common with research and development (R&D) than with software engineering. If we are doing it right, we should have a reasonably clear idea about the business challenges or opportunities that we are trying to address – for example, we may want to try and measure customer sentiment to establish if it is correlated with store performance and to understand which parts of the shopping experience we should try to improve to increase customer satisfaction. Or we might want to predict the failure of train-sets based on patterns in sensor data. But often we won’t know which approach is likely to be most successful, whether the data available to us can support the desired outcome – or even whether the project is feasible at all. And that means – first and foremost – that whatever we call it, analytics is about experimentation. Repeated experimentation.  As Foster Provost and Tom Fawcet put it in their (excellent) textbook Data Science for Business: “the results of a given step may change the fundamental understanding of the problem.”  Traditional notions of scope and requirements are therefore often difficult to apply to analytics projects.

Secondly, whilst many process models have been developed to try and codify the analytic process and so make it more reliable and repeatable – of which the Cross Industry Standard Process Model for Data Mining (CRISP-DM) shown below is probably the most successful and the most widely known – the reality is that analytics is an iterative, rather than a linear process.  We can’t simply execute each step of the process in-turn and hope that insight will miraculously “pop” out of the end of the process. An unsuccessful attempt at modelling, say, customer propensity-to-buy, may cause us to re-visit the data preparation step to create new metrics that we hope will be more predictive. Or it may cause us to realize that we are insufficiently clear in our understanding of the business problem – and require us to start over. One important outcome of all of this is that “failure” rates for analytics initiatives are high. Often, these “failures” really aren’t failures in the traditional sense at all – rather they represent important learning about which approaches, tools and techniques are relevant to a particular problem.  The industry refers to this as “fail fast”, although it might be more appropriate to call it a “learn quick” approach to analytics. But whatever we call it, this high failure rate has important consequences for the way we organize and manage analytic projects that we will return to later in this series.MWFS blog1

There are many ways in which data can mislead, rather than inform us. Sometimes we can find results that appear to be interesting, but that are not statistically significant. We may conflate correlation with causality. Or we may be misled by Simpson’s paradox.  Paradoxically, as Kaiser Fung points out in his book Numbersense, big data can get us into big trouble, by multiplying the number of blind alleys and irrelevant correlations that we can chase – and so causing us to waste precious time and organizational resources.

But something even more basic can also trip us up: data quality. The most sophisticated techniques, algorithms and analytic technologies are still hostage to the quality of our data.  If we feed them garbage, garbage is what they will give us in return.

We cannot automatically assume that data are “true” – in particular, because the data that we are seeking to re-use and re-purpose for our analytics project are likely to have been collected to serve very different purposes.  Analytics of the sort that we are undertaking may never have been intended or foreseen. That is why the CRISP-DM model places so much emphasis on “data discovery”; it is important that we first understand whether the data that are available to us are “fit for purpose” – or if we need either to change our purpose and/or to get better data.

Defining data science

So how then, should we define data science? Spend 10 minutes with Google and you will find plenty of contradictory definitions. Our personal favorite is –

Data Science = Machine Learning + Data Mining + Experimental Method

It may lack mathematical rigor, but it’s short, sweet – and, if we say so ourselves – spot-on!


Martin_WilcoxMartin Willcox –
Senior Director, Go to Market Organisation (Teradata)

Martin is a Senior Director in Teradata’s Go-To Market organisation, charged with articulating to prospective customers, analysts and media organisations Teradata’s strategy and the nature, value and differentiation of Teradata technology and solution offerings.
Martin has 21 years of experience in the IT industry and is listed in dataIQ’s “Big Data 100” as one of the most influential people in UK data-driven business. He has worked for 5 organisations and was formerly the Data Warehouse Manager at Co-operative Retail in the UK and later the Senior Data Architect at Co‑operative Group.

Since joining Teradata, Martin has worked in Solution Architecture, Enterprise Architecture, Demand Generation, Technology Marketing and Management roles. Prior to taking-up his current appointment, Martin led Teradata’s International Big Data CoE – a team of Data Scientists, Technology and Architecture Consultants tasked withassisting Teradata customers throughout Europe, the Middle East, Africa and Asia to realise value from their Big Data assets.

Martin is a former Teradata customer who understands the Analytics landscape and marketplace from the twin perspectives of an end-user organisation and a technology vendor. His Strata (UK) 2016 keynote can be found here and a selection of his Teradata Voice Forbes blogs can be found online, including this piece on the importance – and the limitations – of visualisation.

Martin holds a BSc (Hons) in Physics and Astronomy from the University of Sheffield and a Postgraduate Certificate in Computing for Commerce and Industry from the Open University. He is married with three children and is a lapsed supporter of Sheffield Wednesday Football Club.  In his spare time, Martin enjoys playing with technology,flying gliders, photography and listening to guitar music.

Frank Sauberlich

Dr. Frank Säuberlich – Director Data Science & Data Innovation, Teradata GmbH

Dr. Frank Säuberlich leads the Data Science & Data Innovation unit of Teradata Germany. It is part of his repsonsibilities to make the latest market and technology developments available to Teradata customers. Currently, his main focus is on topics such as predictive analytics, machine learning and artificial intelligence.

Following his studies of business mathematics, Frank Säuberlich worked as a research assistant at the Institute for Decision Theory and Corporate Research at the University of Karlsruhe (TH), where he was already dealing with data mining questions.

His professional career included the positions of a senior technical consultant at SAS Germany and of a regional manager customer analytics at Urban Science International.

Frank Säuberlich has been with Teradata since 2012. He began as an expert in advanced analytics and data science in the International Data Science team. Later on, he became Director Data Science (International).


The Future of Health and Human Services Data Modeling (Part 1)

May 15, 2017

RS3195_shutterstock_276888647The question was put to me “When will the IT industry provide reusable clinical and administrative data warehouses for the Medicaid enterprise?” It was a thought-provoking question that was well worth considering.

Currently the healthcare insurance and provider industries are guided to target reporting that is defined by government regulation and motivated by financial incentives. These specific measures do lead to the creation of tracking and reporting system scoped to meet the requirements for reimbursement from government programs. This does not lead to a comprehensive data warehouse model or analytic platform for the Medicaid enterprise.

Federal government reporting regulations and the new procurement rules for “modularity” do not strictly require a comprehensive data warehousing platform. With an overwhelming task of re-doing MMIS systems, States are struggling with merely defining “modules” and ensuring the basic required functionality is being achieved. Though the business and policy people recognize the need and value for analytics, those are not “requirements” and, thus, are pushed off for future scoping. Unfortunately, that means most of the early modular MMIS EDW RFP’s have been dominated by traditional reporting measures and pre-written applications for tracking statistics, rather than requirements which will set the States up for future flexibility and growth.

The RFP’s lack serious and specific requirements that focus on the future extensibility, data normalization and virtual views of data within a data warehouse platform. Thus, the vendors responding to these RFP’s, in order to provide competitive bids, must often use technologies and infrastructures which are minimally capable of providing reports and data aggregation.

Existing HHS (Health & Human Services) model standards and systems

There are existing data models or standards (for example, MITA, National Human Services Interoperability Architecture (NHSIA) and National Information Exchange Model (NIEM), that attempt to serve as a foundation for a state based HHS data warehouse. While these are a start, they fall short of what is needed to define a data model.

• MITA specifications serve as a good “ruler” to measure the content and capability of a data model for HHS. Running MITA scenarios against the data model in the form of a “data scenario” can provide valuable insight on how the model is designed to work.
• The NHSIA architecture framework does not lead to a granular solution that is specific enough to generate a standard comprehensive HHS data model.
• The National Information Exchange Model (NIEM) serves as a basis for information exchange, but does not lead to a data warehouse model. It only helps determine the requirements needed for development of a data model.
Barriers to producing a common HHS data model
There are barriers to producing a common HHS data model. Some barriers are:
1. The lack of maturity in the Health and Human Services information technology infrastructure and industry groups.
2. The desire of software vendors to gain and hold market share for their proprietary products.
3. The scope and funding of development projects for specific state HHS systems are impermanent and inconsistent due to political issues. Data model process can take many years to complete and it is hard to ensure that money will be available as administrations change.
4. The U.S. historical and constitutional based desire for local control by states and municipalities is a foundation that limits the ability of a Federal government to dictate standards and systems. Consequently, governments other than the U.S. may be more likely to define and force development of a standard data model.

Administration of health and human service programs by states and municipalities creates disparate requirements for standards and systems. From the perspective of a commercial IT development company the market is sparse and fragmented. The funding of development projects for specific HHS organizations can be impermanent and inconsistent. For these reasons and others, the market place has not developed comprehensive standard data models.

How might these barriers be overcome? How might a common data model be created and adopted? I will suggest my favored solution in my part two of this blog.

00-Lee Arnett - blog bio 16 clearLee Arnett specializes in data warehouse architecture and implementation. For the last 5 years he has been the Product Manager for the Teradata Healthcare data model. Previously he implemented Teradata products, involving project management, development, and implementation. Most of that implementation involved building and loading third-normal-form and star-schema data warehouse models, with a focus in financial and insurance systems. Lee also worked for Kaiser Permanente, a Teradata customer.

Machine Networks – Competitive Strength In Numbers

May 11, 2017

Personal experience informs everyday decisions. And the wiser heads among us combat any individual biases that might have influenced their thinking by seasoning judgement with other, more diverse opinions to enable faster, more accurate solutions.

The interesting thing is that this kind of collective intelligence is not an exclusively human trait. Machines do it, too.

The collective computer brain

It hardly needs saying but individual algorithms have strengths and weaknesses. Some are better at dealing with sparse data sets, some handle only numeric inputs, and others consume text like nobody’s business – each attribute colouring the quality of the algorithmic prediction. In the same way, the data source and wrangling method can give one algorithm a clear advantage over others. Not surprisingly then, applying multiple algorithms in concert (aka Ensemble Modelling) can enhance performance considerably.

In fact, more advanced artificial intelligence (AI) algorithms such as neural networks make use of collective intelligence (little networked machines working together towards a common goal).

One hand washes the other

Okay, we know that collective intelligence works with humans and that it can be leveraged between multiple algorithms. But should application be kept within one population or broadened out to include human and machine together? Humans and machines working together can create unique value. For example, when it comes to detecting cancer, medical-imaging analytics have proven to be more accurate than the deductive powers of human pathologists. But a pathologist’s input to image analytic algorithms can help to assess how advanced the cancer is.

So, machine and human decision making are on a par. However, the machine’s ability to automate allows businesses to make millions of decisions that would otherwise be impossible. Speed of execution is a huge benefit and a key differentiator for machine learning techniques.

The power of automation

KPMG predicts that part-automating the insurance-claims journey could cut processing times from months to minutes. Similarly, a SkyFuture drone operator and engineer in the oil industry can complete a rig inspection in five days instead of the eight weeks it usually takes.

Automation allows tens of thousands of decisions to run in parallel. And each business decision has a massive effect on the environment, markets, customer opinion, etc. Making sure a proposed decision is the best possible option requires the execution and observation of multiple decisions in parallel – a challenger methodology.

One vision; multiple viewpoints

The assessment of multiple decisions also benefits machine-learning algorithms. They learn from the positive and negative effects of decisions, altering predictions to mitigate or enhance particular outcomes.

In the field of sports science, analytics companies provide coaches with recommendations to improve the conditioning and performance of individual players. Following a single strategy would become predictable, so athletes are taught different techniques and approaches as part of a programme of continuous improvement.

Intelligent machines 2.0

Machine learning within business is in its infancy, e.g. we still need to manually create and feed algorithms to ensure precision. But before too long, AI will develop two-way interaction. Machines will help us challenge our biases by asking questions that require additional (or more precise) data. Currently, machines are limited by having to learn from the data we decide is relevant. The next wave of supercharged machine learning will be able to navigate its own learning programme. This human : machine partnership will benefit the c-suite considerably by freeing leaders from bias, automating run-of-the-mill management, and allowing them time to develop creative and insightful actions.

Through technological advances such as the cloud, computing power, and the application of data and analytics at scale, machine learning is now available to all. The real challenge for executives will be changing corporate and operational cultures to maximise the benefits of data-driven decision making.

Because human : machine collaboration is the key that’s going to unlock business intelligence for the foreseeable future.

Yasmeen AhmadYasmeen Ahmad – Business Analytics Leader and Partner Practice at Teradata

Over the years, I have worked with many businesses to identify challenges and understand business context, providing an analytical perspective for achieving solutions. This has meant using new, or untapped, sources of data coupled with innovative analytics to enhance competitiveness. Techniques have included event-based analytics, predictive modelling, natural language processing, time-series analysis, and attribution strategy development. My work takes me to all parts of the globe, covering a wide range of industries including finance, retail, utilities, and telecommunications. I speak regularly at international conferences and events. After an undergraduate in computing, I took a tangent into the world of Life Sciences, completing my PhD in Data Management, Mining, and Visualisation, at the Wellcome Trust Centre for Gene Regulation and Expression. Following this, I worked as a data scientist, building analytical pipelines for complex, multi-dimensional data types. Along the way, I’ve provided leadership, training, and guidance, for many analytical teams, creating actionable insights and business outcomes through the development of analytical use cases. I have also lectured on Masters programmes for BI and Data Science. I grew up in Scotland, and love the great outdoors. Consequently, I’m an avid hiker (the Scottish Munros are perfect) and sea kayaker. Oh yes, and a keen traveller, too.