Next Generation Analytics Will Be All About Graphs

Posted on: October 23rd, 2013 by Teradata Aster No Comments

Earlier this month, we announced the 6th major release of our flagship Discovery platform, Teradata Aster 6.0. Perhaps the biggest innovation the new platform brings to market is its impressive graph analytics capabilities.

Before we describe what Aster Graph does, let's get something out of the way. Just like MapReduce and Hadoop 5 years ago, Graph Analytics is a subject that may inspire more doubt than excitement. On the one hand, multiple "graph databases" have existed for years. On the other hand, very few Fortune 500 companies are employing graph analytics to drive significant business value at scale. What is wrong? Are graph analytics useful?

The simple answer is that nothing is wrong; the misunderstanding largely stems from the subtle distinction between graph analytics and graph reporting. The fact is, nearly every graph database currently available is focused on quickly storing and retrieving data that is modeled as a graph. They are good at answering questions like "find me all people that are friends with Joe in this social network." This is referred to as “graph reporting” and, although it sounds interesting, it has limited analytical value.

Graph analytics is fundamentally different.

Graph analytics refers to algorithms that process the whole graph to extract deep insights. For instance, the question "which of the people in this social graph are the biggest influencers" is something that falls squarely in the graph analytics category and something that graph reporting databases can't execute well. Teradata Aster 6.0 is the first commercial product to offer support for true, large scale, graph analytics.

At the interface level, we have a similarly confusing situation. SPARQL is a language that has been dubbed "SQL for Graph"; however SPARQL can only do graph reporting, not graph analytics. It is very hard to write a SPARQL query to find all influencers in a social network or run a Google PageRank-like process to find the most popular websites in the web. However, if what you need is to extract some specific points in the graph based on their immediate relationships, SPARQL will do a good job. Just like SQL is good at structured data and reporting but grossly insufficient beyond that, SPARQL is great at graph reporting but it's the wrong tool to use for graph analytics. Teradata Aster 6.0 introduces a new interface that performs native Graph Analytics while being fully integrated with SQL (more on that below).

When we started thinking about graph analytics at Aster several years ago, we had the ambition to develop a system that could simultaneously satisfy three characteristics:

1. It had to go beyond graph reporting to become a true graph analytics platform, meaning that it had to be able to run nearly every algorithm possible on graphs and scale up to billions of nodes and trillions of edges;

2. Because very few organizations have the skills to write their own distributed graph algorithms, we wanted our graph analytics capability to be fully accessible to SQL analysts and users of visualization tools---not only graph PhDs;

3. It had to work seamlessly with other analytical techniques, like SQL, MapReduce, Statistical Modeling, etc.

I am happy to say that after nearly two years of development, Teradata Aster 6.0 delivers all these requirements through its SQL-GR Graph Analytics engine.

SQL-GR comes essentially with two interfaces. A Java interface allows developers to implement arbitrary graph analytics algorithms. Once an algorithm is implemented through this Java API, our platform can scale up arbitrarily and with ease. In addition, our in-house Aster Analytics Team is developing several graph algorithms that can be used out-of-the box to do things like relationship analytics, social network analysis, fraud analytics etc. These functions will extend our existing library of more than 80 MapReduce functions that already allow SQL analysts to do things like time series analytics, marketing analytics, statistical modeling, text analysis, etc.

Whether a function was written by a customer or by our analytics team, the analytical power is always accessible through a SQL or a BI interface. The Aster SNAP optimizer understands and communicates with our graph engine, and so in a single SQL statement one can combine old-school SQL with Graph & MapReduce analysis. For example, a SQL analyst can use a pre-built graph function that identifies the top influencers in a social network and then use SQL to aggregate, slice/dice and present the results. Once the graph function is developed, its power can be used throughout an organization with SQL and even visualization tools like Tableau and/or Spotfire.

The question that logically comes next is whether you should care about this? And if yes, why care now

Since the beginning of the year we have been talking to several Global 2000 enterprises trying to understand the potential impact that graph analytics has for these organizations. If I could summarize our finding in one word, it would be: transformative. The reason is that the two-way interaction, between a customer and a business, is no longer two-way. What's more important is the interactions that customers are having among each other. Whether a telco whose clients communicate with each other or a financial institution that markets credit cards to clients that discuss its products online and offline, every single business has to deal with relationship graphs that cross the boundaries of the physical and the digital. Every customer interaction is part of a graph, whether we're talking about a social, financial, or informational graph; and every decision is influenced by a graph and creates graph-like interactions between consumers and products. The wealth of information and insight currently hidden in graphs is enormous!

The ultimate question is not whether graph analytics can have an impact to your business. Rather, it is whether you will achieve that impact before your competitors do. Taking advantage of the opportunity to use graph analytics to increase your revenue, improve customer loyalty and reduce waste will help you achieve outstanding business performance but, we believe, perhaps more impressively, will empower you to transform your business in ways that until today, simply weren’t possible.

 

Leave a comment

*