Sick as a parrot

Oh dear, oh dear, oh dear.  Electing to change my flights so that I could watch the England versus Germany game in the Lufthansa lounge at Frankfurt airport en route to Madrid on Sunday turned out not to be my smartest ever move; not only did I have to suffer the indignity of watching England crash out to a technically and tactically superior German team, I was also surrounded by jubilant German fans as the debacle unfolded.  For me at least, the controversy over the disallowed Lampard goal should not obscure the fact that England performed poorly throughout the tournament.  The young German team has earned the right to contest a quarter-final with Diego Maradonna's Argentina team, although I cannot quite bring myself to wish them well in that endeavour.

(This should not upset my German friends and colleagues unduly, as every team that I have cheered for on tour has been ignominiously defeated: France crashed 2-0 to Mexico the day after I chanted "allez les Bleus!" in Paris; in Rome, citing a long-dead maternal Italian Great Grandfather, I melodramatically ripped-off my business shirt to declare my allegiance to the Azurri, only to watch Italy succumb 3-2 to Slovakia; and I have said enough already about the England game.  Football and I may be about to enter a period of trial separation.)

And so to Madrid, for the final stop of our 2010 CTO Road Show.  Madrid's Puerta del Sol, is the centre of the country's road network (el kilómetro cero).  The Spanish traffic administration is spearheading the application of analytics in road and traffic management – and is going well-beyond merely spreading information about congested motorways and on-going construction work in doing so.  The authority responsible for traffic administration – the Dirección General de Tráfico (DGT) - is mining its traffic data to identify trouble spots where significantly more accidents occur than elsewhere and then taking measures to improve road safety at these places.  The DGT started on the journey of developing the analytical capabilities necessary to succeed in this endeavour about three years ago, when it started the implementation an EDW that incorporated data from a variety of sources, including driver, vehicle and meteorological data as well as traffic tickets and data from automated radar devices.  By tracking the impact of speed controls, for example, the authorities can optimize the location of their radar devices; bad news for drivers like myself with a heavy right-foot, but important to improving public safety, nonetheless.  This analysis will become both easier and more sophisticated this year, as DGT is preparing to make use of Teradata's new geospatial capabilities to plot the location of traffic incidents with greater accuracy and to manipulate this data much more efficiently.

All of which is proof – if proof were needed – of what my travelling companion and Teradata technology supremo, Stephen Brobst, has been saying throughout our European tour: the "Internet of things" is upon us and time and space are critical dimensions where the Tsunami of sensor data are concerned.  The accurate and consistent representation of time and location attributes are necessary, but not sufficient; DBMS technology must also be able to support scalable, simple, high-performance manipulation of these data.  As Stephen like to point out, "a write only Data Warehouse is not very interesting" critical, we must also be able to manipulate these data.  Stephen's explanation of Teradata's own implementation of high-performance geo-spatial processing is a model of clarity; if you were unfortunate enough to miss his presentation on tour and would like to know more, please get in touch to discuss your requirements.

While Madrid is focussing on the calles and carreteras, Paris is using analytics to improve travellers' experience of public transport.  The Syndicat des transports d'Île-de-France (STIF) has been recording departure and arrival times, carrier and dates of travel - mainly by capturing anonymous mobile phone data of each individual traveller - since 2008.  By using special algorithims to process the resulting data - more than 17 million events per day - STIF is able to model the impact of construction detours or line outages on consumer usage patterns.  This gives STIF a much better idea of the actual transportation demand, frequency, journey time and even punctuality, for which the authority had to rely on passenger surveys in the past.  The result is greatly improved investment decision-making – not to mention a happier and more relaxed travelling public.

This combination of detailed data and geospatial capabilities can be further enhanced with powerful visualization techniques. For example, 4D visualizations add additional dimensions like time to the picture, turning a detailed event map into a movie.  Traffic authorities can use these visualizations to study how traffic patterns vary between weekday rush hours and Saturdays, for example, or to get an impression of the effects that a closed road would have on the whole road system and schedule maintenance works accordingly.  City planners may also be interested to analyze how average income levels in different areas correlate with traffic patterns, to predict likely traffic volumes as cities develop and evolve.

Good visualization technology – like that provided by Teradata partner Tableau - can help analysts to rapidly uncover meaning and relationships in even very large and complex data-sets.  Several million English fans did not need advanced technology to see clearly on Sunday that which one Italian apparently could not: that Peter Crouch would have been much more likely to score the 3 goals that England needed to avoid elimination from the 2010 World Cup than Emile Heskey.  Probably it made no difference to the final outcome, but still Fabio Capello was blind to the bleedin' obvious.

Until the next time, adios, amigos.

Martin Willcox