Why Can’t Power Engineers and Data Science Just Get Along?

Thursday November 6th, 2014

Having spent much of the summer looking at the ‘fluffier’ customer side of Utilities, I have recently focused more on the ‘hard stuff’, in distribution and generation (although they too need, and indeed are starting to do ‘fluffy stuff’ with analytics… but that is a whole new blog).

I understand these parts of a utility operate (and indeed are incentivised) differently to their retail cousins, but based on the increase in the data their assets and operations will produce going forward, they must need analytics as well right?

My answer is clearly ‘yes’, otherwise I would not pose the question. However, in my role as a ‘fixer’ between the analytics community run by data scientists, and these Utilities sectors run by engineers – I am often more a marriage counsellor than a thought leader. So why is such an obvious fusion so hard to initiate and maintain?


From an engineering perspective, electricity in particular is all about science. The science tells you how to build, how things work, and governs your world. You look at data day to day anyway to make things work, and to be blasé about it you see data scientists as pointed headed, sandal wielding lunatics that claim to understand your world in which you have worked for a long time (and often studied for even longer to get there) with some fancy software and a few algorithms.

Conversely, data science find the engineering community hard work. A bunch of pointy headed, hard hat wielding lunatics who ignore the clear and obvious case ‘in the data’ to do things entirely differently. Why bash your head off a brick wall challenging the premise of science with engineers, when you can talk to retailers, and people in other industries that ‘get it’.

There are of course exceptions on both sides! However, I guess this stand-off will not come as news to many. The reason for this blog is to give an alternative view on why we have got to this stage, and how to resolve it. Many people put the root cause of this disconnect down to a simple difference in approach, and categorise it as ‘too difficult to resolve’. This is both short sighted, and unhelpful.

The key word here is ‘science’. Both parties believe in science, and mathematics. It is just that one party is more mechanically based, and the other more statistical. The fact science exists on both sides of this debate, and that the science is different causes this disconnect. However the science is exactly why both parties can, and should work together well. The key here is not talking about engineering and data science as ‘different’ – but instead for both parties to raise their eyes, and to accept the science the other brings as insight based on which they can constructively challenge each other. Engineers bring the science of the assets, and what is ‘known’ today.

The data scientist should never challenge this viewpoint – but instead bring the science to enable engineers themselves to challenge what they know, by bringing out ‘unknowns’ in the data that allow them to do this. It is arrogant of the data science community to believe they can understand an industry better than those that built it – and they should remember this fact. They are enablers in this relationship. Likewise, engineers have to accept that the demands on their industry from regulators and customers dictate the use of data to understand, build and operate power generation and distribution differently. They need to ‘open up’ to accelerate change, and take guidance from other asset intensive industries where engineers work hand in glove with data science today.

To complement and conclude, I would like to touch upon one related point where I do sympathise with engineers especially. There are too many people coming to them with big data, and analytics for their industry – much of which is paper based marketing pomp. I do not blame many engineers for being sceptical about what can be done in this area based on this. But do remember, not everyone bringing you this message is the same – some of us can make a difference with analytics. Our customers and industry analysts can testify to this. Come speak with them, and us.

Iain Stewart is the principal utilities expert for Teradata in the EMEA region, with over 13 years of experience in utilities sector.Iain also has in depth experience of both smart metering and smart grids, including how these link to and support the wider sustainability agenda. Other areas of experience include renewable energy, and smarter cities.

One thought on “Why Can’t Power Engineers and Data Science Just Get Along?

  1. avatarMajid alDosari

    Trained in science&engineering mainly as well as data science, I struggle with what new insight data science brings to what I know from modeling the physical system.

    However, I think future scientists and engineers should have a component of data science training. I think this should happen across all traditional disciplines.


Leave a Reply

Your email address will not be published. Required fields are marked *