The subject has been already discussed in Daniel Dennett’s excellent article in Scientific American in March this year, How Digital Transparency Became a Force of Nature.
In the article Dennett, the Tufts University philosopher and cognitive scientist, and Deb Roy, the Massachusetts Institute of Technology professor and Twitter’s chief media scientist, compare the developments in Digital Data to the developments of environments in the early-stage Earth. Their main idea is that an emerging trend towards digital transparency will put evolutionary pressure on current companies.
This, in turn, will cause a whole round of “survival of the fittest”, where only companies that can withstand the pressure of digital transparency survive, in the same way that Evolution caused whole families of species to disappear when they couldn’t cope with environmental changes.
Source: “Walking with Dinosaurs film”
Dennett and Roy write: “The impact on our organisations and institutions will be profound. Governments, armies, churches, universities, banks and companies all evolved to thrive in relatively murky epistemological environment, in which most knowledge was local, secrets were easily kept, and individuals were, if not blind, myopic. When these organisations suddenly find themselves exposed to daylight, they quickly discover that they can no longer rely on old methods; they must respond to the new transparency or go extinct.”
Anyone who followed the various governments’ response to Wikileaks would agree that they are struggling to cope with transparency. Will commercial organisations fare any better?
But this is only one way in which Big Data is changing the world.
At present Big Data is Big Promise – it hasn’t yet hit the “killer app”. Remember the early days of the Internet? It was obvious that it is a thing of great promise, but uptake was slow. Then came email and the rest is history.
So what will be Big Data’s killer app?
My money is on Health.
More and more people are collecting personal health data, by smartphone app, by smart watch or by special equipment. I recently spoke to a marathon running university professor who can predict his performance on the next marathon based on data he collects on his daily training runs.
Collect this data and analyse it and you have a treasure trove of information that can help predict epidemics, correctly plan public budgets, improve access to health and discover hidden correlations.
The public is obviously interested, but weary of loss of privacy, which brings us back to Dennett’s article. When large corporations evolve to survive in a transparent world, the individual will need to evolve with them. A society that shares its health data is a healthier society.
Let’s hope that common sense prevails over privacy-made-public fears.
Ben Bor is a Senior Solutions Architect at Teradata ANZ, specialist in maximising the value of enterprise data. He gained international experience on projects in Europe, America, Asia and Australia. Ben has over 30 years’ experience in the IT industry. Prior to joining Teradata, Ben worked for international consultancies for about 15 years and for international banks before that. Connect with Ben Bor via Linkedin.