2 April 2017
We’re drowning in data. Structured data, semi-structured data, multi-structured data, unstructured data, fast data, “Big Data”: in an increasingly digital world we create more and more data every single day. According to an IDC report the global growth in data volumes amounts to about 60% per year. That means it will grow tenfold every five years! That kind of exponential growth is difficult to fathom. Because of all the opportunities for discovery and innovation, the McKinsey Global Institute labelled Big Data “the next frontier for innovation, competition and productivity.”
Every piece of data is stored because someone, somewhere, gets value from it. Storing data itself is not the point, obviously. We extract value for the business by analysing all these data. The importance of analytics is becoming evident to more and more people. Management books like “Super Crunchers” (Ayres, 2006), “Competing on Analytics” (Davenport & Harris, 2007), “Data Driven” (Redman, 2008), or “The Signal and the Noise” (Silver, 2012), have to helped to raise this awareness. The growing appreciation for the value data can bring to an organization has led to these topics now appearing on the agenda of board meetings.
The time has come to take our data science profession as serious as is merited by the value of decisions riding on it. It is about time we start governing our data assets as rigorously as other (tangible) corporate assets. If a balance sheet needs to be auditable, and Registered Accountants have to study for many years to earn their qualification, then how come we don’t hold the same expectations about data quality and the level of professionalism from our data analysts? After all, it is advanced analytics that drive ever more crucial business decisions.
In a business world that is changing ever faster, wins or losses on the competitive battlefield hinge on leveraging data assets slightly better than your rivals. In particular, proprietary data assets are one of the few sustainable sources of competitive advantage. Your talent can be lured away by head-hunters. As soon as innovative product designs hit the market, they can be copied by the competition. But companies never ever share their data with the public, and they will therefore never become available to competitors. Proprietary data are like a gift that keeps on giving.
Most currently available multi-variate statistical procedures were developed in the 80’s when computers started to be commonly available. Yet their adoption in business settings has been relatively slow. It certainly does not seem to have kept pace with the widespread availability of computing power and abundant availability of data. Cloud computing, Moore’s Law, advances in business intelligence, they have all made more data available to more users. And yet the awareness and knowledge of Machine Learning and Artificial Intelligence have developed remarkably slow. The majority of techniques in use today, have been available for decades!
Executives and non-executive boards are being held accountable for the quality of their decision-making. As their actions are scrutinized, wouldn’t you expect them to hold the quality of data that go into these decisions to the same (high) standards? Business Intelligence and data science are increasingly recognized as a source of competitive advantage, and as input to key decisions. I would say it is only normal that practitioners in this field will (must) be held accountable for the quality of their work. Data will only begin to play the leading role we aspire to, if analytics and data governance live up to (professional) business standards that we expect anywhere else along the corporate chain of command.