We are constantly being told that we live in the age of Big Data. The implication is often that this means we are fortunate in some way, since supporting decisions with data is a sure-fire way of reaching the right conclusion.
A very great man – W. Edwards Denning, who is credited with developing the concept of continuous improvement – was one person who firmly believed that every decision should be driven purely by data. He went so far as to say that if decisions were reached in the right way (that is, based on properly selected, analysed and tested data) there were only two types of mistake that could be made.
Making a decision at the wrong time. Or not making a decision at the right time.
Mr Denning sounds like a pretty opinionated kind of guy. But you can see his point. Besides, his thinking underpins proven management methodologies like lean manufacturing and Six Sigma, so it’s hard to knock.
The trouble is, in the world we occupy today, there is so much data to hand that it’s all too easy to get overwhelmed by its sheer diversity and quantity. Recent research from the Chartered Institute of Management Accountants (CIMA) and American Institution of Certified Public Accountants (AICPA) has found that nearly a third of organisations believe Big Data has made decision making more difficult. Even more disturbing, 80% admit that flawed information has been used at least once in strategic decision making during the last three years.
The winning organisations, the research suggests, are those that deeply consider the kinds of data they require, and then carry out the interpretation and analysis that creates the information and insight that in turn support positive action. In addition, they ensure that information is disseminated rapidly across the organisation, using technologies that support person-to-person sharing. Above all, they enable this information to reach the key decision-makers in the organisation at the right time.
A couple of years ago, however, the Harvard Business Review reported that, despite the march of Big Data, ‘hunch and gut-feel’ were still the dominant methods used for decision making in American business.
I don’t suppose much has changed. However, while it’s hard to disagree with Denning’s assertion, I do find myself wondering if he paid enough attention to that creative spark that can ultimately make all the difference between a company and its competitors. After all, would Apple have ever been Apple without Steve Jobs’ vision and all-round off-the-wall wackiness?
My gut feel is telling me ‘probably not’.
I like to think conducting cold, hard data analysis is only one step on the way to making the right decision. The next is to put the resultant insight through the transformative prisms of management experience and – even more important – the human creative imagination.
But please bear in mind that I don’t have much data to support this view…