Over the past few years I’ve been working more and more with the large volumes of data that come from M2M and the Internet of Things. It wasn’t that long ago when “Big Data” was a novelty that was largely a vision of the future – more talked about than done. In a few short years it’s morphed into the “next big thing” that everyone needs to have and which will save our planet and our health systems. Of course, Big Data itself is of limited use. What changes the game is the insight which can be extracted from it. That’s why the headline description of big data can be unhelpful. By concentrating on the “big”, it places the spotlight on the mechanics of database structures, diverting attention from the real skills that the industry needs to make it valuable.
I’d like to share some things I’ve learnt from my experience working in this area. The first is the continuing hype. When I put together a conference on the use of big data at the Cabinet Office last year I was hard pressed to find anyone really doing it commercially – the hype was still far greater than the practice. I don’t think that much has changed since then. We’re still on the lower, gentle slope of the Gartner hype curve. My guess is that the only companies making significant money from big data at the moment are conference organisers and consultants. But attention is being paid.
The second is the type of skills we need to cultivate. We talk about Data Scientists as the new breed of practitioner, but that’s largely a self-invented title from data analysts who want more recognition. Extracting value from big data, or broad data if you want to be more accurate, is more than that. The best definition I’ve heard is that it’s about telling stories with Matlab. It’s not about Hadoop or Cassandra – they’re just the mechanics. The reality is that Big Data needs to be about Data Storytellers if it is going to be transformational.
The third thing is that this is something we do exceedingly well in London. Other places may collect more data, build bigger server farms or invent more capable database structures. But we tell better stories. So if you want to generate value from big data, London’s the place to set up your business.