Big Data: A Garage Full of Old Newspapers

I’m getting a little tired of articles about ‘big data’. Especially the extreme reports – ranging from panic-filled to euphoric – many of which are written by hardware and software suppliers, or by magazines that simply regurgitate what those companies tell them. So far, I’ve failed to discover the essence of big data.

To me, big data is like a garage full of old newspapers: as long as you’ve got enough space available, it’s not a problem to keep them. The question is, are you saving such a large amount of data because you want to or because you have to? How easy or how difficult is it to extend your storage capacity? Are you going to hold onto the old newspapers voluntarily? You could also recycle them. What are you going to do with all those old newspapers anyhow? There’s no point in hoarding them unless you expect to find something interesting among all the articles. But how do you know whether you’ll be able to gain new and surprising insights and make revealing connections from them? Such a scenario immediately makes me think of films such as A Beautiful Mind (2001) and Conspiracy Theory (1997) which respectively featured a schizophrenic maths genius – played by Russell Crowe – and a plot-obsessed amateur journalist – played by Mel Gibson – both of whom collect newspaper clippings and unearth correlations from them. In the case of both the films and old newspapers, I believe it’s all about pattern recognition, and the same goes for big data.

Light Sabre in Star Wars

During a recent congress by software supplier SAS in Amsterdam, The Netherlands, I heard two inspiring speakers who (finally) gave me new insights into big data, albeit indirectly. Tomáš Sedláček, macroeconomist at a Czech bank, described the Western economy as “manic depressed”. “Manic depression is a bipolar disorder with highs and lows, just like alcoholism. You can’t help an alcoholic merely by removing the negative effects of alcohol. You first need to address the manias – the highs. In the Western world, we’ve become used to the good years. The problem in Western civilisation today is that we lack the imagination of previous generations,” he said. It revolves around recognising patterns: which cycles does a business go through. But it’s also essential to name certain assumptions, according to Sedláček: “When you watch Star Wars, you have to believe that a light sabre works; you must accept certain assumptions. Models of reality and processes are also assumptions. However, rather than keeping our distance, we look too closely.” As if you’re standing too close to an impressionistic painting by Monet or Seurat.

Whitney Houston

At the same event, Swedish futurologist Magnus Lindkvist provided some good advice on looking for innovations in a world full of information: “Which ideas are now out of sync in the new reality? Technology has shifted from very expensive to available for everyone, so-called IKEAfication.” Lindkvist illustrated that far from all information is valuable: “You’re well informed, but your head’s full of rubbish. Last year, ‘Whitney Houston’ was the most searched-for term in Google. The human brain prefers sexy lies over the boring truth.” It appears that many people find the gossip pages the most interesting section of a newspaper. “In order to innovate, we need to turn our attention to secrets,” advised Lindkvist.

In other words, we need to carefully consider which old newspaper articles – and not only those on our favourites, the gossip pages – we should read to discover patterns. We need to be very clear on what we want to achieve with big data in order to be able to do anything at all with it. So far, I’ve not had a ‘Eureka’ moment regarding big data. But it could still happen.

Martijn Lofvers, Publishing Director & Editor-in-Chief of Supply Chain Movement