The Rise and Fall and Rise of Big Data

Not so long ago, “big data” was the hottest buzzword in technology and every VP at a major enterprise jonesing to add an “S” to their title was racing to harness the mystical powers of massive yet supposedly manageable data sets. Armed with big data, no problem would prove insurmountable, all forecasts would be met, and every status update would be green.

But lately that luster appears to have faded.

After a sharp rise starting in 2012, a Google Trends query reveals “big data” peaked as a search term in October 2014, and since then interest dropped gradually but consistently and currently stands at less than half its all-time high.

Relative popularity of search term “big data”, July 2008 – July 2018 (Source: Google Trends)

So what happened? Did big data go away? Nope. in fact we now live in a world of even bigger data, of unfathomably large amounts of 1’s and 0’s where so much new digital information is being generated daily — over 2.5 quintillion bytes of Instagram selfies and cat videos, but also boatloads of e-commerce transactions, online product research, and comments/reviews — that according to one estimate from IDC and Dell EMC, the total amount of data in the world is more than doubling every two years, rivaling Gordon Moore’s famous law about increases in computing power.

Clearly there is no shortage of data. In fact, one of the fastest-growing sectors of the $3.5 trillion IT industry — cloud computing — is devoted to wrangling these staggering quantities of bits and bytes in a manner that doesn’t explode the opex line of a major enterprise’s P&L.

Big data has become a fact of life. What’s now driving the conversation are the means to finally achieve those KPI endgames, as the ability of highly sophisticated analytics techniques has, until recently, proved incapable of exploring the depths of these new Great Lakes of corporate data.

Let’s return to Google Trends for another perspective on the “big data needs killer tools” line of thinking:

Relative popularity of search terms “big data” and “machine learning”, July 2008 – July 2018 (Source: Google Trends)

Although not quite a tipping point in the Malcolm Gladwell sense, the moment, in November 2016 — less than two years ago — when machine learning surpassed big data in Google searches nevertheless marks a critically important paradigm shift, as the conversation about the fastest path to a corporate competitive advantage with data pivoted from inputs to outputs.

Before then, none of this was possible at scale. At Cerebri AI we move the border of what’s possible every day using advanced analytics, mapping previously uncharted big data territory.

Our Cerebri Values™ software, powered by machine learning, lifts customer success for major enterprises by turning big data into killer data.