Big Data's Dirty Little Secret

Do you know what you’re doing with your Big Data? Based on my experience, only a small group of companies really do.

There’s just one little problem, as Intel cloud platforms group vice president and general manager Jason Waxman, told investors in a webcast last week.

“This is the dirty little secret about big data: No one actually knows what to do with it,” said Waxman, as ComputerWorld reported. Link (emphasis mine)

While I disagree with his statement that “no one” actually knows what to do with it, I do agree with his overall premise.

It’s been my experience that many companies adhere to the Volume aspect of the Volume, Variety, and Velocity definition. Typically they’re so scared to miss any potential data point that they just save everything and clutter up their clusters.

What many companies lack is a well thought out Data Policy that builds a cluster with the Right Volume, the Right Variety, and the Right Velocity. Unfortunately, formulating that data policy takes up a lot time and requires broad input from across the organization.

Not only will it require input from the Data Science team (i.e. data analysis), but it will require input from the Line of Business (LOB) groups and corporate strategy/vision from upper management. Sounds great! Let’s get to work, right?

Just ask yourself this: when was the last time all three of these groups aligned with minimal friction?

Answer: Usually never.

Tl;dr: Right Volume > Volume