SME Toolkit Logo
Partner Logo
Home  > Is Big Data Analytics All It's Cracked Up to Be?
 Share  Print Version  Email

Is Big Data Analytics All It's Cracked Up to Be?

This post was written as part of the IBM for Midsize Business program, which provides midsize businesses with the tools, expertise and solutions they need to become engines of a smarter planet.

As big data analytics becomes an integral part of just about every IT department, it is important not to get carried away with the hype surrounding the technology. Data-derived insight is already an important aspect of IT, but no data set is without bias; thus, no results are perfect. IT managers at midsize businesses should keep these truths in mind since they will increasingly be called upon to manage and act upon big data results.


The Problem with Big Data

Big data analytics solutions are often sold as the answer to all of a business's problems, or at least as a way to let the data drive the big decisions that a company must make, since the math cannot lie. There is some truth to this, but as big data continues to grow, it is important to keep in mind that analytics are only as good as the imperfect data sets from which they are derived.

This point is driven home in a Boing Boing article. At the heart of the argument is the fact that data sets will always be somewhat imperfect; thus, results should be viewed with this in mind. This is not an issue that the business world hears very much about since those looking to sell big data solutions are more than ready to highlight the fact that big data can offer actionable results and downplay other concerns.

The article, which references an earlier piece from Harvard Business Review, goes on to give a number of real-world examples where big data failed to provide good results. These examples include an analysis of tweets sent during Hurricane Sandy, which gave a distorted view of where the worst of the storm was occurring, and a time when Google missed predicting the breadth of a flu outbreak by almost 100 percent due to irregularities in how people searched for symptoms and treatments.


Clearing the Right Path for Midsize Businesses

The reasons for the poor data sets generally boil down to extreme differences between different sets of people, all of whom must be measured together. Businesses can often avoid some of these pitfalls through the homogenization of data-collecting devices when analyzing in-house processes, but when dealing with the general public or with disparate parts of the business, these issues will persist.

There is hope, however, that as analytics continues to drive innovation, more big data solutions will be partnered with tools derived from the social sciences world, where every data set has to be significantly adjusted to account for input irregularities. Once the big data world turns its attention to the integrity of the data set, expect there to be another data revolution where results get refined even further without the need for a massive amount of new data collection.

For now, midsize businesses will have to live with big data analytics' issues — even an imperfect analytical system is significantly more insightful than the solutions of a few years ago. The level of competition in the midsize space means that IT managers must grab hold of every advantage that they can, and a robust big data solution is still a huge advantage. But every IT manager should keep in mind the imperfection of any analytical solution, work to homogenize the input as much as possible and account for irregularities when acting upon the derived intelligence.

This article was written by Shawn Drew.

 Share  Print Version  Email