Surely by now everyone has heard about Big Data… and the promises of novel tools to wrangle the thorniest and most out-sized data sets. These tools make many of the cutting-edge BI enthusiasts hungry to reap the promised benefits. And as predictably as the seasons, a proliferation of companies selling Big Data solutions and integration services has grown around that enthusiasm. With a dizzying number of vendors fighting for a share of each company’s IT capital purse, many BI architects find themselves at the receiving end of pointed questions from their CIO’s: shouldn’t we have a Big Data solution too?

Big Data isn’t high-tech snake oil though. There are enough documented cases for its value proposition that even the most jaded of BI-Luddites must admit that for some situations, its time has come. For some companies, a Big Data solution is indeed the tool needed to derive fresh, actionable insights from its information assets.

However, Big Data is at the place where BI was perhaps a decade ago. And while BI has always existed in some form as long as we’ve had data to analyse, a structured approach to BI, maturity of upstream and downstream processes, and an industry-proven methodology for effective implementation didn’t come along upon launch in the marketplace. They had to develop organically over time, in response both to the advent of those enabling technologies and to the problems arising after implementations in the field.

Is There a True Need for Big Data?
So, Big Data is (for now) mostly a solution in search of a problem. Many companies are finding that they don’t have a true Big Data candidate: chiefly, so much data is changing so quickly and is of such complexity that they require an innovative approach in order to exploit it — something so profoundly different that existing toolsets cannot deliver sufficient value from it no matter how effectively built and managed. In truth, many prospective Big Data candidates right now are just big data warehousing problems that can be managed with effective, established DW practices if followed diligently.

That said, Big Data will ultimately become the BI of the prior decade. The industry will understand it more clearly, specifically how and when to apply it. As well, companies will grow their capability of sustaining Big Data value drivers by discerning how to leverage the information they can (and cannot) get from Big Data solutions and turn it into effective strategic and tactical operations in their markets.

Therefore, the most valuable tool for companies today in their pursuit of Big Data interest is discernment — the ability to examine their true need and organisational capability, as well as to influence key decision-makers in understanding whether today (or tomorrow) is the right time to move.

The truth is, the longer a company waits, the more likely it is that they can create an effective Big Data solution based on the lessons learned by others, but only if they have a real Big Data problem to begin with!

Companies in these lean-but-growing economic times have many competing priorities for their IT spend, and so it is more important than ever for BI and DW architects and IT leaders to recommend expenditure on Big Data only when it promises to deliver real and sustainable value.

DataHub Writer: Douglas R. Briggs
“Mr. Briggs has been active in the fields of Data Warehousing and Business Intelligence for the entirety of his 17-year career. He was responsible for the early adoption and promulgation of BI at one of the world’s largest consumer product companies and developed their initial BI competency center. He has consulted with numerous other companies and is regard to effective BI practices. He holds a Master of Science degree in Computer Science from the University of Illinois at Urbana-Champaign and a Bachelor of Arts degree from Williams College (Mass).
View Linkedin Profile->
Other Articles by Douglas->

Surely by now everyone has heard about Big Data… and the promises of novel tools to wrangle the thorniest and most out-sized data sets. These tools make many of the cutting-edge BI enthusiasts hungry to reap the promised benefits. And as predictably as the seasons, a proliferation of companies selling Big Data solutions and integration services has grown around that enthusiasm. With a dizzying number of vendors fighting for a share of each company’s IT capital purse, many BI architects find themselves at the receiving end of pointed questions from their CIO’s: shouldn’t we have a Big Data solution too?

Big Data isn’t high-tech snake oil though. There are enough documented cases for its value proposition that even the most jaded of BI-Luddites must admit that for some situations, its time has come. For some companies, a Big Data solution is indeed the tool needed to derive fresh, actionable insights from its information assets.

However, Big Data is at the place where BI was perhaps a decade ago. And while BI has always existed in some form as long as we’ve had data to analyse, a structured approach to BI, maturity of upstream and downstream processes, and an industry-proven methodology for effective implementation didn’t come along upon launch in the marketplace. They had to develop organically over time, in response both to the advent of those enabling technologies and to the problems arising after implementations in the field.

Is There a True Need for Big Data?
So, Big Data is (for now) mostly a solution in search of a problem. Many companies are finding that they don’t have a true Big Data candidate: chiefly, so much data is changing so quickly and is of such complexity that they require an innovative approach in order to exploit it — something so profoundly different that existing toolsets cannot deliver sufficient value from it no matter how effectively built and managed. In truth, many prospective Big Data candidates right now are just big data warehousing problems that can be managed with effective, established DW practices if followed diligently.

That said, Big Data will ultimately become the BI of the prior decade. The industry will understand it more clearly, specifically how and when to apply it. As well, companies will grow their capability of sustaining Big Data value drivers by discerning how to leverage the information they can (and cannot) get from Big Data solutions and turn it into effective strategic and tactical operations in their markets.

Therefore, the most valuable tool for companies today in their pursuit of Big Data interest is discernment — the ability to examine their true need and organisational capability, as well as to influence key decision-makers in understanding whether today (or tomorrow) is the right time to move.

The truth is, the longer a company waits, the more likely it is that they can create an effective Big Data solution based on the lessons learned by others, but only if they have a real Big Data problem to begin with!

Companies in these lean-but-growing economic times have many competing priorities for their IT spend, and so it is more important than ever for BI and DW architects and IT leaders to recommend expenditure on Big Data only when it promises to deliver real and sustainable value.

DataHub Writer: Douglas R. Briggs
“Mr. Briggs has been active in the fields of Data Warehousing and Business Intelligence for the entirety of his 17-year career. He was responsible for the early adoption and promulgation of BI at one of the world’s largest consumer product companies and developed their initial BI competency center. He has consulted with numerous other companies and is regard to effective BI practices. He holds a Master of Science degree in Computer Science from the University of Illinois at Urbana-Champaign and a Bachelor of Arts degree from Williams College (Mass).
View Linkedin Profile->
Other Articles by Douglas->