With Big Data dominating technology and even mainstream news, many Business Intelligence and Data Warehouse professionals are asking themselves how they can add value in this new revolution. Many of us feel the pressure coming from within our own organisations to translate our own skill sets to new technological challenges presented by Big Data, or else find ourselves passed over for new opportunities in favour of external candidates and consultants with hot skills and high-demand experience on real-world implementations at other companies.

As we’ve mentioned before, Big Data in some ways is simply the latest development on the DW/BI technology continuum. Just as BI developed as an organic outgrowth of traditional DW practices, now Big Data is emerging as the cutting edge of BI, the next phase of its ceaseless evolution. While traditional BI is separate from and cannot be supplanted by Big Data, there’s no question that Big Data is a reality we must face, and some of us sooner rather than later. So the question becomes:

What do DW & BI professionals have to contribute to a company’s Big Data agenda?
As it is with almost everything, there are multiple facets of how an interested DW/BI architect or technician can contribute significant value to her company’s Big Data ventures. Some of them are based on developing new skills and preparing for novel and unique challenges posed by Big Data projects. Others are based on leveraging existing skills, which are still applicable on Big Data initiatives.

Skills to develop:

1. Get a solid understanding of Big Data concepts, especially the MapReduce framework and its open-source implementation, Apache’s Hadoop. If you can’t summarise the information in the Wikipedia articles about each of these from memory, you don’t know it well enough yet.
2. Consider a training class in “R”, the programming language widely-used for statistical and analytical applications. Knowing some R permits you to talk intelligibly (or even install and use!) Big Data packages such as pdbR(“Programming with Big Data in R”), a free implementation of R packages compiled and streamlined for use with Big Data sets. If you’ve got some familiarity with Scheme or Matlab, R will look familiar to you. If C, C++, and Java are more your style, you can use these languages to manipulate objects directly.
3. If your educational discipline was science or math, brush-up on collegiate-level courses in statistics and/or database architecture to increase your familiarity with Big Data underlying concepts and techniques. There are many options for this, including high-quality, free, online courses.
4. Many vendors sponsor free colloquia, local presentations and conferences, and presentation events to develop customer interest. For the cost of your time, you can get exposure to and attention from companies solving real Big Data problems for their clients. You can also network with colleagues at other companies. Just by observing which company sends people to these presentations, you can determine which companies have interest in and are pursuing Big Data projects. Depending on your company’s information disclosure policy, it may be possible to have candid discussions with others in similar positions to understand the challenges they face and the skills they are finding valuable as they tackle them.

In the next article, we’ll take a closer look at the kinds of skills you likely already have and will need to leverage so that you can contribute the most to your company’s Big Data initiative.

DataHub Writer: Douglas R. Briggs
“Mr. Briggs has been active in the fields of Data Warehousing and Business Intelligence for the entirety of his 17-year career. He was responsible for the early adoption and promulgation of BI at one of the world’s largest consumer product companies and developed their initial BI competency center. He has consulted with numerous other companies and is regard to effective BI practices. He holds a Master of Science degree in Computer Science from the University of Illinois at Urbana-Champaign and a Bachelor of Arts degree from Williams College (Mass).
View Linkedin Profile->
Other Articles by Douglas->

With Big Data dominating technology and even mainstream news, many Business Intelligence and Data Warehouse professionals are asking themselves how they can add value in this new revolution. Many of us feel the pressure coming from within our own organisations to translate our own skill sets to new technological challenges presented by Big Data, or else find ourselves passed over for new opportunities in favour of external candidates and consultants with hot skills and high-demand experience on real-world implementations at other companies.

As we’ve mentioned before, Big Data in some ways is simply the latest development on the DW/BI technology continuum. Just as BI developed as an organic outgrowth of traditional DW practices, now Big Data is emerging as the cutting edge of BI, the next phase of its ceaseless evolution. While traditional BI is separate from and cannot be supplanted by Big Data, there’s no question that Big Data is a reality we must face, and some of us sooner rather than later. So the question becomes:

What do DW & BI professionals have to contribute to a company’s Big Data agenda?
As it is with almost everything, there are multiple facets of how an interested DW/BI architect or technician can contribute significant value to her company’s Big Data ventures. Some of them are based on developing new skills and preparing for novel and unique challenges posed by Big Data projects. Others are based on leveraging existing skills, which are still applicable on Big Data initiatives.

Skills to develop:

1. Get a solid understanding of Big Data concepts, especially the MapReduce framework and its open-source implementation, Apache’s Hadoop. If you can’t summarise the information in the Wikipedia articles about each of these from memory, you don’t know it well enough yet.
2. Consider a training class in “R”, the programming language widely-used for statistical and analytical applications. Knowing some R permits you to talk intelligibly (or even install and use!) Big Data packages such as pdbR(“Programming with Big Data in R”), a free implementation of R packages compiled and streamlined for use with Big Data sets. If you’ve got some familiarity with Scheme or Matlab, R will look familiar to you. If C, C++, and Java are more your style, you can use these languages to manipulate objects directly.
3. If your educational discipline was science or math, brush-up on collegiate-level courses in statistics and/or database architecture to increase your familiarity with Big Data underlying concepts and techniques. There are many options for this, including high-quality, free, online courses.
4. Many vendors sponsor free colloquia, local presentations and conferences, and presentation events to develop customer interest. For the cost of your time, you can get exposure to and attention from companies solving real Big Data problems for their clients. You can also network with colleagues at other companies. Just by observing which company sends people to these presentations, you can determine which companies have interest in and are pursuing Big Data projects. Depending on your company’s information disclosure policy, it may be possible to have candid discussions with others in similar positions to understand the challenges they face and the skills they are finding valuable as they tackle them.

In the next article, we’ll take a closer look at the kinds of skills you likely already have and will need to leverage so that you can contribute the most to your company’s Big Data initiative.

DataHub Writer: Douglas R. Briggs
“Mr. Briggs has been active in the fields of Data Warehousing and Business Intelligence for the entirety of his 17-year career. He was responsible for the early adoption and promulgation of BI at one of the world’s largest consumer product companies and developed their initial BI competency center. He has consulted with numerous other companies and is regard to effective BI practices. He holds a Master of Science degree in Computer Science from the University of Illinois at Urbana-Champaign and a Bachelor of Arts degree from Williams College (Mass).
View Linkedin Profile->
Other Articles by Douglas->