Hadoop survey results reveal use cases, needs and trends

By Anne-Lindsay Beall, Insights Editor

In a recent TDWI survey of more than 300 data management professionals, respondents were unanimous in their zeal for Hadoop. But with adoption rates still fairly low, the burning question is, why? How are companies using Hadoop? And why is implementing Hadoop important?

The TDWI Best Practices Report, Hadoop for the Enterprise by Philip Russom, answers both questions – first by asking survey respondents to name the most useful applications of Hadoop if their organizations were to implement it (they were to select four or fewer). Here are the top responses:

What are companies doing with Hadoop?

Data warehouse extensions: 46 percent.
Data exploration and discovery: 46 percent.
Data staging area for data warehousing and data integration: 39 percent.
Data lake: 36 percent.
Queryable archive for nontraditional data (web, machine, sensor, social): 36 percent.
Computational  platform and sandbox for advanced analytics: 33 percent.

Not surprisingly, data warehousing and BI are well represented, but non-DW/BI applications – such as archiving traditional data (19 percent), content management (17 percent) and operational applications (11 percent) – also showed up on the list. Russom points out that these applications are becoming more common among Hadoop users and is a sign that Hadoop usage is diversifying across enterprises.

Why is Hadoop important?

To get a broader picture of Hadoop usage – and users’ unvarnished opinions – the TDWI survey also asked an open-ended question:

“In your own words, why is implementing Hadoop important (or not important)?”

As you’ll see from the few excerpts below, the respondents’ comments reveal a number of benefits, needs and trends:

“Cost savings. Linear scalability. Evaluate ‘the hype’ practically. Complement BI.”
—BI architect, telecom, Europe

“Reduces cost of data. New ability to query big data sets. Supply chain improvements. Predictive analytics.”
—Vice president, food and beverage, Asia

“Our existing infrastructure cannot handle the tenfold increase in data volumes.”
—Data strategy manager, hospitality, US

“It’s important to realize the potential of big data and to explore new business opportunities.”
—Data specialist, consulting, Asia

The bottom line? Hadoop has enormous potential that’s only beginning to be tapped, and it’s on its way to becoming mainstream. Of those surveyed, 44 percent expect to have HDFS in production within 12 months, and another 14 percent will have it up and running within 24 months.

To learn more about benefits, barriers and best practices for Hadoop, download the full TDWI Best Practices Report: Hadoop for the Enterprise.