Enterprising Developments

Breaching Big Data's Biggest Challenge: The Skills Gap

Joe McKendrick
Insurance Experts' Forum, August 23, 2012

Last week, I read that Gartner designated “big data” as one of the most over-hyped terms of the year, right behind cloud computing.

Nevertheless, the analyst group acknowledges that big data will have a profound, transformational impact on businesses. This is advice to which the insurance industry needs to pay close attention.

Where and how does big data make a difference? The bottom line is that big data helps organizations make better business decisions. A majority of respondents to a new survey, sponsored by Rainstor, found 75 percent agree that big data is extremely important for improving overall business value.

However, there are challenges. Thirty-seven percent of survey respondents indicate that the biggest challenges around managing big data are the speed of data creation (velocity), increase in types of data (variety), and the ability to provide analytics against data.

Perhaps the most vexing challenge is simply finding people with the skills to translate big data into big analytics and big decisions. Switching from data warehouse to Hadoop-style environments is one of the moves being considered—whereas standard SQL still appears to be the enterprise standard when running queries and analysis against existing data warehouses, new skills are needed to make new approaches like this possible.

When asked what the limiting factors are for those looking to adopt and deploy Hadoop, more than half of the survey's respondents agreed that they either lack the skilled resources or they simply have limited bandwidth to take on new technologies and projects. Despite the fact that managing and analyzing big data is high on the business and IT priority list, it is still challenging to achieve inside a large enterprise. Certainly, existing IT personnel are seasoned users, administrators and operators of relational and columnar-type databases, yet they are not seasoned java programmers or engineers that can easily deploy and support open-source Hadoop and then provide the analytics that the business demands.

The ability to find people to make the transformation to an analytics-driven culture will be a top priority for insurance and financial services companies in the years to come. Remember, data management is no longer some role in the database or IT department. Insurance companies, perhaps more than other types of companies, are now data companies. It is the core of the business.

Joe McKendrick is an author, consultant, blogger and frequent INN contributor specializing in information technology.

Readers are encouraged to respond to Joe using the “Add Your Comments” box below. He can also be reached at joe@mckendrickresearch.com.

This blog was exclusively written for Insurance Networking News. It may not be reposted or reused without permission from Insurance Networking News.

The opinions of bloggers on www.insurancenetworking.com do not necessarily reflect those of Insurance Networking News.

Comments (3)

Joe, very insightful article. We are seeing an increase in businesses seeking specialized skills to help address challenges that arose with the era of big data. The HPCC Systems platform from LexisNexis helps to fill this gap by allowing data analysts themselves to own the complete data lifecycle. Designed by data scientists, ECL is a declarative programming language used to express data algorithms across the entire HPCC platform. Their built-in analytics libraries for Machine Learning and BI integration provide a complete integrated solution from data ingestion and data processing to data delivery. More at http://hpccsystems.com

Posted by: HAANA M | August 31, 2012 3:31 PM

Report this Comment

Pretty good article. You missed the reflexive nature of the definition of 'Big data' though - when you make a statement like:

"limited bandwidth to take on new technologies and projects"

That is inherent in the definition of 'big data' - it is a volume/complexity of data which exceeds your organizational abilities to absorb it; if it was easy, it wouldn't be 'big'. Organizations which used to have Terabytes of data & gradually worked their way up to Petabytes while keeping their toolset built out don't consider their data to be 'big'

Posted by: Aborth | August 23, 2012 1:07 PM

Report this Comment

Joe -- great blog and so very true. Knowing data and then being able to understand it or so critical. Many times we don't have the building blocks/foundation from even to start. Proper skills and the building blocks are key.

Posted by: Cindy M | August 23, 2012 12:29 PM

Report this Comment

Add Your Comments...

Already Registered?

If you have already registered to Insurance Networking News, please use the form below to login. When completed you will immeditely be directed to post a comment.

Forgot your password?

Not Registered?

You must be registered to post a comment. Click here to register.

Blog Archive

Living with the Internet of Things (and crowd funding)

The Internet of Things has it’s teething problems.

6 Technology Priorities for Individual Life Carriers

While many aging, generally mainframe-based systems, remain capable of supporting basic policy processing and accounting functions, the costs associated with enhancing them are becoming increasingly problematic.

With Google Favoring Mobile, Will The Industry Take it Seriously?

Google’s search engine will now will favor mobile friendly content over traditional website content; within the insurance industry, the greatest initial impact is likely to be felt by insurance distributors.

Why Some Technologists Get Cold Feet on Mobile

There are those who believe that favoring one channel or mode over another will lead to even more silos and dysfunction than we already have in many organizations.

Insurance IT Spending and Budgeting Benchmarks

New research from Novarica highlights areas of concern and offers insights on insurers spending and budgeting decisions.

Enterprise Mobilemania Continues Unabated

More than half of companies are spending more on developing mobile applications -- but are they more efficient?