Enterprising Developments

Breaching Big Data's Biggest Challenge: The Skills Gap

Joe McKendrick
Insurance Experts' Forum, August 23, 2012

Last week, I read that Gartner designated “big data” as one of the most over-hyped terms of the year, right behind cloud computing.

Nevertheless, the analyst group acknowledges that big data will have a profound, transformational impact on businesses. This is advice to which the insurance industry needs to pay close attention.

Where and how does big data make a difference? The bottom line is that big data helps organizations make better business decisions. A majority of respondents to a new survey, sponsored by Rainstor, found 75 percent agree that big data is extremely important for improving overall business value.

However, there are challenges. Thirty-seven percent of survey respondents indicate that the biggest challenges around managing big data are the speed of data creation (velocity), increase in types of data (variety), and the ability to provide analytics against data.

Perhaps the most vexing challenge is simply finding people with the skills to translate big data into big analytics and big decisions. Switching from data warehouse to Hadoop-style environments is one of the moves being considered—whereas standard SQL still appears to be the enterprise standard when running queries and analysis against existing data warehouses, new skills are needed to make new approaches like this possible.

When asked what the limiting factors are for those looking to adopt and deploy Hadoop, more than half of the survey's respondents agreed that they either lack the skilled resources or they simply have limited bandwidth to take on new technologies and projects. Despite the fact that managing and analyzing big data is high on the business and IT priority list, it is still challenging to achieve inside a large enterprise. Certainly, existing IT personnel are seasoned users, administrators and operators of relational and columnar-type databases, yet they are not seasoned java programmers or engineers that can easily deploy and support open-source Hadoop and then provide the analytics that the business demands.

The ability to find people to make the transformation to an analytics-driven culture will be a top priority for insurance and financial services companies in the years to come. Remember, data management is no longer some role in the database or IT department. Insurance companies, perhaps more than other types of companies, are now data companies. It is the core of the business.

Joe McKendrick is an author, consultant, blogger and frequent INN contributor specializing in information technology.

Readers are encouraged to respond to Joe using the “Add Your Comments” box below. He can also be reached at joe@mckendrickresearch.com.

This blog was exclusively written for Insurance Networking News. It may not be reposted or reused without permission from Insurance Networking News.

The opinions of bloggers on www.insurancenetworking.com do not necessarily reflect those of Insurance Networking News.

Comments (3)

Joe, very insightful article. We are seeing an increase in businesses seeking specialized skills to help address challenges that arose with the era of big data. The HPCC Systems platform from LexisNexis helps to fill this gap by allowing data analysts themselves to own the complete data lifecycle. Designed by data scientists, ECL is a declarative programming language used to express data algorithms across the entire HPCC platform. Their built-in analytics libraries for Machine Learning and BI integration provide a complete integrated solution from data ingestion and data processing to data delivery. More at http://hpccsystems.com

Posted by: HAANA M | August 31, 2012 3:31 PM

Report this Comment

Pretty good article. You missed the reflexive nature of the definition of 'Big data' though - when you make a statement like:

"limited bandwidth to take on new technologies and projects"

That is inherent in the definition of 'big data' - it is a volume/complexity of data which exceeds your organizational abilities to absorb it; if it was easy, it wouldn't be 'big'. Organizations which used to have Terabytes of data & gradually worked their way up to Petabytes while keeping their toolset built out don't consider their data to be 'big'

Posted by: Aborth | August 23, 2012 1:07 PM

Report this Comment

Joe -- great blog and so very true. Knowing data and then being able to understand it or so critical. Many times we don't have the building blocks/foundation from even to start. Proper skills and the building blocks are key.

Posted by: Cindy M | August 23, 2012 12:29 PM

Report this Comment

Add Your Comments...

Already Registered?

If you have already registered to Insurance Networking News, please use the form below to login. When completed you will immeditely be directed to post a comment.

Forgot your password?

Not Registered?

You must be registered to post a comment. Click here to register.

Blog Archive

Trends in P&C and L/H/A Policy Administration Systems

Novarica research shows that nearly 40 percent of P&C and life/health/annuity carriers are currently replacing or planning to replace a policy administration system.

Product Configurators: Moving Insurers toward Self-Sufficiency

Insurers may like a vendorís full service model for updating policy content rules, but they donít want to be held captive if the vendor doesnít offer fast speed-to-market.

How Quote Data Can Deliver Powerful Business Insights

Quote data often is disregarded due to its volume, but properly managed can offer insights into product and pricing strategy, expense control, cross selling and upselling.

Insurers: Let's Be The Best

I donít like when insurance companies are hectored by people inside or outside of the industry about how they arenít innovative. Many insurers are leading the way in gleaning real results from emerging technology disciplines, including big data, analytics, mobile technology, and telematics.

6 Crucial Guidelines for Digital Insurers

Going digital isnít just something that can be accomplished by decree. It takes finesse to keep everything in sync.

Top Stories in Property/Casualty

Novarica Commentaries are available to clients only, but weíve posted direct links to some of the most important stories below.