Enterprising Developments

Breaching Big Data's Biggest Challenge: The Skills Gap

Joe McKendrick
Insurance Experts' Forum, August 23, 2012

Last week, I read that Gartner designated “big data” as one of the most over-hyped terms of the year, right behind cloud computing.

Nevertheless, the analyst group acknowledges that big data will have a profound, transformational impact on businesses. This is advice to which the insurance industry needs to pay close attention.

Where and how does big data make a difference? The bottom line is that big data helps organizations make better business decisions. A majority of respondents to a new survey, sponsored by Rainstor, found 75 percent agree that big data is extremely important for improving overall business value.

However, there are challenges. Thirty-seven percent of survey respondents indicate that the biggest challenges around managing big data are the speed of data creation (velocity), increase in types of data (variety), and the ability to provide analytics against data.

Perhaps the most vexing challenge is simply finding people with the skills to translate big data into big analytics and big decisions. Switching from data warehouse to Hadoop-style environments is one of the moves being considered—whereas standard SQL still appears to be the enterprise standard when running queries and analysis against existing data warehouses, new skills are needed to make new approaches like this possible.

When asked what the limiting factors are for those looking to adopt and deploy Hadoop, more than half of the survey's respondents agreed that they either lack the skilled resources or they simply have limited bandwidth to take on new technologies and projects. Despite the fact that managing and analyzing big data is high on the business and IT priority list, it is still challenging to achieve inside a large enterprise. Certainly, existing IT personnel are seasoned users, administrators and operators of relational and columnar-type databases, yet they are not seasoned java programmers or engineers that can easily deploy and support open-source Hadoop and then provide the analytics that the business demands.

The ability to find people to make the transformation to an analytics-driven culture will be a top priority for insurance and financial services companies in the years to come. Remember, data management is no longer some role in the database or IT department. Insurance companies, perhaps more than other types of companies, are now data companies. It is the core of the business.

Joe McKendrick is an author, consultant, blogger and frequent INN contributor specializing in information technology.

Readers are encouraged to respond to Joe using the “Add Your Comments” box below. He can also be reached at joe@mckendrickresearch.com.

This blog was exclusively written for Insurance Networking News. It may not be reposted or reused without permission from Insurance Networking News.

The opinions of bloggers on www.insurancenetworking.com do not necessarily reflect those of Insurance Networking News.

Comments (3)

Joe, very insightful article. We are seeing an increase in businesses seeking specialized skills to help address challenges that arose with the era of big data. The HPCC Systems platform from LexisNexis helps to fill this gap by allowing data analysts themselves to own the complete data lifecycle. Designed by data scientists, ECL is a declarative programming language used to express data algorithms across the entire HPCC platform. Their built-in analytics libraries for Machine Learning and BI integration provide a complete integrated solution from data ingestion and data processing to data delivery. More at http://hpccsystems.com

Posted by: H. M | August 31, 2012 3:31 PM

Report this Comment


Pretty good article. You missed the reflexive nature of the definition of 'Big data' though - when you make a statement like:

"limited bandwidth to take on new technologies and projects"

That is inherent in the definition of 'big data' - it is a volume/complexity of data which exceeds your organizational abilities to absorb it; if it was easy, it wouldn't be 'big'. Organizations which used to have Terabytes of data & gradually worked their way up to Petabytes while keeping their toolset built out don't consider their data to be 'big'

Posted by: Aborth | August 23, 2012 1:07 PM

Report this Comment


Joe -- great blog and so very true. Knowing data and then being able to understand it or so critical. Many times we don't have the building blocks/foundation from even to start. Proper skills and the building blocks are key.

Posted by: Cindy M | August 23, 2012 12:29 PM

Report this Comment

Add Your Comments...

Already Registered?

If you have already registered to Insurance Networking News, please use the form below to login. When completed you will immeditely be directed to post a comment.

Forgot your password?

Not Registered?

You must be registered to post a comment. Click here to register.

Blog Archive

The Internet of Things: Helping Insurers Make Better-Informed Decisions about Risk

The IoT is a major game changer for the insurance industry, and will likely affect every part of the insurance value chain. After all, insurance is data-driven, and that’s exactly what the IoT can deliver—relevant, actionable, real-time data that can provide an accurate picture of what is being—or may be—insured.

Software-Defined Everything

What does it take to virtualize all the key components in your data center?

On Thanking the Regulator … Really

The Financial Conduct Authority is demanding higher standards of consumer protection from insurers, which could lead to greater customer engagement and understanding.

Competing with the Coasts for Tech Talent

Are heartland-based insurers at a recruiting disadvantage for tech skills?

Putting Your Investments Where Your Transformation Is: Part 2: Optimizing Your IT Investments Portfolio

Sam Medina continues a 3-part series on Transforming the IT Investment Budget in order to fund new programs and initiatives without the necessity of additional capital expense.

Boosting Performance with Integrated Underwriting Tools

A unified, comprehensive platform can help underwriters perform their jobs more efficiently — and profitably.

Advertisement

Advertisement