Enterprising Developments

Just How Big is Big Data? It's All Relative

Joe McKendrick
Insurance Experts' Forum, March 21, 2011

Lately, the term du jour in IT management circles has been “Big Data,” used to describe the huge surge in information coming into enterprises from all directions, from the Web, sensors, transactional applications, end user files, and an abundance of other places. 

But when we talk about “big data,” is there anything new under the sun? In many ways, Big Data is a relative term that changes over time. Think about our threshold of “expensive.”  In 1980, a jaw-droppingly expensive car in a showroom may have had a price sticker of $28,000. Nowadays, no one blinks an eye at such as price.

Likewise, our idea of what Big Data is also subject to a form of inflation. I remember having a chat about a decade ago with Richard Winter of Winter Group, which tracked the world's largest databases. At that time, there were some gi-normic databases cracking the 1 TB mark. Now, that's almost what one can run on a PC.

A few months back, I attended an industry panel, sponsored by Teradata, which featured a debate on the notion of Big Data. And it is indeed a very fluid term. Donald Feinberg, analyst with Gartner, put it this way: 10 years ago, a company with gigabytes of data would have been considered to be handling Big Data. Today, he said, “If you asked 200 people for a definition of 'Big Data,' you would get 200 definitions,” he says. “Everything is relative.”

However, another panelist said there may be a threshold as to what constitutes Big Data—a moving threshold, but a threshold nonetheless. Mark Jefferey of Northwestern's Kellogg School of Management says that systems begin to behave differently once that threshold is crossed.

“When you get into systems more than a couple hundred terabytes into the petabyte range, the rules are different,” he said. “You have to think about scalability, concurrency and humungous tables. And you need some serious horsepower to be able to do that.”

The ability to store Big Data is evident; but the real challenge is being able to filter, process and turn such voluminous material into actionable insights.

Such capabilities open up possibilities for new types of applications that could come out of capturing and analyzing Big Data, which could be very different and far more advanced than what we have now.

For example, we're seeing carriers putting telematics to work, enabling data to be streamed in from customers' vehicles to help set incremental pricing. A few years ago, carriers' database infrastructures would have been incapable of managing and storing all the data associated with millions of customers' driving habits. Today, it's just another data set.

So, every era has its Big Data. But as our capacity to handle and make sense of this data grows, it is opening the door to some very interesting ways it can be put to work.

Joe McKendrick is an author, consultant, blogger and frequent INN contributor specializing in information technology.

Readers are encouraged to respond to Joe using the “Add Your Comments” box below. He can also be reached at joe@mckendrickresearch.com.

This blog was exclusively written for Insurance Networking News. It may not be reposted or reused without permission from Insurance Networking News.

The opinions of bloggers on www.insurancenetworking.com do not necessarily reflect those of Insurance Networking News.

Comments (1)

Big data is here now and market leading global insurers are moving forward today! These carriers are enriching traditional "structured forms of data", like transactional data, with the new "unstructured forms of data" for sensors (telematics), web applications, and social networking channels like Twitter and Facebook. Early adopters are gaining new insights in customer/agency social and web behavior, fraud, risk management and underwriting - seeing breakthrough opportunities in the marketplace! Carriers are now able to market and service their customers in a whole new light.

Posted by: JohnBurke0123 | March 28, 2011 2:28 PM

Report this Comment

Add Your Comments...

Already Registered?

If you have already registered to Insurance Networking News, please use the form below to login. When completed you will immeditely be directed to post a comment.

Forgot your password?

Not Registered?

You must be registered to post a comment. Click here to register.

Blog Archive

Digital Vision vs. Harsh Reality

Much work remains to reconcile insurers' digital vision with the digital reality that seems to be arriving for other industries.

Vendors Embrace Mobile Technology

IT leaders at software firms clearly recognize the importance of mobility to drive their businesses forward. Almost 70 percent see mobility as mission critical or important to their organization today.

Big Data Is Paying Off

Insurers are getting business benefit out of their big data projects, but these projects alone won't grow their business.

What Can Insurers Learn from Home Depot?

The latest cyber-attack highlights the importance of helping policy holders defend themselves.

Not Your Father’s Insurance Company

Carriers need to look at new and impactful ways to be there for their customers.

How to Attract Top Tech Talent

When it comes to rankings of the best places to work, insurers are few and far between. Here’s what those who make the lists do to appeal to IT professionals.