Covering Disruptive Technology Powering Business in The Digital Age

Home > DTA news > Blog > Using Data to Find That Proverbial Needle in a Haystack
Using Data to Find That Proverbial Needle in a Haystack
June 5, 2020 Blog


“Why does my insurer calculate my premium based on which group I fit into rather than based on me as an individual?” That’s the question many people ask themselves when they look at their annual insurance renewal quotation.



Traditionally, insurers calculate risk based on indicators like age, location, job and gender. If you meet the criteria, you’re placed in a group and you’ll get a premium similar to everyone else in that group. If you happen to be in a high-risk group, you can expect high premiums.

Even if you are a safer driver, typically travel on slower roads and more regularly inflate your tyres, your insurer won’t take that into account, or will they?

Octo Telematics uses Cloudera’s Enterprise Data Cloud to provide data to motor insurance companies all over the world. By analysing 170 billion miles of driving data from five million connected cars, they are able to help insurers do things like offer completely different premiums to people of the same gender and age, driving the same car, living in the same location, but with varying styles of driving.


They are able to look for and find that needle of insight in a haystack of data to increasingly personalise the customer experience.

Increased personalisation is rapidly moving to be more than a generic recommendation engine at an online checkout. Personalisation is about giving individuals the right offers and information at the right time and doing so at scale – and the only way they can do this is to look at, and apply, the data in real time.


How might this look?

Take retail as an example. Sales promotions used to be as simple as a 30% discount on all items. A data-driven, personalised retail sales promotion could mean every customer gets unique discounts across different products. Information about stock items, the time of day, number of people in the shop, supply chain analysis and personal shopping habits of individual customers could all contribute to that personalised discount offer.

To be able to deliver such a personalised customer experience, retailers may literally need to gather thousands of data points per customer per minute across hundreds of thousands of customers. That data may be coming from POS, CCTV, mobile devices, previous activity history and social media, to name a few. At the same time, this may be combined with data about stock, sales and supply chain to build a rounded moment-by-moment picture of what to offer each customer and what impact that will have on the bottom line.

The common thread about being able to find that needle in the haystack is that, by design, the amount of data is huge and the insight derived can be personalised as down to individuals.

Delving a little deeper, this means capturing data from multiple edges, combining it with historical data that is likely to sit in a centralised data store and compiling data from business systems. After the data is analysed, the insights and results are pushed back out to the edge as individual offers.



Keep in mind that all this is being done securely, in a timeline that allows an offer to be put to customers in the time they spend in your shop.

The only way this is possible is to leverage an enterprise data cloud. Cloudera has built its enterprise data cloud with these use cases in mind. Businesses can now have the power of massive data processing capability, along with the ability to connect to multiple edges to pull vast real-time data streams, and all done with uncompromising attention to governance and security.

Find out more about Cloudera’s enterprise data cloud here.