Technology - Natural language processing: Getting to know you


The data gained from knowing your customers can be invaluable, improving sales, services and customer retention. But how can unstructured data be analysed?

In the current competitive and challenging insurance environment, created by the persistent soft market, the advantage of knowing your customer and being able to respond to their needs effectively and quickly is clear. Often, the data that can help insurers improve sales, services and customer retention already exists within an organisation. But not all data passing through the various channels is nicely ordered and easily accessible to employees.

For example, data contained in written correspondence, conversations in call centres and written and verbal customer feedback is often overlooked. Ideally, all of this information provided by the customers and agents themselves would be used to augment the existing structured data. The potential in this unstructured data — which represents 80% to 90% of all data — for early identification and forecasting of customer trends could be of significant value.

Unstructured data takes various forms, including video, voice, photos and text. In order to exploit the huge amount of unstructured text, techniques such as ‘natural language processing’ are often deployed to derive deep insights. NLP is a field of computer science and linguistics concerned with the interactions between computers and human (natural) languages.

So, how can an insurance call centre, for example, use NLP to analyse voice conversions after having converted them to text, as well as derive insights from raw text inputs, such as police reports, emails or call centre logs?

With these centres handling a large volume of customer calls, obtaining a better understanding of what customers are looking for and providing high quality customer service are imperative to remain competitive. While traditional call centre applications effectively capture some of the structured details of the call for later use, key information about the customer emotions, their satisfaction or call quality is often lost.

Converting conversations
To overcome this situation, it is possible to convert verbal conversations to text and then apply machine-learning or statistical techniques to classify and detect similarities in word patterns to make the information easier to use. This text is compared against a variety of familiar, industry-specific vocabulary to decipher the true meaning of conversations. Finally, it is combined with structured data to provide enhanced insights for insurers.

The sentiments and emotions of callers are identified through key words. For example, positive sentiments are often expressed with the words ‘thanks’ and ‘appreciate’, whereas negative sentiments can be recognised through the phrases ‘want to speak to a supervisor’, ‘can’t get’ or ‘why not’. Analysing these trends can help insurers discover if there are any unifying factors that are user based; time based – for example, dissatisfied customers call during the evenings; business process based; or regional – for example, whether customers from specific regions are more content or disappointed. This analysis helps the call centre to ensure appropriate staffing.

Fraud indicators
Another area where NLP is increasingly playing a role in optimising call centre services is fraud detection. Using NLP, text in call notes or claims forms is searched for the presence of words, phrases or entities relating to the claim. The presence of certain words or phrases can highlight unusual patterns and even give early indications of a fraudulent pattern in the claim.

Those claims records in which unusual patterns are found are then provided as input to a probabilistic scoring engine, which is used to compute the risk score for each claim. The higher the score, the higher the probability the claim is fraudulent. Claims with a high probability of fraud can then be investigated by special investigative units, thus greatly increasing their productivity. Equally, these fraud insights can be integrated with call centre processes to enhance customer experience, protect the organisation’s brand, and lead to improved profitability.

Applying NLP-based analytics can provide tangible business benefits. These include: higher call centre operating efficiency, meaning improved first-call resolution rates and reduced average handling times. NLP can also help monitor employee quality to reduce agent turnover and improve agent training.

Other benefits include enhanced customer service, reducing customer dissatisfaction and increasing retention; more effective compliance monitoring; increased profits for the call centre due to improved up-sell results and cost reductions by automation of large-volume speech and text processing.

In addition, NLP could generate more effective marketing campaigns and increased market intelligence; improved business processes, through the identification and repair of those that are broken and the identification of specific challenges; as well as improved fraud detection and brand protection, while also improving customer service.

The key differentiators between a manual approach to interrogating unstructured data and an automated NLP approach are that the NLP-based solutions can be less labour intensive, provide a higher degree of consistency, drive down operational costs, be less prone to wrong interpretations and be less likely of missing key observations.

There is also the possibility of identifying additional findings beyond manual interpretation. Since an analytics engine would be performing the analysis on the text, it is easy to add new parameters based on which analytics needs to be performed and integrate the insights into call centre operations processes. This allows problems to be addressed immediately, especially when customer sentiment needs to be tracked in situations like the reaction to the launch of a new product.

To help maximise the delivery of the above-mentioned benefits, companies should consider some hygiene factors when mobilising NLP initiatives. Practical experience of applying NLP techniques for a large insurer suggests the following considerations should be taken into account before embarking on such an initiative.

First, the time taken to auto-transcribe calls into text can be time-consuming and considering a large sample can initially be prohibitive. So, targeting the appropriate problem statement and the relevant supporting data set is a key decision when deploying NLP techniques for the first time.

Sensitive issues
One critical success factor is the age of the sample data set. The gap between the period when the calls actually happen and the study takes place should always be minimised, enabling sensitive issues that require immediate attention to be identified.

Another key success factor is the level of training the tool goes through. It will need to be trained not only in the domain-specific vocabulary but also company-specific vocabulary relating to the scope of the exercise.

Until recently, the technology to understand natural language communication between a customer and a company has largely been restricted to a few niche players in the industry. Advances in, and adoption of, NLP is re-addressing this situation, allowing insurers to process internal and external unstructured data that exists with more confidence.

Improved recognition of a larger variety of dialects and better understanding of intonation are two areas of progress. In addition to spoken content, sentiment analysis with acoustic features is used to map emotions like anger, surprise, nervousness or dissatisfaction. Similarly, one can expect ‘audio keyword spotting’ features to be added to tools in the future, to limit the benefit realisation period when dealing with large volume text analysis, and help give real-time guidance to agents.

The good news is that insurers are finally benefiting from the explosion of unstructured data that is flowing through the veins of their organisations. NLP as a technology has matured in the past few years and, with the research & development plus training investments made by insurance IT service firms in its adoption and implementation, NLP is now ready to help insurers optimise their business processes and mine unstructured data for meaningful insights.

  • LinkedIn  
  • Save this article
  • Print this page  

You need to sign in to use this feature. If you don’t have an Insurance Post account, please register for a trial.

Sign in
You are currently on corporate access.

To use this feature you will need an individual account. If you have one already please sign in.

Sign in.

Alternatively you can request an indvidual account here: