AI hallucinations cover launched in first wave of new insurance products for AI

Armilla has launched a new insurance policy to cover legal claims by customers or third parties caused by malfunctioning artificial intelligence (AI) relating to chatbot errors and ‘hallucinations’ (essentially when AI language models ‘make things up’).

The insurance is designed specifically to cover situations where the AI has performed below expectations as a result of a decline in the performance of the product over time (known as ‘model drift’ or ‘model collapse’). Model drift can happen due to various factors, including:

1. real world data evolving so that the data that the model was trained on no longer accurately represents the current situation;
2. changes in the underlying relationships between input and output variables, making the model less reliable;
3. the model's own internal mechanisms;
4. model collapse, where the model relies too much on patterns in its own outputs rather than on human-generated data; 
5. data pollution, where the increasing prevalence of AI-generated content online makes it more difficult to find high-quality, human-generated data for training, which can lead to a decline in model quality; and 
6. unintended consequences of updates. 

The policies developed by Armilla are underwritten by several Lloyds insurers. 

High profile AI errors 

As companies rush to develop and adopt AI there have been some costly mistakes. High profile incidents include:

1. chatbots swearing at customers and criticising their own organisation;
2. chatbots incorrectly accusing high profile individuals of committing criminal damage; 
3. an AI virtual assistant for an airline giving incorrect information to passengers about bereavement fares; 
4. ChatGPT was used for legal research to find precedents to support a case filed by an Avianca employee for injuries they sustained. At least six cases submitted to court did not exist; and
5. AI recruitment tools displaying bias against women.

The new policies are designed to cover some such errors (although by no means all), which historically have been difficult to insure.

What does this mean for insurers?

The availability of insurance could encourage more companies to adopt AI. Some insurance for AI related losses is already available, however these typically have low limits and very often do not cover losses caused by model drift.  

This latest product development is an important and exciting step for the insurance industry and may well hail a new era of AI-focused insurance products. As ever with the development of new products, very careful underwriting and policy drafting will be the key to success.