Addressing Hallucinations in AI

As generative AI (GenAI) tools grow in availability and usability, businesses are making new strides in automation, decision-making, and customer interactivity. However, these tools—and the large language models (LLMs) powering them—aren’t infallible. Sometimes, they generate responses that are incorrect or entirely fabricated. These AI hallucinations can pose real risks, especially when GenAI systems are trusted to perform critical tasks or interact with customers.

In this post, we’ll unpack the important concept of AI hallucinations—what they are, why they can be a problem, and how your business can mitigate their risks. We’ll also look at how Twilio AI Assistants incorporate features and guardrails to ensure your AI usage remains trustworthy.

Understanding AI Hallucinations

AI hallucinations occur when an LLM generates information that appears plausible but is actually inaccurate or entirely fabricated. Because GenAI applications typically offer their responses with a tone of certainty and confidence, distinguishing a hallucinated response from a correct one can be difficult for everyday users. This can lead to the spread of misinformation.

Why do AI hallucinations occur? LLMs are trained on vast datasets, but ultimately, they don’t “know” facts. Instead, they predict the next word or phrase based on patterns learned during training. When an LLM lacks sufficient context or the right data to respond to a prompt, those predictive abilities can take a wild turn.

This turn might happen because of an over-eager LLM, ready to agree with any fact presented in a prompt:

It can also happen when the training data set contains factually incorrect information. For example, let’s say a training set contained an old Reddit thread which included advice on getting pizza cheese to stay on pizza. You can see where we're going with this – that happened, and one sarcastic response suggested using Elmer’s glue to help hold the cheese in place:

And what do you know? That advice ended up as an answer in Google’s early AI summary feature when users asked about making their cheese stick:

The community noticed this (and it galvanized a response!), but it is a prime example of when a poisoned data set leads to hallucinations down the line.

Why Hallucinations Are a Problem

If your business uses GenAI for operational efficiency or customer-facing services, then it’s important to be aware of how AI hallucinations may crop up and how to mitigate these risks.

Misinformed business decisions

When misinformation from an AI hallucination creeps into your internal data, it can lead you to make business decisions based on faulty information.

In one example, an AI demo tasked with summarizing a financial report responded with several factual inaccuracies. What if a business used this inaccurate response to inform its own strategies? In another example, a law firm submitted AI-hallucinated legal research in a court filing and was fined for the mistake.

Erosion of customer trust

Your business brand and reputation depends heavily on customer trust. What would happen to that trust if your GenAI-powered customer service chatbot provided occasionally hallucinated responses to customer inquiries? Or consider the repercussions if you published AI-generated marketing content that wasn’t checked for factual errors.

A classic case of this was an AI-generated travel article from Microsoft. In recommending tourist destinations in Canada, the article recommended visiting the Ottawa Food Bank on an empty stomach. You can see how the effects of AI hallucinations can quickly erode customer trust and damage your business's reputation.

Strategies to Mitigate AI Hallucinations

GenAI is still an emerging technology. And while LLM advancements to reduce the likelihood of AI hallucinations have borne fruit, the phenomenon continues to be a concern. At present, eliminating AI hallucinations altogether is not a reasonable outcome to shoot for. Instead, businesses using GenAI should adopt strategies to mitigate them.

Retrieval-augmented generation (RAG)

Retrieval-augmented generation (RAG) is an approach that enhances a GenAI system by fetching information—in real time—that’s relevant to a user’s prompt. By retrieving additional information to supplement the LLM’s original training dataset, a RAG system provides additional knowledge and context to the AI. This decreases the opportunity for hallucinations and increases the relevance and accuracy of the response.

RAG can go a long way in reducing AI hallucinations, though it may not entirely eliminate them as most models will fall back to their training data.

Prompt engineering

In many cases, carefully crafting your prompts can help minimize the instances of AI hallucinations. Basic prompt engineering techniques include:

1. Give clear and specific instructions.
2. Ask direct questions.
3. Provide sufficient background information or context.

For example, if you’re building an AI agent to help retail shoppers, you might be tempted to give it a system prompt like this:

That prompt is vague and lacks specificity and context. Instead, here’s an example of a clear and specific system prompt for the LLM that will perform more effectively:

Applying prompt engineering techniques like these provides expectations and guardrails for a GenAI system, guiding it toward a correct response.

Intentional handling of highly critical data

When working with highly critical data, treat LLMs like untrusted clients, similar to browsers or mobile apps. This approach involves using a secure middle layer to manage API interactions, ensuring that LLMs don’t have direct access to critical systems.

Just as you wouldn’t expose backend APIs to an unsecured front-end, don’t allow LLMs free rein over critical operations and don’t leave room for interpretation of critical data. For example, simply embedding critical numbers and dates as part of an unstructured text prompt can open the system up to possibly mishandling the data and hallucinating.

To mitigate this issue, pass critical data to the system programmatically and grant the model minimal access necessary for task completion. Purpose-built tools, such as Tools in the Twilio AI Assistant framework can help ensure the data is handled properly.

Along with being able to execute complex calculations or dynamically retrieve real-time information from external APIs, Tools can act as proxy APIs to handle authentication, rate limiting, and data validation before reaching an external API. This layered security approach helps prevent unauthorized activity and allows you to pass critical data programmatically within your system rather than as part of the prompt body to the model.

Human-in-the-loop

GenAI technology is rapidly evolving, but it’s not yet in a place where businesses can use it unchecked. Human oversight is still a necessity. The level of human oversight over GenAI activity may depend on the criticality of the tasks being performed. For example, a healthcare provider using GenAI to craft a patient care plan should include a high level of oversight from qualified physicians and healthcare professionals.

Twilio AI Assistants and Hallucinations

Twilio understands that while there’s no single solution to completely eliminate AI hallucinations today, we believe that with strong data control, ongoing testing, and vigilant monitoring, GenAI can be trusted to accurately and responsibly manage customer communications. With this philosophy in mind, Twilio AI Assistants employs a robust suite of tactics specifically designed to reduce the occurrence of AI hallucinations and enhance the reliability of your interactions.

One of the key measures in place is the restriction on the model’s access to sensitive fields such as identity. For security reasons, our model does not handle the identity directly, and instead it is provided from a trusted source and sent as an HTTP header for each tool request, isolating critical information from the model.

To further ensure consistency and reliability, we actively monitor model performance using a standard set of evaluations known as evals. These assessments measure performance, effectiveness, and the accuracy of language models. Diligent evaluation allows us to iteratively improve our models, ensuring they can reliably navigate a variety of scenarios.

Twilio also offers tools to help businesses ensure that their Conversational AI performs reliably and meets customer expectations. For example, for customers using Twilio Voice products, features like Voice Intelligence, language operators, and real-time transcription are available to monitor and optimize the conversational experience within the Twilio platform. Twilio aims to empower businesses to manage and control the performance of their GenAI systems, fostering a trustworthy environment for both businesses and their customers.

Conclusion

While we cannot yet eliminate AI hallucinations entirely, that doesn’t mean they should be a major obstacle to your business. Techniques like RAG and prompt engineering can help you reduce the chances of inaccurate responses. Combine these strategies with a human-in-the-loop approach to using GenAI, and you can limit the potential damage caused by hallucinations, creating a more reliable AI experience.

Twilio AI Assistants takes these concerns seriously, incorporating advanced tools and safety features to minimize hallucinations. Give AI Assistants a try, and let us know what you think at twilioalpha@twilio.com.

Emily Shenfield (eshenfield [at] twilio.com) brings her background in software engineering, teaching, and theater to her role as a Technical Marketing Engineer at Twilio on the Emerging Tech and Innovation team. She's excited about exploring the future of customer engagement and how it impacts developers. Outside of work, she enjoys yelling the answers at the TV during Jeopardy and eating cookies.

Alvin Lee is a full-stack developer based in Phoenix, Arizona. He specializes in web development, integrations, technology consulting, and prototype building for startups and small-to-medium businesses.