How to Integrate Twilio AI Assistants with Communication Channels

How to Integrate Twilio AI Assistants with Communication Channels

This post explores how Twilio AI Assistants integrates with different communication channels, providing a seamless generative AI conversational experience. It covers the integration patterns for voice and messaging channels ( SMS, WhatsApp, Twilio Flex Webchat 3.x), and how to facilitate these configurations using an open source quickstart middleware. Additionally, it explains how to hand over conversations to a live agent on Twilio Flex when needed. Optimizing for flexibility, customers can also choose to hand over conversations to any contact center of their choice. In the later sections, a step-by-step guide walks through the configuration process, leaving you with a working AI Assistants integration on the channels you choose.

Understanding Twilio AI Assistants’ integration patterns

Twilio AI Assistants is designed to be omnichannel by default, enabling integrations across various different channels. This section explains the underlying mechanisms of integrating Twilio AI Assistants with voice and messaging channels.

Voice integration pattern

Twilio AI Assistant natively integrates with voice calls through TwiML (Twilio Markup Language). This is achieved using the <Assistant> noun inside the <Connect> verb, allowing incoming calls to be routed directly to Twilio AI Assistants.

To connect a Twilio AI Assistant to a voice-capable Twilio phone number, two primary options are available:

1. Using TwiML Bins – Hosted TwiML instructions

   • TwiML Bins provide a way to host static TwiML responses directly within Twilio.

   • This option allows a straightforward integration without additional infrastructure.

2. Using Webhook Integration – Dynamic TwiML via Twilio Functions

   • A Twilio Function can dynamically generate TwiML responses based on business logic.

   • This approach offers more flexibility, such as selecting different Twilio AI Assistants based on context or user preferences, including language selection.

The following TwiML snippet connects an incoming call to a Twilio AI Assistant:

In this example, when a call reaches the Twilio phone number, Twilio executes the <Connect> verb, routing the call to the Twilio AI Assistant specified by its id. The welcomeGreeting attribute sets an initial greeting for the caller.

Twilio AI Assistants leverages ConversationRelay for voice interactions, allowing additional attributes to be specified within the <Assistant> noun, such as language and transcriptionProvider:

Understanding these integration patterns enables Twilio AI Assistants to be configured for voice interactions.

Messaging channels integration pattern

Integrating Twilio AI Assistant with messaging channels involves two key components: 

1. Twilio Conversations – Acts as an omnichannel framework that consolidates all messages exchanged between participants (e.g., customer and Twilio AI Assistant) into a unified conversation session

2. Middleware Implementation with Twilio Functions – Acts as the bridge between Twilio AI Assistants and the messaging channel

Twilio Conversations

Twilio Conversations is a unified messaging platform that connects multiple channels into a single, seamless conversation experience. Each messaging channel sender, such as a phone number for SMS or a WhatsApp Sender for WhatsApp, is assigned a Conversation Address within Twilio Conversations. This address determines how incoming messages are processed. 

Webhooks can be configured at the Conversation Address level, allowing for more granular control over which channels are integrated with Twilio AI Assistants. This ensures that only specific Conversation Addresses are routed to Twilio AI Assistants, while others remain independent.

Twilio Conversations provides a user interface for configuring Conversation Addresses, where webhooks can be set up without requiring direct API calls. When integrating Twilio AI Assistants, Autocreate Conversations for new messages must be enabled in your Twilio Console to ensure that incoming messages automatically initiate a conversation. The webhook URL should point to the middleware (i.e., the Twilio Function) handling Twilio AI Assistant’s API calls, with the onMessageAdded event selected to trigger processing when a new message is received.

Integrating Twilio AI Assistants with messaging channels using Twilio Conversations and middleware enables scalable message processing across multiple platforms.

Middleware implementation via Twilio Functions

A middleware layer is required to facilitate communication between Twilio Conversations and Twilio AI Assistant. Its primary function is to: 

1. Receive incoming messages from Twilio Conversations’s webhook that is triggered via onMessageAdded event

2. Invoke the AI Assistant’s Messages API to generate a response. 

   1. Within the body payload, it is essential to set the webhook parameter to the middleware URL so as to receive Twilio AI Assistant’s response

3. Add the Twilio AI Assistant's response back into the conversation.

The sample middleware implementation can be be found in my repo:

• Incoming Webhook: Link

• AI Assistant Response Webhook: Link

Agent handover (aka Twilio Flex Handover) integration pattern

Twilio AI Assistant’s Tools feature can be used to trigger a middleware service responsible for executing the handover process to Twilio Flex. The tool configuration calls a webhook (the middleware) that performs the necessary operations to transition the conversation to a live agent.

Voice handover to Twilio Flex

For voice calls, the middleware utilizes the <Enqueue> TwiML verb to transfer the call to Twilio Flex. The <Enqueue> verb requires the Twilio Flex Workflow SID, ensuring the call is routed to an available agent.

Additional metadata can be passed to Twilio Flex using the <Task> noun, which will appear as Task attributes within Twilio Flex, providing agents with relevant context about the call.

Messaging Channels handover to Twilio Flex

For messaging channels, the middleware invokes the Twilio Interactions API to create a task within Twilio Flex. Once the task is created, the middleware must remove the Twilio AI Assistant webhook from the conversation. This prevents the AI Assistant from responding while a live agent is handling the conversation.

A sample implementation of messaging channels handover to Flex is available here.

Getting started

This section outlines the steps to configure Twilio AI Assistants with various communication channels using twilio-aia-adapters, a quickstart sample middleware for Twilio AI Assistants.

twilio-aia-adapters is deployed using Twilio Functions which requires no external hosting service and it provides the following features:

1. Integration webhook for voice – Supports multilingual by just passing a language query parameter. Editable by modifying the voice-languages-config.private.json file within the function asset folder.

2. Integration webhook for conversations – Designed for secure and high-performance connection with Twilio AI Assistant. 

3. Tool - Integration webhook for Twilio Flex handover – Supports both voice and messaging, transferring all attributes from Twilio AI Assistant as task attributes within Twilio Flex.

Prerequisites

Before you can start configuring the integration, you'll need the following:

• A Twilio Flex account (Guide)

• Node.js v18.x.x  (Guide)

• Twilio CLI v5.22.9 or above (Guide)

• Twilio CLI Serverless Toolkit v3.2.0 or above (Guide)

• Twilio CLI Flex Plugin v7.1.0 or above (Guide)

• Twilio AI Assistant created (Guide)

Step 1: Deploy twilio-aia-adapters

In your terminal, run the following commands to clone the repository and deploy Functions:

Step 2: Take note of the Twilio AI Assistant SID

• Open your web browser and login to Twilio Console
• Navigate to Twilio Console > AI Assistants > My Assistants
• Ensure that you have already created a Twilio AI Assistant. If not, please follow the instructions outlined here to create one
• Under SID, take note of the SID which starts with aia_asst_xxxxxxx.

Step 3a: Configure Voice Channel

1. Navigate to Twilio Console > Phone Numbers > Manage > Active Numbers

2. Select your DESIRED-VOICE-CAPABLE-NUMBER

3. Go to Configure > Voice Configuration > A Call Comes In

4. Set the Webhook URL to: https://twilio-aia-adapters-XXXX-dev.twil.io/voice/incoming?aiAssistantSid=aia_xxxxx with HTTP POST

5. Click Save configuration

To use with Twilio AI Assistant with a different language, append the query parameter language into the Webhook URL. 

Example: https://twilio-aia-adapters-XXXX-dev.twil.io/voice/incoming?aiAssistantSid=aia_xxxxx&language=zh-CN

The list of out-of-the-box configured languages are:

• en-US

• en-AU

• es-ES

• hi-IN

• ja-JP

• ko-KR

• pt-BR

• th-TH

• vi-VN

• zh-CN

• zh-HK

Step 3b: Configure Conversations (SMS/WhatsApp) Channel

• Navigate to Twilio Console > Conversations > Addresses > Configure addresses
• Under your DESIRED-MESSAGING-CHANNEL-ADDRESS, click on the Pencil Icon
• Under Step 1: Autocreate Conversations for new messages?, Enable Autocreate a conversation
• Under Step 2: Which Conversations Service should the Conversation be created in?, select Flex Chat Service
• Under Step 3: Want to set up an integration?, select Webhook
• Webhook URL for incoming messages: https://twilio-aia-adapters-XXXX-dev.twil.io/conversations/messageAdded?aiAssistantSid=aia_xxxxx
• Webhook method: POST
• Webhook filters: onMessageAdded only
• Click Update

Step 3c: Configure Flex Webchat 3.0 Channel

• Ensure you have Twilio Flex Webchat 3.x configured (Guide)
• Navigate to Twilio Console Flex Manage Messaging
• Under Filter by Address type, select Chat
• Under your DESIRED-FLEX-WEBCHAT-ADDRESS, click on the Pencil Icon
• Under Flex Integration
• Integration Type: Webhook
• Webhook URL: https://twilio-aia-adapters-XXXX-dev.twil.io/conversations/messageAdded?aiAssistantSid=aia_xxxxx
• Click Update Address

Step 4: Configure Twilio Flex Handover

• Navigate to Twilio Console > TaskRouter > Workspaces to obtain both workspaceSid (i.e., WSxxxxx) and workflowSid (WWxxxx). workflowSid can be retrieved after clicking into the default workspace named Flex Task Assignment and then Workflows.
• Once done, navigate to Twilio Console > AI Assistant > My Assistants
• Select your DESIRED-AI-ASSISTANT
• Go to Tools and click the Add Tool button
• Name: Twilio Flex Handover
• Description: When the user requests to talk to a live agent or an actual human. This is meant to transfer call to a contact center
• Select HTTP Method: POST
• Webhook URL: https://twilio-aia-adapters-XXXX-dev.twil.io/tools/flex-handover?workfspaceSid=WSxxxxx&workflowSid=WWxxxxxx
• Input Schema: export type Data = {conversationSummary: string, conversationSentiment: "positive" | "neutral" | "negative",};
• Authorization Provider: None
• Click Save Changes button

[Optional] Step 5: Deploy Twilio Flex Plugin - AI Assistant Summary

In your terminal, run the following commands:

Conclusion

Integrating Twilio AI Assistants with voice and messaging channels enables an AI-driven conversational experience across multiple platforms, allowing you to reach customers on their preferred channel. With the provided step-by-step guide and middleware, setting up AI Assistants for various channels was hopefully a straightforward process.

As a next step, you can learn how to enhance the reliability of your Twilio AI Assistant by integrating a robust knowledge base. We’re excited to see what you build next!

Leroy is a seasoned solution architect with a knack for designing scalable architectures on the cloud. He is currently part of the Solution Engineering team for APJ. Leroy can be reached at lechan [at] twilio.com.