Connect Twilio with your Dialogflow Agent

Twilio Media Streams provides the ability to access the audio in real-time from your active phone calls. Media Streams powers the integration between Twilio and Dialogflow. The Dialogflow Agent receives the stream, processes the request for intent, and responds with audio. It can then optionally escalate back to your Twilio application so that you can customize subsequent actions in the call flow, such as forwarding the call to an agent or routing the call into a conference.

The following pictures depict a very high-level interaction between a Caller, Twilio and Google Dialogflow.

Expand image

1. Caller asks a question.
2. Twilio streams the audio stream via Websocket to Node app running in Google AppEngine.
3. The Node app streams the audio to Dialogflow API.
4. The Node app receives the audio response stream from Dialogflow.
5. The Node app streams the audio back to Twilio.
6. Twilio plays audio back on the phone call and the caller hears the response. This process continues until the caller hangs up or the end of interaction is reached.

This guide provides step by step instructions to integrate Twilio with Google Dialogflow.

Make sure the following prerequisites are completed before continuing with the rest of the instructions in this guide:

• Setup your Dialogflow Agent. You can build your own or use a pre-built Dialogflow Agent. For the purpose of this guide, make sure to select region US - USA (Global) in the top left corner of Dialogflow.

• Copy the Google Project ID mapped to your Dialogflow agent, you will need it later. You can find this in the settings page for your Dialogflow Agent

• Download your Google Credentials with permissions to access Dialogflow API and name the file as google_creds.json, you will need it later. To find this:

  • Click on the Project ID in the Dialogflow Agent settings
  • This will open a new window on Google Cloud Platform. From the sidebar select "API" and then "Credential". You'll fine
  • In the page that open, you will find a service account called "Dialogflow Integrations". Click on it
  • In the page that opens, delete the existing key and click on "Add Key"
  • Select "JSON"
  • A new json file will be downaloded

• Setup and configure gcloud command line tool to be able to deploy code in Google AppEngine.

• Sign up and activate your Twilio account.

• Copy Twilio Account Sid and Auth token from Twilio console page.

• Setup Twilio CLI using the Quick Start.

We've provided a Node application that uses Twilio Media Streams to integrate with Dialogflow. The following instructions walk you through configuring and deploying the app into Google AppEngine.

Clone the Github repo

git clone https://github.com/twilio/media-streams

Change directories

cd media-streams/node/dialogflow-integration

Install the dependencies

npm install

Copy the google_creds.json file you downloaded earlier, into this folder. Configure the app environment: this command will prompts you for information that will be used by the app in runtime,

npx configure-env

1
exports.handler = function(context, event, callback) {
2
  const twiml = new Twilio.twiml.VoiceResponse();
3
  const dialogflow = JSON.parse(event.dialogflowJSON);
4
  switch (dialogflow.intent.displayName) {
5
    case 'speak-to-an-agent':
6
      twiml.dial(process.env.AGENT_NUMBER);
7
      break;
8
    default:
9
      console.error(`Intent: "${dialogflow.intent.displayName}" (${dialogflow.intent.name}) was not handled.`);
10
      twiml.hangup();
11
  }
12
  callback(null, twiml);
13
};

Deploy the App to Google App Engine with command

gcloud app deploy

Verify all the details and type Y and press Enter to initiate the deployment process. The deployment process typically takes between 2-5 minutes.

Once the deployment is completed successfully, you will see a success message. Copy the URL displayed, as it will be used when configuring Twilio number.

At this point your app is deployed in AppEngine and ready to accept traffic. You can validate to make sure your app is serving incoming traffic by running command:

gcloud app browse

In this section you will purchase a Twilio phone number and configure it with the URL you received in the previous section after deploying to App Engine. This will be completed using the Twilio CLI.

If you have already have a number that you'd like to use, you can skip both the Search and Purchase steps.

Search for a number for available numbers to choose from in the 650 area code that provide voice capabilities. Change the area code to match your needs.

twilio api:core:available-phone-numbers:local:list  --area-code="650" --country-code=US --voice-enabled

Purchase a Twilio Phone Number of your liking. Keep track of the SID (it starts with a PN) that is displayed on creation, you'll use that in the next step.

twilio api:core:incoming-phone-numbers:create --phone-number="+16505551234"

Set up your phone number to point to your AppEngine instance. Replace the SID with your actual SID. Replace <YOUR APP ENGINE URL> with your AppEngine URL.

twilio api:core:incoming-phone-numbers:update --sid=PNXXXXXXXXXXXXXXXX --voice-url="https://<YOUR APP ENGINE URL>/twiml"

If you currently have an on-premise system for your IVR and contact center, you have a couple of options to integrate your on-premise system with Twilio to leverage Google Dialogflow.

You can follow the instructions described in this document and once you successfully purchased and configured a Twilio phone number, you can forward the calls from within your existing infrastructure to the configured Twilio phone number.

Alternatively, you can set up a SIP Domain using the instructions from this tutorial and route calls from your existing infrastructure to Twilio over SIP.

Billing for Twilio usage is additional to, and will occur separately from, Dialogflow CX usage billing from Twilio. (Certain other Google advanced features like analytics and metrics are also charged separately by Google). With Twilio you only pay for what you use. For example, if, based on the instructions in this guide, you purchased a Twilio US Local Inbound number, your Twilio and Dialogflow CX costs based on the current list price as of August 1, 2023, would be:

$1/month for Twilio Local Inbound Number + Inbound PSTN minutes + $0.085/min* to receive calls in Dialogflow
*Billing for Dialogflow is rounded up to the next minute, (e.g. 65 seconds = 2 minutes Dialogflow charges billed by Twilio)

To learn more about our pricing, visit https://www.twilio.com/en-us/voice/pricing/us.