Skip to contentSkip to navigationSkip to topbar
On this page

TwiML™ Voice: <VirtualAgent> with Dialogflow CX


The <VirtualAgent> TwiML noun, used with Twilio's Google Dialogflow CX Connector Add-on, allows you to connect callers to a Google Dialogflow agent. <VirtualAgent> nests inside the <Connect> verb to direct a call to a Dialogflow agent.


Set up the integration between Twilio and Dialogflow CX

set-up-the-integration-between-twilio-and-dialogflow-cx page anchor

Before using <VirtualAgent>, you must complete the Dialogflow CX one-click integration.


<Connect> attributes

connect-attributes page anchor

<VirtualAgent> is a TwiML noun that is nested within the <Connect> verb.

<Connect>'s action and method attributes allow you to execute a new set of TwiML instructions for a call after the <Connect><VirtualAgent> session ends.

Attribute NameAllowed ValuesDefault Values
actionRelative or absolute URLCurrent TwiML document URL
methodGET, POSTPOST

action

action page anchor

The action attribute takes an absolute or relative URL as a value. After a <Connect><VirtualAgent> session ends, Twilio will send an HTTP request to this action URL and will expect a new set of TwiML instructions in the response. If you do not provide an action attribute, Twilio will request TwiML from the current TwiML document.

The following code sample is an example of using the action attribute with <Connect> and <VirtualAgent>.

Specify an action URL with <Connect>Link to code sample: Specify an action URL with <Connect>
1
const VoiceResponse = require('twilio').twiml.VoiceResponse;
2
3
const response = new VoiceResponse();
4
const connect = response.connect({
5
action: 'https://myactionurl.com/twiml'
6
});
7
connect.virtualAgent({
8
connectorName: 'project',
9
statusCallback: 'https://mycallbackurl.com'
10
});
11
12
console.log(response.toString());

Output

1
<?xml version="1.0" encoding="UTF-8"?>
2
<Response>
3
<Connect action="https://myactionurl.com/twiml" >
4
<VirtualAgent connectorName="project" statusCallback="https://mycallbackurl.com"/>
5
</Connect>
6
</Response>

For Dialogflow CX, this HTTP request's body will contain additional information about the result of the <Connect><VirtualAgent> session, along with Twilio's standard request parameters. A <Connect><VirtualAgent> session ends (and thereby triggers Twilio's HTTP request to the action URL) in the following ways:

  • The Dialogflow CX agent session ends successfully and the caller did not hang up
  • The conversation with the Dialogflow CX agent requires a live agent handoff
  • The caller hangs up
  • An error occurred during the VirtualAgent session

Twilio's request to the action URL

twilios-request-to-the-action-url page anchor

The body of Twilio's request to the action URL contains properties about the <Connect><VirtualAgent> session, along with Twilio's standard request parameters. You can use this information to configure the call behavior after a <Connect><VirtualAgent> session, or to log information for analytics purposes.

PropertyDescription
VirtualAgentProviderThe VirtualAgent provider e.g. DialogflowCX
VirtualAgentStatusThe reason a VirtualAgent session ended (see below for more information)
VirtualAgentProviderDataJSON string containing the provider specific action callback event data (see below for more information)
VirtualAgentErrorCodeA Twilio error code. See Twilio's Error and Warning Dictionary for more information. This property will only appear if the value of the VirtualAgentStatus property is failed.
VirtualAgentErrorA message describing the error that caused the VirtualAgent session to fail. This property will only appear if the value of the VirtualAgentStatus property is failed.
VirtualAgentProviderErrorA Dialogflow-provided error message. This property will only appear if the value of the VirtualAgentStatus property is failed and the VirtualAgentErrorCode is 32601.

This property describes why the VirtualAgent session ended. Possible values are:

  • completed: VirtualAgent session was terminated by one of the following:
    • The Dialogflow CX agent indicated end of conversation (by matching the end of conversation intent)
    • The caller hung up
  • live-agent-handoff: VirtualAgent returned a live agent handoff response
  • failed: An error occurred during VirtualAgent processing
  • paused: VirtualAgent session was paused with the intention of the session being resumed later

action VirtualAgentProviderData

action-virtualagentproviderdata page anchor

This property is a JSON string containing Dialogflow-specific data from the agent interaction:

  • ConversationId: Unique identifier for this conversation provided by Google
  • EndUserId: Unique identifier for the end user participant provided by Google
  • AgentHandoffParameters: Parameters included from the Dialogflow CX agent if the agent returned a live agent handoff response
  • PauseParameters: Custom parameters included from the Dialogflow CX agent if the agent returned a custom payload response with "TWILIO_ACTION": "PAUSE".

Respond to Twilio's request to the action URL

respond-to-twilios-request-to-the-action-url page anchor

If you want a call to continue after a <Connect><VirtualAgent> session ends, your application should provide new TwiML instructions based on the <Connect><VirtualAgent> session.

This optional <Connect> attribute indicates the HTTP request method to be used when Twilio sends a request to the action URL. If you don't specify a method attribute in <Connect>, POST is used by default.

The allowed values are POST and GET.


<VirtualAgent> attributes

virtualagent-attributes page anchor

The table below lists all of the <VirtualAgent> attributes with their allowed values. The Dialogflow CX integration allows additional customization of the call flow using <Parameter> and <Configuration> TwiML nouns.

NameAllowed ValuesRequired?Default Value
connectorNameA stringYesempty
statusCallbackAn absolute URLNoempty
statusCallbackMethodGET, POSTNoempty

The connectorName is a string, configured in Dialogflow CX connector instance in the Unique Name field. You can review your Dialogflow CX connector instances in the Twilio Console(link takes you to an external page).

The statusCallback attribute is an absolute URL where Twilio will send HTTP requests. While the Dialogflow agent interaction is in progress, Twilio will send HTTP requests to this URL each time an intent matches in the Dialogflow agent's conversation.

The body of Twilio's requests to your statusCallback URL contains information about intents, transcripts (caller text vs. virtual agent text), sentiment analysis (if enabled), and custom parameters that were sent to Dialogflow to build custom logic.

The body of Twilio's request to your statusCallback URL contains the following properties:

ParameterDescriptionTypeRequired?
AccountSidTwilio Account SIDstringYes
CallSidTwilio Call SIDstringYes
TimestampTime of the event, conformant to UTC ISO 8601 TimestampstringYes
StatusCallbackEventThe event type that triggered the status callback request (e.g. intent)stringYes
VirtualAgentProviderThe VirtualAgent provider (e.g. DialogflowCX)stringYes
VirtualAgentProviderDataJSON string with data from Dialogflow CX. See more information below.JSON stringYes

statusCallback VirtualAgentProviderData

statuscallback-virtualagentproviderdata page anchor

This parameter in the status callback is a JSON string containing the Dialogflow CX-specific intent event data. The JSON string can contain the following fields:

KeyDescriptionTypeRequired
ConversationIdUnique identifier for this conversation provided by GooglestringYes
EndUserIdUnique identifier for the end user participant provided by GooglestringYes
ReplyTextThe Dialogflow CX agent's response to the callerstringYes
LanguageCodeThe language that was triggered during intent detectionstringNo
ParametersJSON object with key-value pairs containing the collected session parametersobjectNo
ConfidenceThe intent detection confidence. Values range from 0.0 (completely uncertain) to 1.0 (completely certain)numberNo
ResolvedInputFinal text input matched against. This value may be different from the original input because of spelling correction or other processing.stringNo
IntentIdIntent ID provided by GooglestringNo
IntentDisplayNameThe human readable name of this intentstringNo
SentimentAnalysisScoreSentiment score between -1.0 (negative sentiment) and 1.0 (positive sentiment)numberNo
SentimentAnalysisMagnitudeA non-negative number in the [0, +inf) range, which represents the absolute magnitude of sentiment, regardless of score (positive or negative)numberNo

This is the HTTP method to use when requesting the statusCallback URL. Accepted values are GET or POST, and the default value is POST.


You can pass custom key-value pairs to your Dialogflow CX agent by using the nested <Parameter> TwiML noun within <VirtualAgent>. The <Parameter> noun allows you to send custom session parameters(link takes you to an external page) to your Dialogflow CX virtual agent, which can be referenced in your agent as $session.params.parameter-name.

<Parameter> has two attributes: name and value, both of which must be used every time you use <Parameter>.

For example, if you want to provide the end caller with a personalized agent greeting, you can supply a nested <Parameter> element with the customer name to the Dialogflow CX agent as in the following code sample.

This key-value pair will be passed in as a session parameter(link takes you to an external page) which can be referenced under $session.params.cutomer_name at runtime in your Dialogflow CX agent.

Add custom parameters to a Dialogflow CX interactionLink to code sample: Add custom parameters to a Dialogflow CX interaction
1
const VoiceResponse = require('twilio').twiml.VoiceResponse;
2
3
const response = new VoiceResponse();
4
const connect = response.connect();
5
const virtualagent = connect.virtualAgent({connectorName: 'uniqueName'});
6
virtualagent.parameter({name: 'customer_name', value: 'Burton Guster'});
7
8
console.log(response.toString());

Output

1
<Response>
2
<Connect>
3
<VirtualAgent connectorName="uniqueName">
4
<Parameter name="customer_name" value="Burton Guster"/>
5
</VirtualAgent>
6
</Connect>
7
</Response>

Pause and resume a conversation session

pause-and-resume-a-conversation-session page anchor

You can pause and resume a Dialogflow CX agent conversation session at any time. This allows you to pass a call back and forth between Dialogflow and Twilio while keeping the session context intact. As a result, the caller doesn't need to repeat themselves or start from the beginning when the call is passed back to Dialogflow.

Note that sessions have a 30 minute TTL (time-to-live) for inactivity between pausing and resuming.

To pause a session, your Dialogflow CX agent must return a custom payload(link takes you to an external page) JSON response that contains the key TWILIO_ACTION with the value PAUSE. This signals to Twilio that the session should be paused. Twilio then sends a request to the action URL you've specified. You can add additional parameters in the custom payload from your Dialogflow agent, which Twilio will include in the request to the action URL.

When a session is paused, the request from Twilio to your action URL has VirtualAgentStatus set to paused and VirtualAgentProviderData contains a PauseParameters property with any custom parameters provided in the custom payload.

For example, if your Dialogflow CX agent returns the following custom payload response to Twilio:

1
{
2
"TWILIO_ACTION": "PAUSE",
3
"memberId": "$session.params.member_id",
4
"paymentAmount": "$122.34"
5
}

Twilio's request to your action URL will contain the following values in the PauseParameters field:

1
{
2
"memberId": "abc123",
3
"paymentAmount": "$122.34"
4
}

To resume a session, send a request to Twilio with the TwiML <Connect><VirtualAgent> containing a nested <Config> noun. The <Config> noun can include the following settings:

  • resumeEndUserId: This configuration setting is required. It represents the existing conversation participant that is resuming the session. Its value should be set to the EndUserId property from the VirtualAgentProviderData of the paused event.
  • resumeEventName: This configuration setting is optional. Its value should be set to the name of a custom event to trigger in the Dialogflow CX bot when the session is resumed. A custom event allows you to prompt a specific response from the agent or progress the conversation with additional context that was collected during the pause period. A custom event handler(link takes you to an external page) can be defined and triggered when the session is resumed.

See the dynamic connector configuration section for more information on <Config>. Note that you can only resume a session on the same Call SID which initiated the conversation.

Below is an example of how to resume a session with both the resumeEndUserId and resumeEventName configuration settings:

1
<?xml version="1.0" encoding="UTF-8"?>
2
<Response>
3
<Connect>
4
<VirtualAgent connectorName="uniqueName">
5
<Config name="resumeEndUserId" value="projects/ccai-test-customer/locations/us-east1/conversations/119qL_EbVa6QqascUqOqi0taw/participants/40aMiI3SQZaEbby-EIyhrw"/>
6
<Config name="resumeEventName" value="customEventName"/>
7
</VirtualAgent>
8
</Connect>
9
</Response>

Dynamic connector configuration

dynamic-connector-configuration page anchor

TwiML noun allows you to override your underlying Dialogflow CX Connector's configuration or pass additional parameters that change the behavior of your virtual agent. The <Config> noun is nested inside of <VirtualAgent>. <Config> has two attributes: name and value, both of which must be used every time you use <Config>.

For each configuration option you want to override, you must include a new <Config> noun. The name attribute then corresponds to one of your Dialogflow CX Connector's configuration options, such as language, sentimentAnalysis, voiceName, and welcomeIntent.

Additionally, there are attributes that are not present in your Dialogflow CX Connector configuration such as voiceModel, speechModel, speechModelVariant, but you can still set these attributes using <Config> noun nested inside of <VirtualAgent>.

Below are the configuration settings supported within <Config> TwiML noun:

Configuration SettingDescriptionDefault
languageLanguage in which the caller will be interacting with the Dialogflow CX Virtual Agent, e.g. "es-es". When choosing a language, you must apply "STT" and "TTS" filters in the Dialogflow CX languages(link takes you to an external page)If not specified, this defaults to the setting in the Dialogflow CX connector configuration
sentimentAnalysisBoolean indicating whether to enable sentiment analysis or not. If you want Dialogflow to perform sentiment analysis on the intents, you must select language that supports "Sentiment" filter in addition to STT/TTS in the Dialogflow CX languages(link takes you to an external page)If not specified, this defaults to the setting in the Dialogflow CX connector configuration
voiceNameThe TTS voice(link takes you to an external page) to use for the Dialogflow CX Virtual Agent, e.g. "en-US-Standard-C"If not specified, this defaults to the setting in the Dialogflow CX connector configuration
welcomeIntentThe name of the event to trigger when connecting to the virtual agent. You would set this to the built-in system event(link takes you to an external page) "WELCOME" to trigger the Default Welcome Intent for your agent. However, Dialogflow CX allows you to configure custom events(link takes you to an external page) on your Dialogflow CX agent to programmatically denote where the flow should start.empty
voiceModelThe TTS custom voice model(link takes you to an external page) to use. If specified in addition to voiceName, this attribute will take precedence. This feature allows you to train a custom voice model using your own studio-quality audio recordings to create a unique voice.empty
speechModelThe STT model(link takes you to an external page) used in the processing of human speech. If not specified, the "default" speech-to-text transcription model is used.This option is not exposed in the Dialogflow CX Connector, but uses the "default" speechModel under the hood
speechModelVariantThe speech model variant used in speech-to-text. If an enhanced model variant is specified and an enhanced version of the specified model for the language does not exist, this omits an error. See the Enhanced Models(link takes you to an external page) guide for more information.This option is not exposed in the Dialogflow CX Connector and not explicitly set by Twilio. Dialogflow defaults to "USE_BEST_AVAILABLE"
resumeEndUserIdThe ID of the end user to resume a paused session with.empty
resumeEventNameThe name of the custom event to trigger in the Dialogflow CX bot when resuming a paused session.empty

For example, if you want to customize the TTS voice and language for the virtual agent interaction, you can supply the respective configuration settings inside the <Config/> attributes as follows:

Override Dialogflow CX Connector ConfigurationLink to code sample: Override Dialogflow CX Connector Configuration
1
const VoiceResponse = require('twilio').twiml.VoiceResponse;
2
3
const response = new VoiceResponse();
4
const connect = response.connect();
5
const virtualagent = connect.virtualAgent({connectorName: 'uniqueName'});
6
virtualagent.config({name: 'language', value: 'en-us'});
7
virtualagent.config({name: 'voiceName', value: 'en-US-Wavenet-C'});
8
9
console.log(response.toString());

Output

1
<Response>
2
<Connect>
3
<VirtualAgent connectorName="uniqueName">
4
<Config name="language" value="en-us"/>
5
<Config name="voiceName" value="en-US-Wavenet-C"/>
6
</VirtualAgent>
7
</Connect>
8
</Response>

Need some help?

Terms of service

Copyright © 2024 Twilio Inc.