Build Automations for Utilities from Call Insights with Voice Intelligence

Build Automations for Utilities from Call Insights with Voice Intelligence

At SIGNAL 2021, Twilio launched Voice Intelligence, an AI-powered language analysis tool that transforms voice call data into actionable insights. Since then, we've bolstered our NLU with sentiment analysis, call summarization, and improved entity recognition. Additionally, we've expanded language support to include 15 additional languages and introduced a new Language Operator API for enhanced configuration flexibility.

Twilio's Voice Intelligence has the power to revolutionise how utility companies handle customer calls. This AI-powered tool automatically transcribes, analyses, and extracts insights from voice conversations, turning previously inaccessible call data into a goldmine of customer intelligence.

Imagine analysing every customer conversation, quickly identifying competitive churn risks, and empowering your support teams to respond in real time. That's the power of Voice Intelligence.

By combining pre-built and custom Language Operators, businesses can create automated workflows that transform customer service. From detecting key phrases to classifying calls, Voice Intelligence is the secret weapon for understanding your customers better and delivering personalised, efficient customer experiences at scale.

In this post, you’ll learn how to use Voice Intelligence to surface data from call content to identify potential customer churn to a competitor. You’ll use a combination of Twilio’s pre-built Language Operators together with a Custom Operator that we will create. 

To do this, you will be doing the following:

1. Setting up a Voice Intelligence Service and create a custom Language Operator to detect competitor references.

2. Creating a custom middleware to receive webhook status callbacks when the call transcription has been processed. I’ll be using Twilio Serverless Functions for this, but you can use the backend of your choice.

3. Using the middleware fetch the Transcript and Language Operators for the Transcript and if a competitor reference is detected, send an SMS to the customer using Twilio Programmable Messaging

Language Operators

If you’re not familiar with Language Operators, they are modularized Natural Language Understanding capabilities, which analyse your customer calls and turn the voice data into structured information that can be used programmatically.

You can use the pre-built language operator for out-of-the-box functionality like voicemail detection and call recording disclosures, or create your own operator to classify calls and detect key phrases in a call.

The inferences from the operators will show up in the call transcript and you can leverage operators to build automated workflows—which is what you’ll do in this guide.

Prerequisites

You will need:

• A Twilio Account. If you don't have one, sign up using this link to get a trial account. 

• A Twilio Phone Number.  You’ll be provided with a number in your trial account.  If you want to purchase an alternative number you can release the number provided and purchase an alternative number or upgrade your account.

• The ability to forward an incoming call to another phone in order to have the customer and agent conversation.  To keep things simple we’re going to forward the call to another phone number that we own using a TwiML Bin and Twilio’s Programmable Voice <Dial> Verb.

• Make sure your calls are recorded because Voice Intelligence processes recordings in order to transcribe the call.  We’ll set recording up in our TwiML Bin, the steps are provided below.

Set up the TwiML Bin

Creating a TwiML Bin in Twilio is a straightforward process that allows you to define how incoming calls are handled. In this case, the TwiML Bin will make a call to your mobile number and ensure the call is recorded with dual-channel recording. Here are the steps and the example TwiML code you'll need:

1. Navigate to TwiML Bins in the Twilio Console.

2. Create a New TwiML Bin: Click on the Create New TwiML Bin button or the blue plus (+) icon.

3. Name Your TwiML Bin: Enter a friendly name that describes the purpose of your TwiML Bin, such as Call-Forwarder-Recorded.

4. Enter Your TwiML Code: In the TwiML editor, input the XML code that defines the behaviour of your calls. Use the example provided below.

5. Save Your TwiML Bin: Once you've entered the TwiML, click Create to save your new TwiML Bin.

6. Configure Your Twilio Number: Go to the Phone Numbers section, select the number you want to use, and configure it to execute your TwiML Bin when a call comes in.

Example TwiML Code:

In this TwiML:

• The <Dial> verb is used to call the specified mobile number.

• The record attribute is set to record-from-ringing-dual to start recording from the ringing phase with dual-channel recording, which records the caller and recipient on separate audio channels.

Build a Webhook Endpoint with a Serverless Function

To utilise Voice Intelligence Service webhooks, you'll need a destination for the events. In this example, I’m using Twilio Functions, a serverless platform that simplifies creating event-driven webhooks without worrying about infrastructure management.

Start by setting up a new Service in Twilio Functions. You can name it something like voice-intelligence-handler or any other descriptive title. Follow Twilio's guide for creating a new Service if you need assistance.

After creating your Functions Service, add a new Protected Function. Choose a path for your Function's URL - it can be something simple like /webhook since it won't be publicly accessible. Replace the provided boilerplate code with this basic version:

This code will respond with a 200 OK to any request from Twilio. The function's protected status ensures it returns a 403 Unauthorised to requests from other sources, as Twilio requests are authenticated using the X-Twilio-Signature header. While this empty 200 response isn't particularly useful yet, it provides us with a URL for your service that you can use in the Voice Intelligence Service configuration next. You'll enhance this function later to handle sending SMS messages to customers.

After deploying the service (by saving the function and clicking Deploy All), you can obtain the URL for your new voice intelligence handler. Look for the Copy URL option below the code editor. If you named your service voice-intelligence-handler, the URL will resemble https://voice-intelligence-handler-XXXX.twil.io/webhook.  Keep the URL to hand as you’ll need it in the next step.

Configure a Voice Intelligence Service

For the purposes of this blog you’ll be using the Twilio Console UI to configure Voice Intelligence, but you could configure the same using the Twilio REST API or even by the Twilio CLI if you wanted to.

In the Twilio Console, navigate to Explore Products on the left hand menu. In this screen underneath the Programmable Communications section you’ll find Voice Intelligence.

Create the Service

Once pinned, navigate to Voice Intelligence and on the Overview page scroll towards the bottom to the Getting Started Guide, expanding the drop down for Create Service. Click the View Services button. 

Once inside the Services page click the Create new Service button. Give your Service a Unique Name, an optional Friendly Name and choose from one of the available languages presented in the drop down list. For this guide, I’m going to leave the language as the default English (United States) as my customer is American; you can choose a language that is most appropriate for your call parties.

In order to use the service you’ll need to accept the Twilio Predictive and Generative AI/ML Features Addendum by checking the box.  You also have the option to:

• Redact Personally Identifiable Information PII from both transcripts and recordings (for this guide leave this disabled)

• Enable auto transcription on your Twilio Call Recordings - be sure to enable this as this is the mechanism we’ll use for this guide.

• Opt in for Twilio’s data use option.

Click Create to create the service and you’ll be taken to the Service configuration page.

Set up the Language Operators

The first tab lists the Language Operators available to you out of the box. You’ll need to add any to the service that you wish to use by clicking the adjacent Add button next to each.  For this guide you’ll use the Entity Recognition, Conversation Summary and Sentiment Analysis. Once added, you also need to create a custom Language Operator so that you can detect if any of Owl Utilities’ competitors are mentioned on a call.

To create our Competitor Reference custom Language Operator, click the Create Custom Operator button at the top of the page and when prompted enter your Operator Name Competitor Reference and choose the Phrase Matching type. Once completed click Next.

On the next screen you’ll enter your phrases to match against. Start by giving your phrase set a name (we’ll call this Competitors) then add the phrases you want to use to train the Operator. For this example enter a phrase to match each of Owl Utilities four main competitors: Owl Utilities, Hummingbird Energy, Buzzard Gas and Woodpecker Energy.

You have the option of specifying an Exact match which will match exact phrases or a Fuzzy Match which will find words or phases in transcripts using machine learning techniques even if that match is less than 100%. For this example use a fuzzy match for each. When completed click the Create Button.

The final few steps are to add the Operator to the Service by clicking the Add to Service button. Then on the Settings tab you can view more information about the Operator including its unique ID, created date and time plus supported languages.  

You can choose to specify to apply this Operator on a single channel of both. This is useful if you only want to leverage this with one party, e.g., checking whether the agent has introduced themselves by name. For this example we’ll leave this set to Both.

Configure the webhook

Next navigate to the Webhooks tab and paste in the URL of our Function that we copied earlier.

Update the Function Code

Now you’ll need to copy the code from below and update your Function. Simply replace the placeholder code you have with this. Be sure to hit Save and then Deploy for your changes to take effect.

Here’s a Sequence diagram of the end to end call flow annotated with the steps numbered in the code.

To summarise what’s happening, the code runs when Twilio sends a webhook event to indicate a transcript is complete. It parses the webhook event and then makes two requests to the Voice Intelligence API: Firstly to Fetch the Transcript Resource and Secondly to Fetch the Operator Results. It performs a check on the Operator Results to see if:

1. There is a match against our Competitor Reference Language Operator; and

2. The sentiment is anything other than positive (which would catch any neutral, mixed or negative sentiment).

If both of the above statements are true, the code makes an API call to the Twilio Programmable Messaging API to send an SMS message to the customer.

Take a closer look at the API Responses

After the webhook is received we take the transcriptSid and use this to fetch the Transcript Resource via API. The API response looks something like the example below.  We’ll be using the mediaParticipant_id of the first channelParticipant to obtain the customer’s phone number as we’ll be needing that when it comes to sending the SMS to them.

Next, we’ll make the API call to fetch the Operator Results. The API response can be quite large depending on how many Language Operators you have added to your service but the part we’re interested in for this example is the presence of:

1. The Competitor Reference and we’re checking for an extract_match value of  true; and

2. Sentiment Analysis, specifically the value of the predicted_label being anything other than positive. In this example it is neutral.

What happens if the Function fails?

In this particular case, I am ignoring errors in my function, i.e. if the Competitor Reference Language Operator isn’t detected, we return a HTTP 200 and continue regardless. This was a decision based on it not being mission critical that these messages get through.

Test with a call

Now you have everything in place, it’s time to make a test call and perform some role play!  Playing the role of the customer, dial your Twilio number you configured to route to your TwiML Bin, a call should arrive on the phone number you configured in the TwiML Bin.

As the customer, mention you’ve received an offer from a competitor Hummingbird Energy and you’re thinking of switching to them. You can role play this out further—perhaps pretending the agent will need to transfer you to another department. Pretend you don’t have time to hold and hang up the phone.

As soon as you hangup, Twilio will process the recording and the transcript will be available shortly after, at which point Twilio will send a request to your webhook and the customer should receive an SMS.

What if I didn’t receive the SMS?

Check your Transcript

If you didn’t receive the SMS, you can find your transcript in the Twilio Console UI Transcripts Viewer to see if the Competitor Reference language operator was successfully matched. 

If it wasn’t, you may want to try again.

Other things to check

• If you don’t see your transcript

  • Check you have a call recording for the call in the Recording Logs.

• If you see your transcript and the Language Operator match but don’t receive the SMS

  • Check for SMS errors.

  • Check there are no webhook errors—any error will likely mean there is an issue executing your Twilio Function.

  • Debug your Function for errors.

     

Wrapping Up

Building with AI to mitigate customer churn automatically is just one example of how Voice Intelligence can help businesses. Voice Intelligence could also help evaluate customer satisfaction, decoding customer frustrations, identifying missed opportunities, agent quality monitoring and training, enriching customer FAQs, predictive maintenance based on trends, automated data entry or identifying compliance risks—not just in a Utilities setting but across any industry taking voice calls.

You don’t even need to use Twilio to host your calls, you can bring your recordings to Twilio and start mining your historic calls for rich data insights. To learn more about Voice Intelligence check out our developer documentation. If you want to bring your own recordings to Twilio you can find out how to do that here.

Ben Datlen is a Presales Solutions Architect at Twilio, where he helps businesses unlock the full potential of communication technologies.  When not architecting solutions, Ben enjoys playing the guitar, riding his bike and spending time with his family and 2 dogs. He can be reached at bdatlen@twilio.com.