Working on the Pipeline: Sending Twilio Voice Data to Segment using Event Streams

Working on the Pipeline: Sending Twilio Voice Data to Segment using Event Streams

Unless you want to be stuck renting your customer data from Facebook or Google, the future of your customer engagement depends on having your own first-party data. With the death of the third-party cookie by the end of 2023, you need to adapt to a first-party future, and take steps now to build the integrations between your websites, apps, email interactions and customer data platforms (CDP) such as Segment to help deliver personalized experiences to your current and prospective customers.

Once you've gone through to work of building the pipelines to shunt this first-party data around and aggregate it using a CDP like Segment, the next step is to find new sources of first-party data to enhance the internal representation of your customers to more accurately and effectively deliver the personalized experience 83% of customers say they prefer.

Someone clicking a banner on your site, filling up a shopping cart in your app, or opening a link in an email are obvious sources of first-party data to help construct a profile. What can be less obvious is how you can utilize metadata generated by Twilio communication channels like Messaging or Voice to understand your customer's communication preferences and tune your interactions with them appropriately to maximize the likelihood of a delightful experience.

For example, you've got a phone number from a customer and they have opted in to receive messages from you, but you see zero engagement with the messages you send. Are the messages being delivered? If not, could it be due to the customer providing you a landline number that is not able to receive SMS in the first place? If you're doing outbound calls, are they being blocked by the carrier due to lack of STIR/SHAKEN attestation, or being labeled as potential spam when they arrive at the handset? Does the customer not recognize the calling number and send every call straight to voicemail? These are the types of questions you can answer using the call data generated by Twilio Voice Insights, and answers you can now get into Segment using the Event Streams sink pilot.

Let's make it happen.

Enable Voice Insights Advanced Features via API

Since we're talking about sending Voice Insights data to Segment, we need to enable Voice Insights Advanced Features. The Voice and Conference Insights dashboards, and Call, Conference, and Participant summaries you know and love from Console are included at no additional cost with every minute on the platform, but programmatic access via API or Event Streams requires Voice Insights Advanced Features to be enabled. You no longer need to go into Console and enable onesie-twosie on accounts, we have exposed a Settings API endpoint that allows you to enable with a simple request:

You can also pass an account SID in the request using the SubaccountSid parameter to enable Voice Insights Advanced Features on your subaccounts, which makes life a lot easier for ISVs with potentially thousands of subaccounts. The response you receive from Twilio will confirm the operation was successful:

Enable Voice Insights Advanced Features via Console

If you don't have a bunch of subaccounts you need to manage, or just prefer using Console, you can also manage Voice Insights Advanced Features status on the Monitor tab under Call Insights Settings .

Setting up Segment

Once Voice Insights Advanced Features is enabled, we need to have someplace to land the data. Go into the Connections section of the Segment app and add a source. If you search for Twilio you will see Twilio Event Streams (Beta) source as an option; select this and proceed.

Give your Source a friendly name and click Create Source. Once the source is created you will receive a Write Key that you will need later when we're configuring Event Streams, so write that down and keep it handy.

Now that you have a source set up you can send the data wherever you would like; in my case I chose to send it to an existing Redshift destination. You will need to provide configuration details for your destination like host, port, auth credentials, etc.

Once those config deets are provided, your Source->Segment->Destination pipeline is set up and ready to party.

Create an Event Streams Sink via API

Now that you've got Voice Insights Advanced Features enabled you can use the Twilio CLI and configure Event Streams to send data to Segment. The first thing we need to do is create Sink, where we will pour our call data into the pipeline on its way to Segment. You will need to pass the Write Key you got when you set up the Source in Segment. You will get a SID that starts with DG in the response; copy this for use later on.

Create an Event Streams Sink in Console

You can also create a Sink directly in Console. Just provide the description, select the type, and provide the Write Key for your Segment Source and voila!

Subscribe to Voice Insights Call Summary records with Event Streams via Twilio CLI

In this example I am signing up to have Voice Insights Call Summary records sent to Segment whenever the summary is completed; you can also have partial records sent as the summary is being generated, or you can use our ML-based predicted complete event to receive the summary data sooner.

You are by no means limited to receiving Call Summary records, either. You can set up subscriptions to Call Events, Call Metrics, Participant Summary, and Conference Summary records as well. We keep parity between the public API schema and what we send to Event Streams, so anything you're used to consuming via the REST API, you can also interact with over Event Streams. This post is focused on Voice, but you are certainly not restricted to sending Voice events, you can see the full catalog of subscriptions here.

Subscribe to Voice Insights Call Summary records with Event Streams via Console

If command line interfaces do not pass your vibe check, you can always create a new subscription directly in Console.

Validating the Pipeline

Once the Subscription is created, data will start to flow to Event Streams from Voice Insights, and then through to Segment. Call Summary records are not marked as complete until 30 minutes after the call has ended, so if you do not already have calls being placed on the account you are configuring this pipeline on, then you will need to place a test call and wait for the summarization to complete, Voice Insights to emit the summary, Event Streams to receive and process it, and for it land in Segment. Based on my testing it takes seconds for a completed Call Summary to land in Segment; you can validate that your call data is landing in Segment by going into your Sources and clicking on Debugger to see the records that have been sent.

Use Cases

So now that you've got your Twilio Voice data landing in Segment, you can start to do some of the cool stuff described above, like determining the preferred communication channel for your users. Other ideas you might consider are:

• You can use the start_time of calls to track when consumers answer the phone and store a custom trait to use for next engagement, or use the answered_by parameter to identify customers who actually pick up their phone instead of letting it go to voicemail and mark them as priority targets for the Voice channel.
• If you're using Twilio's Voice SDK to provide in-app calling, you can correlate SDK events published to Voice Insights with in-app activity to understand user behavior; i.e. for a rideshare app is the rider using the inbound call notification from that app as the "notification" that the driver is downstairs, and thus you shouldn't sweat your answer rates?
• In a retail business, are your customers calling in immediately after visiting a store, or conversely, are their phone interactions ending unsatisfactorily and causing them to come in for resolution?
• If your finserv app has in-app calling built into it to connect investors with advisors, what's the relationship between subjective call quality and conversion rates? Is there a threshold of quality degradation where it makes sense to bail on the VoIP call and switch to PSTN to save the day?
• In healthcare missed appointments are a huge opportunity cost for the providers; you know your scheduling system is sending out notifications in advance of appointments, but are they being received? What's the answer rate on your appointment reminders? What percentage of the answered calls are going to voicemail? What's the correlation between reminders going to voicemail and missed appointments, if any?
• If you want to measure the hit rate of your lead generation campaign and you have the lead's phone number, it's possible to monitor whether or not leads contacted your company via the voice channel, regardless of the original channel they were contacted over. This data can then be used to help tune the effectiveness of the marketing campaign.
• Identify churn risks and opportunities to salvage fraught customer relationships by monitoring repeated calls into your contact centers. First call resolution is de rigueur, so any customers you see blowing up your phone represent a high likelihood of a dissatisfied customer and a customer will only stay dissatisfied for so long before they churn. Define a threshold for what you consider to be too many calls, and when that is breached, flag that account for review and potential outreach from your resolution desk or saves team.

You can also use this Event Streams -> Segment pipeline to personalize your Flex IVRs, capture Studio Flow progress, or build something no one else has thought of using the Segment Developer Toolkit.

Once the flow of data from calls made using Twilio to Segment is established, your organization can start to tease out the relationships between your customers' call behavior and your desired business outcomes, unlocking the power of streaming data pipelines and customer data platforms.

We can't wait to see what you build!

Michael Carpenter (aka MC) is a telecom API lifer who has been making phones ring with software since 2001. As a Product Manager for Programmable Voice at Twilio, the Venn Diagram of his interests is the intersection of APIs, SIP, WebRTC, and mobile SDKs. He also knows a lot about Depeche Mode. Hit him up at mc (at) twilio.com