Unleash the Power of Precise Conversational Insights with Call Annotations

Unleash the Power of Precise Conversational Insights with Call Annotations

We are excited to announce the general availability of Call Annotations, a powerful new tool for customers to enrich Voice Insights Call Summary records with subjective contextual information, facilitating analysis and reporting, ensuring compliance, enhancing customer experience, and enabling seamless integration with other systems. Call annotations provide valuable context and insights that drive actionable outcomes and contribute to the overall success of voice communication strategies.

What is Voice Insights Call Annotations?

Voice Insights Call Annotations allows Twilio Voice customers to add subjective experience details to the Voice Insights Call Summary record, including:

• Call quality issues
• Connectivity issues
• Call score
• Spam labeling
• Answering machine detection
• Your internal tags, keys or identifiers and metadata for a single call such as custom comments and incident ticket URLs

Refer to this API doc for more information on each parameter.

With Call Annotations, you can also get useful insights on calls via the Voice Insights API by filtering call summaries using any of the labels used to annotate the calls (more on this later in this post).  

Call Annotations also supersedes the soon-to-be-deprecated Feedback API and the Feedback widget in the Twilio Console.

How to Annotate Calls

You can annotate calls anytime after the call initiates (and a call_sid is available), which means a call can be annotated during an active call and after the call has ended for up to 30 days post call initiation. In the below example we are annotating a call with customer feedback on: Quality Issues, Call Score, Spam labeling, Answering Machine detection, and a support ticket associated with the call.

Sample Command

For instance, in the sample command you can see that the call was tagged by the user for quality issues such as low volume, choppy audio, DTMF issues, latency, echo, OWA (one way audio) and static noise.

Sample Response Body

How to filter calls based on Annotated values

Once a call has been provided with subjective feedback by the user using call annotations, the call summaries can then be filtered on one of the annotated properties. For instance, in the code sample, the call summaries were filtered on calls which were answered by a human being and calls which had an echo in the line as experienced by the users.

Why is this important?

Annotated call summaries provide valuable context and insights that drive actionable outcomes and contribute to the overall success of voice communication strategies. Some of which are:

1. Training and Coaching Contact Center Agents: Annotated voice calls serve as valuable training resources for new agents and ongoing coaching for existing agents. By reviewing recorded calls with annotations, agents can learn best practices, understand customer preferences, and gain insights into effective communication techniques. Supervisors can use annotations to highlight examples of exceptional customer service or identify areas where agents need additional training and support. The workflow below illustrates a potential workflow to leverage LLM models to do automatic sentiment or intent classification on the call transcriptions then tag back the call summary, using call annotations api, with the extracted output. Alternatively, an agent can use call annotations to augment call summary based on their subjective experience of the call with the customer. Those annotated calls can then form the pool of calls to be leveraged for future agent training.

Here are a few samples of calls tagged for agent training using call annotations:

• Example of a call which was labeled as a training call for new agents

• Example of a call which was labeled as a training call for new agents in which the agent managed to turn the customer sentiment favorably. The customer sentiment could be derived from an actual conversation fed to an NLU engine and the output of which can be tagged back to the call summary using call annotations api.

• Example of a call which was labeled as a training call for new agents in which the agent managed to upsell a new solution to the customer.

2. Compliance monitoring: Call Annotations can play a crucial role in compliance monitoring and auditing.

• Compliance Tagging: Use annotations to tag calls that involve compliance-related aspects including specific disclosures for calls related to regulated industries.

• Data Protection and Consent: Annotate calls where customer consent for data processing or marketing purposes is obtained. Include details such as the specific consent given, the date, and any related information required for compliance with data protection regulations. These annotations may serve to identify that proper consent was obtained during the call.

• Spam Tagging: You can also tag calls which were deemed to be mislabeled as spam by end user feedback or via 3rd party sources. Call Annotations serve as markers to identify calls that require closer scrutiny from a compliance perspective. They can be used to track and flag calls that may be violating regulatory requirements or company policies. For instance, if an outbound sales outreach call was deemed as unwanted in nature by the end customer, the same experience tag can be associated with the call as shown below:

3. Call Quality Monitoring:  The  Voice Insights APIs, along with the Annotation Resource, can be effectively utilized for call quality monitoring in the following ways:

• Call Quality Metrics: Retrieve call quality metrics such as jitter, packet loss, latency, and experience details such as calls which had silence detected. These metrics provide quantitative data on call quality and user behavior.

Here is a sample code to retrieve calls which had call tags:

• Real-time Monitoring: Set up real-time monitoring of call quality by subscribing to events using Twilio Voice Insights Event Streams. Receive notifications or trigger actions based on call quality events, such as high packet loss or significant audio quality degradation. Monitoring enables you to proactively identify and address call quality issues as they occur.
• Annotation for Quality Markers: Use the Annotation Resource to add quality markers to calls. Annotate specific moments in the call where there were quality concerns, such as audio distortion, dropped audio, or other audio anomalies. Annotations provide a reference point for quality analysis and can help in identifying patterns or common issues affecting call quality.

• Post-call Quality Assessment: Analyze call quality after the call ends by retrieving call records and associated annotations. Use the Twilio Call Summary API to retrieve call details and annotations for further analysis. By reviewing the annotations and call quality metrics, you can assess the overall call quality and identify areas for improvement. The example below can be used to list all call summaries in the last 4 weeks which were annotated for Quality Issue of the type “audio echo.” This data feedback is also extremely useful for Twilio support to do a root cause analysis of calls experiencing quality issues, reducing the overall time to resolution.

4. Marketing Analytics: (A/B Testing Outbound sales campaign effectiveness): Call Annotations allows for easy categorization and analysis of voice calls based on specific criteria. For example, customers can seamlessly A/B test outbound campaign effectiveness by using call annotations to label outbound calls for each campaign. By tracking call answer rates for the respective campaigns, customers can gauge the effectiveness of each. Here is an example of a call being tagged as a part of a Test A campaign in a A/B test scenario:

5. Enhanced customer experience: Call Annotations can contribute towards improving the overall customer experience by providing valuable insights about each call. Contact center agents can leverage call annotations during their interactions with customers and label calls with useful subjective feedback.

Capturing Agent tags for the call:  For example, call annotations can capture both the agent as well as the customer feedback, sentiment, or specific issues raised during the call. This information enables businesses to personalize interactions, address customer concerns promptly, and identify areas for service improvement.

Furthermore, call annotations can be leveraged to elevate a sales dialer's user interface, allowing real-time capture of invaluable feedback from the agent's viewpoint. Here is an illustration that showcases a sales/agent dialer with an enriched UI that presents call annotation parameters. Agents can seamlessly record their subjective insights for each call, and once annotated, this information updates the call summary, accessible through Voice Insights. Thanks to Voice Insights' support for event streams, these call summary events can also be integrated with Segment, enhancing end user profiles and traits.

Capturing end user feedback at the end of the call: Imagine the immense value in collecting feedback from both agents and end customers, gaining dual perspectives on each call. This dynamic capability empowers contact centers with a comprehensive view, a potent asset for enhancing customer experiences. For instance, an agent's perception of a call's success may differ from the end user's, and call annotations offer a means to capture user feedback post-call. Post call-surveys can gather insights through DTMF responses or speech input using the TwiML Voice <gather> . These responses can then be used to augment the call summary using call annotations as illustrated below:

With this workflow, call annotations provide a distinct advantage by effortlessly uniting feedback from both agents and end customers. This integration effectively eradicates the prevalent issue of data silos, sparing the considerable effort typically required to consolidate call details. This enables contact centers to adopt a more unified and well-informed strategy for enhancing customer engagement and service excellence.

6. Customer Intelligence and Attributions using Machine Learning: Call annotation effectively crowdsources first-party tagged data, a goldmine in the realm of machine learning development. Building ML engines often hinges on the value of this first-party data. With call annotations, you can turbocharge your first-party tagging initiatives by exposing call annotation parameters to client UIs and dialers. Once a call is tagged, these annotations become retrievable through the call summary API invocation. This data serves as a vital resource for crafting intriguing attribution models.

7. Feedback Resource EOL: Call annotations supersedes the Feedback resource in the API and in Twilio console. Call Annotation enriches the existing Voice Insights Call Summary records. Customers can use the call summary API to filter calls based on annotated values thereby making it useful to analyze specific calls of interest.  Customers should look to transition away from the Feedback API to the Call Annotation API before the end of the year.

What's next for using call annotations?

In a nutshell, the insights and information derived from call annotations can significantly enhance various facets of call management, including troubleshooting, customer support, efficiency, monitoring, compliance, agent training, performance optimization, and overall contact center refinement. These improvements, in turn, have the potential to boost call answer rates and elevate the overall customer experience.

Refer to our public docs to start trying out call annotations.

We can’t wait to see what you build!

Akshit Dayal is a Product Manager for Twilio Voice Insights and a die hard fan of Twilio products. He is an engineer by training and a builder at heart and is dedicated to building products which drive desirable business outcomes for the customers. He is deeply interested in the intersection of voice and ML/AI and is constantly exploring solutions which efficiently and effectively solve customer problems. He can be reached at akdayal [at] twilio.com.