Harnessing Contextual Data with Predictive Intelligence: Lessons from Customers using Predictions

Harnessing Contextual Data with Predictive Intelligence: Lessons from Customers using Predictions

First-party data is the most reliable source for understanding and predicting customer behavior. At Twilio Segment, we have seen this firsthand with our Predictions product, which helps businesses leverage their first-party data to predict customer behavior and drive meaningful business outcomes.

Predictions in action

We built Predictions to make it easy for businesses to generate AI-driven insights quickly and efficiently. Each model built by our customers is built using only their data - see nutrition facts for Predictions for more details. It is a fully managed solution that simplifies model creation and manages the entire model lifecycle. Our platform automatically:

• Retrains models regularly to ensure accuracy.
• Evaluates model performance against benchmarks.
• Provides transparency by displaying key insights for customers.

We present users with clear, actionable details, including:

• Prediction distributions to help businesses interpret and act on results easily¹.

• Key model drivers for deeper insights into their first-party data².

• Historical vs. new data comparisons to build trust in predictive accuracy³.

Driving results with AI-Powered Predictions

Since launching Predictions, our customers have built over 500 predictive models. To assess their effectiveness, we analyzed how well these models identify high-converting users.

The following graph illustrates this by comparing the conversion rates of users predicted to be high-value (top decile) versus the average conversion rate across the entire user base⁴.

• The X-axis represents the observed "lift" in conversion for the top 10% of users based on historical predictions relative to the average user.
• The Y-axis represents the proportion of models in production that achieved a given lift.

A lift value greater than 1 indicates that the model successfully distinguishes high-converting users from lower-converting ones. The higher the lift, the better the model’s predictive power. Our analysis shows that more than 99% of models consistently achieve strong lift, confirming their ability to provide actionable insights.

By leveraging these predictions, businesses can:

• Target high-value users more effectively.
• Optimize marketing and engagement strategies.
• Improve customer retention and revenue growth.

Customer wins

While it is satisfying to see that the models that customers build using our platform are highly predictive, seeing businesses achieve real results with Predictions is the true measure of success. Following are a couple of examples where customers have leveraged Predictions to drive meaningful business outcomes.

• Reduced acquisition costs and higher lifetime value. The Motley Fool, an investing and stock market research company, quickly saw results after deploying Twilio’s CustomerAI Predictions. During the quarter, the company's acquisition team was able to easily identify and target ads to people with a high predictive lifetime value, which reduced their lead acquisition costs by 34% on Facebook. Plus, the customers they acquired with this predictive audience had a 9% higher customer lifetime value than those acquired by non-AI audiences — meaning they're gaining better quality customers at a lower cost.
• Higher return on ads (ROAS) and clickthrough rate CTR improvement in campaigns. TradeMe reported a 10% improvement in click-through rates (CTR) and over a 20% increase in open rates for campaigns built with AI Predictions. They also reported that audiences built with AI Predictions delivered 2-3x higher return on ad spend (ROAS).

Looking ahead

We are constantly working to improve our models and expand their capabilities to cover more use cases. Our goal is to make predictive AI even more accessible and effective for businesses of all sizes.

Want to see how Predictions can work for you? Check out our documentation.

Pilar Fernández Gallego is a Machine learning engineer at Twilio. She can be reached at pgallego [at] twilio.com

Ankit Awasthi is a Director of Engineering at Twilio. He can be reached at aawasthi [at] twilio.com

¹ The distribution shows the probability of conversion on the Y-axis and the respective percentile on the X-axis. We calculate the average probability of conversion for the selected range and divide it by the average probability of conversion over all the users. The data shown in this image is simulated data.

² The pie chart shows the top events contributing to the performance of the model. The values for each event is calculated using Shap values for individual features derived from respective events and then combined to get the value for each. Other metrics are recorded at the time of training and inference. The data shown in this image is simulated data.

³ This is computed by looking at conversions that happen after the prediction is made and by bucketing users in deciles based on their predictive scores. Higher deciles should ideally have higher conversions, showing the model is predictive in out of sample data. The data shown in this image is simulated data.

⁴  The data is collected over the 7 day window following the predictions made by the model. Corresponds to a single snapshot of the data. Filters out models where the total number of conversions were less than 100 over the 7 day window.