Introduction to Application Testing with Twilio

Introduction to Application Testing with Twilio

In this article you’ll learn how to simplify the maintenance of your Twilio application by using various types of application testing. We’ll focus on testing SMS & Voice applications, but these principles can be applied to other Twilio services as well.

Let’s get started.

Unit Testing

Unit tests are automated tests written to ensure that a section of an application (known as the "unit") behaves as intended. Traditionally, a unit test is defined as a test on the smallest piece of code that can be logically isolated in a system. In procedural programming, a unit could be an entire module, an individual function or a procedure. In object-oriented programming, a unit is often an entire interface or class.

Many developers do not consider unit tests when they rely on external systems, but when building applications with Twilio’s API, the unit includes the API call to the Twilio endpoint.

Twilio Test Credentials allow you to exercise parts of Twilio’s API without charging your account. You can send mock requests and receive a response complete with HTTP status codes & error messages. This provides you with confirmation that the parameters of your request to Twilio are properly formed and the unit is functioning as intended.

Here’s an example of a Node.js application utilizing Test Credentials:

And the response:

Here are a few articles that provide a deeper dive on Unit Testing with Twilio:

• Getting Started with Unit Testing a Laravel API using PHPUnit
• Unit Testing Your Twilio App Using Python’s Flask and Nose
• Unit Testing Twilio Programmable SMS in .NET Core with xUnit, Moq, and Visual Studio 2019
• 6 Awesome Reasons to Use nGrok when Testing Webhooks
• Interactive Voice Response (IVR) Testing With Python and pytest

Error Detection

End-to-end testing is a prerequisite for integration testing and live testing. This will require you to make live requests to the Twilio API. However, you need to understand how to detect when an error occurs before you run a full end-to-end test.

Error Logs

Error logs are available in your Twilio Console (see Debugging Tools). You can use these out of the box dashboards to manually review logs at any time.

Error Alerts

Checking error logs manually is manual. For a more automated approach, you may want to consider building an error handler that notifies your team when errors occur. This makes testing much easier and, as a bonus, you can re-use it for production error alerting!

Twilio Phone Numbers & Messaging Services allow you to define a Fallback URL, i.e. a URL that Twilio requests in the event of a fatal error. Checkout our Availability Reliability Documentation and Twilio Error and Warning Dictionary for more details.

First, you’ll build a handler and publish it to a URL endpoint. I will use Twilio Serverless Functions in this example but feel free to use whatever tools you prefer.

Below is an example of an SMS alerting handler that sends an SMS message to each of the engineers whenever a Primary Handler webhook of a Phone Number or Messaging Service receives a response w/an error status. Keep in mind, this can look many different ways depending on your unique application.

Here is the payload expected for the call above:

Events API

Twilio’s Event Streams API allows you to tap into a unified stream of every interaction sent or received on Twilio, including errors. Implementing an error detection system with Event Streams is probably not worth it unless you are already using Event Streams for some other reason. Thus, we won’t cover an implementation here.

However, here are a few Event Types that you can subscribe to listen for errors:

Integration Testing

With error detection out of the way, you can begin building end-to-end tests of your Twilio application. The first type of end-to-end test we will focus on is integration testing.

There are many different integration testing philosophies: big-bang, sandwich, risky-hardest – to name a few. The big-bang approach tests all individual modules of an application at the same time. Since integration testing with Twilio requires a live end-to-end test, the big-bang approach is the most applicable.

Simulating Production Traffic

Conducting live integration tests with Twilio is simple because you can mimic production traffic by using Twilio’s API to programmatically send SMS messages and make live voice calls.

As an example, this is all you need to execute an integration test for an SMS application. Remember, we have already built an alerting system so once these tests are executed we will be notified when a failure occurs.

Here are a few additional resources on this topic:

• Get Started Testing a Yii2 API Using Codeception
• Testing Twilio SMS Requests for .NET Applications with the Rider REST Client
• Building Better Phone Trees With Twilio Using Multivariate Testing

Live Testing

Live testing is designed to help you identify outages before your customers do.

Implementing live tests is fairly straightforward. You simply mimic production traffic by using Twilio’s API to send SMS messages and make voice calls, as described above in Simulating Production Traffic.

Designing live tests in a way that provides value is more challenging. Generally, live testing is best used for applications with a high Delivery Time Objective (DTO), i.e., the average time an outage would be undetected in a production environment. High volume use cases benefit very little from live testing because outages will be discovered quickly by production traffic.

Here are a few strategies to maximize the effectiveness of your live testing.

Low Volume Use Cases

The use cases which benefit the most from live testing typically have irregular usage patterns. For example, an outage notification use case will typically have long periods without usage, followed by short bursts of high throughput.

These use cases are ideal for live testing because we have an opportunity to be notified of an outage long before it affects production traffic.

Off Hour Use Cases

Imagine a delivery notification system that sends an average of one message per minute during business hours. Even without live testing, this use case has a DTO of 1 minute (assuming the messages are sent consistently). In other words, performing live-testing during business hours would have a maximum ROI of 1 minute of advanced notice – probably not worth it.

However, it would be valuable to perform live tests before the start of each business day. This would give your engineering team time to address any outages before the application hits peak times.

Overlapping Capabilities

In circumstances where an application serves multiple use cases, we can maximize the effectiveness of our live testing by considering the overlapping dependencies of each use case.

For instance, say we have built a two-factor authentication application that operates in the United States, Canada, UK, and Japan. Sending a test message to a United States phone number, we are inadvertently testing the services that the other regions use and are, in part, testing them as well.

Final Thoughts

In this post you’ve seen how to implement unit tests with Twilio Test Credentials, perform end-to-end integration testing and how to maximize the effectiveness of live testing. Application testing is a large topic but hopefully this has provided you with a better understanding of how to get started.

We can't wait to see what you'll build with Twilio. Let us know on social media and don't forget to @TwilioDevs on Twitter or LinkedIn.

Phil Bredeson is a Principal Solution Engineer with Twilio. He helps Twilio customers design scalable and performant solution architecture.

Mark Shavers is the Manager of Solutions Engineering at Twilio and his team supports Twilio's Enterprise customers.  He has been in the technology industry since before MySpace was popular. He enjoys working with builders to create innovative solutions, and when he's not building he'll be volunteering with local youth sports teams or charities. Get in touch with him by connecting with him on LinkedIn.