Interactive Voice Response (IVR) Testing With Python and pytest

Interactive Voice Response (IVR) Testing With Python and pytest

In a previous tutorial you learned how to build an IVR system with Python, Django and Twilio. In that project you created an IVR system that provides users with information about movie show times over the phone.

In this article you are going to add automated tests for the IVR system using Pytest.

IVR Testing: Why It Matters

Maintaining good and comprehensive tests is crucial for your project's success. First, as a developer, writing tests gives you an opportunity to experience what it's like to be a customer of your own system. Whether you are developing an API, a website, a dashboard or an IVR system, writing tests is like using your own system. If there is something wrong, or something that can be improved, you are going to notice it when you try to test it.

Good tests are tests you can trust. If once all the tests pass you feel safe releasing to production, it means you have good tests. Having reliable tests will provide you with the confidence to make changes in the future without breaking your system. A system with bad tests, or no tests at all, can easily fail in production without you knowing.

Writing tests for processes that rely on third-party services can be especially challenging. However, if you structure your system correctly, you should be able to isolate and test only your business logic. In this article you are going to experiment with techniques to do just that with IVR testing.

Requirements

To follow along with this tutorial you are going to need:

• Python: you can find installation instructions on Python's website.
• Django: A web framework for Python.
• Twilio Account: A free Twilio account. If you are new to Twilio create a free account now. You can review the features and limitations of a free Twilio account.
• A Twilio phone number: To develop and eventually deploy your IVR system you need a phone number associated with your Twilio account. You can purchase a phone number from Twilio.
• A phone: to test your IVR system you'll need a phone to call into the system.

Let's get started!

Setup

In this tutorial you are going to write automated tests for the movie showtimes IVR system you created in my previous tutorial. If you didn't do the first tutorial and you want to follow along, go ahead and create the project as described in the tutorial Building an Interactive Voice Response (IVR) System with Python, Django and Twilio.

To source code for this tutorial is available here. To follow along, clone the project and follow the setup instructions in the README file.

Working With a Virtual Environment

Virtual environments are useful for creating a separate workspace for each project. Using virtual environments, every project can maintain its own dependencies and projects don't get mixed together. It's a good idea to use a separate virtual environment for each project.

In the previous tutorial you created a virtual environment called venv. To activate it, go to the project's folder twilio-ivr-test, and run the following command from your terminal:

If you are using Windows, enter the following commands to activate the virtual environment:

While the virtual environment is activated, any Python package you install will be installed only in the virtual environment.

Install Pytest

Pytest is a popular testing framework for Python. Unlike other testing frameworks, such as the built-in unittest, Pytest encourages small, function based tests, and uses dependency injection.

To install Pytest, execute the following command from your terminal:

Great! Pytest is now installed in your virtual environment.

Pytest has a wide variety of third-party plugins. One of the most useful plugins is pytest-django. The plugin is maintained by the Pytest team, and it makes it easier to write tests for Django projects.

To install the plugin, execute the following from your terminal:

NOTE: There is a package with a very similar name called “django-pytest”. This is not the package you want to install.

Now that you have installed all necessary packages and plugins, you are ready to configure your Pytest project.

Configure Pytest

Pytest uses a configuration file called pytest.ini. Create the file in the root directory of your project, where the manage.py file is, and add the following content:

The configuration above tells Pytest where to find your Django settings file. Pytest provides many more configuration options, but for our purposes this is enough.

Write Your First Test Case

To make sure the project is set up correctly, and to understand how Pytest collects and executes tests, you are going to write a dummy test. Create a new file called test.py in the root directory of your project and add the following tests to it:

Before you execute the test, notice a few things:

• Tests are functions: Pytest encourages function based testing. Unlike other test frameworks that use classes, in Pytest tests are just simple functions.
• Test functions are prefixed with test_*: To mark a function as a test case, the function name must start with test_. This is how Pytest decides which functions to collect.
• Test cases are using assert: To match an observed value against an expected value you use Python's built-in assert statement. Other frameworks sometimes provide utility functions to compare values, but in Pytest you can just use assert.

To execute your tests, open the terminal and run the pytest command:

Pytest provides a lot of information in the output:

• Pytest executed two tests in the file test.py. Dot (.) means a test passed and F means a test failed. In your case, one test passed and one test failed.
• The output includes an entire section devoted to failed tests. Under FAILURES you can see that the test test_should_fail failed. The test expected the variable a to equal 2, but variable a equaled 1, so the test failed.

Now that you executed your first Pytest test, you are ready for the next step. Before you move on delete the file test.py. We won't be using it anymore.

Writing Tests

So far you've configured Pytest to work with your Django project. You wrote a dummy test and saw how successful and failed tests look like. We are now going to write real tests.

Creating a Tests Module

To start writing tests for the movies application, create a new tests module inside it. From your terminal, create a new directory called tests inside the movies directory:

To make this directory a Python module, add an empty file called __init__.py:

On Windows, create these files using the file manager or using your IDE. This is what the tests module in your movies app should contain at this point:

It's a good idea to put your tests in a module of their own inside the application. Just like any other Python module, as it grows bigger you can split tests into separate files.

Testing the Request Validator

In the previous tutorial we discussed the different security measures provided by Twilio such as the request validator. A RequestValidator is used to authenticate requests from Twilio. It works by generating a signature locally using the contents of a request and your Twilio auth token, and comparing it with a signature attached to every request from Twilio. If the signatures don't match, the request is considered unauthenticated and is rejected.

To validate requests in your views you created an instance of RequestValidator using your Twilio Auth Token. You then added a function that accepts an HttpRequest and uses the request_validator instance to validate it. If the request is not valid, an error is raised:

The function handles three distinct cases:

1. The request does not contain a signature: A SuspiciousOperation error is raised.
2. The request contains an invalid signature: A SuspiciousOperation error is raised.
3. The request contains a valid signature: no error is raised.

In the following sections you are going to test these three scenarios.

Using RequestFactory

Create a new file called test_validate_django_request.py in the tests module, and include a test to check that a request with no header is rejected:

This is your first test, so let's break it down:

• You prefixed the test name with test_: this will make Pytest recognize this function as a test case.
• You provided a meaningful name for the test case: the name of the function contains both the tested scenario and the expected result. If this test fails, you should be able to quickly figure out what went wrong.
• You injected Django's RequestFactory fixture to the test case: RequestFactory is a special object provided by Django to create requests in tests. The plugin pytest-django has a built-in fixture that provides an initialized instance of a RequestFactory called rf.
• You created an HttpRequest for testing: using the request factory rf, you created a POST request object to the path /. The path does not matter in this case because you only test the validation, not the routing.
• You used pytest.raises to test that an error was raised: this test case expects a SuspiciousOperation exception to be raised. Using pytest.raises, if this error is not raised, the test case will fail with a helpful message.
• You validated the request using the function validate_django_request: you finally got to use the function being tested.

Now, run your tests and see if it worked:

Great! The test passed.

You still have two more scenarios to cover, so next, write a test for when a signature is provided, but it is invalid:

This function is very similar to the first test function, only this time, you provided a signature in the request header HTTP_X_TWILIO_SIGNATURE. If you run your tests now, you'll see that they pass:

Notice that now two tests pass.

Using unittest.Mock

The last scenario is when a valid signature is provided. In this case, the function should not fail and should return nothing.

To be able to provide a valid signature the way Twilio does, you would have to replicate the entire signing process and include the entire content of the payload in the request. This sounds like a lot of work for such a simple scenario. In fact, if you think about it, testing the signing process is not your job because you did not actually implement it, you are just using it.

The RequestValidator is a great example of an external dependency you are using, but is outside the scope of your tests. You want to test how your code handles different outcomes, without testing the external dependency itself. To eliminate the effect of external dependencies in tests, you can mock external functions and objects. A mock is an object that you use instead of another object, and that you control its output.

The best way to understand how and when to use mock, is with your third test case:

In this scenario you test what happens when a valid signature is provided to the function. You created a request object using the request factory, and you included a signature in the appropriate header.

The function validate_django_request uses an instance of RequestValidator to validate the request. The instance is stored in the variable request_validator in the module movies.views. To validate the request, the function invokes request_validator.validate that returns a boolean indicating whether the request is valid or not:

To implement your scenario, you need this function to return True. So first, you created a mock context using mock.patch. To mock the variable your function uses internally, you provided the mock context with the path to that variable: movies.views.request_validator. While this context is active, any call to this path will result in a call to the patched object mock_request_validator. To make validate return True, you assigned the expected result to the mock object:

When you patched the object you used autospec. The autospec directive ensures that the mock looks like a real RequestValidator object. It is not required, but it helps to catch mistakes so it's a good idea to add it.

Now, when you call validate_django_request inside the mock context, it will invoke your mock object instead of the real object and return True.

Execute your test to see if it worked:

All the tests pass! You successfully mocked Twilio's RequestValidator.

Another benefit of using a mock in this case is that if the internal implementation of RequestValidator changes, your tests will not be affected. By mocking this dependency, you keep your tests isolated and focused on your code rather than the library's code.

Testing IVR Views

So far you've tested the request validation, which is the first thing that happens when one of your views accepts a request. By mocking the validation function, you are now able to "fake" a successful authentication and focus on testing the view's business logic.

Creating Fixtures

To provide your users with information about showtimes, your IVR system uses data stored in the database. Data created for testing is often called a "test fixture". Fixtures are objects and data you use in tests to implement different scenarios.

Your IVR system stores data about movies, theaters and showtimes. Movies and theaters change less often than showtimes, so they are good candidates for a test fixture.

To create your first test fixture, create a file called conftest.py inside the tests module. The file conftest.py is a special file used by Pytest. Fixtures you declare in this file are automatically made available to tests.

Include the following content in your conftest.py file:

Let's take a closer look at your first fixture:

• You decorated a function with @pytest.fixture to register it as a Pytest fixture.
• The fixture function is using another fixture called db. This special fixture is provided by the pytest-django plugin, and it is necessary if the fixture needs access to the database.
• The name of the fixture is the name of the function, theater_A. It creates a new Theater object in the database and returns it.

You can now use this test fixture in Pytest test cases.

Create a new file called test_showtimes_ivr.py inside the tests module, and include the following test:

Before you run this test, let's understand what it does:

• You added a test case by defining a function that starts with test_.
• The function requests the fixture theater_A.
• Inside the function you checked that the values you've set in the fixture are correct.

Now, run the test from your terminal:

Great! The fixture worked as expected. Now that you know how to use fixtures, you can remove this test from the file.

To be able to create different scenarios you are going to need more than just one theater. Edit the file conftest.py and include the following fixtures:

The file now contains fixtures of two theaters, theater_A and theater_B, and two movies, movie_A and movie_B. With these four fixtures you can create different test scenarios.

Creating a Test Client

Twilio IVR system uses a special markup language called TwiML. Your IVR views use the Twilio Python Helper Library to generate TwiML markup using the VoiceResponse class. Unlike APIs and views you are used to working with, the call to your views is not conducted from a terminal or a browser, it's happening from Twilio's IVR service in response to a phone call from a caller.

One way to test your interaction with Twilio without actually making any requests to Twilio in your tests, is to process responses in the same way Twilio does.

To process a Twilio call in tests, create a new file movies/tests/twilio_phone_call.py and add an empty class to it:

You are going to use this class in your tests to interact with your views the same way Twilio does. The TwilioPhoneCall class is going to make requests to your IVR views, so it's going to need a Client.

Phone calls from Twilio include metadata such as the caller's phone number, and a unique identifier for the call. With that in mind, add a constructor to the class:

Before you move on, take a closer look at the arguments to __init__:

• start_url: The first URL that Twilio should make a request to when a call comes in. This is the URL you configured in the Twilio Dashboard.
• call_sid: A unique identifier for the call. This identifier persists across all requests in the same call.
• from_number: The caller's phone number.
• client: A Django test client to make requests to your IVR views. Notice that we don’t need to make requests on behalf of a Django user, so the client can be an anonymous client.

The __init__ function also sets some internal members to manage the state of the call:

• next_url: The next url to issue a request to. The first URL is always the start_url. The next URL depends on the user input, and it will change during the call.
• call_ended: Indicate whether the call has ended.
• _current_twiml_response: We will get to this in a bit. It will be used to process the response.

To create a new TwilioPhoneCall, you create an instance of the class and provide it with the relevant information:

Now that you provided all the necessary information to the TwilioPhoneCall, you are ready to initiate a call. To initiate a call you want the TwilioPhoneCall object to make a request to the start_url. In the __init__ you assigned the start_url to the next_url, so make a request to next_url with the call sid and the caller's phone number:

The function _make_request is making a request to an IVR view in the same way Twilio does. Let's break it down:

• Patch request_validator with a mock object to make the request pass the validation. You already tested the validation process earlier, so you don't need to do it here as well.
• Make a post request to the next_url with the call SID and the caller's phone number, as well as any other payload provided to the function.
• Process the request and return the response. This is not implemented yet, we will get to it in a bit.

The function is internal to the class, and should not be used by anyone outside of it. To declare a function "private" in Python it is customary to prefix it with an underscore _. This won't actually prevent it from being called outside the class, but it is a known convention for declaring non-public methods.

With the ability to make a request you can now implement a method to initiate a new call:

To initiate a call you issue a request to the start url and return the response. There is no payload other than what was already provided in __init__, so you can issue a new request with no additional data.

The last piece of the puzzle is processing the response. Remember, your IVR views produce TwiML markup, and you want to process it the same way Twilio does.

Processing TwiML Responses

In your IVR system, when a call is made to your Twilio phone number, Twilio issues a POST request to the URL /movies/choose-movie. This is the contents of the response:

The TwiML markup is built of verbs. The first verb in the response above is Say. Using this verb, you greet the user "Welcome to movie info!".

The next verb is Gather, which is used to accept input from the caller. Inside the Gather tag there are additional Say verbs that list theaters for the caller to choose from. At this point, if the user enters the digits of one of the theaters, Twilio will make another POST request to the action URL provided in the Gather tag, along with the digits the caller entered. In this case, the action URL is /movies/choose-movie.

If the caller did not enter any digits for the number of seconds provided by the attribute timeout, Twilio will skip to the next verb. In this case, if the user did not enter any digit for 20 seconds, Twilio will say "We did not receive your selection", and redirect.

Implement this logic in your TwilioPhoneCall class:

Let's see what we have here:

• Declare a non-public function _process_twiml_response.
• Accept an HttpResponse and return an iterator of responses. A TwiML response can have different outcomes, depending on the input from the caller. One way to maintain state while we wait for the user input is to use an iterator.
• Check the response status code and if it failed, mark the call as ended and return the failed response.
• Parse the XML content of the response using the built-in ElementTree module.
• Iterate over verbs that require an action and handle them according to their type:
  • When a Hangup verb is reached, mark the call as ended and return the response.
  • When a Redirect verb is reached, set the next URL and make a request. Notice that you handle a special case where the verb does not contain an explicit action, in which case the URL is the current URL.
  • When a Gather verb is reached, set the next url from the tag's action URL, and return the response.

Now that your class can process a TwiML response, you can add two more user interactions to your TwilioPhoneCall class:

The new user interactions are:

• enter_digits: make a request to the next URL with the digits the caller entered.
• timeout: simulates a case where the server is waiting for input from the caller and it times out. In this case you skip the current action, and execute the next one. This is where the iterator comes in handy.

This is the complete code for the TwilioPhoneCall class:

This class implements everything you need to test your movie info IVR system. The class does not cover all of the verbs available in TwiML, but you now know how to add them yourself.

Writing Test Cases for the IVR System

You now have a test class that simulates a call from Twilio and test fixtures for theaters and movies. You can finally start to implement some test scenarios.

In the file test_showtimes_ivr.py, start by adding a fixture that provides an instance of TwilioPhoneCall for the show times IVR:

The fixture showtimes_phone_call creates a new TwilioPhoneCall with a start URL of your movie info IVR system. Unlike the other fixtures you created in conftest.py, this fixture is intended to be used only in this file, so we create it here.

Notice that to provide a client for the object, you injected another fixture provided called client. This fixture is also provided by the pytest-django library. If you have multiple IVR systems in the project, you can reuse the TwilioPhoneCall object for them as well.

Now write your first scenario:

To provide data to the test case, you injected it with the data fixtures you created before in conftest.py, and the showtimes_phone_call fixtures you just added:

• The test starts by instantiating a new call to your IVR system.
• You first check that the user is asked to choose a theater from the list of available theaters.
• Next, you choose "theater A" by entering its digits.
• Then, you check that the user is asked to choose a movie from the list of available movies.
• You choose "movie A" by entering its digits.
• You didn't create any show times in the database, so you check that system is telling the caller there are no upcoming shows.
• Finally, you make sure the call has ended.

This is your first test, elegant and straightforward.

Let's add another test to check that when upcoming show times are available, the system is listing them for the caller:

Like the previous test, you used the showtimes_phone_call fixture, and the theater and movie fixtures. To get to the showtimes selection you initiate a call and choose the first theater by entering its digits.

This time you want to check that show times are suggested to the caller, so you create a show time that starts soon. To search for upcoming show times, the view function queries the database for show times in the selected theater and movie, in the next 12 hours.

To get the current time, the function is using Django's timezone internally. To make sure the show time you created is found, you patch Django's timezone module and provide a date that should return results. Finally, you select the movie and check that the correct show time is provided to the caller.

The next scenario you can check is what happens when the caller doesn't press any digit during the theater selection phase:

This time, after initiating the call, you used the timeout function to jump over the Gather verb, and redirect instead. In this case you expect to be redirected to the same view again.

The next scenario you might want to test is what happens when the caller entered digits for a theater that does not exist:

After initiating the call, you entered the digits 10. These digits do not belong to any theater, and the test expects to be redirected to the same view again.

Similar scenarios can be added for the movie selection:

Just like for theaters, when the caller did not enter any digits, or if the caller entered digits that do not exist, you expect to be redirected back to the same menu.

Previously you added a scenario to check that a show time is found. Your IVR system also handles a case where multiple show times are found, so add a test for that as well:

You expect your IVR system to handle the multiple results.

Finally, you want to make sure the system in only listing upcoming show times and not shows that already started, or that are playing in more than 12 hours:

To set up the data for the test, you create two shows: one that already started and one that starts in 12 hours and one minute. In this case, you expect the system to say there are no upcoming shows.

Now, fire up your terminal and execute the tests:

Great! All the tests pass.

Conclusion

In this IVR testing tutorial you learned how to:

• Setup Pytest to test a Django project.
• Use the special fixtures provided by the django-pytest plugin such as rf, db, and client.
• How to use a RequestFactory to test Django views.
• How to create test fixtures for Django models in Pytest.
• How to mock external dependencies using unittest.mock.
• How to test a Twilio IVR system using Pytest.

You are now ready to test your own IVR system!

Haki is a software developer and a technical lead. Haki takes special interest in databases, web development, software design and performance tuning.

• Personal Website - https://hakibenita.com
• Twitter - https://twitter.com/be_haki
• Github - https://github.com/hakib