Automated Survey with Python and Django

Time to read: 5 minutes

January 10, 2017

Written by

Andrew T. Baker

Twilion

Reviewed by

Paul Kamp

Twilion

Jose Oliveros

Contributor

Kat King

Twilion

Orlando Hidalgo

Contributor

Samuel Mendes

Contributor

Brianna DelValle

Twilion

Automated Survey with Python and Django

Have you ever wondered how to build an automated survey for Voice or SMS for your Python Django web app? This tutorial shows how it's done using the Twilio Python SDK. We promise a quick build with our Python Helper Library and your Python skills - but handling the eventual results is up to you.

Here's How Our Survey Will Work:

The end user calls or sends an SMS to the survey phone number.
Twilio receives the call or text and makes an HTTP request to your application asking for instructions on how to respond.
Your web application instructs Twilio (using TwiML) to gather or record the user input over the phone, and prompt for text input with message if you are using SMS.
After each question is answered, Twilio makes another request to your server with the user's input. That input is stored on the application's database.
After storing the answer, your Django server will instruct Twilio to redirect the user to the next question or finish the survey.

Creating a Survey

In order to perform an automated survey we first need to have some questions configured. For your convenience, this sample application's repository already includes one survey that can be loaded into the database. If the database is configured correctly, this survey will be loaded into the database when the app starts.

You can modify the questions from the survey by editing the bear_survey.json file located in the root of the repository and re-running the app's load_survey command:

$ python manage.py load_survey your_survey.json

Editor: this is a migrated tutorial. Clone the original code from https://github.com/TwilioDevEd/automated-survey-django/

import json
from automated_survey.models import Survey, Question


class SurveyLoader(object):

    def __init__(self, survey_content):
        self.survey = json.loads(survey_content)

    def load_survey(self):
        new_survey = Survey(title=self.survey['title'])
        questions = [Question(body=question['body'],
                              kind=question['kind'])
                     for question in self.survey['questions']]
        new_survey.save()
        new_survey.question_set.add(*questions)

We want users to take our survey, so we still need to implement the handler for SMS and for calls.

The Interview Loop

The user can answer a question for your survey over the phone by using either their phone's keypad or voice. After each interaction Twilio will make an HTTP request to your web application with either the string of keys the user pressed or a URL to a recording of their voice input.

For SMS surveys the user will answer questions by simply replying with another SMS to the Twilio number that sent the question.

It's up to the application to process, store, and respond to the user's input.

Let's dive into this flow to see how it actually works.

Configure a Twilio Number

To initiate the interview process, we need to configure one of our Twilio numbers to send our web application an HTTP request when we get an incoming call or text.

Click on one of your numbers and configure Voice and Message URLs that point to your server. In our code, the route is /automated-survey/first-survey/ for both Voice and Message. Ifyou don't already have a server set up to run your application, ngrok is a helpful tool for testing your webhooks locally.

from automated_survey.models import Survey, Question
from django.http import HttpResponse, HttpResponseRedirect
from django.core.urlresolvers import reverse
from django.shortcuts import render_to_response
from django.views.decorators.http import require_POST, require_GET
from django.views.decorators.csrf import csrf_exempt
from twilio.twiml.voice_response import VoiceResponse
from twilio.twiml.messaging_response import MessagingResponse


@require_GET
def show_survey_results(request, survey_id):
    survey = Survey.objects.get(id=survey_id)
    responses_to_render = [response.as_dict() for response in survey.responses]

    template_context = {
        'responses': responses_to_render,
        'survey_title': survey.title
    }

    return render_to_response('results.html', context=template_context)


@csrf_exempt
def show_survey(request, survey_id):
    survey = Survey.objects.get(id=survey_id)
    first_question = survey.first_question

    first_question_ids = {
        'survey_id': survey.id,
        'question_id': first_question.id
    }

    first_question_url = reverse('question', kwargs=first_question_ids)

    welcome = 'Hello and thank you for taking the %s survey' % survey.title
    if request.is_sms:
        twiml_response = MessagingResponse()
        twiml_response.message(welcome)
        twiml_response.redirect(first_question_url, method='GET')
    else:
        twiml_response = VoiceResponse()
        twiml_response.say(welcome)
        twiml_response.redirect(first_question_url, method='GET')

    return HttpResponse(twiml_response, content_type='application/xml')


@require_POST
def redirects_twilio_request_to_proper_endpoint(request):
    answering_question = request.session.get('answering_question_id')
    if not answering_question:
        first_survey = Survey.objects.first()
        redirect_url = reverse('survey',
                               kwargs={'survey_id': first_survey.id})
    else:
        question = Question.objects.get(id=answering_question)
        redirect_url = reverse('save_response',
                               kwargs={'survey_id': question.survey.id,
                                       'question_id': question.id})
    return HttpResponseRedirect(redirect_url)


@require_GET
def redirect_to_first_results(request):
    first_survey = Survey.objects.first()
    results_for_first_survey = reverse(
        'survey_results', kwargs={
            'survey_id': first_survey.id})
    return HttpResponseRedirect(results_for_first_survey)

Up next, let's see how Webhooks can help us with the survey flow.

Respond to a Twilio Request with Python and Django

Right after receiving a call or an SMS, Twilio will send a request to the URL specified on the number's configuration.

The first endpoint will check if the user has already answered a question or not. If not, it means this is the first request, so we need to welcome the user and redirect them to the first question. This is done in show_survey which will use the Say verb for calls and the Message verb for SMS.

The is_sms property comes from some simple middleware that checks if request has Voice or Message data.

This endpoint will also include a Redirect to the question's endpoint so that the user can continue the survey.

from automated_survey.models import Survey, Question
from django.http import HttpResponse, HttpResponseRedirect
from django.core.urlresolvers import reverse
from django.shortcuts import render_to_response
from django.views.decorators.http import require_POST, require_GET
from django.views.decorators.csrf import csrf_exempt
from twilio.twiml.voice_response import VoiceResponse
from twilio.twiml.messaging_response import MessagingResponse


@require_GET
def show_survey_results(request, survey_id):
    survey = Survey.objects.get(id=survey_id)
    responses_to_render = [response.as_dict() for response in survey.responses]

    template_context = {
        'responses': responses_to_render,
        'survey_title': survey.title
    }

    return render_to_response('results.html', context=template_context)


@csrf_exempt
def show_survey(request, survey_id):
    survey = Survey.objects.get(id=survey_id)
    first_question = survey.first_question

    first_question_ids = {
        'survey_id': survey.id,
        'question_id': first_question.id
    }

    first_question_url = reverse('question', kwargs=first_question_ids)

    welcome = 'Hello and thank you for taking the %s survey' % survey.title
    if request.is_sms:
        twiml_response = MessagingResponse()
        twiml_response.message(welcome)
        twiml_response.redirect(first_question_url, method='GET')
    else:
        twiml_response = VoiceResponse()
        twiml_response.say(welcome)
        twiml_response.redirect(first_question_url, method='GET')

    return HttpResponse(twiml_response, content_type='application/xml')


@require_POST
def redirects_twilio_request_to_proper_endpoint(request):
    answering_question = request.session.get('answering_question_id')
    if not answering_question:
        first_survey = Survey.objects.first()
        redirect_url = reverse('survey',
                               kwargs={'survey_id': first_survey.id})
    else:
        question = Question.objects.get(id=answering_question)
        redirect_url = reverse('save_response',
                               kwargs={'survey_id': question.survey.id,
                                       'question_id': question.id})
    return HttpResponseRedirect(redirect_url)


@require_GET
def redirect_to_first_results(request):
    first_survey = Survey.objects.first()
    results_for_first_survey = reverse(
        'survey_results', kwargs={
            'survey_id': first_survey.id})
    return HttpResponseRedirect(results_for_first_survey)

Lets see how we can keep track of the user's survey progress.

Question Controller

This endpoint will check if the survey is taken via SMS or a Call and build the next survey question as a TwiML response. Each type of question and interaction (Call/SMS) will produce different instructions on how to proceed. For instance, we can record voice or gather a key press during a call, but we can't do the same for text messages.

When the user is interacting with us over SMS we don't have something like an ongoing call session with a well defined state. It becomes harder to know if an SMS is answering question 2 or 20, since all requests will be sent to our /automated-survey/first-survey/ endpoint. To solve that, we can use Twilio Cookies to keep track of which question is being answered at a given moment. This is done by setting an answering_question_id session key and leaving cookie management to Django.

from django.core.urlresolvers import reverse
from twilio.twiml.messaging_response import MessagingResponse
from twilio.twiml.voice_response import VoiceResponse
from django.http import HttpResponse

from automated_survey.models import Question
from django.views.decorators.http import require_GET


@require_GET
def show_question(request, survey_id, question_id):
    question = Question.objects.get(id=question_id)
    if request.is_sms:
        twiml = sms_question(question)
    else:
        twiml = voice_question(question)

    request.session['answering_question_id'] = question.id
    return HttpResponse(twiml, content_type='application/xml')


def sms_question(question):
    twiml_response = MessagingResponse()

    twiml_response.message(question.body)
    twiml_response.message(SMS_INSTRUCTIONS[question.kind])

    return twiml_response

SMS_INSTRUCTIONS = {
    Question.TEXT: 'Please type your answer',
    Question.YES_NO: 'Please type 1 for yes and 0 for no',
    Question.NUMERIC: 'Please type a number between 1 and 10'
}


def voice_question(question):
    twiml_response = VoiceResponse()

    twiml_response.say(question.body)
    twiml_response.say(VOICE_INSTRUCTIONS[question.kind])

    action = save_response_url(question)
    if question.kind == Question.TEXT:
        twiml_response.record(
            action=action,
            method='POST',
            max_length=6,
            transcribe=True,
            transcribe_callback=action
        )
    else:
        twiml_response.gather(action=action, method='POST')
    return twiml_response

VOICE_INSTRUCTIONS = {
    Question.TEXT: 'Please record your answer after the beep and then hit the pound sign',
    Question.YES_NO: 'Please press the one key for yes and the zero key for no and then hit the pound sign',
    Question.NUMERIC: 'Please press a number between 1 and 10 and then hit the pound sign'
}


def save_response_url(question):
    return reverse('save_response',
                   kwargs={'survey_id': question.survey.id,
                           'question_id': question.id})

Now let's see how the response is built.

Building Our TwiML Verbs

If the question is of a "numeric" or "yes-no" nature then we use the <Gather> verb. However, if we expect the user to record a more robust answer, we use the <Record> verb. Both verbs take an action attribute and a method attribute.

Twilio will use both attributes to define our response's endpoint to be used as a callback. This endpoint will be responsible for receiving and storing the caller's answer.

During the Record verb creation, we can also ask Twilio for a Transcription. Twilio will process the recording and extraction of useful text, making a request to our response endpoint when this transcription is done.

from django.core.urlresolvers import reverse
from twilio.twiml.messaging_response import MessagingResponse
from twilio.twiml.voice_response import VoiceResponse
from django.http import HttpResponse

from automated_survey.models import Question
from django.views.decorators.http import require_GET


@require_GET
def show_question(request, survey_id, question_id):
    question = Question.objects.get(id=question_id)
    if request.is_sms:
        twiml = sms_question(question)
    else:
        twiml = voice_question(question)

    request.session['answering_question_id'] = question.id
    return HttpResponse(twiml, content_type='application/xml')


def sms_question(question):
    twiml_response = MessagingResponse()

    twiml_response.message(question.body)
    twiml_response.message(SMS_INSTRUCTIONS[question.kind])

    return twiml_response

SMS_INSTRUCTIONS = {
    Question.TEXT: 'Please type your answer',
    Question.YES_NO: 'Please type 1 for yes and 0 for no',
    Question.NUMERIC: 'Please type a number between 1 and 10'
}


def voice_question(question):
    twiml_response = VoiceResponse()

    twiml_response.say(question.body)
    twiml_response.say(VOICE_INSTRUCTIONS[question.kind])

    action = save_response_url(question)
    if question.kind == Question.TEXT:
        twiml_response.record(
            action=action,
            method='POST',
            max_length=6,
            transcribe=True,
            transcribe_callback=action
        )
    else:
        twiml_response.gather(action=action, method='POST')
    return twiml_response

VOICE_INSTRUCTIONS = {
    Question.TEXT: 'Please record your answer after the beep and then hit the pound sign',
    Question.YES_NO: 'Please press the one key for yes and the zero key for no and then hit the pound sign',
    Question.NUMERIC: 'Please press a number between 1 and 10 and then hit the pound sign'
}


def save_response_url(question):
    return reverse('save_response',
                   kwargs={'survey_id': question.survey.id,
                           'question_id': question.id})

Now let's learn how to handle our user's response.

Handling responses.

After the user has finished submitting their answers, Twilio sends a request telling us what happened and asking for further instructions.

At this point, we need to recover data from Twilio's request parameters (_extract_request_body does this) and store that data using the save_response_from_request function.

Recovered parameters vary according to what we asked in our questions:

Body contains the text message from an answer sent over SMS.
Digits contains the keys pressed for a numeric question.
RecodingUrl contains the URL for listening to a recorded message.
TranscriptionText contains the text of a recording transcription.

from twilio.twiml.messaging_response import MessagingResponse
from twilio.twiml.voice_response import VoiceResponse

from django.http import HttpResponse
from django.core.urlresolvers import reverse
from django.views.decorators.http import require_POST

from automated_survey.models import QuestionResponse, Question


@require_POST
def save_response(request, survey_id, question_id):
    question = Question.objects.get(id=question_id)

    save_response_from_request(request, question)

    next_question = question.next()
    if not next_question:
        return goodbye(request)
    else:
        return next_question_redirect(next_question.id, survey_id)


def next_question_redirect(question_id, survey_id):
    parameters = {'survey_id': survey_id, 'question_id': question_id}
    question_url = reverse('question', kwargs=parameters)

    twiml_response = MessagingResponse()
    twiml_response.redirect(url=question_url, method='GET')
    return HttpResponse(twiml_response)


def goodbye(request):
    goodbye_messages = ['That was the last question',
                        'Thank you for taking this survey',
                        'Good-bye']
    if request.is_sms:
        response = MessagingResponse()
        [response.message(message) for message in goodbye_messages]
    else:
        response = VoiceResponse()
        [response.say(message) for message in goodbye_messages]
        response.hangup()

    return HttpResponse(response)


def save_response_from_request(request, question):
    session_id = request.POST['MessageSid' if request.is_sms else 'CallSid']
    request_body = _extract_request_body(request, question.kind)
    phone_number = request.POST['From']

    response = QuestionResponse.objects.filter(question_id=question.id,
                                               call_sid=session_id).first()

    if not response:
        QuestionResponse(call_sid=session_id,
                         phone_number=phone_number,
                         response=request_body,
                         question=question).save()
    else:
        response.response = request_body
        response.save()


def _extract_request_body(request, question_kind):
    Question.validate_kind(question_kind)

    if request.is_sms:
        key = 'Body'
    elif question_kind in [Question.YES_NO, Question.NUMERIC]:
        key = 'Digits'
    elif 'TranscriptionText' in request.POST:
        key = 'TranscriptionText'
    else:
        key = 'RecordingUrl'

    return request.POST.get(key)

Finally, we redirect to our Question controller, which will ask the next question in the loop. Our next_question_redirect function will handle this for us until the loop is finished.

Display the Survey Results

You probably don't want your survey results to just sit buried in your database. For this show_survey_results route we simply query the database for survey answers using the Django ORM and then display the information within a template. We display a panel for every question in the survey, and inside each panel we list the responses from the different calls or messages.

You can access this page in the application's root route.

from automated_survey.models import Survey, Question
from django.http import HttpResponse, HttpResponseRedirect
from django.core.urlresolvers import reverse
from django.shortcuts import render_to_response
from django.views.decorators.http import require_POST, require_GET
from django.views.decorators.csrf import csrf_exempt
from twilio.twiml.voice_response import VoiceResponse
from twilio.twiml.messaging_response import MessagingResponse


@require_GET
def show_survey_results(request, survey_id):
    survey = Survey.objects.get(id=survey_id)
    responses_to_render = [response.as_dict() for response in survey.responses]

    template_context = {
        'responses': responses_to_render,
        'survey_title': survey.title
    }

    return render_to_response('results.html', context=template_context)


@csrf_exempt
def show_survey(request, survey_id):
    survey = Survey.objects.get(id=survey_id)
    first_question = survey.first_question

    first_question_ids = {
        'survey_id': survey.id,
        'question_id': first_question.id
    }

    first_question_url = reverse('question', kwargs=first_question_ids)

    welcome = 'Hello and thank you for taking the %s survey' % survey.title
    if request.is_sms:
        twiml_response = MessagingResponse()
        twiml_response.message(welcome)
        twiml_response.redirect(first_question_url, method='GET')
    else:
        twiml_response = VoiceResponse()
        twiml_response.say(welcome)
        twiml_response.redirect(first_question_url, method='GET')

    return HttpResponse(twiml_response, content_type='application/xml')


@require_POST
def redirects_twilio_request_to_proper_endpoint(request):
    answering_question = request.session.get('answering_question_id')
    if not answering_question:
        first_survey = Survey.objects.first()
        redirect_url = reverse('survey',
                               kwargs={'survey_id': first_survey.id})
    else:
        question = Question.objects.get(id=answering_question)
        redirect_url = reverse('save_response',
                               kwargs={'survey_id': question.survey.id,
                                       'question_id': question.id})
    return HttpResponseRedirect(redirect_url)


@require_GET
def redirect_to_first_results(request):
    first_survey = Survey.objects.first()
    results_for_first_survey = reverse(
        'survey_results', kwargs={
            'survey_id': first_survey.id})
    return HttpResponseRedirect(results_for_first_survey)

Congratulations! Now you can send automated surveys to your users with the power of Twilio's API, the Twilio Python Helper Library, and Django.

If you have configured one of your Twilio numbers with the application built in this tutorial you should be able to take the survey yourself and see the results under the root route of the application.

Where to Next?

We hope you found this sample application useful. If you're a Python developer working with Twilio, you might enjoy these other tutorials:

Appointment Reminders

Automate the process of reaching out to your customers in advance of an upcoming appointment.

Click to Call

Click-to-call enables your company to convert web traffic into phone calls with the click of a button.

Did this Help?

Thanks for checking this tutorial out! If you have any feedback to share with us, we'd love to hear it. Reach out to us on Twitter and let us know what you build!

Related Resources

Twilio Docs

From APIs to SDKs to sample apps

API reference documentation, SDKs, helper libraries, quickstarts, and tutorials for your language and platform.

Resource Center

The latest ebooks, industry reports, and webinars

Learn from customer engagement experts to improve your own communication.

Ahoy

Twilio's developer community hub

Best practices, code samples, and inspiration to build communications and digital engagement experiences.

Automated Survey with Python and Django

Here's How Our Survey Will Work:

Did this Help?

Related Posts

Related Resources

From APIs to SDKs to sample apps

The latest ebooks, industry reports, and webinars

Twilio's developer community hub