Voice Biometric Authentication With Twilio

June 04, 2014
Written by

Twilio Bug Logo

What if you could use your voice as a password – saying a phrase that only works if you said it?  That would be more secure than even a complex password because a person’s voice is harder to duplicate than any password.

2494667_orig

Well you can, using voice biometric authentication.  Voice biometric authentication techniques – using digital sampling techniques to recognize a  speaker’s voice – are gaining popularity in a number of industries.   The organizations that have typically implemented voice authentication technology are large ones such as banks and government departments. Organizations that have heightened security concerns and the funds to implement rather complex technology of voice authentication.

But the field is changing – and now voice authentication is accessible via APIs, the way that Twilio has made communications available via API.

What Is It?

I wanted to set up a proof of concept with voice bio authentication from the first time heard about  it, but I struggled to find a API that was useable without a large overhead.  Luckily Noel Grover from https://voiceit.io reached out to us, and I was able to set up a Twilio based voice authentication demo in no time.

More information on VoiceIt:  https://showcase.twilio.com/partner-listing/a8E1W00000097U6UAI 

 

A common use case is two factor authentication.  For example, during a bank transfer or application login the flow would be similar to existing Twilio two factor authentication, but instead of entering a code via SMS or IVR, a voice must be recognized.

You can try it yourself by calling:  +1 612-400-7423

Basics:

One needs to become familiar with some key concepts I struggled with a bit initially, but are  pretty standard in the field.

1.  Enrollment – this is the process of creating a voice print. Typically, a  user will be asked to repeat a phrase multiple times, and the recording of these utterances will be compared to future authentication attempts.

2. Authentication – This is the comparison of a users phrase to recorded enrollment.  These algorithms are tunable, a company and dictate how strict or loose the matching algorithm should be.   And VoiceIT has the ability to authenticate via a REST api and wav file, which is particularly convenient with Twilio.

3. Users – Self explanatory, to use VoiceIt you need a user created.  This user is A tip for using Twilio with VoiceIt – instead of making a user create a username, you likely will want to use their phone number, and put it in the required email format for them.

4. These concepts and more are covered in the VoiceIt API.

The Code

In this demo, I’m going to use a system that when called, recognizes the number, and if the user exists, allows them to log in via their passphrase.  If their voice matches the pre-recorded phrase, the demo simply says “thanks your voice has been recognized”.  If callers aren’t recognized, they can enroll as a new user.

To REST based API of VoiceIT made integration with Twilio super easy.  Here is an example in Node, which made making a request from one service to another very easy.

We will start – with some basic Node requirements, and use Express so we can receive web requests from Twilio.

 

 

// # Twilio & VoiceIt Demo

// This application demonstrates how Twilio integrates with the VoiceIt
// Voiceprint Portal, allowing for biometric authentication with your voice
// applications.

// Standard Operating Procedure
// -------------------------------

var twilio     = require('twilio'),
    SHA256     = require('crypto-js/sha256'),
    bodyParser = require('body-parser'),
    express    = require('express'),
    request    = require('request');

// Prepare the Express server and body parsing middleware.
var port = process.env.PORT || 1337;
var app = express();
app.use(bodyParser());


 

 

You will need a Developer ID from VoiceIt – you can register directly on their website to set that up.  In this code sample the DeveloperId is stored in an environment variable called VOICEIT_DEV_ID.

var VOICEIT_DEV_ID = process.env.VOICEIT_DEV_ID;

 

Next, we write a helper function, callerCredentials, that will be used later, every time we receive a new call to the system.

 

/ Stubbing VoiceIt Profiles with Phone Numbers
// --------------------------------------------
// VoiceIt authentication requires an email address, so we will make a fake
// one for this caller using the response body posted from Twilio.
var callerCredentials = function(body) {
   // Twilio's `body.From` is the caller's phone number, so let's use it as
   // identifier in the VoiceIt profile. It also means, the authentication is
   // bound only to this phone number.
   return  {
     number   : body.From,
     email    : body.From + '@twiliobioauth.example.com',
     password : SHA256(body.From)
   };
};

 

Next,  we set up the section of the code that will accept requests from Twilio when an incoming call comes in.  Notice that the method is /incoming_call. If your server is running on  http://yourtwilioserver.yourcompany.com/, you would configure your Twilio phone number voice URL to = http://yourtwilioserver.yourcompany.com/incoming_call.

That will send inbound calls to this section of the code.  From there, a few things will happen:

1. The incoming phone number from the caller will be used to create user credentials with the callerCredentials function. Note, the only unique part of the user record in this demo is the phone number they are calling from.

2. And api call is made against VoiceIt using the to GET information about user. If they don’t exist, create the user in VoiceIt.

3. If the VoiceIt user does not exist, start off the process to /enroll the user.

4.  Play some feedback to the caller using TwiML

// Accept Incoming Calls
// ---------------------
// We need to accept incoming calls from Twilio. The fully-qualified URL should
// be added to your Twilio account and publicly available.
app.post('/incoming_call', function(req, res) {
  var caller  = callerCredentials(req.body);
  var twiml   = new twilio.TwimlResponse();
  // Prepare options for the VoiceIt `GET /sivservice/api/users` API request.
  var options = {
    url: 'https://siv.voiceprintportal.com/sivservice/api/users',
    headers: {
      'VsitEmail'       : caller.email,
      'VsitPassword'    : caller.password,
      'VsitDeveloperId' : VOICEIT_DEV_ID
    }
  };

  request(options, function (error, response,  body) {
    // When VoiceIt responds with at `200`, we know the user's account profile
    // exists in the VoiceIt system.
    if (!error && response.statusCode == 200) {
      var voiceIt = JSON.parse(body);

      // Greet the caller when their account profile is recognized by the VoiceIt API.
      twiml.say(
        'You have called Voice Authentication. Your phone number has been recognized.'
      );
      // Let's provide the caller with an opportunity to enroll by typing `1` on
      // their phone's keypad.
      twiml.gather({
        action    : '/enroll_or_authenticate',
        numDigits : 1,
        timeout   : 3
      }, function () {
        this.say(
          'You can now log in, or press 1 now to enroll for the first time.'
        );
      });
      twiml.redirect('/enroll_or_authenticate?digits=TIMEOUT');

      res.send(twiml.toString());
    } else {
      switch(response.statusCode) {
        // Create a VoiceIt user when the HTTP status is `412 Precondition Failed`.
        case 412:
          // Prepare options for the VoiceIt `POST /sivservice/api/users` API request.
          var options = {
            url: 'https://siv.voiceprintportal.com/sivservice/api/users',
            headers: {
              'VsitDeveloperId' : VOICEIT_DEV_ID,
              'VsitEmail'       : caller.email,
              'VsitFirstName'   : 'First' + caller.number,
              'VsitLastName'    : 'Last' + caller.number,
              'VsitPassword'    : caller.password,
              'VsitPhone1'      : caller.number
            }
          };

          request.post(options, function (error, response,  body) {
            if (!error && response.statusCode == 200) {
              var voiceIt = JSON.parse(body);
              console.log(voiceIt);
            } else {
              console.log(response.statusCode);
              console.log(body);
            }
          });

          twiml.say(
            'Welcome to the Voice Authentication system. You are a new user, ' +
            'you will now be enrolled.'
          );
          // Then we'll want to send them immediately to enrollment.
          twiml.redirect({ digits: '1' }, '/enroll');

          res.send(twiml.toString());
          break;
        default:
          new Error('An unhandled error occured');
      }
    }
  });
});

 

The experience will vary depending on whether the user is enrolling for the first time, or they are authenticating.

The /enroll function, if selected, is used to create a voice  print for the user. They must say the phrase 3 times to enroll.  This enrollment will be saved on VoiceIt, and be compared with future /authentications.

// Routing Enrollments & Authentication
// ------------------------------------
// We need a route to help determine what the caller intends to do.
app.post('/enroll_or_authenticate', function(req, res) {
  var digits = req.body.digits;
  var twiml  = new twilio.TwimlResponse();

  // When the caller asked to enroll by pressing `1`, provide friendly
  // instructions, otherwise, we always assume their intent is to authenticate.
  if (digits == 1) {
    twiml.say(
      'You have chosen to create a new account with your voice. You will be ' +
      'asked to say a phrase 3 times, then you will be able to log in with that phrase.'
    );
    twiml.redirect('/enroll');
  } else {
    twiml.redirect('/authenticate');
  }

  res.send(twiml.toString());
});

app.post('/process_enrollment', function(req, res) {
  var caller       = callerCredentials(req.body);
  var enrollCount  = req.query.enrollCount;
  var recordingURL = req.body.RecordingUrl + ".wav";
  // Prepare options for the VoiceIt `POST /sivservice/api/enrollments/bywavurl API request.
  var options      = {
    url: 'https://siv.voiceprintportal.com/sivservice/api/enrollments/bywavurl',
    headers: {
      'VsitDeveloperId' : VOICEIT_DEV_ID,
      'VsitEmail'       : caller.email,
      'VsitPassword'    : caller.password,
      'VsitwavURL'      : recordingURL
    }
  };

  request.post(options, function (error, response, body) {
    var twiml = new twilio.TwimlResponse();

    if (!error && response.statusCode == 200) {
      var voiceIt = JSON.parse(body);

      if (voiceIt.Result == 'Success') {
        enrollCount++;
        // VoiceIt requires at least 3 successful enrollments.
        if (enrollCount > 2) {
          twiml.say(
            'Thank you, recording recieved. You are now enrolled. You can log in.'
          );
          twiml.redirect('/authenticate');
        } else {
          twiml.say(
            'Thank you, recording recieved. You will now be asked to record your phrase again.'
          );
          twiml.redirect('/enroll?enrollCount=' + enrollCount);
        }
      } else {
        twiml.say('Sorry, your recording did not stick. Please try again.');
        twiml.redirect('/enroll?enrollCount=' + enrollCount);
      }
    } else {
      twiml.say('Sorry, your recording did not stick. Please try again');
      twiml.redirect('/enroll?enrollCount=' + enrollCount);
    }

    res.send(twiml.toString());
  });
});

 

The /authenticate route is the heart of the actual voice authentication system.  When a user reaches this section, they will speak their Twilio phrase, and the Twilio recording will be sent to VoiceIt to match their enrollment.

If it matches, they get a simple message “Great Success!”, and some JSON returned from VoiceIt will be played, stating the authentication match percentage. In a real world application, this would then go on to the next function, for example allowing you to access your bank balance, etc.

And that’s it! The machine is now recognizing your voice.  I, for one, welcome our new robot overlords.

Code also available on GitHub:

https://github.com/voiceittech/voiceit-ivr-demo

app.post('/authenticate', function(req, res) {
  var twiml = new twilio.TwimlResponse();

  twiml.say('Please say the following phrase to authenticate.');
  twiml.pause(1);
  twiml.say('Never forget tomorrow is a new day.');
  // We neeed to record a `.wav` file. This will be sent to VoiceIt for authentication.
  twiml.record({
    action    : '/process_authentication',
    maxLength : '5',
    trim      : 'do-not-trim',
  });

  res.send(twiml.toString());
});

app.post('/process_authentication', function(req, res) {
  var caller       = callerCredentials(req.body);
  var recordingURL = req.body.RecordingUrl + '.wav';
  var options      = {
    url: 'https://siv.voiceprintportal.com/sivservice/api/authentications/bywavurl',
    headers: {
      'VsitAccuracy'              : 5,
      'VsitAccuracyPassIncrement' : 2,
      'VsitAccuracyPasses'        : 4,
      'VsitConfidence'            : 87,
      'VsitDeveloperId'           : VOICEIT_DEV_ID,
      'VsitEmail'                 : caller.email,
      'VsitPassword'              : caller.password,
      'VsitwavURL'                : recordingURL
    }
  };

  request.post(options, function(error, response, body) {
    var twiml = new twilio.TwimlResponse();

    if (!error && response.statusCode == 200) {
      var voiceIt = JSON.parse(body);
      console.log(voiceIt);

      switch(voiceIt.Result) {
        case 'Authentication failed.':
          twiml.say('Your authentication did not pass. Please try again.');
          twiml.redirect('/authenticate');
          break;
        default:
          twiml.say("Great Success!");
          twiml.say(voiceIt.Result);
      }
    } else {
      twiml.say('API Error. Your authentication did not pass. Please try again.');
      twiml.redirect('/authenticate');

      new Error(response.statusCode, body);
    }

    res.send(twiml.toString());
  });
});

app.listen(port);
console.log('Running bioauthentication on port ' + port);