Encrypting and Storing Twilio Flex Recordings Off-site

Encrypting and Storing Twilio Flex Recordings Off-site

By default, Flex stores recordings in a Twilio-owned Amazon S3 bucket. Some customers may have security requirements that require recordings to be stored in a location which they control. This blog post will guide you through how to accomplish just that. Included is a sample project which allows your recordings to be stored in your own Amazon S3 bucket, as well as a Proxy Service which will play back the recordings from your Amazon S3 bucket.

Enabling Recordings

First things first. The easiest way to enable recordings in Flex is to enable WFO first. Enabling WFO requires you to select a Flex Billing plan. Once WFO is enabled, in the Twilio Console, go to Flex -> WFO or just click here. Make sure that Call Recording is turned on:

If you are still in the trial mode, you can enable Flex Recordings by using the Actions Framework. You will need to overwrite the AcceptTask Action and tell Flex to record the call. Here is a sample plugin which will enable recordings:

Actions.replaceAction('AcceptTask', (payload, original) => {
  payload.conferenceOptions.record = 'true';
  payload.conferenceOptions.recordingStatusCallback = 'https://your-twilio-runtime-domain/recording-status-callback';
  return original(payload);
});

Process Overview

This solution has two parts: storing the recording and playing back the recording. When the recording is completed, we will run a function which will copy the recording from the Twilio-owned Amazon S3 bucket to a different Amazon S3 bucket. In the sample project, we will be utilizing Twilio Functions for the callback function. Once the recording is copied, the task attribute will need to be updated with a proxy address which will be able to playback the recording over the HTTP protocol when called.

When recordings are played back in Twilio WFO, the proxy server will need to retrieve and decrypt the recording and stream it back to the WFO Player.

Creating a Plugin that will call a Twilio Function when recording is complete

The code sample above also sets the recordingStatusCallback URL. This tells Flex to call the given URL when the call recording file has been processed and is available for you. This will need to be set in the AcceptTask Action. You can add the code snippet above to either a new or an existing Flex plugin to call the Function which will copy the recording. Just keep in mind that each Action can only be replaced from a single plugin. If your plugin is already replacing the AcceptTask action, you will need to merge this snippet with your existing customizations.

Creating the Function

This project was created as a proof of concept. Twilio Functions are limited to 10 seconds of runtime and therefore they may not be suitable for production environments where large recordings are used. You will need to do your own scalability testing to see if Twilio Functions are suitable for your environment, but for the purposes of this post, they will work just fine.

The function relies on a couple of different node packages, more specifically aws-sdk, axios and s3-upload-stream. In the Twilio Console, go to Runtime -> Functions -> Configure or click here. In the Dependencies section, add each of the npm packages. When you're finished, your Dependencies should look like this:

While you are on the Configure page, it is also a good time to add the Environmental Variables that your Function will rely on:

• AWSaccessKeyId - the key used to access your S3 bucket
• AWSbucket - the name of your S3 bucket
• AWSsecretAccessKey - the secret for your S3 bucket

When you are finished, your Environmental Variables section should look like this:

Remember to save your configuration by clicking the `Save` button at the bottom of the page.

Next, create a new function here and assign it the path /recording-status-callback. Paste the following snippet of code and save it:

const axios = require('axios');
let AWS = require('aws-sdk');
const S3UploadStream = require('s3-upload-stream');

exports.handler = async function(context, event, callback) {
	Object.keys(event).forEach( thisEvent => console.log(`${thisEvent}: ${event[thisEvent]}`));

    // Set the region
    AWS.config.update({region: 'us-east-1'});
    AWS.config.update({ accessKeyId: context.AWSaccessKeyId, secretAccessKey: context.AWSsecretAccessKey });

    const s3Stream = S3UploadStream(new AWS.S3());

    // call S3 to retrieve upload file to specified bucket
    let upload = s3Stream.upload({Bucket: context.AWSbucket, Key: event.RecordingSid, ContentType: 'audio/x-wav'});

    const recordingUpload = await downloadRecording(event.RecordingUrl, upload);

    let client = context.getTwilioClient();
    let workspace = context.TWILIO_WORKSPACE_SID;

    let taskFilter = `conference.participants.worker == '${event.CallSid}'`;

    //search for the task based on the CallSid attribute
    client.taskrouter.workspaces(workspace)
      .tasks
      .list({evaluateTaskAttributes: taskFilter})
      .then(tasks => {

        let taskSid = tasks[0].sid;
        let attributes = {...JSON.parse(tasks[0].attributes)};
        attributes.conversations.segment_link = `https://your-proxy-address/?recordingSid=${event.RecordingSid}`;

        //update the segment_link
        client.taskrouter.workspaces(workspace)
          .tasks(taskSid)
          .update({
            attributes: JSON.stringify(attributes)
          })
          .then(task => {
            callback(null, null);
          })
          .catch(error => {
            console.log(error);
            callback(error);
          });

      })
      .catch(error => {
        console.log(error);
        callback(error);
      });
};

async function downloadRecording (url, upload) {

  const response = await axios({
    url,
    method: 'GET',
    responseType: 'stream'
  })

  response.data.pipe(upload);

  return new Promise((resolve, reject) => {
    upload.on('uploaded', resolve)
    upload.on('error', reject)
  })
}

That should take care of the first part of the process, copying the recording. Next, we will tackle playback of the recording using the Twilio WFO Player.

Playing back recordings using the Twilio WFO Player

When a recording is accessed in the Twilio WFO Player, a request is made to the location set on the Task attribute conversations.segment_link. The Function above set this attribute after the recording was copied. This URL needs to point to a Proxy Server which will retrieve and decrypt the recording, and return an audio stream over the HTTP protocol as a response. The Proxy Server also needs to support partial content. 

This project uses a Node Application which acts as the Proxy Server for the Recording Playback. Here is the source code for the Proxy Server:

const express     = require('express');
const app         = express();
const aws         = require('aws-sdk');
const downloader  = require('s3-download-stream');
require('dotenv').config();
require('log-timestamp');

aws.config.update({ region: 'us-east-1' });
aws.config.update({ accessKeyId: process.env.accessKeyId, secretAccessKey: process.env.secretAccessKey });

const s3 = new aws.S3({apiVersion: '2006-03-01'});

app.listen(8080, function() {
  console.log('[NodeJS] Application Listening on Port 8080');
});

app.get('/', function(req, res) {

    const recordingSid = req.query.recordingSid;

    console.log('Received playback request');
    console.log(recordingSid);

    let range = req.headers.range;
    console.log(range);

    let config = {
      client: s3,
      concurrency: 6,
      params: {
        Key: recordingSid,
        Bucket: process.env.bucket
      }
    }

    if (range !== undefined || range !== 'bytes=0-') {
      config.params.Range = range;
    }

    s3.headObject(config.params, (error, data) => {
      if (error) {
        console.log(error);
      }

      console.log(data);

      if (range !== undefined) {

        let contentRange = data.ContentRange;
        if (range === 'bytes=0-') {
          contentRange = `bytes 0-${data.ContentLength - 1}/${data.ContentLength}`;
          config.params.Range = `bytes=0-${data.ContentLength - 1}`;
        }

        res.status(206).header({
          'Accept-Ranges': data.AcceptRanges,
          'Content-Type': 'audio/x-wav',
          'Content-Length': data.ContentLength,
          'Content-Range': contentRange
        });
      } else {
        res.header({
          'Accept-Ranges': data.AcceptRanges,
          'Content-Type': 'audio/x-wav',
          'Content-Length': data.ContentLength
        });
      }

      downloader(config).pipe(res);
    })
});

Extending the Sample Project further

This project was created as a Proof of Concept. In order to use it in a production environment, it will likely need to be extended further. For instance, the original recording in the Twilio S3 bucket is never deleted. Please note, if you would like to delete the original recording, it is recommended that you allow the recording to stay for up to 2 hours after completion. This will allow certain Twilio WFO processes to access the recording. These processes are used for calculating the agent talk time, customer talk time and crosstalk time.

Not included in the project is Access Control for the Proxy Server. In the current form, it will respond to every request, regardless of origin.

The source code for this project is also available on Github.

Lehel Gyeresi is a part of the Expert Services team and works as a Contact Center Solutions Consultant. You can reach him at lgyeresi [at] twilio.com.