Enhancing Twilio HTTP Request Authentication Using PKCV and Enterprise/Security Editions

Enhancing Twilio HTTP Request Authentication Using PKCV and Enterprise/Security Editions

Public Key Client Validation (PKCV) is a security mechanism that leverages asymmetric cryptography to authenticate HTTP clients and validate their requests. As cyber threats evolve, Public Key Client Validation provides a robust alternative to using Basic Authentication alone, offering enhanced security for your Twilio app. At Twilio, customers using our Enterprise or Security Editions can leverage PKCV. In this article we will review what PKCV is made of, how it works, and how it actually happens.

Prerequisites

Before starting, be sure to have the following ready:

• A Twilio account - sign up here if you haven’t yet
• Security or Enterprise Editions
• A Main API Key/ Secret pair
• Account and Key management operations cannot be performed with Standard keys.
• Python 3.x
• Twilio Python helper library
• Full working code examples (optional)

Key terms

Implementing PKCV is a multi-step process, using several different technologies. Your exact implementation will depend on your organization’s needs. This guide aims to break down each concept into defined pieces that can be used to build an implementation of Public Key Client Validation from scratch.

But before writing any code, it is important to agree on some key terms. In the next few sections, we’ll define some terms you’ll want to be familiar with before implementing PKCV.

Asymmetric Cryptography

Asymmetric encryption, also known as public-key cryptography, is a type of encryption that uses a pair of keys for secure communication - a public key and a private key. The Private Key is owned by the issuer and is used to compute the signature. The Public Key is shared with Twilio, which we use to verify that the token was indeed signed with the corresponding private key, thus affirming the token's authenticity and integrity.

RSA

RSA is a cryptographic algorithm, originally introduced in 1977, that uses a pair of keys to encrypt and decrypt data. RSA stands for Rivest-Shamir-Adleman, after its inventors Ron Rivest, Adi Shamir, and Leonard Adleman.

To digitally sign a request with RSA, the sender encrypts a hash of the request with their private key. The recipient of the message can use the sender's public key to decrypt this hash. If it matches the hash of the received request, it confirms the request’s integrity and verifies the sender's identity.

Canonical Request

A Canonical Request is a standardized representation of an HTTP request. This representation ensures consistent formatting of the request for cryptographic signing and validation purposes. A Canonical Request will look something like this:

Auth Token

By default, all requests to Twilio require Basic Authentication. Your Account SID is the username and your Auth Token is the password.

Like this:

Each Account SID only has one Auth Token. Auth Tokens are not scoped to specific permissions, and can perform all account actions. Using Auth Tokens can be specifically challenging for organizations where shared secrets are not allowed. However, if compromised they can be rotated immediately via Console or the REST API.

API Keys

An Account can have multiple API Key/ Secret pairs, scoped for different users or purposes.

They are used the same way as the Account SID and Auth Token in Basic Authentication.

In the event an API secret becomes compromised or is no longer in use, it can be revoked to block unauthorized access without impacting other keys or other parts of your application. API Keys give more granular control for applications with more complex security needs.

JWT

From jwt.io:

"JSON Web Token (JWT) is an open standard (RFC 7519) that defines a compact and self-contained way for securely transmitting information between parties as a JSON object. This information can be verified and trusted because it is digitally signed. JWTs can be signed using a secret (with the HMAC algorithm) or a public/private key pair using RSA or ECDSA.

Although JWTs can be encrypted to also provide secrecy between parties, we will focus on signed tokens. Signed tokens can verify the integrity of the claims contained within it, while encrypted tokens hide those claims from other parties. When tokens are signed using public/private key pairs, the signature also certifies that only the party holding the private key is the one that signed it.”

( RFC 7519)

Public Key Client Validation

While Basic Authentication works great for some applications, it can introduce risks that some organizations are unwilling to accept.

Public key validation uses asymmetric encryption, allowing secure communication without the need to share a secret key.

Public key cryptography involves encrypting the request data, which protects sensitive information during transmission. Basic Authentication typically involves sending credentials in a header, which can potentially be intercepted.

A request to Twilio made with PKCV uses a JWT in the Twilio-Client-Validation header, like this:

We will review how to construct this JWT later, but for now just know it is a hashed version of a request that can be validated by Twilio.

PKCV also requires including timestamps in the signed messages, helping prevent replay attacks. Any tampering after the request is sent would invalidate the signature on the receiving end.

Let’s begin!

Public Key Client Validation is implemented using the following general steps:

• Create a new API Key/ Secret Pair
• Generate an RSA Key Pair
• Submit the RSA Public Key to Twilio
• Canonicalize and Hash the Request
• Generate the JWT
• Attach the JWT to the request header

For demonstration purposes, the first method of achieving these steps will be outlined from scratch in Python, without the use of Twilio libraries.

Then, the Twilio Python Helper Library will be used with a more straightforward implementation.

Steps 1-3 only need to be performed once per HTTP client. For example, if you have multiple users, you would create an API Key/ Secret Pair and an RSA Key Pair, and submit the Public key for each user. Then, each user can configure their HTTP client with this information when making requests to Twilio’s APIs.

Steps 4-6 relate to formatting and sending requests to Twilio, in a way where Twilio can validate the contents of the requests, and authorize the user or application making the request.

The full working code for both halves of the process can be found in my github repo.

• Steps 1-3: api_rsa_keys.py
• Steps 4-6: homemade_pkcv_requests.py

Method A - Homemade PKCV with Python

PKCV can be implemented from scratch, without using Twilio or other external libraries. This approach is beneficial for understanding the underlying process and to minimize dependencies in environments where reducing external libraries is required for security or compatibility reasons.

1. Create new API Key/ Secret Pair

Each HTTP client should have its own API Key/ Secret Pair, which can be obtained via Console, or using the API Keys Resource.

Be sure to note the secret for later use as it is only returned once in the initial response, and cannot be obtained later.

2. Generate an RSA Key Pair

A private key is generated with math. A public key is then derived from the private key. For demonstration purposes, the .pem files can be saved in the current working directory.

With Python, RSA key pairs can be generated with the cryptography package.

Cryptography is not a standard Python library, so an RSA key pair can also be generated using the command line if needed.

When prompted, enter a passphrase to encrypt the private key.

The private key is then used to create the public key:

3. Submit the Public Key to Twilio

RSA Key pairs, when shared with Twilio, are scoped to a specific API Key, enabling granular revocation and renewal.

Twilio’s CredentialPublicKey Resource can be hit with a POST containing the public key, and will return a Credential SID which is later used to ‘link’ the JWT to the associated Private Key.

For the purposes of curiosity (or necessity), the standard http client library can be used instead of the external requests package.

Additionally, Public Keys can also be uploaded and managed via Console.

Done! We have all the information needed to configure a new HTTP client to make requests using PKCV.

• Twilio API Key
• Twilio API Secret
• RSA Public Key
• RSA Private Key
• Credential SID from submitting the RSA Public Key to Twilio

Save this information as environment variables in your app to use in the following steps, and to make future requests. For example, if using virtualenv, add the following to the end of venv/bin/activate, and reactivate the virtual environment.

4. Canonicalize and Hash the request

A standardized representation of an HTTP request ensures consistent formatting for cryptographic signing and validation purposes. For this demonstration, a Canonical Request should look like this:

In Python, each element of the canonical request can be handled with its own function.

Let’s break this down, function by function.

Canonicalized HTTP Method

Force the method to be uppercase with .upper()

• Any leading or trailing whitespace is removed with .strip()

Canonicalized Resource Path

• urllib.parse.urlparse(path).path parses the URL string, and isolates the path
• urllib.parse.quote(normalized_path, safe='/-_.!=') isolates and encodes the path, ensuring that special characters are percent-encoded, while allowing specific characters defined in safe to remain unencoded.
• For Twilio PKCV, you must ensure the following:
  • A space should always be '%20'
  • An asterisk should always be '%2A'
  • A tilde (‘~’) should always be '%7E'
  • An empty string path should always be '/'

Given a URL like https://api.twilio.com/2010-04-01/$TWILIO_ACCOUNT_SID/Calls.json?Status=completed, this function will return /2010-04-01/$TWILIO_ACCOUNT_SID/Calls.

Canonicalized query string

• urllib.parse.parse_qsl(query_string, keep_blank_values=True) breaks down the query string into a list of key-value pairs, and ensures that parameters with empty values are retained.
• For each key-value pair (k, v), both k and v are URL-encoded using urllib.parse.quote, with safe='/-_.!=' ensuring all characters are encoded using the same considerations as above.
• sorted() sorts the list of encoded pairs alphabetically
• In the return statement, the sorted and encoded key value pairs are reconstructed into a query string with join()

This function would take a query string

And return:

Canonicalized Headers

• Lower-case and trim each header key with .lower() and .strip()
• Trim each header value and reduce blank spaces
• Sort header values that correspond to the same key
• Combine the formatted key/values - {key}:{values}\n
• .sort() is used to sort alphabetically if the original list is not needed
• Each header line is terminated with a \n for a blank line between the canonical-headers and the canonical-hashed-headers in the final canonical request

A Header object like this:

Would be converted to:

Canonicalized hashed headers

Some use cases may only require certain headers to be validated, but in this example all of them are hashed.

The hashed-headers are canonicalized by the following operations:

• Isolate each header
• Lowercase and trim
• Sort alphabetically
• Join with semicolon

Hashed and Encoded Request Body

Hash the request body using SHA-256 and hex-encode the resulting hash. (Classic move.)

If the request body is empty, omit hashing it.

This JSON request body…

…becomes

5. Generate the JWT

A JSON Web Token (JWT) is a URL-safe string, used to securely transmit information between two parties, consisting of three parts:

1. Header - specifies the token type and the algorithm used for signing, such as HMAC SHA256 or RSA.
2. Payload - contains the data or information about the user or session, organized in key-value pairs.
3. Signature - ensures the token's integrity and authenticity.

• The request_hash variable is the encoded hash of the entire canonical request constructed in step 4. This is included in the JWT so Twilio can compare its computed request hash for validation.
• The final jwt_token is generated and signed with the Private Key associated with the Public Key and Credential SID from Steps 1-3.

• The resulting JWT looks like this:

6. Attach the JWT to the request header

Before we send the request, ensure PKCV is correctly configured on your Account. Once you have Security or Enterprise Editions of your Twilio account, the PKCV setting can be found under General Settings.

Once this setting is enabled, all requests using just Basic Authentication will fail.

With this setting disabled, PKCV can still be used, and requests made using Basic authentication will also be allowed.

Now what we have all been waiting for - let’s use PKCV to list Call Logs!

• The JWT is included in the Twilio-Client-Validation header of the request
• A GET request is made to https://api.twilio.com/2010-04-01/Accounts/{ACCOUNT_SID}/Calls.json using the defined querystring.
• The Authorization header uses a Base64 encoded string where the username is the API Key generated in step 1. This same API Key is used to construct the JWT to match against request credentials.

The results aren’t spectacular - just the response as expected. A list of calls.

But the implication is much more clear: you can now ensure a request comes from a sender who is in control of the private key and that the message has not been modified in transit. 👍

Method B - Help from Twilio SDK

The Twilio Server-side SDKs fully support PKCV implementation via the built-in Validation Client.

A request made using Basic Authentication with the Twilio Python Helper Library looks like this:

With a few small changes, we can use the Validation Client instead, which is instantiated using the keys generated in Steps 1-3 above.

The example included in the twilio-python source code “just works”.

Conclusion

By securing communications and verifying client authenticity, PKCV plays a crucial role in safeguarding systems against unauthorized access. We encourage you to integrate these practices within your infrastructure to bolster security. For further exploration, consider diving into some of the following additional resources:

• https://www.twilio.com/docs/usage/api
• https://docs.openssl.org/master/man7/ossl-guide-libcrypto-introduction/
• https://jwt.io/introduction
• https://en.wikipedia.org/wiki/RSA_(cryptosystem)
• https://archive.org/details/arxiv-cs9903001/mode/2up

Matt Coser is a Senior Field Security Engineer at Twilio. His focus is on telecom security, and empowering Twilio’s customers to build safely. Hit him up on linkedin to connect and discuss more.