How to Manage Application Secrets With PHP Using Vault

Time to read: 10 minutes

February 08, 2022
Written by
Matthew Setter
Twilion
Reviewed by
Niels Swimberghe
Twilion

How to Manage Application Secrets With PHP Using Vault

For far too many years, PHP developers stored application credentials and secrets, such as usernames, passwords, and API keys, alongside their code.

While extremely convenient, this practice was a security nightmare just waiting to happen; if someone could access an application’s source code, they had access to all of its sensitive data too.

Nowadays, this practice is nowhere near as common as it once was. Rather, it's now incredibly common to store credentials separately from code in dotenv files (.env) which makes them available as environment variables.

However, while this is a significant improvement, this practice still isn't the best way to keep credentials and secrets secure. For example, if the .env file is accidentally committed to version control, then the credentials and secrets are once again stored alongside code.

Alternatively, if a malicious actor can access the environment where an application is running from, they can access the credentials by inspecting the environment; such as by running env | sort.

A far more secure way of storing application credentials and sensitive data is by using a dedicated secrets manager!

Secrets managers securely store all of an application’s secrets and confidential data. The application code then retrieves secrets from the secrets manager as required. Given that, it only needs credentials to authenticate itself, such as a TLS certificate or an authentication token.

Should a malicious actor gain access to the authentication credentials, they can quickly be invalidated or rotated, rendering them useless. This avoids the need to update a potentially large number of application secrets.

In this tutorial, you're going to learn the essentials of securely managing application secrets with PHP using HashiCorp Vault. Specifically, you’ll learn how to perform five essential operations: creating, retrieving, updating, and deleting a secret, as well as retrieving all available secrets.

Prerequisites

To follow this tutorial, you will need the following:

  • PHP 7.4 (ideally 8.1) installed on your development machine
  • Composer installed globally
  • cURL
  • jq

What is Vault?

According to the Vault website, Vault is:

Vault is an identity-based secrets and encryption management system. It allows you to secure, store and tightly control access to tokens, passwords, certificates, encryption keys for protecting secrets and other sensitive data using a UI, CLI, or HTTP API.

Some of its essential functionality is:

  • Secrets storage
  • Dynamic secrets
  • Automation of credential rotation
  • Rolling of encryption keys; and
  • API-driven encryption

Using a unified interface, secrets are stored in an underlying secrets engine. These secrets engines provide a lot of power and flexibility for storing secrets, including being able to integrate with existing infrastructure, such as Microsoft Azure, Google Cloud, a database, or RabbitMQ.

For the purposes of this tutorial, however, the application will use, arguably, the simplest engine, the KV Secrets Engine (kv).

It is a generic Key-Value store used to store arbitrary secrets within the configured physical storage for Vault.

A handy feature of the engine is that, when enabled, secrets can be versioned, allowing them to be rolled back.

Install and start Vault

The first thing you need to do is to install HashiCorp Vault.

If you're using Windows, download a precompiled binary appropriate for your operating system's architecture. After the download is finished, unzip the downloaded file.

Then, double-click vault.exe in the extracted directory to start Vault. If you prefer, after you’ve downloaded and extracted vault.exe, move it into your system path, so that it’s easier to use later on.

If you're using macOS or Linux, however, follow the instructions to install it using your preferred package manager.

Regardless of your operating system, however, once installed, test that it works by running the following command.

vault

If successful, you will see the command's help message, which lists the supported options.

Next, start Vault in “Dev” Server mode, by running the following command.

vault server -dev

This starts Vault in “Dev” Server mode so that you can get started quickly. Never use this mode in production, however, as it isn’t secure.

From the output which Vault prints to the terminal, note the "Root Token" which is written towards the end of the output, e.g., Root Token: s.XmpNPoi9sRhYtdKHaQhkHP6x. It will be required later.

Integrate Vault with PHP

With Vault running, it's now time to build a small PHP application that will manage its application secrets in Vault. The app will use the Slim Framework to provide routing and a DI container, so that no unnecessary code needs to be written.

Create the application’s directory structure

The first thing that you need to do is to create the application’s directory structure. As the application returns JSON responses, the structure is quite minimal, containing just one directory, public, which will contain the de facto bootstrap file, index.php.

Create it and change into the top-level directory, by running the two commands below.

mkdir -p hashi-vault/public
cd hashi-vault

If you’re using Microsoft Windows, don’t use the -p option.

Install the required dependencies

With the project's directory structure created, it's time to install the project's seven external dependencies:

Library

Description

csharpru/vault-php

This is a PHP client for Vault, which supports different authentication backends with token caching and re-authentication.

laminas/diactoros

Diactoros is used to simplify the creation of requests, URIs, and responses.

PHP Dotenv

This package merges variables set in a .env file into PHP’s $_ENV and $_SERVER Superglobals

PHP-DI

This is a small and intuitive dependency injection (DI) container.

php-http/curl-client

This provides the HTTP connection to Vault, as csharpru/vault-php doesn’t natively provide that functionality.

Slim PHP

If you're basing the application on the Slim framework then, naturally, you have to use the framework.

Slim PSR7

In addition to Slim PHP, you'll use this library to integrate PSR-7 into the application. It's not strictly necessary, but I feel it makes the application more maintainable and portable.

To install these dependencies, run the command below in your terminal, in the root directory of the project.

composer require --with-all-dependencies \
    csharpru/vault-php \
    laminas/laminas-diactoros \
    php-http/curl-client \
    php-di/php-di \
    slim/psr7 \
    slim/slim \
    vlucas/phpdotenv

Depending on the speed of your internet connection, all the packages should be installed quite quickly.

Add the required environment variables

Now that the dependencies are installed, create a new file, named .env, in the root directory of your project, then paste the code below into that file.

VAULT_TOKEN=<<VAULT ROOT TOKEN>>

After you’ve done that, replace the <<VAULT ROOT TOKEN>> placeholder with the root token from Vault’s console output, that you took note of earlier.

Create the core application logic

Now, in your editor or IDE create a new file, named index.php in the /public directory, and in it paste the code below.

<?php

declare(strict_types=1);

use DI\Container;
use Http\Client\Curl\Client;
use Laminas\Diactoros\{RequestFactory,StreamFactory,Uri};
use Laminas\Diactoros\Response\{EmptyResponse,JsonResponse};
use Psr\Http\Client\ClientExceptionInterface;
use Psr\Http\Message\{
    ResponseInterface as Response,
    ServerRequestInterface as Request
};
use Slim\Factory\AppFactory;
use Slim\Routing\RouteCollectorProxy;
use Vault\AuthenticationStrategies\TokenAuthenticationStrategy;
use Vault\Exceptions\RequestException;

require __DIR__ . '/../vendor/autoload.php';

$dotenv = Dotenv\Dotenv::createImmutable(__DIR__ . '/../');
$dotenv->load();

$container = new Container();

AppFactory::setContainer($container);
$app = AppFactory::create();

$app->addBodyParsingMiddleware();

$app->get('/', function (
    Request $request, Response $response, array $args
) {
    return new JsonResponse('It works');
});

$app->run();

The code starts off by importing all the required classes and Composer's autoloader. Following that, it uses PHP Dotenv to load the Vault token in .env into PHP's $_ENV and $_SERVER Superglobals, so that the application can authenticate against Vault later.

After that, it initialises a new dependency injection (DI) container (PHP-DI), and sets the container as the global container instance. A new Slim PHP object, $app, is initialised with the container transparently passed to it.

With the SlimPHP app ready to use, it does two more things:

  1. It calls the addBodyParsingMiddleware() method to ensure that requests using methods other than GET and POST (PUT, PATCH, and DELETE, etc) are supported.
  2. It calls the get() method to create the default route (/) for the application. The route doesn’t do a lot; it just returns a JSON encoded string, “It works”, which confirms that the application is working.

With that done, it’s time to test the application. Before you can do that, however, start it, using PHP’s built-in web server, by running the command below.

php -S 127.0.0.1:8080 -t public

Then, run the curl command below to call the application’s default route.

curl http://localhost:8080

If successful, you will see the output below.

"It works"

Assuming that you received the expected output, press ctrl+c to stop the application.

Register a Vault client as a DI dependency

Before the application can connect to Vault, it needs to initialise a \Vault\Client::class object. To do this, add the code below in public/index.php after the call to $container = new Container();.

$container->set(\Vault\Client::class, function(Container $c) {
    $client = new \Vault\Client(
        new Uri('http://127.0.0.1:8200'),
        new Client(),
        new RequestFactory(),
        new StreamFactory()
    );
    $client
        ->setAuthenticationStrategy(
            new TokenAuthenticationStrategy(
                $_SERVER['VAULT_TOKEN']
            )
        )
        ->authenticate();

    return $client;
});

The object is initialised with four parameters, but the only one worth focusing on is the first one, which provides the hostname and port where Vault is listening for connections.

Note: the tutorial uses laminas-diactoros to provide the underlying HTTP connectivity. Use another package, if you prefer, so long as it provides implementations for the following PSR-7 interfaces:

  • \Psr\Http\Client\ClientInterface
  • \Psr\Http\Message\RequestFactoryInterface
  • \Psr\Http\Message\UriInterface
  • \Psr\Http\Message\StreamFactoryInterface

After that, the client’s authentication method is set, in this case the Vault token which you made available as an environment variable earlier.

Vault supports a number of authentication methods, including token, username and password, and TLS certificates, as well as a number of external services, such as AWS, LDAP, and Kerberos.

Finally, so that the object is available throughout the application, it’s registered as a service in the DI container, using \Vault\Client::class as the service’s name.

Add the ability to create and update a Vault secret

Now, with all the infrastructure code in place, it’s time to add the ability to create and update an application secret inside Vault. To do that, in public/index.php, replace the initial route definition with the one below (formatted for readability).

$app->group('/secret', function (RouteCollectorProxy $group) {
    $group->post('/{key}', function (Request $request, Response $response, array $args) {
        $result = $this->get(\Vault\Client::class)->write(
            sprintf(
                '/secret/data/%s', 
                $request->getAttribute('key')
            ),
            ['data' => $request->getParsedBody()]
        );
        return new JsonResponse($result->getData());
    });
});

The code starts off by creating a route group with the prefix /secret. The reason for this is so that the base route path for all the CRUD operations will be the same, i.e., /secret.

Creating a route group ensures that any route registered within the group has that prefix. This is a handy way of reducing code and avoiding potential bugs.

After that, a POST route is registered which requires a route parameter named key. The route parameter doesn’t have any restrictions on the characters it can contain. I did this for the sake of simplicity.

The route’s handler retrieves the \Vault\Client::class service from the DI container and uses its write() method to create a new secret using key’s value for the secret’s name, and the request’s body as the secret’s value.

If the secret already exists, it’s updated instead. This might seem strange, as you might, quite reasonably, expect to use a method named update() to update a secret — not create(). However, as secret versioning is enabled by default, any write requests to an existing secret create a new version of that secret — in effect, updating it.

Regardless of whether a new secret was created or an existing one updated, the code then initialises and returns a JsonResponse object with the write() method’s response.

Doing this returns a JSON-encoded version of the data returned from $result->getData(), sets the response’s Content-Type header to application/json, and its status code to 200. It’s a handy way of ensuring a valid response for very little effort.

With the route defined, it’s time to test it. Do that, by running the command below. This will create a new secret at the path secret/music-vault, with a key named "band", which has the value "foofighters".

Feel free to change the key name, and the key/value pair to something that you’d prefer.

curl -X POST -d band=foofighters \
    http://localhost:8080/secret/music-vault | jq

When the command completes, you should see output similar to the example below, indicating that the secret has been created, along with its version, among other things.

{
    "created_time": "2022-01-26T08:47:13.495687Z",
    "custom_metadata": null,
    "deletion_time": "",
    "destroyed": false,
    "version": 1
}

Now, test updating the secret. Run the cURL command below in your terminal. Note the request data passed to the command and the HTTP method (PUT) are different from the previous call.

curl -X PUT -d band=the-foo-fighters \
    http://localhost:8080/secret/music-vault | jq

The response returned from the request is almost exactly the same as for the first request, however, note that the version property is now set to 2, instead of 1.

{
    "created_time": "2022-01-26T08:56:07.81223Z",
    "custom_metadata": null,
    "deletion_time": "",
    "destroyed": false,
    "version": 2
}

Add the ability to retrieve a Vault secret

Now that a secret’s been created, it makes sense to add the ability to retrieve it. To do that, in public/index.php, add the route definition below, after the definition of the previous route, which you just created.

$group->get('/{key}', function (
    Request $request, Response $response, array $args
) {
    $result = $this->get(\Vault\Client::class)->read(
        sprintf(
           '/secret/data/%s', 
           $request->getAttribute('key')
        )
    );
    return new JsonResponse($result->getData());
});

This route, similar to the previous one, defines a route variable named key, which is used to determine the secret to return.

It retrieves the \Vault\Client::class service from the DI container to interact with Vault and calls its read() method to retrieve the secret, using the value of key for the secret’s name. The response to the call is returned, JSON-encoded, by the JsonResponse object.

To test the route, run the curl command below in your terminal.

curl http://localhost:8080/secret/music-vault | jq

If successful, you should see the JSON output below.

{
    "data": {
        "band": "the-foo-fighters"
    },
    "metadata": {
        "created_time": "2022-01-26T08:47:13.495687Z",
        "custom_metadata": null,
        "deletion_time": "",
        "destroyed": false,
        "version": 2
    }
}

Add the ability to delete a Vault secret

Now that the application can create, read, and update a secret, it’s time to add functionality to delete one, completing the core CRUD functionality.

To do that, add the route definition below, after the previous route in /public/index.php.

$group->delete('/{key}', function (Request $request, Response $response, array $args) {
    $result = $this->get(\Vault\Client::class)->revoke(
        sprintf(
            '/secret/metadata/%s', 
            $request->getAttribute('key')
        )
    );
    return new EmptyResponse(204);
});

There are two key differences in this route from the create and update route. These are:

  1. The call to the \Vault\Client::class service’s revoke method on line three, which deletes the secret matching the value supplied in the key route variable.
  2. The return of an EmptyResponse object, with an HTTP 204 No Content status code. The reasons for this are that the underlying response from Vault doesn’t return anything in the body, so there’s nothing to pass on, and that the request has succeeded.

To test the route, run the cURL command below in your terminal.

curl --verbose -X DELETE \
    http://localhost:8080/secret/music-vault | jq

As the --verbose flag is passed, you will see the response headers returned from the request, which display the returned HTTP status code, confirming the successful deletion of the secret.

Add the ability to retrieve all secrets in Vault

Finally, create one, final, route, this time to retrieve all secrets in Vault. To do that, add the route definition below before the call to $app->group('/secret', function (RouteCollectorProxy $group) {.

$app->get('/secrets', function (Request $request, Response $response, array $args) {
    try {
        $vault = $this->get(\Vault\Client::class);
        $result = $vault->keys('/secret/metadata');
        return new JsonResponse($result->getData());
    } catch (RequestException|ClientExceptionInterface $e) {
        return new JsonResponse($e->getMessage(), $e->getCode());
    }
});

The code defines a route with the path /secrets, which does not accept any route variables. Like all the other routes, it retrieves the \Vault\Client::class service from the container.

It then calls the service’s keys() method, passing in a string composed of two parts. The first part, /secret, is the key to retrieve the data for. The second part, /metadata, is the suffix required to query information about that secret.

To test the route, run the command below in your terminal.

curl http://localhost:8080/secrets | jq

If successful, you will see the following output. There, you can see that the secret has a single key named music-vault.

{
    "keys":[
        "music-vault"
    ]
}

More keys would be listed if there were more stored for that secret.

A parting word about security

Two things are worth bearing in mind.

Firstly, the examples in the tutorial's code intentionally used HTTP to avoid setting up a self-signed SSL certificate and configuring a web server to use that certificate.

Secondly, while the application uses a token to interact with Vault, requests to the API itself are not secured. Consequently, anyone who can access the API can access your secrets.

In a production application, only make requests over HTTPS, and use proper authentication and authorisation to ensure information is only available to valid users with appropriate access.

That's how to manage application secrets with PHP using Vault

While it might seem like an intense topic — at least at first — managing application secrets and secure credentials can be, almost, as easy as managing any other application data.

I hope that this quick tour of HashiCorp Vault using the KV Secrets Engine will encourage you to use a secrets manager in your PHP applications and in your organisation, if you’re not already doing so.

P.S., There’s a lot more to know about storing sensitive application data securely — and Vault especially. So, watch this space for future Vault tutorials which will build on this one.

Matthew Setter is a PHP Editor in the Twilio Voices team and (naturally) a PHP developer. He’s also the author of Deploy With Docker Compose, which shows the shortest path to deploying apps with Docker Compose. When he’s not writing PHP code, he’s editing great PHP articles here at Twilio. You can find him at msetter@twilio.com, and on Twitter, and GitHub.

Related Posts

Related Resources

A newspaper article

Twilio Docs

From APIs to SDKs to sample apps

API reference documentation, SDKs, helper libraries, quickstarts, and tutorials for your language and platform.
An Open book

Resource Center

The latest ebooks, industry reports, and webinars

Learn from customer engagement experts to improve your own communication.
User group reactions

Ahoy

Twilio's developer community hub

Best practices, code samples, and inspiration to build communications and digital engagement experiences.