Managing Development Environment Variables Across Multiple Ruby Applications

Managing Development Environment Variables Across Multiple Ruby Applications

There is one thing I’ve noticed on the Twilio blog since I started here. We are always telling you to store your account SID and auth token in environment variables. What we don’t tend to do is spend too much time talking about how best to manage those environment variables.

In my previous role, I worked at a digital agency contributing to many different applications over time. Each of these applications relied on various different APIs and stored varying amounts of different credentials. Best practices such as the Twelve Factor App and the constraints of certain hosting platforms like Heroku lead to us using environment variables to store these secrets. It took us a long time to standardise how we did that though, from hacking the Rails boot process to eventually extracting out a gem that handled it for us.

It is very easy to set an environment variable in simple cases, however with many apps, many credentials and a fast paced environment, setting and resetting env variables becomes unwieldy very quickly. But that’s where Ruby and the community comes to the rescue. There are a number of ways to store and manage your environment variables. Let’s take a look at how they can improve your development environments.

The easiest way to use environment variables

Let’s start with the easy way to use environment variables. Say you’re building an application and using Twilio, you’re going to need your account SID and auth token. So let’s load them into the environment. On a Mac or Linux command line, you would type:

I don’t personally develop on Windows but there’s a very comprehensive answer on superuser.com about environment variables for various operating systems that might help you if you do.

When you run your application you can refer to ENV['TWILIO_ACCOUNT_SID'] and ENV['TWILIO_AUTH_TOKEN']. Let’s try that with a simple one-liner to print out the variables. Adding the -e flag to the ruby command allows us to execute a string of code right on the command line, which is useful for this example.

Good stuff, we have environment variables!

The hardest way to use environment variables

If we carry on developing from our terminal window we were just using, then everything will be fine. However, open a new terminal window and run the same Ruby command again and see what happens:

The environment variables are gone. In order to use them again, we are going to have to export them again for every terminal session in which we want run our application. There has to be a better way!

The most awkward way to use environment variables

It is, of course, possible to set your environment variables a bit more permanently. On Mac and Linux you can add those same export lines to your ~/.profile file and they will be loaded as you create a new terminal session. I defer again to the experts on superuser.com about Windows environment variable management.

The problem I have here is while I have many projects that may use the same services, like Twilio, Twitter, Facebook, Mailchimp, etc, they don’t all share the same credentials. You could prefix your environment variables with the app name as a sort of namespace, but that gets very messy over time.

Saving the environment

There are a three things we need in order to make environment variables manageable:

• We need to load credentials, secrets and other configuration items into the environment on a per project basis
• We need to be able to do this from the simplest scripts up to full Rails applications
• We need to keep secrets out of source control so that they don’t get spread around unintentionally

Ruby environment loaders

As I said at the beginning, the Ruby community has indeed provided for us. A look through The Ruby Toolbox’s list of configuration management gems shows us a lot, although not all of them are of use to us in this situation. The two that stand out for loading config into the environment are dotenv, Figaro. I’d like to submit envyable as an entrant to this list, as it grew out of my previous agency in order to cover this use case too.

Here’s a quick rundown of how each of these libraries works and how they can take the pain out of loading configuration into your environment.

Envyable

For envyable, all you need to do is add the gem to your Gemfile:

Run bundle install, then add a env.yml file to your config directory. If you’re not working within Rails you might need to create this directory too.

Add that file to your .gitignore file (or the ignore file for your version control system of choice)

And then start adding your secrets to env.yml.

If you are using Rails, then the config will be loaded for you when you restart your application. If you are using this within another framework or application, you just need to require the gem and call:

before you need your config. Now, within your application you will have access to ENV['TWILIO_ACCOUNT_SID'] and ENV['TWILIO_AUTH_TOKEN'].

Figaro

Figaro is similar to envyable but with a few more features. Its downside is that it only works on Rails. You add the gem to your Gemfile, as usual:

After you run bundle install, run the Figaro install script:

This will create the config/application.yml file for your config and add it to your .gitignore file all in one go. Now you can start adding secrets to the file:

As you can see in this case, Figaro lets you add default environment variables and you can then override them on a per environment level. Figaro has some other features which can be very useful too, such as required keys and a script to set config on Heroku automatically.

Dotenv

Unlike the previous gems, dotenv does not rely on yaml for a config file placed within the config directory, instead opting for a .env file in a project’s root. To get started, add the gem to your Gemfile:

Or for Rails applications:

Run bundle install and create yourself a .env file.

You can now add your config to the file in a format that is intentionally reminiscent of exporting environment variables on the command line:

You can even add the export statement at the front in case you want to source the file in bash:

I recommend adding the .env file to your .gitignore file as well, though Brandon Keepers, the creator of dotenv, suggests you commit the file to the project with just your development credentials saved. This is a bit of a philosophical difference between dotenv and envyable/figaro. My development credentials tend to be based on my accounts so I don’t want to share them around and would prefer to keep any secrets out of source control.

If you’re using Rails, the environment variables will be automatically loaded, but in other applications you can simply:

You can also provide environment specific config by creating .env.production or .env.development files, which will override any config set in the .env file in the respective environment.

What about secrets.yml?

Ah yes. Since version 4.1 Rails has had its own way of loading secrets. Newly created Rails 4.1 projects come with a config/secrets.yml file. Any key within that file can be accessed by calling Rails.application.secrets.key_name. In fact, in Rails 4.2 there is also Rails.application.config_for which can be used to load arbitrary yaml files from the config directory. Whilst his post is just about secrets.yml, I think Steve Richert (the author of Figaro) describes best why Rails still hasn’t got it right.

So which is the best?

There is, of course, no simple answer to this question. Frankly each of the gems I’ve talked about above, and there are probably more available that I haven’t found, do a great job of keeping your secrets safe and loading them into the environment on a per project basis. Each of them solve the problems I described when working across many applications and allow you to stick to the Twelve Factor methodology.

I personally like to use envyable (I suppose that’s no surprise since it was born out of our methodologies at my last place of work). I like the comfort of a yaml file in a config directory as opposed to a hidden .env file. I also like that it works across all Ruby applications and that, at heart, it really is very simple. Your favourite might turn out to be Figaro, with its extra features, or dotenv, which is far and away the most popular of the three according to the Ruby Toolbox.

If you have a different way of loading environment variables that I haven’t covered here, I’d love to hear about it and why you chose it. Drop me a comment here or grab me on Twitter or email. And if you think envyable is too simple, I’m always open to pull requests.

Whichever method you choose, you will no longer have to mess about exporting variables on the command line over and over or cluttering your dotfiles. You can just get on with creating your next great application!