Let’s state the obvious here: you don’t want to store secrets in a public Git repository (usually on GitHub) but you need to use those secrets to run your app. It’s a conundrum. How do you keep API keys, hash salts, passwords, and other sensitive information out of your public repository but still available to your app at runtime? The answer that most people have settled on is the very popular dotenv module/package/whatever your language of choice calls libraries. This module keeps your secrets safe in a
.env file ignored by Git that’s stored in your project’s root directory and it’s contents are automatically loaded as environment variables when the system starts up. It’s a great idea but we just don’t need it anymore. Here I’ll offer up an alternative to dotenv, when dotenv makes sense to use, and why most projects can safely and easily ditch this library.
How it works
Dotenv is simple. It only takes a few minutes to browse the code and figure out how it works under the hood. From a user’s point of view keeping your app’s secrets safe in a public repo requires you to follow these steps:
- Install dotenv (in your
composeror whatever). It comes in flavors for almost every language you can imagine so it’s probably available through your language’s package manager of choice
- Create a file called
.envin the root of your project. Fill it with secrets (shell script style like
MY_VAR=some-string). The syntax is the same as how you’d set environment variables in a shell script except without the
.gitignorefile so none of your secrets get committed to your public repository.
- Load the
dotenvmodule when your application starts up.
When I first heard about this I thought it was the most amazing solution to a common problem ever! I still do think it’s a great idea but if you understand how your terminal environment works it’s just not necessary anymore.
First off, I don’t like the idea of installing yet another library to keep my secrets safe. Your system’s environment is built to hold variables already. You don’t need an external tool to set env vars.
But maybe you think installing a library is worth the extra time, effort, or whatever you’re trading in to get the module installed. That’s fine but now you’re relying on a Ruby/PHP/Python/Node library to load your environment variables for you. My questions is this: why don’t you just set your environment variables yourself? Why have a module loaded to do it? I get that development and production configuration values will differ but relying on a module like this leads to ugly environment checking code and it does a job that you could just as easily have done yourself in a few lines of code.
There’s a better way to set environment variables
Libraries like dotenv became popular because the way they solve the environment variable problem is actually simple and elegant. They require no thought and “just work”. Add secrets to your
.env file, add
.env to your
.gitignore, then somewhere high up in the app’s startup routine you call a method on the
dotenv object to load your environment variables safely and securely. There’s nothing wrong with that — I just want to show you there are other, often better ways to handle per-environment configurations whether you’re working solo, on an open source project, or on a team that needs to share
.env secrets. Here’s what I’m doing in my latest Node.js project.
I created a shell script called
envvars.sh (I could have just called it
.env but I didn’t want to confuse anyone used to using that library). It’s an executable script written in Bash that simply exports some variables to my session then tells me when and what it’s done. It does exactly what dotenv does for the most part but it actually sets the values in your environment, not just Node’s
process.env object. Some veterans might gasp and gag at the thought of me reinventing the wheel. We already have a module that loads our environment variables, why write she’ll scripts to do it, they might ask. The answer is because it’s easier, flexible and more transparent. The added benefit is that by setting the variables in the terminal environment itself rather than just shoving values into Node’s
process.env object I can run other programs concurrently that use the same values.
Implementing your own dotenv
For my Node project I have a need to keep the source code free and open source while still protecting secrets like the API credentials I use to connect to a third party API plus some other things like session hashes, encryption keys, salts and whatnot.
First I create the
envvars.sh file in the root of my project. I then add that to my list of ignored files by running
echo "envvars.sh >> .gitignore. Now my secrets are safe. The inside of my
envvars.sh file looks like this:
1 2 3 4 5 6 7 8 9 10 11 12
Make sure to
chmod +x envvars.sh (make the file executable) just to be on the safe side once that’s written.
Integrating into an Express app with Gulp
Disclaimer: At this point the gulp task I describe does not set the environment variables in such a way that it makes them available to my gulp tasks. I’d love it if someone could help solve this problem. The rest of this post goes on to describe the desired solution…
The next step on my journey is to source the envvars file when I start my default gulp task so my entire app and gulp task runner has access to those environment variables. I do this using Node’s built in
In my gulp file I have tasks for transpiling LESS to CSS, running browserify to bundle my JS and React components, and of course start a server. I won’t bore you with the details of those. Here’s the very first task that gets run as part of my default Gulp task when I run
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19
That gulp task right there is the equivalent of having dotenv installed and running. The only problem is that the variables don’t actually stay set after the task runs. More on that later.
In production I’m hoping you use a proper process manager like PM2 which will set your envvars for you but in development mode simply using Node’s built in child process module is all it takes to load up your environment with the right envvars. What I just explained is basically what you think dotenv is doing behind the scenes (remember, dotenv is setting props on the
process.env object, not really setting env vars) except with more flexibility because your shell scripts can do more than simply declare keys and values.
Dotenv isn’t bad
If you’re new to development then please do use dotenv. If you’re not comfortable with the concepts of processes and environment variables then start off using dotenv. But learn how these modules work and experiment with different ways of doing things like my method described here. It’s fun to learn new things.
The Fork in the Road
At this point (as of this writing on May 12, 2016) I haven’t found a way to make the environment variables set from the
exec method callback to stick to the environment or even stick in the current Node process. That means this method technically doesn’t work yet. Because of this, I cannot officially integrate it into my gulpfile and tell people it works. Instead, my solution is to
source envvars.sh before running my Node app. I only need to do this once per terminal session so it’s not a big pain and it has the added benefit of allowing other applications (think: background jobs, related APIS) to use these same exported variables. It’s one extra command to run (
source envvars.sh && gulp but it gets me where I want to go which is having
process.env populated with any environment variables I’ve set.
You’ll notice that the
exec method takes a callback function so many of you will be thinking, “it doesn’t work because of Node’s async behavior”. I don’t think this is the case because even when I start the Express server within the
exec callback my
process.env is still missing the exported variables I had set in my
envvars.sh file. I’m still searching for a true all in one solution that’ll let me set environment variables from a gulp file or, rather, run shell scripts from the gulp file.
In the future I’ll be optimizing this code so look out for updates to this method of setting envvars. Happy coding and let me know if you can finish the final piece of this puzzle.