UPDATE: This post contains decent ideas but I recommend you check out the updated article here. It’s more secure and easier to follow.
The Problem in a Nutshell
I mentioned previously that I was rewriting Write.app in Ruby using Rails. The new Write.app project has now diverged into the Write.app core API and the core client. The core client is just like any other API client – its provides a UI to interact with the back end API and is completely disconnected from the app logic. As such I need a way to uniquely identify it and prevent others from abusing the Write.app API keys. Before explaining how I solved the problem I want to warn you about some of the insecure solutions I’ve seen people suggest on StackOverflow and other sites.
The Don’t List
Don’t rely on sessions as your only security method. Session IDs can be stolen, fixated, or otherwise hijacked. If your app creates a session any time a request is made to it how are you going to identify that only an authorized client requested it and it isn’t just some guy running
curl -I at his terminal?
Don’t rely on custom headers. I flirted with the idea of sending a custom header on every request. The problem with this is that if you’re generating the header to be sent client-side then an attacker only needs to visit your app once to browse through the code to get it. If you use a proxy to inject a custom header in every request from the client-side to the server side you’re still generating it on every request regardless of where it originated.
Don’t trust the referrer header. First off, not every browser sends a referrer header but more importantly, they’re easily spoofed.
Don’t rely on IP addresses. Some people suggest you store a unique token on the client based on their IP address in order to identify them. In a best case scenario you may ensure requests only come from a specific IP but you end up pissing off people on mobile devices, behind proxies, or anyone using an ISP that routinely reassigns IP addresses by rejecting their requests when their IP changes. IPs can be spoofed anyway so trusting them in this case is not a good idea.
SSL is not enough. Some people thinking slapping an SSL certificate on a site automatically makes it secure and means they no longer need to worry about security. This isn’t the case. Man in the middle attacks, XSS, and CSRF attacks are all still possible when using SSL, especially during the initial handshake. Using SSL puts you pretty far ahead of the game but it’s not the end of the line.
In the case of Write.app, much of the client is a public site so there’s only a need to authenticate the client upon certain requests. Because there’s no need to authenticate a client that’s just browsing the homepage or the features page a key is not required there. With this in mind, here’s Write.app’s solution:
Requesting the key
The first thing that happens is that the client will request a key. This will only happen on certain pages like the sign up and log in pages. The idea here is that we want to make sure that only users browsing with a known client (in this case the official website or core client as it’s called) are allowed to take actions like creating or authenticating a user.
So when the client app requests the login page the server generates a unique token based on information sent in the request. The information used is always something the server knows, something the client knows, and something both know. So for example the server can generate a unique key based on
User agent + current time + secret key. The server generates a hash based on this information and then stores a cookie containing only the hash on the client machine.
At this point our key really isn’t a key anymore. It has been transformed into an access token. The server should then take this access token and store it for later retrieval. You can put the key in a database but since data of this type needs to be retrieved often I would suggest using a key-value store like Redis to cut down on database reads/writes and boost performance.
When you store the token you should also store a separate piece of data to indicate what permissions are associated with the token. In this case our token is acting only as a way to register and authenticate users so we store it next to a value that indicates who the token belongs to (the app’s web UI) and what permissions it has (limited to create and authenticate users). We treat it just like we would any other API client that way we can capture stats and control how it is used.
Authorizing a request
When the client then makes the POST request to create a new user or log in the server will check to see if the client sent an identifying cookie along with the request. If not, we reject the request. If it does send the cookie, the server should once again generate the hash using the values used previously (these values are either already known or sent with the request anyway so we’re not really taxing the server much) compare it to the cookie being sent to us, and if the values match allow the request to proceed.
Now that we’ve completed an authorized request using a valid cookie we no longer need it. Remember that because we’re using a client to access an API there really is no concept of a session on the server. Authorized clients should be able to make requests based on single use tokens, not sessions stored on the server. Instead we should be handling application state on the client which is beyond the scope of this article but is important to mention.
When the user finishes taking any of the actions approved for our client (login or sign up) we throw away the token. Before we do, however, we need to generate a new one this time tied to a specific user with different permissions. In our API we give each user their own API key which allows them to take actions through our client app. During a signup or login action we look up the secret key from the user’s database table (or generate a new one if its a sign up action) and from then on this key will be used in conjunction with the other data we used to generate our core client token (user agent and current time in our example) to generate a new single use token each time a user logs in. This process is identical to how we manage the app’s own key.
In Write.app’s case, all core client keys expire after a certain amount of time that never exceeds 24 hours. A cron job runs periodically to disable and delete keys that have expired.
This kind of setup works for Write.app because the client itself first needs to authenticate with the server to take any action and even then does not have sufficient privileges to modify or return sensitive data. It’s essentially a read-only application that never leaks data outside of the server side API.
These measures alone are not enough for full security however and are beefed up on the server side with other techniques to enhance security. The core client keys are still susceptible to leakage under certain conditions but the system is set up in such a way that even if an attacker were to get the keys they couldn’t do anything useful with them without significantly more time and effort. The fact that keys change automatically over time shortens the time an attacker has to find the three values used in creating the access token and then once discovered the attacker needs to be able to set up a way to get a user to hand over their username and password. An attacker could create their own account but the value of an attack on Write.app would be in exposing or modifying existing user data.
Remember too that the site uses SSL and other measures besides just limited-time tokens to allow requests. All in all this model is actually more secure than the current server side implementation of Write.app. I’ll be writing in more detail with example code how this system works on the Write.app blog soon.
In implementing this strategy for my own project, I realized that some of my original idea is not as good as I thought, there are areas that could cause confusion (like the values used for hashing), and much of my original idea is only applicable when you are in control of both the client code and the API. So here are some refinements to my original idea:
1. Clarity on Keys and Tokens
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23
First off, I know that code can be more concise – that’s not the point, the point is to get the idea of what’s going on. I also know that separating these two functions and forcing the client to make two requests instead of one can be kind of stupid. I’m okay with this and here’s why – only the official Write.app client will be making two requests. Other clients will only need to call the
tokens/exchange function because they’ll already know their API key.
So what’s going on in that code above is that the official client requests its own API key. This is because the official client’s key will change often throughout the day. So an attacker would need to check for the existence of new keys multiple times a day at random intervals to impersonate the official client and even then the public key can only be exchanged for tokens with very limited permissions anyway. So yes, there is a window of time where a third party can impersonate Write.app but they can only create new accounts and log users into their accounts. The latter vulnerability can be quite serious however since Write.app uses SSL everywhere (with HSTS and Perfect Forward Secrecy enabled) the user should still be able to confirm the identity of the site they’re on by checking the certificate. That said, I do have a solution for that as well which is beyond the scope of this post update.
When a user logs in using the official client app, the API first validates that the token being passed belongs to the official API client then automatically generates a new, auto-expiring, access token from the logging-in user’s API key. This key never leaves the server and the only way to access the token it creates is to actually log in as the user it’s assigned to.
What happened to all that hashing?
In the original post I mentioned that our token would hash some values and token authentication would depend on the server matching the hash sent to the data it had. I was basically describing a method to use HMAC to validate tokens. The current code is set up to support such a scheme but it currently doesn’t use it. Right now we’re just validating what’s been sent in the
Authorization: header with what we have stored in our datastore (Write.app uses Redis because token lookups are incredibly slow when you have thousands of users making millions of requests and each one needs to match a token header to what’s stored locally).
What about 3rd party clients?
The solution I’m describing above assumes that you are in control of both the client and the API. That is to say, you have built both the API and client. If you were to build a third party app to access your Write.app account through the API you would have a very hard time. You could use the public
request_key -> exchange_for_token -> log_in workflow secure. After the user is logged in, the official client runs purely in the client without any server-side help beyond that API actually validating tokens and returning data.
So finally, here’s the solution to the third-party client problem:
A final word on client-side apps using third-party APIs
You really shouldn’t do it if you’re requesting sensitive information. It only really works if you are the creator of both the client and the API. Otherwise you’re most likely just duplicating functionality that the official client already has. The point of giving out API keys to users in our case, is so that Write.app users can access their account data from the command line, mobile apps, or use specific functionality of the API to enhance the experience of another app. I know that sounds very Twitter-esque but it’s a good point. Your API generally shouldn’t be built for creating clones of the official client but rather as a way to use the data contained inside of it in new and different ways.
I do plan to do another write-up when the final implementation of our API authentication is complete. Hopefully this helps some people for now. If anyone has better ideas please speak up in the comments. It’s actually pretty hard to find good information about client-side API key security online.