JSON Web Tokens (JWT)
A common situation in software is verifing information that was provided to a backing service is accurate, and was produced by or on behalf of a specific person, organisation, software, or any two parties. Services can't rely on client validation for this kind of thing, so we need to be able to transfer this information, ensure its integrity, and verify the source. JSON Web Tokens allow us to just that.
In this article I'm going to walk through how these tokens work by breaking down each component and discussing them individually. I'm only going to talk about the box-standard features, so if you're looking for nitty-gritty information around everything you can do with a JWT, this article won't be covering all that.
JSON
The first part of JWT is JSON, JSON allows us to encode the two of the parts of the token, the payload, and how we created the signature.
I could assume that the majority reading this article will already know what JSON is, but its always good to give a quick run down of something to ensure we're all on a level playing field. If you're familar with JSON, feel free to skip to the next section.
JSON, short for JavaScript Object Notation, is a way to encode data in a friendly way that is easily readable by both humans and "machines" into a string. The format is very straightforward and has two structure types, and 4 scalar types.
Firstly we will quickly go through the scalar types, these include textual data, numbers, booleans, and null.
Textual data, or strings are represented by a sequence of characters contained within double quotes, as you would see in many programming languages, for example "Hello! Lets talk about that!", the characters can be any unicode character except for ", \, or any control characters, " and \ can both be escaped by preceding the character with \, for example "We can use the \" and \\ characters like this!", which would represent the value We can use the " and \ characters like this!, we shouldn't need to worry about this too much though, as JSON encoders will take care of this for you.
Numeric data is represented using its decimal value, for example 12345, 0.12345, and -1 are all valid numbers in JSON.
Boolean values are easy, they're represented using the values true and false.
The null value is just that, null, nothing special here.
The first "structure type" we'll talk about is an object, where you can specify key value pairs within the braces { and }. Following the key and preceding the value is a colon (:) seperating the two; and each key pair is seperated by a comma (,). The key is always a string, following the same rules as above, yet the value can be any valid JSON value, for example:
js{ "key": "string", "number": 1, "boolean": false, "null": null, "object": { "key": "value!" }, "array": [1, 2, 3] }
The second "structure type" is an array, where you can specify values within the square brackets [ and ]. Each value within an array is seperated by a comma (,). The values withn an array can be any JSON value, just like an object. For example:
js[ "string", 1, false, null, { "key": "value!" }, [ 1, 2, 3 ] ]
Something to note as well is that the whitespace between either key:value pairs in objects or values in arrays isn't taken into account when decoding the value.
Base64URL
Base64 is a way to encode data into a format that is string friendly, meaning as developers we don't need to handle binary data at every turn, for JWT a variant is used that is URL friendly, the URL friendly variant excludes the usage of +, /, and =, Base64URL achives this by utilising - instead of +, _ instead of _, and omitting the padding characters (=).
JWTs make use of Base64URL for encoding the header and payload JSON values, and encoding the resulting signature into a string friendly value.
A big thing to know about Base64 is that the input to output ratio of 3:4, meaning that every 3 bytes you put in, you're going to get 4 bytes out, this makes a big difference when dealing with large amounts of data. I mention this because you should be mindful about what information you put into your payload as large payloads could mean large output tokens.
A real simple example of Base64 is that if I input the value "Lets talk about that!" I'll receive the ouput TGV0cyB0YWxrIGFib3V0IHRoYXQh, the input length was 21 characters, while the output length was 28 characters.
Signatures
Signatures are used everywhere, if you have ever had to mark a legal document with a value that can be verified as your own, you sign it, anyone that holds that document will now be able to verify that your signature matches others they have, or could use your signature in the future to verify future documents.
Digital signatures are like our own signatures, they provide a way to verify a document without needing to ask the source every time. With our tokens we want to be able to sign the value with a secret, and to be able to verify that signature anywhere without needing to know the secret, we can do this by utilising RSA signatures which utilise asymmetric keys, one private, and one public.
JWTs do support algorithms that only require a single secret shared between multiple parties, the differences between the two popular algoritms can be found here. My suggestion is to try and utilise RSA with JWT if you can, however if you only want a single secret with no paired certificate, then HMAC is still a viable solution, just more care is needed when handling keys.
With RSA we can take any data, and create a unique signature for it using a private key, which can then be verified using the asymmetric public key.
In JWT we utilise this by first running the header and payload JSON values through a Base64URL encoder, joining them with a period (.) and then passing this to our RSA signing algorithm. It is a bit hard to show this in an example, but you'll see this in action later in this article.
Something to note when using asymmetric keys is that we can rotate our private key over time, while still retaining the public key information.
Headers
For our tokens to be verified we first need to know how the signature of our token was created, to do this we utilise a header, this header is a JSON object with typically two key value pairs, the key typ which tells us the media type used for the token, this is the value "JWT" if the payload is a JSON object, and the key alg where its value tells us what algorithm was used. The header could aslo contain additonal pairs, for example you may see kid which allows the signer to identify the key pair that was used.
Heres an example of a header value:
json{ "alg": "RS256", "typ": "JWT" }
You can read about the various keys that can be used here.
Payloads
A payload can be anything in theory, in practice this is generally a JSON object with key pairs where the value is generally a scalar value, however any JSON value is valid.
The payload is where the sources claims are held, for example their user identifier (sub). It can also hold other values like who the intended audience (aud) of the token is, who created it (iss) and when (iat), when the token's validity starts (nbf) and ends (exp), and also the identifier of the token (jti) if available. JWTs don't require any of these values to be contained in the payload, yet there are standards around this, for example the JSON Web Token Claim Regsitry.
You're free to add whatever key value pairs to your payload, however if you're dealing with something like OpenID then you will want to prefix your keys with a namespace, for example your domain name, this ensures your custom values aren't confused with anyone elses:
json{ "sub": "23311fbb-5d90-4bef-b76c-36ea30c22dac", "example.com/type": "Mythical" }
Of course if your service or client is going to be the only consumer of these payloads, feel free to use whatever you like!
The information that's put into these tokens however should be only information that is normally readable within the security context where these tokens are used, for example if used with a browser, treat the information as publicly accessible. There is an additional format that can be used with JWTs called JWE, which you can read more about here (Said link is to the RFC for JWE, so reader beware that its not a beginner friendly document!).
Tokens!
We've talked about everything we need to know to make & utilise our tokens. Now we just need to make one!
We're going to utilise the npm module jsonwebtoken.
For these examples we're going to use the private key:
default-----BEGIN RSA PRIVATE KEY----- MIICXAIBAAKBgQDf2KBojuyUPdcOv1Spa43M+BsUTh+FBJPJMFgYgZREA8nsspiZ sVX1BFTZ/B5imWUsNvS8WA39uaPWa+IcBU8qOo7k4KyvHp+PwT5SzLaarhVNAgTx g936zO7fP6LlkAqupq7hUXYRFeIns4CngI6h9OIV+k8Io6hc58bl17AJoQIDAQAB AoGBALicjl7tUQxJnC4behVoEMC09pBehfxMdB3/cwhzBfa6MTS3bseCy+OROPG2 ztB+tkQq5tjWvmM9UXQr8YUuk8QzPTsP8X1fXQarx+Ri27d8+N7WzTm5QtkORBhg 4FV+cPD4eDzJVp6RkWOLBwLXvv9KSlyFh1uIL9mM8IscMvepAkEA8RdMB5BtcwyM X5BZuVAiK4jHH2ZFcePp4jpMQNYW3tmNzCXxgUHrPSSzhSyF8QqpUyPRs/xjDbTM gwlj5T4tcwJBAO2wU+QCxzZirutHCqwqi3O9ypx+N1Xw2P5/dSlA2xaYX8sKkwe1 BkXxELwHPeFjj2cgtAjFDZ4Bk06v9h3qJ5sCQBn9cfPKzRG/A471x8ZjbhuVVin7 Y3cgo8EAmeHPco25lECywnM1wP9JapTrtNDEXnaZAO1PQvpiSD3EEGHRLyMCQApm 0QUFahpjLyx0q27lXbzu7VLz4xALvjNE+KeZgvz2JhsIl26a6W9eIVFZL8gRR1FI CRjpJrNndj7XTHn6qUkCQClCm9dvA6qNHN+/y6q2PNNhSJVFKEEM0V9BL8ba2khK bX4eXScclTvNm9xUHF6LdRElLYdtrF3MwJZKQTyrhlo= -----END RSA PRIVATE KEY-----
And the public key:
default-----BEGIN PUBLIC KEY----- MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQDf2KBojuyUPdcOv1Spa43M+BsU Th+FBJPJMFgYgZREA8nsspiZsVX1BFTZ/B5imWUsNvS8WA39uaPWa+IcBU8qOo7k 4KyvHp+PwT5SzLaarhVNAgTxg936zO7fP6LlkAqupq7hUXYRFeIns4CngI6h9OIV +k8Io6hc58bl17AJoQIDAQAB -----END PUBLIC KEY-----
These keys aren't going to be used for anything else except for this article, so sleep easy!
Please don't use these keys for anything else except for running these examples, instead generate your own to use within your projects.
Typically the issuing service (or client!) will hold the private key in a secure way, for example a Vault, the public key can then be distributed to consumers of the tokens, the public key doesn't necessarily need to be kept secure because it doesn't allow anyone to sign tokens using it (instead they would need the private key). The issuer could have multiple key pairs, for example one for each customer, audience, individual, or even each token, if using multiple they'll most likely have included the kid key in the header allowing for use of a JSON Web Key Set which can be used by external parties to find a matching certificate to verify a token with without needing to have a copy of it first.
We're going to create token with the payload using the key pair above:
json{ "sub": "23311fbb-5d90-4bef-b76c-36ea30c22dac", "example.com/type": "Mythical" }
To generate the token we need to invoke sign on the jsonwebtoken module:
jsconst JSONWebToken = require("jsonwebtoken"), fs = require("fs"); const PRIVATE_KEY = fs.readFileSync("./jwtRS256.key", "utf-8"); const payload = { sub: "23311fbb-5d90-4bef-b76c-36ea30c22dac", "example.com/type": "Mythical" }; const options = { algorithm: "RS256" }; console.log({ token: JSONWebToken.sign(payload, PRIVATE_KEY, options) });
This produced the value (the signature section will change every time its generated):
defaulteyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiIyMzMxMWZiYi01ZDkwLTRiZWYtYjc2Yy0zNmVhMzBjMjJkYWMiLCJleGFtcGxlLmNvbS90eXBlIjoiTXl0aGljYWwiLCJpYXQiOjE1NTUxNTMwMzN9.HLC1JxTZ-En9nOnl2jzXiM40tU0HynnNVqhlv4T478nVpPgp8LHnByzJYSeva2tP5no1nXflVbJMqAD5T3P57JAQ110tnXiqTTiynMd3k_PnE10PpLoX5CdNzFIrop-jDk2yo2IrlCAeb1JwChj4Nk0c7T4PZaCROJhAobQpIdM
We can see there are three distinct sections seperated by periods (.), the first being the header value (decoded):
json{"alg":"RS256","typ":"JWT"}
The second being our payload:
json{"sub":"23311fbb-5d90-4bef-b76c-36ea30c22dac","example.com/type":"Mythical","iat":1555153033}
And the third being our signature (which is binary data).
We can verify our token is valid by instead invoking the verify method, and passing our public key. We must also be sure to limit the algorithms we expect our token to be utilising, ensuring that the algorithm in the token can't be changed by a bad actor.
jsconst JSONWebToken = require("jsonwebtoken"), fs = require("fs"); const PUBLIC_KEY = fs.readFileSync("./jwtRS256.key.pub", "utf-8"); const token = "eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiIyMzMxMWZiYi01ZDkwLTRiZWYtYjc2Yy0zNmVhMzBjMjJkYWMiLCJleGFtcGxlLmNvbS90eXBlIjoiTXl0aGljYWwiLCJpYXQiOjE1NTUxNTMwMzN9.HLC1JxTZ-En9nOnl2jzXiM40tU0HynnNVqhlv4T478nVpPgp8LHnByzJYSeva2tP5no1nXflVbJMqAD5T3P57JAQ110tnXiqTTiynMd3k_PnE10PpLoX5CdNzFIrop-jDk2yo2IrlCAeb1JwChj4Nk0c7T4PZaCROJhAobQpIdM"; const options = { algorithms: ["RS256"] // We know we only ever used RS256 with this token }; console.log({ validPayload: JSONWebToken.verify(token, PUBLIC_KEY, options) })
The verify method returns the payload if the token was valid, so you'll see the payload value in your console when running this code.
We can copy our header, payload, public, and private key into the debugger here, which will also help visualise the different sections explained above.
As we can see from this article, JSON Web Tokens are just a few bits of straightforward standards brought together to make something powerful. They provide a interoperable way for us to easily verify claims from third party software without reducing security. They're used by a wide array of organisations, and are at the core of many other standards like WebID, and OpenID Connect. You can read more about JWTs here and view all the available libraries for various languages.
