Validate your Node/Express.js REST API Calls with yup

Photo by Gabriel Crismariu on Unsplash

Overview

When your Express API receives HTTP requests, how do you check whether the request is valid? Especially for POST and PUT requests, where you expect the request body to contain an entire object to process, how do you know whether it has the right fields and valid values?

In some of my early APIs, it seemed natural enough to either write my own validation code in my routes to check for fields and types, or to even have some code in the model layer that would check an object for various conditions before it got put into the database, for example. These solutions always wound up messy / unreliable / incomplete. This article shows you how to create custom Express middleware to validate HTTP request bodies using a cool validation library called yup. I’ll also show an example of nested objects and conditional validation in yup schemas.

The code for all of these examples can be found in this repository:

neightjones/express-yup-validation

This repository is built off of https://github.com/neightjones/node-babel-template — I wrote about the process of setting up this babel /eslint / prettier simple boilerplate project in these 2 posts:

• https://medium.com/javascript-in-plain-english/a-minimal-node-js-express-babel-setup-part-1-6be7b3f3bb55
• https://medium.com/javascript-in-plain-english/a-minimal-node-js-setup-part-2-eslint-prettier-vs-code-7963768dbb30

1. Our New “Product“ Model

To start, let’s create a new type of object that we can use throughout our examples. Let’s pretend we’re creating an eCommerce site where we have lots of product listings for people to buy. For now, our product model will be really simple and have the following fields:

• id: this is the unique identifier for each product — this is required
• name: a string to represent the product’s name — required
• description: a string to describe the product — this is optional
• price: how much does this product cost? — this is required
• category: even though we probably wouldn’t do it this way, this is going to be a simple string to specify which type of product this is (sporting goods, clothing, etc.) — this is required

So here’s an example of a valid product:

// Looks good!
const product = {
  id: 1,
  name: 'The Imitation Game',
  description: 'Movie about Alan Turing',
  price: 19.99,
  category: 'movie',
};

The only optional field is description, so we didn’t need to add it to get a valid product. All other fields (with proper types!) were required.

2. Product Schema with yup

With our new product model in mind, let’s see how we can encode this information into a yup schema.

Yup states that it is “a JavaScript schema builder for value parsing and validation.” We’ll be focusing on the validation use case in this post. Two things need to happen:

1. We define a schema to represent our product object model — this means encoding its fields, types, what is required or not, etc.
2. We’ll see how to use the yup schema to test out whether a JavaScript object is valid

The code in this section isn’t found in the linked repo — this is just to get a feel for how yup validation works with some examples.

Check out this initial example:

• Line 1: import everything from yup so we can use yup.<x> syntax
• Line 3: this is our first yup schema… we’ll expand on this shortly, but for now we’re saying the productSchema simply has a field called name, which is of type string
• Lines 7, 8: these are two simplified product examples
• Line 10, 14: these two blocks use the isValid method on the productSchema to check whether the objects we pass in are valid (it returns a Promise, so we use .then to wait for the result and log it out

As you’d expect, product1 is valid — it has the name field, which is the only field mentioned in the schema so far. However, product2 is also valid… in our schema, we didn’t say that the name field is required, so it doesn’t matter that product2 only has a field called title, and extra fields are okay.

Luckily, we can chain together different operators on each yup schema field, including the .required() call. With that in mind, let’s create the full productSchema with the fields we described initially, and use .required() where we need it:

• Line 3: our productSchema now has all the fields we discussed. All fields except for description have the required() call chained onto them, since description is the only non-required field. Also, you can see that the number fields have some extra validation (both are positive numbers, and one is an integer)… you can find all these options e.g. for numbers here.
• Line 11: product1 has all required fields, and each field has the proper type
• Line 19: product2 is missing description, which is fine, since it’s not required. However, look at its price field… the schema calls for a number, not a string.
• Based on that, the console logs will show that product1 is valid, but product2 is not. When we add the actual code to our project, you’ll also see how to capture what exactly was the validation error for cases when an object is invalid.

That’s a basic look at yup to get us going, so we’ll dive into the actual code. Later on we’ll run through a slightly more complex example to show off how we can do conditional validation in a yup schema.

3. Add a Simple products Router

As I mentioned before, you can see all the final code in this repo, or if you’d like to follow along you can work off of this boilerplate I wrote about previously.

First, we’ll add a new routes file for products. Under src/routes, we’ll create a new folder called products (we’re using a folder rather than a simple products.js file because we’ll have more code grouped with this new route logic)… inside of src/routes/products, add an index.js file, and add this to your new file:

Next, you’ll need to hook this up with src/app.js by adding 2 lines. First, import the productsRouter where the other routers are imported, like this:

import indexRouter from '#routes/index'; // this was already here
import usersRouter from '#routes/users'; // this was already here
import productsRouter from '#routes/products'; // this is new!

Second, you’ll tell the app to use this router for /products routes by adding a line after the other routers are used, like this:

app.use('/', indexRouter); // this was already here
app.use('/users', usersRouter); // this was already here
app.use('/products', productsRouter); // this is new!

Now we’re all set up to make some calls to our new productsRouter. Nothing too exciting is going on here. It’s a simple POST endpoint that logs out the object sent in the request body, and then echoes the object back to the requester… let’s test it out to see that it works before moving on. Assuming you’ve started your Express server on port 3000, run the following from your terminal:

curl -v localhost:3000/products \
-X POST \
-H "Content-Type: application/json" \
-d '{ "hello": "world" }'

If all goes well, you’ll see your Express server print out the object { "hello": "world" }, and see from your terminal that your request received a 200 response with the { "hello": "world" } JSON echoed back to you.

Now we’re ready to add in some validation.

4. Express Middleware to Validate POST / PUT

Middleware in Express lets you add layers of logic to the request / response cycle before you handle the request in your main API endpoint code. What’s our goal with adding some custom Express middleware? We want to check whether the POST request contains a valid product object before our main handler. That way, there are 2 outcomes:

1. If the product is valid, the rest of our route / service / model code can be written cleanly, since we’ll be guaranteed the product has the fields it needs to, or
2. If the product is not valid, it’s ideal to fail early, and return a 400: Bad Request back to the requester. If we do this, we’ll have successfully isolated the validation logic in one spot, which is a big win once it gets complex.

Create a validator File

Create a new file in our new folder src/routes/products that we made above, and call it validator.js. This file will simply contain yup schemas for any objects we need to validate in the corresponding API endpoints in that folder. For now, we’re only dealing with products, so we’ll take our productSchema we made earlier and make it the default export, like this:

Of course, this doesn’t do anything yet… we’ll need to create the actual Express middleware and eventually use this productSchema in tandem with the middleware.

Create the middleware

Even though we’re only working on the products routes right now, let’s create the middleware in a spot where any new routers can use it to validate their objects. Create a new folder in src/routes called middleware, and within that folder create a file called validateResource.js. Add in this code:

Let’s go through how this works:

The normal signature for Express middleware looks like this:

function myMiddleware(req, res, next) {
  // logic for my middleware here
}

Here’s an Express guide to middleware. Importantly, you have access to both the request and response objects. That way, you can see things like the request body, method, etc., and well as return a response if you wish via the res parameter. Lastly, the next parameter is how you chain middleware together — when you call next();, you’re telling Express to move on to the next middleware in the chain (or, in our case, move on to the main route handler, which is essentially just another middleware).

One interesting attribute of our validateResourceMW middleware above is that it returns a function that has the proper form to be an Express middleware. We do this so that we can parameterize our middleware. So, the function that is exported takes an argument called resourceSchema as its single argument, and returns a valid middleware function that utilizes resourceSchema in its logic. This way, different routers can pass different object schemas to this generic middleware, which enables each router to get the correct custom middleware for the object types it cares about. In our case, we’ll be passing in our productSchema that we just made in the validator.js file that sits alongside the product router.

Let’s look at the logic within the middleware function:

• Line 7: the ‘resource’ that we’re trying to validate is the request body. In the case of our productsRouter, it’ll be a product object that gets POST’ed
• Line 10: We’re using yup slightly differently now to validate the resource. You can see the async call to resourceSchema.validate(resource);. This is convenient because if the resource is invalid, it’ll throw an error, which is why we wrap this in a try catch block.
• Line 11: If we get here, we know that the resource was valid, so we call next(), which forwards the request to our route handler
• Lines 13–14: We hit the catch block if the validate call throws an error. We use validate here rather than isValid (from before) in order to capture errors. You can see that we return a 400 error with a custom JSON response that uses e.errors.join(', ') — these errors are useful strings from yup about what went wrong during validation. We’ll see some examples.

Update Product Schema

Before we tie all the pieces together, let’s look back at our productSchema that we created in src/routes/products/validator.js. It made perfect sense when we were running some basic tests earlier… specifically, it made sense that a product would have an id field, since we were answering the question “Is this a valid product?” However, in the case of a POST request, there shouldn’t be an id associated with the request body — the database will generate a new, unique id for us. So let’s remove the id field, leaving us with an updated validator.js file like this:

Tie it all Together

Finally, we need to use the middleware in our products router index file, which in turn needs the productSchema we defined in validator.js… the new products router file will look like this, now:

We import validateObjectMW (the middleware) and productSchema (the yup schema) into our route index file. Then, on line 7, you can see that the second argument to the post is now validateObjectMW(productSchema), which tells Express to run this middleware before it runs the logic in our POST endpoint.

Restart your Express server and let’s run a couple of curls to test it out.

First try:

curl -v localhost:3000/products \
-X POST \
-H "Content-Type: application/json" \
-d '{ "name": "The Imitation Game", "price": 19.99, "category": "movie" }'

This passes successfully, since our request body is a valid product (I didn’t bother adding description but that’s not required). Since it’s successful, you should see the log “Nice! We made it past the validation.” as well as a 200 response along with the JSON body echoed back.

Next, let’s prove that our validation is working by testing this invalid body:

curl -v localhost:3000/products \
-X POST \
-H "Content-Type: application/json" \
-d '{ "name": "The Imitation Game", "price": "$19.99", "category": "movie" }'

I receive a 400 status response with the following JSON body:

{"error":"price must be a `number` type, but the final value was: `NaN` (cast from the value `\"$19.99\"`)."}

Perfect! Whether or not you want to return something like this to the user is a different story, but now we know that we can successfully capture validation errors from our schema. Note as well that there is no log on the server that says “Nice! We made it past the validation.” The middleware returns the 400 response before any of the route handler code can run, which is exactly the outcome we wanted.

POST and PUT requests will work very similarly. The only difference with PUT is that you’ll likely have a path variable to represent the id of the resource being updated (e.g. PUT /products/:id). In that case, you’ll still use the request body with the same productSchema we defined.

We’ve achieved our goal of failing as early as possible if a request hits our API to POST or PUT an invalid object. I like this solution because it separates the concerns of validating the object vs. any business logic on that object nicely. The simple case is taken care of, but let’s look at a couple more interesting features we can use in yup for more complex schemas.

5. Nested Objects in yup Schema

So far, our product object only has simple field types like numbers and strings. How does a schema look with a more complex type? Let’s add another field to our product model called locations. The locations key will contain an array of objects representing the locations at which the product is sold. Each location will have the following fields:

• city: the city of the location
• state: the 2-letter state code for the location

Luckily, we can nest yup objects and keep our schema clean by separating the sub-object (location) information. Here’s our updated validator.js file for our products routes:

We now have a new yup schema called locationSchema which mirrors the description above. I used a feature on yup’s string api called matches — this takes a regex and ensures that the string is valid relative to the regex. This isn’t a perfect solution for the 2-letter state codes, but at least we’re checking that we have exactly 2 capital letters.

On line 16, we tie the 2 schemas together. productSchema now has locations, which is an array of objects that must follow our locationSchema. Note that the value is not .required(), so a product doesn’t need to have locations to be valid. Here’s a test which will pass validation successfully:

curl -v localhost:3000/products \
-X POST \
-H "Content-Type: application/json" \
-d '{ "name": "The Imitation Game", "price": 19.99, "category": "movie", "locations": [{ "city": "New York", "state": "NY" }, { "city": "Denver", "state": "CO" }] }'

What happens here?

curl -v localhost:3000/products \
-X POST \
-H "Content-Type: application/json" \
-d '{ "name": "The Imitation Game", "price": 19.99, "category": "movie", "locations": [{ "city": "New York", "state": "NY" }, { "city": "Denver", "state": "Colorado" }] }'

We get a 400 with a useful error message:

{"error":"locations[1].state must match the following: \"/^[A-Z]{2}$/\""}

The full state “Colorado” was sent rather than it’s 2-letter code “CO,” which triggered a validation error.

6. Conditional Logic in yup Schema

Let’s do one final example. What if we want to add a subCategory field to our product model, but it’s only required for certain category values? For example, what if the sporting goods category requires a sub-category like basketball or football, and similarly the electronics category requires a sub-category like laptops or tvs? We’ll assume no other category requires a subCategory.

We can handle this pretty easily in yup. Here’s another updated version of our schema:

We take advantage of yup’s .when clauses. This reads as follows: “subCategory is a string… when the category field is ‘sporting goods,’ we know that subCategory is a required string, and the same holds true if category is ‘electronics’ — however, if neither condition holds true, subCategory is not required.”

Let’s test a few curls…

curl -v localhost:3000/products \
-X POST \
-H "Content-Type: application/json" \
-d '{ "name": "The Imitation Game", "price": 19.99, "category": "movie" }'

This first call is successful because category is movie, so there is no need to add a subCategory field. Let’s incorporate a category with new requirements:

curl -v localhost:3000/products \
-X POST \
-H "Content-Type: application/json" \
-d '{ "name": "The Imitation Game", "price": 19.99, "category": "sporting goods" }'

This example fails validation. category is “sporting goods,” but we don’t have subCategory included as specified in our new schema. Finally:

curl -v localhost:3000/products \
-X POST \
-H "Content-Type: application/json" \
-d '{ "name": "The Imitation Game", "price": 19.99, "category": "sporting goods", "subCategory": "basketball" }'

This is a successful call, because we use “sporting goods” again for category, and this time include a subCategory string as required. The idea is the same for the other interesting category “electronics” in our schema.

The .when option in yup provides a lot of flexibility. There’s even an escape hatch if you need more flexibility, where you can use a more generic function inside the when clause — see these examples in the docs.

Thanks for following along, and let me know if you spot any issues or have other suggestions to improve this workflow.