Express

Tutorial: Getting Started With Express

Express is a powerful framework that helps you write servers in JavaScript on Node.js. To demonstrate, we'll build a toy server that keeps track of a list of users.

Before jumping into our actual API, we'll need to do some setup.

Hello from Express!

Create a directory for your express project and change into it. Make sure you have yarn installed, then run the following terminal commands to install express in our project:

$ yarn add express express-async-errors

We've just installed two external libraries that will help us:

• express is the library we'll use to implement the server.
• express-async-errors is another library that will make error handling nicer. More on this later.

Now, create a file in the directory called index.js and give it the following contents:

const express = require('express');
require('express-async-errors');
const app = express();

app.get('/', (req, res) => {
  res.send('Hello from Express!');
});

const listener = app.listen(3000, () => {
  console.log(`Listening on port ${listener.address().port}`);
});

In about 6 lines, we've successfully implemented our first web server! You can run it by running node index.js in your terminal, at which point you should see "Listening on port 3000" printed to your console. The server will run indefinitely, listening for incoming HTTP requests from clients, until either you shut it down (on most computers you can do this by pressing Ctrl+C in your terminal window) or your computer shuts down. Open a browser and go to http://localhost:3000 in your address bar. You should see the text "Hello from Express!" on your screen.

Let's break down the example piece by piece:

Initialization

const express = require('express');

This line imports Express into our project. require is a built-in function that imports JS code in your files in Node.js, just like import in Python. You can import libraries you've installed through yarn like we see above, or import relative files by including . and .. in the file path.

require('express-async-errors');

This line imports the express-async-errors library, which is a utility library that will make error handling nicer. Without it, errors thrown from inside Promises are unfortunately not recognized by Express.

const app = express();

Here we initialize our server (by convention we call it app, short for application).

Handlers

app.get('/', (req, res) => {
  res.send('Hello from Express!');
});

This is where things get exciting: we define a route handler on our server. When the server receives an incoming HTTP request, it will route the request to a handler based on information like the request's method, path, and potentially other information (more on this in the Routing section). The handler will then be in charge of responding to that request. Most of our server-side logic will either be in handlers directly or in functions that are directly called by handlers.

This warrants looking at each piece in a bit more detail:

• app.get(...) defines a handler that responds to the GET HTTP method. Recall that every HTTP request must specify a method.. You can also use the app.post(...), app.put(...), and app.delete(...) methods to define methods that respond to those requests with POST, PUT, and DELETE methods. You can also use app.use(...) to register a handler that will respond to any HTTP method. By default, when you use a browser to go to a web page, it sends an HTTP request to that server using the GET method. This is why we can see that our handler is called when we visit our server in the browser.
• The first argument, /, is the path that our handler should respond to. Paths in HTTP requests look like paths in the filesystem on UNIX-like machines, and they sometimes mean the same thing. If you use a static web server, accessing a path like /myfolder/myfile.txt will actually look for a folder called myfolder, then for a file called myfile.txt on the server. We'll be implementing a dynamic web server so the paths in our API won't correspond to actual files on our computer.
  • / is a special path that denotes the root of the server. When a client requests http://mywebsite.com, they'll be implicitly directed to the root of your server. localhost is a special address that your computer uses to refer to servers that are running on your machine.
• Finally, the second argument to app.get(...) is our handler: a callback that is executed whenever a matching request comes in. The arguments to this callback are req, short for Request, and res, short for Response.
  • req represents the HTTP request that was received by the server. You can access properties like the request method, path, headers, query string, and body by accessing req.method, req.path, req.headers, req.query, and req.body. All of these are available by default with the exception of req.body, which will require a little bit of additional configuration that we'll see shortly.
  • res represents the HTTP response that the server will send back to the client. You can use methods like res.status() to modify the HTTP response code, res.send() to send back an HTML string, and res.json() to send an object that Express will serialize to JSON for you.

Try adding console statements to log the request properties for requests you receive! Make sure to stop the server with Ctrl+C before restarting it. You can refresh your browser tab to send a new request to your server. You can test receiving query params by sending a request like http://localhost:3000/?mykey=myvalue.

Listening and Serving

const listener = app.listen(3000, () => {
  console.log(`Listening on port ${listener.address().port}`);
});

Finally, this last bit instructs Express to loop infinitely and listen for incoming requests. You can specify a port here (3000 in our case), which will allow multiple servers to listen on the same machine (so long as they all have a unique port).

Our "Database"

Now that we have the basics covered, we'll move on to actually implementing our server. We'll simulate a database by using a file on disk. In our project directory, create a file called database.json and give it the following contents:

[]

Next, add the following import to the top of the file:

const fs = require('fs'); // This will let us read and write files on our filesystem.

Under the const app = express(); declaration, add the following two methods:

/**
 * Read all the users in our database.
 */
const readDatabase = () =>
  new Promise((resolve, reject) => {
    fs.readFile("database.json", (err, data) => {
      if (err) {
        reject(err);
      } else {
        resolve(JSON.parse(data));
      }
    });
  });

/**
 * Overwrite all the users in our database.
 */
const writeToDatabase = (new_users) =>
  new Promise((resolve, reject) => {
    fs.writeFile("database.json", JSON.stringify(new_users), (err) => {
      if (err) {
        reject(err);
      } else {
        resolve();
      }
    });
  });

By default, fs functions read and write files asynchronously using callbacks. Here, we've simply converted them to use Promises to make them nicer to use.

In the real world, using files for databases like this is a very bad idea because it's easy for multiple, simultaneous requests to interfere with each other (which could potentially corrupt our file or yield incorrect results). Real databases are designed with this need for concurrency in mind, but they are out of scope for now.

Note that our writeToDatabase function completely replaces the contents of our file, so if we want to do something like add a single user, we'll first need to read the existing contents so we can append our new user to the end of our list.

Using our database

With our "database" in place, we're ready to start defining some handlers. Replace our existing handler with the following:

app.get("/users", async (req, res) => {
  const users = await readDatabase();
  res.json(users);
});

Note that we've made our handler callback async, which lets us use await when we read from our database.

You should now be able to go to http://localhost:3000/users on your browser, at which point you should see [] on your screen. Your server is now reading from the database and returning the results to its clients!

Testing your API

Before we go further, we'll need to discuss how best to test your API. You can use your browser to run simple GET requests, but for anything beyond that, you'll either want to use something like the Fetch API or Postman. I recommend Postman for a good mix of simplicity and power; see if you can call our GET http://localhost/users endpoint from it.

CORS

If you decide to use the Fetch API, you should add the following code underneath your "database" code. This is due to CORS, which is a security measure in your browser that by default will disable network requests to different sites. If, for example, you have an HTML file where you've implemented an API client using Fetch, your browser will interpret this as a different website than your server. Different ports are also considered different websites (so http://localhost:3000 is considered a different website than http://localhost:3001).

We can tell Express to enable these requests despite coming from different sites with the following middleware (more on middleware later):

app.use((req, res, next) => {
  res.setHeader("Access-Control-Allow-Origin", "*");
  next();
});

Creating users

Let's add a handler to create new users in our database. First, we'll want to tell Express to enable reading from request bodies. Add the following line just below our database functions:

app.use(express.json());

This is an example of something called middleware, which is a central concept in Express. More on this later.

Now we're ready to define our handler for creating new users. The convention here is to use GET /users to fetch all our users and POST /users to create a new user. This looks like the following:

app.post("/users", async (req, res) => {
  // We need a unique identifier for the new user. We can "generate" one by
  // taking the highest known user ID, then adding 1 to it.
  const new_id =
    users.length === 0 ? 1 : Math.max(...users.map((user) => user.id)) + 1;
  const new_user = {
    id: new_id,
    name: req.body.name,
    email: req.body.email,
    is_admin: req.body.is_admin
  };

  const users = await readDatabase();
  const new_users = users.concat(new_user);
  await writeToDatabase(new_users);

  res.json(new_user);
});

This is our first example of some more complex server-side logic. Namely:

• In general, everything you store in any database should have a unique ID. We're using numbers in this case, starting at 1 if our database is empty and otherwise adding 1 to the highest known ID.
• We construct our user object by reading their name, email, and admin status from the request body, alongside the ID that we just generated.
• We read the current list of users from the database, add our new user to the list, then write everything back to the database.
• By convention, after successfully creating our user we return it to the client. This is often useful so the client will know which ID we assigned to the user we just created.

Try calling our API with something like the following request body:

{
  "name": "Jacob",
  "email": "jacob@example.com",
  "is_admin": false
}

You should find that a new user with the ID 1 is returned. Create a couple more users with varying data, then call the GET /users method to retrieve all of the users you just created.

Deleting users

Let's add a handler to delete users. By convention, we'd like our endpoint to look something like this, where 1 is the ID of the user we'd like to delete:

DELETE /users/1

However, we can no longer represent our path as a fixed string. Instead, our path must accept a single ID as a parameter so the server knows which user to delete (as deleting all of the users in the database is probably not what the client wants to do). Express handles this by using route parameters:

app.delete("/users/:id", async (req, res) => {
  const users = await readDatabase();
  const new_users = users.filter((user) => user.id !== Number(req.params.id));
  await writeToDatabase(new_users);
  res.end();
});

Now, our route has special syntax at the end ":id" that Express will recognize as a dynamic route. This is commonly used to capture numeric parameters, but will also capture certain strings (with the notable exception of strings with periods . and hyphens -). Once captured, parameters can be accessed in the handlers through req.params like we see above.

Try calling the new endpoint to delete some of the users we've created.

Routing

To recap, we can register different request handlers to respond to various paths and HTTP verbs. Express also lets us define handlers to respond to multiple HTTP verbs and dynamic paths that accept parameters. It's entirely possible that you'll define two or more handlers that match a single request. For this reason, Express will remember the order that you defined your handlers; if more than one handler matches a request, Express will direct it to the first one you defined. To see this, you can add a dummy handler below all of the others for DELETE /users/jacob that responds with some garbage text. Since the DELETE /users/:id handler is already defined above, any request to DELETE /users/jacob will trigger the dynamic endpoint and we'll never reach the dummy handler we just created.

If you find that the handler you want to call isn't being called, this is probably why.

A "not found" handler

You may have noticed that if you try to call a method and path for which we haven't defined a handler, you'll get an error message like Cannot "METHOD" "PATH". Express provides a default handler out of the box for when endpoints don't exist. Our API currently returns JSON for all of its existing endpoints, so let's override this behaviour such that our "not found" response is also JSON. Under all other handlers, add the following:

app.use((req, res) => {
  res.status(404).json({ error: "endpoint not found" });
});

Since we don't define a path and we use app.use to accept any HTTP method, this handler will serve as a catch-all to process any requests that don't match our defined endpoints. Try it out!

Middleware

Middleware in Express are a powerful way to add functionality to multiple handlers to your server. In real Express servers, middleware are used to implement things like authentication, request authentication, and more.

Up to this point, we've had no way to tell which clients have made requests to our server. In the real world, we often want to log requests that are made so we have a sense of how much load our server is currently under. Under app.use(express.json()), add the following code:

const requestLoggerMiddleware = (req, res, next) => {
  const started = Date.now();
  res.on("finish", () => {
    const took = Date.now() - started; // This measures how long the request took to complete.
    console.log(`${req.method} ${req.url} ${res.statusCode} ${took}ms`);
  });
  next();
};

app.use(requestLoggerMiddleware);

There are two parts here:

• First, we define our middleware. This is simply a function that accepts the request and response, just like our handlers, only it also accepts a third parameter, next. In Express, middleware form a chain, which is essentially a pipeline of middleware functions. Requests go through each middleware function in the chain before reaching a handler. The next function tells Express that the current middleware has completed, so the client request will continue in the next registered middleware (just like with Routing, the order in which you register middleware functions is important).
• Next, we register the middleware function we just created.

Every middleware should either call the next() function or end the request through a command like res.json() or res.send(). You might want to do the latter if the user needs to be logged-in but isn't, in which case your authentication middleware would immediately respond with an error instead of continuing down the middleware chain.

If you omit the call to next() in a middleware and fail to end it using a res method, the client's request will hang. This is generally undesirable.

Now, when you call your API, you should see a record of your requests printed in the server's terminal window.

Error Handling

Errors can happen for any number of reasons. As an example, maybe our database file gets deleted. Try renaming it to something like "database_removed.json" and fetching our list of users.

You'll likely receive a response like:

Error: ENOENT: no such file or directory, open 'database.json'

Notice that by default, Express takes the error and directly returns it to the user. In addition to not being useful for our users, this is a security vulnerability because you're giving potential hackers detailed information on what went wrong. In real servers, any such clues can help hackers break into our servers and read privileged data that they shouldn't have access to.

The proper procedure for error handling is:

1. Log the error somewhere, so we developers know it happened.
2. Return a generic error response to the user that doesn't give any more information than necessary.

To do this, add the following code below our "not found" handler:

app.use((err, req, res, next) => {
  console.log(err);
  res.status(500).json({ error: "something went wrong :(" });
});

Try it out!

Response Codes

You'll likely have noticed that in our "not found" and error handlers, we're calling the res.status() method. This sets the HTTP response code, which is a standard of communicating the status of the request. Here is a detailed list of the different response codes used by servers.. The main ones to keep track of are:

HTTP Code	Title	Meaning
200	OK	The request was successful, all is well.
404	Not Found	The requested resource could not be found.
500	Internal Server Error	Something went wrong on the server, like our database file going missing.
418	I'm a teapot	The client has received an HTTP request to brew coffee, which is cannot fulfill because it is a teapot.

In general, response codes from 200 to 299 indicate success, 300-399 indicate some kind of redirect (the browser can often follow these automatically!), 400-499 indicates some problem on the client's side, and 500-599 indicates some problem on the server's side.

Next Steps

Our tutorial is complete, but our API is far from production ready. In particular:

• There's nothing to indicate when the client has tried to delete a user that doesn't exist. Ideally, the server would return a 404 error in this case.
• Right now the client can only fetch all users in the database at once. A common feature is to fetch a single user by their ID. See if you can implement this.
• Our POST handler assumes that the client has given all the required information in its request body, but there's no guarantee of this. Clients can be from anywhere in the world since our server will ultimately be hosted on the internet, so there's nothing stopping someone from sending a request without a name, email, or is_admin email. Request validation is the process of getting our server to verify that requests contain valid data. See if you can add this, and return a 400 Bad Request error if you receive a malformed request. In particular:
  • Make sure req.body.name exists and is a nonempty string.
  • Make sure req.body.email exists and is a nonempty string.
  • If req.body.is_admin is not provided, assume it is false. Make sure it is stored in the database as such.