[RFC] API routes

[timneutkens]

Feature request

Is your feature request related to a problem? Please describe.

Currently a lot of users create a custom server.js to handle creating their API.

However this causes some issues:

• React rendering is sync, meaning that while rendering performance for requests to the API can differ
• The API and routing solely depend on the custom server, meaning that it can only be deployed as one server
• Because it's one server the bundle size of this server is quite heavy (slow bootup)
• Because it's one server your complete API goes down if one handler has a uncaught exception

There's also some other ergonomic issues:

• Because the server runs outside of Next.js it is not compiled, meaning you can't use import / export
• Because import / export is not supported we've seen many users start compiling their custom server.js which introduces another build step, that is different from Next.js. Furthermore they add nodemon, to sort of get hot reloading of the server file, which means Next.js is rebooted every time you make changes to the API, losing all compilation state.
• We've seen this extra build step being implemented wrong many times, for example in doing so tree-shaking was disabled because of Babel options being different for client/server compilation.

The solution to most of these problems revolve around 3 issues:

• Dynamic routing
• Being able to create APIs
• Middleware (see [RFC] server(less) middleware #7208)

Describe the solution you'd like

This proposal goes into the API creation part of the issue.

The solution that I have in mind involves a few small changes to Next.js. It comes down to:

• The user will be able to create a api directory inside of pages. This is because pages/api has to do with routing and we might change pages to routes in the future.
• This api directory is automatically mapped to /api (as expected)
• So for example you can create api/posts.js and it will be mapped to /api/posts
• The handle will look like: export default (req, res) => this means that you can use any of the http server frameworks like express micro etc.
• In case of micro I think we should change the API of the programmatic usage to this:

import micro from 'micro'

export default micro((req, res) ⇒ {
   return 'Hello World'
})

Current invoking micro() will return a Node.js http.Server which doesn't work well with just req, res as a handler. This also ties into using micro standalone with @now/node . Doing this will allow us to deprecate micro-dev too in favor of using Next.js / now dev + @now/node

Usage for express would look like:

import express from 'express'

const app = express()

export default app

Note that express would not be in charge of routing, just the request handling, meaning that in most cases you actually want to use micro as it's much smaller and has better performance.

[kylehotchkiss]

This is a great idea. I've been using next.js for well over a year and have needed server.js for custom routing nearly the entire time. When I started moving some projects over to Zeit Now, i realized my server.js convention didn't work and begun to use routes in the now config file. I then needed an API to do a fs read, and since server.js wasn't there (a convention i've been somewhat used to), I had to learn how to make my API the Now way. I'm enjoying using Now, but having to learn the new convention for just that platform was somewhat of a bummer.

Big thing I like about this solution: it reduces lock-in to Now service slightly since there'd be a standardized way to do parameterized routing and APIs within next.js. Woo! (and as a marketing thing, it'd be easier for people to bring their projects to Now later on down the line since both apis and routing are standardized)

[damusnet]

This is interesting. I would love to do away with server.js yet I always find myself needing it eventually. I'm hoping that now dev will alleviate some of the use cases but that's only for use with now hosting. At work we will continue to have to come up with our own solution.

Some things I currently use server.js for:

1. URL rewriting aka vanity URLs (/posts/1)
2. API proxy (mostly in dev though)
3. Trailing slash removal (could be done in Next.js I suppose)
4. Asset prefix voodoo (we have to serve everything from /new)
5. Healthcheck and logging (would be moot if there was no server)

[timneutkens]

@damusnet

1. That's the next proposal I'm going to post 🤐
2. You can do that on /api I think.
3. That would be handled if you had global middleware right? Check this [RFC] server(less) middleware #7208
4. I'd like to see an example, could you share the custom server on DM, thanks!
5. healthchecks can be handled by /api/healthcheck or similar right?

[Janpot]

Would it make sense then to add a next version of fetch, that calls those api functions directly when used serverside, instead of hitting the network? (Here's one reason to go for a ServiceWorker/fetch style interface, rather than a node.js http style interface 🙂 )

[damusnet]

@timneutkens yes, all great points. I do think that in between URL rewriting (next proposal), this proposal and middleware proposal you are covering all bases. That is super exciting! Great work!

[leerob]

Having an /api directory inside of /pages would greatly improve the DX for me 😍

[developit]

Echoing @Janpot here - fetch('/API/foo') from within Components/getInitialProps should allow accessing API routes directly.

[BasitAli]

I like how the api interface looks very similar to serverless functions. To make the most use out of those we could look into maybe exporting Express routers (or whatever equivalent micro routers are) which would give us the power of defining custom paths (with middlewares) extending upon the filename. Or, maybe a configuration file (similar to serverless.yml) for defining the routes (which may not be the preferred method given the how next uses pages for routes).

[timneutkens]

I like how the api interface looks very similar to serverless functions.

It is, because it'll be deployed as serverless functions.

To make the most use out of those we could look into maybe exporting Express routers

As said in the proposal "Note that express would not be in charge of routing, just the request handling, meaning that in most cases you actually want to use micro as it's much smaller and has better performance."

Or, maybe a configuration file (similar to serverless.yml) for defining the routes (which may not be the preferred method given the how next uses pages for routes).

The file path is the configuration here. We'll soon introduce dynamic routing to cover most cases.

[lucleray]

Echoing @Janpot here - fetch('/API/foo') from within Components/getInitialProps should allow accessing API routes directly.

@developit
In that case, isn't it better to do fetch(fooApiFunction) and import the function (import fooApiFunction from './api/foo'), so that we can only bundle the necessary api functions for each serverless app pages ? I'm wondering how feasible this is 🤔

[ericclemmons]

Im happy to see this direction, as it's been a very helpful pattern with https://github.com/ericclemmons/polydev

Some suggestions, from my experience:

• Keep functions simple. I support Express out of the box (because req.params and res.json are simpler that way), but that's too opinionated for Next. (As noted, that doesn't stop a user from exporting express()).

• I find "index" routing (e.g
  /api/user/:id/index.js) better for organization. I know Next converts about.js into about/, but server routes often have shared logic and middleware (auth), which you don't want to automatically turn into a route. (You'll need to find a way to ignore these files, otherwise there'll be ../../../../auth.js all over the place).

• Done well, I could see Next for pages+api to potentially replace the need for an explicit now.json, assuming builders were supported as they are for "now dev".

• I also agree that path-based fetch should be a goal.

(Sorry, on mobile so poorly formatted)

[Janpot]

@lucleray I was kind of thinking in the direction of having an interface that aligns more closely to fetch, but that also translates well to node style req/res, along the lines of:

// /api/test.ts
import serve, { Response } from 'next/serve';
export default serve(async ({ request }) => {
  return new Response('hello world');
});

on the server, serve would return a function (req: IncomingMessage, res: ServerResponse) => void to run as a serverless function as it exists now. This would serve requests to the usual fetch API on the browser side.

import fetch from 'next/fetch';
// ... in getInitialProps
  const response = await fetch('/api/test');

During server render though, when called in getInitialProps, that code would do something differently. It would dynamically import /api/test.ts like import(pagesDir + '/api/test') and directly call the function that was passed to serve(). Since we already defined the handler in terms of the fetch API, I guess this should be low overhead. Calls to absolute urls just do a regular fetch as we know it. Basically the function would be compiled twice, once as a serverless request handler, and once to be imported in the next.js server bundle.
Not sure if it makes sense and if it's completely feasable, it's still a bit of a rough idea. But maybe it can spark some discussions.

edit

🤔 I guess in getInitialProps you could also make fetch directly bridge to (req: IncomingMessage, res: ServerResponse) => void as well, you probably don't exactly need an inverse fetch abstraction. @lucleray going over your comment again, I now realize this is probably exactly what you are saying.

[jescalan]

Is there any reason the api directory needs to be inside pages? It feels like it would make more sense to me to have it be outside 😀

[Janpot]

@jescalan It seems like they're thinking of renaming "pages" to "routes".

[jescalan]

I saw that, but this is a massive breaking change and probably will take a bit to roll out. I feel like just keeping api outside of pages would make it non-breaking, be a still very clear structure, and make the change to routes no longer needed...

[PullJosh]

To play devil's advocate: If api belongs outside of routes, people could also create a routes/api directory (in addition to the regular /api directory) and cause a conflict.

I still prefer putting api outside of pages, but it's a problem to consider.

[timneutkens]

Initially my idea was a separate directory but the conflicts + significantly more complex compilation + not having a single source of truth for routing is not worth creating another directory. Also people could get confused by api as a separate directory and put more than just routes in there. We've even seen this with pages where users started adding components inside pages.

Furthermore adding api would be breaking in terms of we'd suddenly be exposing code that is in that specific directory, whereas if you had pages/api (unlikely) before it was already exposed.

Changing pages to routes is not a breaking change btw. We'd just support both and document routes only. Still not sure about it though.

[Janpot]

@timneutkens have you give any thought on how to pass environment variables to these routes? It seems like build time configuration works with the current implementation in canary, but unlike with pages, it feels like some runtime config should be possible.

[lucleray]

One of the limitation with the api/ folder is that it requires you to have your api under /api (and consequently you can't have pages under /api).

Maybe it's possible to decide in each file whether it's a page or an api endpoint, like this:

// pages/index.js

import { api, page } from 'next'

// an api endpoint
export default api((req, res) => 'hello')

// a page
export default page((props) => 'hello')

It also helps to:

• type pages and api exports
• show a warning if someone adds in pages/ a file that's not supposed to be there (for example a component or a lib file)
• configure at a granular level the way a page or an api is "optimized" at build time by passing an options object

[sergiodxa]

I agree with @jescalan @PullJosh to move api outside pages, a reason for this could be what if someone has a real folder called api, probably to document its product API, inside pages? This would break that website, and basically make impossible to have a /api* URL rendering HTML, except of course your API handle that manually.

An example website using /api* for rendering HTML could be Material-UI (which is using Next.js right now). Other examples includes Airtable, Akamai, angular.io website, remove.bg or Ghost.

Maybe Next could be to have both api and pages and use pages when the Content-Type or/Accept expect text/html (like a browser) and default to the api if it' not specified the Content-Type/Accept or is another value.

[iamstarkov]

I would like to chime in about one common "non-pages but also non-api" use case—oauth2 flow.

You dont need client side pages for it, but its also not an api.

https://github.com/iamstarkov/topics-manager/tree/master/pages/auth/github

Right now with Next you are forced to use getInitialProps on both start and callback stages of OAuth2 flow, which is acceptable. Although Next will allow non react routes, then it maybe worth to look into less friction OAuth support

[zorrme]

What about support for middleware like passport ?

[talbet]

I've started playing with the API routes on the canary channel and already they have made the dev experience much nicer. One comment I would make: there may need to be a way to prevent api routes from being disposed when using the development server.

I have several API routes that cache their requests to speed up subsequent fetches. However, without increasing maxInactiveAge and pagesBufferLength for all pages, the api routes get disposed of and the cache needs to be re-created.

[Timer]

API routes have been released in Next.js 9! Read the blog post to learn more and give them a try!

Thank you to everyone who provided feedback.

[slavb18]

Is there ideas to make regular page to listen for api (POST/PUT/.. etc requests)
e.g.

GET /posts/p leads to rendering (using getInitialProps)
POST /posts/p leads to request procession (using some variant like getInitialProps: processRequest (req, res) )

[huv1k]

What is the reasoning behind it @slavb18?

[slavb18]

Initially I was looking for basepath support for api #4998, becouse currently API route is unprefixed and cannot be used in multi-application-deployment case without redirecting proxy.

But after that I realized that really I need page controllers.
My workflow is

1. USER gets some page
2. USED do some action on page

this resolves to

1. GET /path/to/page
2. POST /path/to/page

So I need to GET and POST on page URL.

Is there some example/workaround to make custom routing for POST?

[slavb18]

May be there should be more generic conception instead of "api/pages" - Web Resource

E.g.
/pages/XXX used to auto-generation of GET enpoints (implemented)

/resources/XXX could be used as handler for GET/POST/PUT/other request types

Then

GET /YYY routes to /resources/YYY.js (is exists GET handler), otherwise to /pages/YYY

POST/PUT/etc /YYY routes to /resources/YYY.js

where YYY.js could contain:

per-http-methods, like get(),post(),put(),options(), etc
or default "all methods" handler

[danechitoaie]

According to the docs https://nextjs.org/blog/next-9#api-routes we get req refers to NextApiRequest which extends http.IncomingMessage

And NextApiRequest does have a body which I assume it will be populated only on POST, PUT, requests. You could post a JSON and see what needs to be done based on the data inside the JSON (ex: {"action": "add/update/delete/etc", "data": {...}}

Other option is that since they extend the http.IncomingMessage you should have access to req.method (see https://nodejs.org/api/http.html#http_message_method) so you can do your own routing.

Ex:

export default function handle(req, res) {
    switch (req.method) {
        case "GET":            
            break;
    
        case "POST":            
            break;
    
        default:
            // https://nodejs.org/api/http.html#http_response_writehead_statuscode_statusmessage_headers 
            // Set status to 400 bad request
            break;
    }
}

[slavb18]

According to the docs https://nextjs.org/blog/next-9#api-routes we get req refers to NextApiRequest which extends http.IncomingMessage

And NextApiRequest does have a body which I assume it will be populated only on POST, PUT, requests. You could post a JSON and see what needs to be done based on the data inside the JSON (ex: {"action": "add/update/delete/etc", "data": {...}}

Yes, I can handle any http method, but only on hardcoded path "/api"

This is enough if there is only one application on host. But in cause of multi-application scenario this leads to path conflicts. (eg. corporate infrastructure with lots of interacting microservices: /app1, /app2, each with own api)

Currently this can be fixed by proxy configuration (e.g. on Apache I can configure two separete ProxyPass /app1/api -> backend:port/api, /app1 -> backend:port/app1/ but this is workaround. Resource paths on backend should be symmetric to proxied app )

[djnorrisdev]

@timneutkens have you give any thought on how to pass environment variables to these routes? It seems like build time configuration works with the current implementation in canary, but unlike with pages, it feels like some runtime config should be possible.

This is more for people reading about the pages api features than a response to Tim.

As of me testing it today, that works from variables set in next.config.js, and those variables are consumed in /api/whatever as process.env.whatever.

[timneutkens]

In case of API routes you don't have to inline the variables at build-time, we're making it easier to use this pattern in development also: #11106

[kripod]

Echoing @Janpot here - fetch('/API/foo') from within Components/getInitialProps should allow accessing API routes directly.

@developit
In that case, isn't it better to do fetch(fooApiFunction) and import the function (import fooApiFunction from './api/foo'), so that we can only bundle the necessary api functions for each serverless app pages ? I'm wondering how feasible this is 🤔

I just stumbled across this discussion and wondering whether there's an idiomatic solution to this nowadays.

How may I interact with a database through getServerSideProps while making the same query available through API routes? Should I fetch data on the server side, ignoring the additional network costs?

Also, how should errors be handled when abstracting database queries away into separate files? My initial idea is using a discriminated union as the return type, e.g.:

import { PrismaClient } from "@prisma/client";

const prisma = new PrismaClient();

type QueryResult<T, E = unknown> =
  { success: true, data: T } |
  { success: false, error: E };

export async function findUserById(id: string): QueryResult<User> {
  const user = await prisma.user.findOne({ where: { id } });
  return user
    ? { success: true, data: user }
    : { success: false, error: "An error has occurred." };
}

[balazsorban44]

This issue has been automatically locked due to no recent activity. If you are running into a similar issue, please create a new issue with the steps to reproduce. Thank you.