Serverless Next.js

[timneutkens]

This does not change existing behavior.

building to serverless is completely opt-in.

• Implements target: 'serverless' in next.config.js
• Removes next build --lambdas (was only available on next@canary so far)

This implements the concept of build targets. Currently there will be 2 build targets:

• server (This is the target that already existed / the default, no changes here)
• serverless (New target aimed at compiling pages to serverless handlers)

The serverless target will output a single file per page in the pages directory:

• pages/index.js => .next/serverless/index.js
• pages/about.js => .next/serverless/about.js

So what is inside .next/serverless/about.js? All the code needed to render that specific page. It has the Node.js http.Server request handler function signature:

(req: http.IncomingMessage, res: http.ServerResponse) => void

So how do you use it? Generally you don't want to use the below example, but for illustration purposes it's shown how the handler is called using a plain http.Server:

const http = require('http')
const page = require('./.next/serverless/about.js')
const server = new http.Server((req, res) => page.render(req, res))
server.listen(3000, () => console.log('Listening on http://localhost:3000'))

Generally you'll upload this handler function to an external service like Now v2, the @now/next builder will be updated to reflect these changes. This means that it'll be no longer neccesary for @now/next to do some of the guesswork in creating smaller handler functions. As Next.js will output the smallest possible serverless handler function automatically.

The function has 0 dependencies so no node_modules are required to run it, and is generally very small. 45Kb zipped is the baseline, but I'm sure we can make it even smaller in the future.

One important thing to note is that the function won't try to load next.config.js, so publicRuntimeConfig / serverRuntimeConfig are not supported. Reasons are outlined here: #5846

So to summarize:

• every page becomes a serverless function
• the serverless function has 0 dependencies (they're all inlined)
• "just" uses the req and res coming from Node.js
• opt-in using target: 'serverless' in next.config.js
• Does not load next.config.js when executing the function

TODO:

• Compile next/dynamic / import() into the function file, so that no extra files have to be uploaded.
• Setting assetPrefix at build time for serverless target
• Support custom /_app
• Support custom /_document
• Support custom /_error
• Add next.config.js property for target

Need discussion:

• Since the serverless target won't support publicRuntimeConfig / serverRuntimeConfig as they're runtime values. I think we should support build-time env var replacement with webpack.DefinePlugin or similar.
• Serving static files with the correct cache-control, as there is no static file serving in the serverless target

[lucleray]

This is amazing 🙏

[tonyxiao]

This is awesome. I would love an example of how to deploy dynamic next.js to netlify with function support also - (https://www.netlify.com/docs/functions/).

[jesstelford]

@timneutkens

Compile next/dynamic / import() into the function file, so that no extra files have to be uploaded.

Does this mean client side import() calls are no longer dynamically loaded? Ie; is the client side for a page now a single static bundle when using the 'serverless' target?

[timneutkens]

@jesstelford No, this PR changes nothing to the client compilation, only the server compilation, meaning that the serverless output can be ran with just the single file.

[jesstelford]

🎉 Thanks for clarifying!

♥️ The changes.

[Vadorequest]

I'll try to migrate my existing Next.js app running on 5.0.1-canary.16 to the latest of Next and see how I can use this new target with AWS Lambda, in the following days.

Not that the production doesn't work well, but the local development is a real pain (express, next, serverless-offline, hmr, all those don't just play well with my current setup)

Kinda worried about environment variables, as I rely on them though. And I use custom stages as well, I remember Next wasn't so friendly about that, by forcing either production or development.
Who can I ping when I run into troubles, and where? :)

[adamszeptycki]

question what about the CSS and JS deps?

Is there a step by step tutorial how to deploy to lambda aws?

[Skaronator]

@adamszeptycki I think the assetPrefix could help with this: https://nextjs.org/blog/next-7/#static-cdn-support

But a example how to deploy to lambda with serverless would be well welcome (including s3+cloudfront for assets)

[jakewies]

Does the target: 'serverless' config support any existing environment variable solution for nextjs at the moment?

[Skaronator]

@jakewies Yes you can.
next.config.js

module.exports = {
  target: process.env.TARGET,
};

package.json

  "scripts": {
    "dev": "next",
    "now-build": "cross-env TARGET=serverless next build",
  },

[chirag04]

@timneutkens noticed we don't follow the same build directory structure as server target for the serverless target. more specifically, we don't output pages dir under the build_id

here is the build dir structure for server

.next
    server
         static
               <build_id>
                     pages
                           index.js

vs serverless target:

.next
    serverless
          pages
              index.js

having that pages dir under a build dir can be useful for a variety of use cases. One such is that it allows serving different versions of the codebase at the same time for testing purposes. thoughts?

[maggo]

@Skaronator I think @jakewies meant passing configuration down to the runtime environment, like next/config does.

I've got a temporary solution using webpack.EnvironmentPlugin to replace the env variables at build time, as @timneutkens mentioned in the PR description. It's working but there probably should be an integrated solution to it.

webpack(config) {
  config.plugins.push(
    new webpack.EnvironmentPlugin(['FOO', 'BAR'])
  );
  return config;
}

and then access in your code like always process.env.FOO

[Vadorequest]

@maggo For the record, I believe that's what https://github.com/mrsteele/dotenv-webpack is about.

I'm also struggling with ENV variables. The very particular way Next.js handles them makes it unfriendly to work with, from my experience.

For instance, one of my devs created a SENTRY_DSN env variable today, and stored its value in the .env file that is git tracked and shared amongst developers. He also added require('dotenv').config({ path: '.env.${process.env.NODE_ENV}' }) in the next.config.js file, so that those variables would be loaded, and then added SENTRY_DSN: process.env.SENTRY_DSN in next.runtimeConfig.js inside the exported publicRuntimeConfig:

const serverRuntimeConfig = {};

const publicRuntimeConfig = {
  GROUP_NAME: process.env.GROUP_NAME,
  NODE_ENV: process.env.NODE_ENV, // XXX Used in utils/env
  SENTRY_DSN: process.env.SENTRY_DSN
};

module.exports = {
  serverRuntimeConfig,
  publicRuntimeConfig,
};

I really don't like it. First, the design is broken because loading require('dotenv').config({ path: .env.${process.env.NODE_ENV} }) assumes you have a specific env file for your environment, since I rely on the .env, it didn't find my SENTRY_DSN env variable because loaded the wrong file and failed silently.

Also, this particular env variable isn't a secret, it's the same in localhost, staging, production and therefore I don't care about git tracking it, it just makes things easier and isn't a security issue. So I don't even see the point storing it in a .env file. (but that's another issue I guess)

Finally, do you see what's happening here? I gotta define a ENV var in a .env file, which is then loaded by next.config.js, injected by next.runtimeConfig.js, then use the next/config getConfig method to retrieve it, to finally be able to use it. Why? Because I need it on the frontend part of the app, and there is no other way to load env vars for the frontend. If I would only need it on the backend, I could just reference it as we usually do in a node app, using process.env.SENTRY_DSN.

See how overcomplicated this all got? Not transparent, fails silently, overcomplicated, multiple references and configuration for a single ENV var, and a hell of a brainfuck to debug when it doesn't work because it's a needle in a haystack issue.

So, I tried hard to find another way and didn't fine anything better within a hour of work. But in the process of finding for a better solution, I found https://github.com/mrsteele/dotenv-webpack and I believe it would just solve my issues. It fixes the issue completely differently, by replacing the references to any ENV var and hardcoding them in the bundled version. It's not something I usually like, but it's most likely a better way to handle ENV vars than relying on the publicRuntimeConfig and alike, and is definitely more straight-forward, IMHO.

So, if you have some feedback regarding its usage, I'm interested. You seem to consider this as a temporary solution, while I'm thinking migrating towards it because I haven't found any better solution really.

[maggo]

@Vadorequest yeah I didn't like the next/config way either, though it does solve the issue with different environment variables for runtime and server build. I'd prefer the dotenv/EnvironmentPlugin way as it's such a common pattern, see Create React App and Gatsby as two static site builders that use this pattern.

I meant temporary as in my own local configuration and I'm hoping that we'll see something like this in next soon. Maybe we should set up an RFC for this.

[mkondel]

@timneutkens How would you pass any parameters from the server to the functions? I am able to get the actual values from the request, but nextjs never sees them.

I tried to do that using the page.render() call, but I think it only accepts req and res. I want to be able to use url params like /item?something=123, and also handle custom routes like /item/:id. This is the code I was trying to use:

page.render(req, res, '/item', {something: '123', id:'someID'})

In my view code, I can then grab both 'something' and 'id' from props.query like so:

function Item({ query }) {...}
export default Item

[timneutkens]

It should be handled on the proxy level. As documented page.render only accepts req and res. It's completely different from the custom server API.

This is to prevent many of the common pitfalls people currently have when using the custom server API.

[mkondel]

Where would this proxy need to be? Wouldnt it still be before the page.render call? My understanding is that I need to wrap the functions in a serverless handler. For example:

exports.handler = async (event) => 
  event.path.indexOf('_next') > -1 ? handleStaticAssets(event) : await handleRoute(event);

handleRoute() uses page.render. This is why I am looking for a solution through this method. I have also tried using the functions produced by this PR directly as lambdas. That did not work, so I am following the bits and pieces I have been able to find.

Could anyone else please shed some light on how to pass parameters. Or is this impossible with the current implementation?

[timneutkens]

I'm not entirely sure what you're trying to do based on the code, but it seems like you're creating 1 lambda to handle multiple routes instead of having a separate lambda for each route.

req.url is parsed by the render function: https://github.com/zeit/next.js/pull/5927/files#diff-ece49b808112109a6e8f9a93eef072e8R51

This will then be passed onto the render function (including query). However the wrapper should only be in charge of converting the incoming payload to a Node.js http req/res compatible format, in case of AWS Lambda it'll look like this: #6070 (comment)