server middleware

[jasonkuhrt]

Perceived Problem

• users resort to server.custom in every app to add express middleware
• server.custom should be a rare escape hatch
• server.custom widespread use would make Nexus framework de-facto coupled to express
• plugins need to be able to add middleware, but they cannot
• no middleware that spans serverless and serverful

Idea

• works like a FILO (first in last out) stack

• beforeware can short-circuit, aftwares of beforewares that ran will run

• await next to transition seemlessly from beforeware to afterware

• return to short-citcuit

  • in afterware has no effect, Nexus will log warning

• third parameter is ctx, intentionally mimicks position in resolver sig

• mutations to ctx are visible in resolvers, super simple

  • no more schema.addToContext

• plugins can extend req object for use at app level [1] via typegen [2]

  • so a heroku plugin could add types for the req headers heroku provides and likewise for a now plugin for now headers.

• plugins can extend res object for use at app level [1] via typegen [2]

• plugins can extend ctx object [3]

• everything you see here will work as well for a serverless deployment (lambda) as it will a serverful one (ec2)

• req/res/ctx augmentation by plugins will be visible in app middleware types

• app level augmentation of req/res/ctx in middleware will not be visible in later middleware since the type of middleware can be altered based on syntactic order.

  server.middleware((req, res, ctx) => { ctx.foo = 1 })
  server.middleware((req, res, ctx) => { ctx.foo }) // type check error, but actually correct

• app level augmentation of ctx in middleware will be visible in resolver sigs

• does not handle errors. Errors are fatal and handled by the framework. Approach akin to fastify

[1] "for use at app level" because we don't want plugins to start depending on features from other plugins, or do we?

[2] We cannot use global interface merging b/c we don't know if the plugin will be enabled or disabled. We resort to typegen.

[3] They already can, but ctx is only for resolvers right now. This point is saying that plugins will be able to modify context for middleware too.

Ideas from other middleware systems we align with

koa:

however a key design decision was made to provide high level "sugar" at the otherwise low-level middleware layer. This improves interoperability, robustness, and makes writing middleware much more enjoyable.

fastify

Uncaught errors are likely to cause memory leaks, file descriptor leaks and other major production issues. Domains were introduced to try fixing this issue, but they did not. Given the fact that it is not possible to process all uncaught errors sensibly, the best way to deal with them at the moment is to crash. In case of promises, make sure to handle errors correctly.

Fastify follows an all-or-nothing approach and aims to be lean and optimal as much as possible. Thus, the developer is responsible for making sure that the errors are handled properly. Most of the errors are usually a result of unexpected input data, so we recommend specifying a JSON.schema validation for your input data.

Note that Fastify doesn't catch uncaught errors within callback-based routes for you, so any uncaught errors will result in a crash. If routes are declared as async though - the error will safely be caught by the promise and routed to the default error handler of Fastify for a generic Internal Server Error response. For customizing this behaviour, you should use setErrorHandler.

Examples

Beforeware

server.middleware((req, res, ctx) => {
  const token = extractToken(req)
  ctx.user = {
    id: token.claims.userId
  }
})

Beforeware & Afterware

server.middleware(async (req, res, ctx, next) => {
  const start = Date.now()
  await next
  const end = Date.now()
  res.headers.set('x-response-time', end - start)
})

Beforeware short circuit

server.middleware((req, res, ctx) => {
  const token = extractToken(req)

  if (!token) {
    res.status = 401
	res.body = '...'
	return res
  }

  ctx.user = {
    id: token.claims.userId
  }
})

plugin augmnet req

module {
  interface NexusRequest {
    /* extend type here */
    foo: string
  }
}


export default NexusPlugin.create(project => {
  project.runtime((augment) => {
    augment.request({
      type: `
        foo?: string
      `,
      term(req) {
        if (math.random() > 0.5) {
          req.foo = 'bar'
        }
      }
    })
    augment.response({
      type: `
        foo?: string
      `,
      term(res) {
        if (math.random() > 0.5) {
          res.foo = 'bar'
        }
      }
    })
  })

Lifecycle

imagine three middlewares

normal

before:   middleware 1
before:   middleware 2
before:   middleware 3
graphql handler
after:    middleware 3
after:    middleware 2
after:    middleware 1

short-circuit ex 1

before:   middleware 1
before:   middleware 2 RETURN
after:    middleware 1

Notes

Alt: Dedicated before/afterware functions

• pro: can statically type that Res is only for beforeware
• con:

server.before((req, res, ctx) => {})
server.after((req, res, ctx) => {})

type Before = (req:Req, res:Res, ctx:Ctx) => Res
type After = (req:Req, res:Res, ctx:Ctx) => void

Plugin-level type extraction

We will run type-extraction at the app level. Why not the plugin level too then?

// before
export default NexusPlugin.create(project => {
  project.runtime((augment) => {
    augment.request({
      type: `
        foo?: string
      `,
      term(req) {
        if (math.random() > 0.5) {
          req.foo = 'bar'
        }
      }
    })
  })

// after
export default NexusPlugin.create(project => {
  project.runtime((hooks) => {
    hooks.onRequest(req => {
      if (math.random() > 0.5) {
        req.foo = 'bar'
      }
    })
  })

Why not use an existing system

1. Most are not optimized for TypeScript (exception: https://github.com/gcanti/hyper-ts)
2. None are not optimized for TypeGen
3. None have the concept of context that spans server middleware + resolvers
4. None have the concept of being deployed to serverful or serverless environments
5. None have the higher-order concept of how to handle being within a plugin, which brings an ordering challenge.

Links

• https://zeit.co/docs/v2/network/headers
• https://devcenter.heroku.com/articles/http-routing#heroku-headers
• https://koajs.com/
• https://github.com/koajs/koa/wiki#middleware
• https://github.com/gcanti/hyper-ts
• https://github.com/fastify/fastify/#fastifyusemiddlewarereq-res-next
• https://github.com/fastify/fastify/blob/master/docs/Errors.md
• https://expressjs.com/en/guide/using-middleware.html

[jasonkuhrt]

About middleware ordering

The order of middleware is important. For example authorization middleware should come after something like response-time middleware. If the auth fails, we don't want to lose response time checking!

Developers need to control manage this.

This is hard when plugins can add middleware.

We could add a z-index concept where plugins can state the priority of their middleware. An arbitrary int would be too unwieldy. But say it could be 1 or 2.

Nexus could guarantee that all level 1 gets run (so order does not matter). While level 2 would be potentially mutually exclusive to other middleware in its layer.

Layer 2 could be ordered by plugin name by default. Its deterministic.

If two plugins the user is using put middleware in an order that doesn't work for them the user could be given an api to customize it:

server.middlewareOrder.current // good for debugging
server.middlewareOrder.change(
  'pluginA.middleware1',
  'pluginB.middleware2',
  '...rest',
)

If ordering needs to bring something to the top or bottom it could use the ...rest arg.

varg function.

union of strings based on typegen (all middleware in use).

Each arg provided would remove itself from the union set those encoding set-like behaviour

Otherwise ordering changes would work by making relative changes

server.middlewareOrder.set(
  'pluginA.middleware1',
  'pluginB.middleware2',
)

Meaning here for example the location of pluginA.middleware1 does not change, and pluginB.middleware2 is put directly after it.

This would allow the user to organize the middleware in their app how they like, maybe distributed for some reason, while still being able to control the final order precisely. This aligns well with code-order-independence goal of Nexus.

[osirvent]

Hi everybody,

I'm a newbie in software development. Recently I've been developing some very simple projects with feathersjs and I love the power of hooks to manage server-side logic.

Early this year @daffl launched repo to create an "Async middleware for JavaScript and TypeScript". Maybe this can be used in nexus middleware or this could also be a total non-sense.

Nevertheless it would be great to see a nexus-plugin-feathersjs.

Anyway, thanks for all your work.

[jasonkuhrt]

Hey @osirvent, thanks for that very relevant share. Skimmed it but I'll need to take more time to understand why purity and functional composition with strong data primitives (like Either type) isn't good enough of a direction. fp-ts is probably leading the way in bringing these ideas to TS in a principaled way.

Happy to hear concrete examples of what we could leverage.

I do think getting paralyzed by analysis is a risk at this stage for us, and that starting to ship quickly and often is probably a good option right now for us to get traction.

[paniavula]

This is a good feature. Support for accessing field and args as well would help. Nexus plugin as well can be used but that sounds complex for a regular simple use.

Any timeline for this as we are moving aggressively in adopting nexus-future. Is there any place to track the release notes for releases?

[jasonkuhrt]

This is a good feature.

Which parts are most interesting to you? The more detail the better, its very helpful for us :)

Support for accessing field and args as well would help.

What does field and args mean here? Do you mean you want GraphQL info inside your server middleware?

Is there any place to track the release notes for releases?

GH releases soon. We just need prisma-labs/dripip#21. CC @Weakky we can discuss prioritizing this again.

[paniavula]

Thanks for your quick response. Here is more detailed description of the requirement.

In general here are the different middlewares I could see for a graphql api.

1. middlewares needing req/res only -> eg: cors, other response header sanitisers
2. middlewares needing req/res, graphql fields -> authentication, authorization etc. For User signup, login they will be disabled and are enabled otherwise
3. middlewares needing req/res, graphql fields and args -> business validations, transformations like password encryptions, triggering post commit actions like sending to mq etc.

1 is more server level middleware. 2 and 3 are more graphql resolver level middlewares.

3 would be extremely helpful as it provides a layered architecture. It also helps with simplicity, configurability, reusability giving a good structure to code. After this middleware it would only be persistence using "t.crud...". If these middlewares can be enabled/disabled using some configuration it would be even more helpful. This helps to write a single codeline and use it for different applications, enable/disable middlewares depending on business need of the application.

Some of these are achievable in the old nexus using graphql-middleware. In nexus-future as we are using express, 2 and 3 have become a bottleneck. Nexus plugin could be a choice. This however seems to be requiring to be packaged separately as the codeline seems to be scanning package.json for something like nexus-plugin-... Loading from a local folder like "plugins" in similar to how we are loading from app.ts would be helpful. This however may need to be reviewed with your other larger goals in mind.

In case the above need can be achieved easily with other means, please suggest.

[jasonkuhrt]

@paniavula I think 1 & 2 in your list can already be solved by Nexus Schema middleware:

• https://www.nexusjs.org/#/components/schema/plugins-api?id=oncreatefieldresolverconfig
• ex: https://github.com/graphql-nexus/schema/blob/93ccf7c499a114dac33da94d549d20157b9d0163/src/plugins/fieldAuthorizePlugin.ts#L119

But you seem to already be aware of this!

Nexus plugin could be a choice. This however seems to be requiring to be packaged separately as the codeline seems to be scanning package.json for something like nexus-plugin-... Loading from a local folder like "plugins" in similar to how we are loading from app.ts would be helpful. This however may need to be reviewed with your other larger goals in mind.

Yep we're very aware of this and want to address limitations around this soon!

What's not clear to me is, should the framework provide a higher level abstraction that makes writing middleware spanning server/schema easy, or, should users be left on their own to write server middleware + schema middleware to achieve their ends. I think this is tbd, and requires us to do the basics before we can get fancy, if at all.

[paniavula]

Yep we're very aware of this and want to address limitations around this soon!

What's not clear to me is, should the framework provide a higher level abstraction that makes writing middleware spanning server/schema easy, or, should users be left on their own to write server middleware + schema middleware to achieve their ends. I think this is tbd, and requires us to do the basics before we can get fancy, if at all.

I think the current plugin architecture solves most of the immediate problems. The only critical need for now would be to write simple plugins locally and complex plugins through a separate dependency. The fancy stuff could also bloat the build size. My alpine based docker image for previous nexus (with prisma) was around 140MB when uncompressed. Now it is around 300+MB.

[jasonkuhrt]

The fancy stuff could also bloat the build size

Totally, that's why we're working toward built in tree-shaking. We recently changed the plugin system to be tree-shakable. So those big worktime prisma deps will be droppable before shipping to prod.

The only critical need for now would be to write simple plugins locally

👌