New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make Feathers server framework independent to work with Express, Koa and Hapi #258

Closed
daffl opened this Issue Mar 12, 2016 · 21 comments

Comments

Projects
10 participants
@daffl
Member

daffl commented Mar 12, 2016

Now that Feathers works library independent on the client already (see #193), we don't really need Express as hard dependency for the server anymore either. Instead, we can create separate modules for Express, Koa and potentially Hapi that can be configured just like any other plugin:

Express

The only thing to change upgrading from a Feathers 2 application would be to app.configure(express()):

const feathers = require('feathers');
const express = require('feathers-express');

const app = feathers()
  // Make this app Express compatible
  .configure(express())
  // Configure REST API that uses Express
  .configure(express.rest());

// Use any Express middleware
app.use(bodyParser.json());
// Use Feathers services normally
app.use('/todos', {
  get(id) {
    return Promise.resolve({ id, description: `You have to do ${id}!` });
  }
});

Koa

Koa support came up several times (see #83 and #58). This can now be done very similar:

const feathers = require('feathers');
const koa = require('feathers-koa');

const app = feathers()
  // Make this app Koa compatible
  .configure(koa())
  // Configure Koa REST handler
  .configure(koa.rest());

// Use normal Koa middleware
app.use(function *(){
  this.body = 'Hello World';
});

// Use a Feathers service through app.service
app.service('/todos', {
  get(id) {
    return Promise.resolve({ id, description: `You have to do ${id}!` });
  }
});

@daffl daffl added this to the 3.0.0 milestone Mar 12, 2016

@ekryski

This comment has been minimized.

Show comment
Hide comment
@ekryski

ekryski Mar 12, 2016

Member

The biggest barrier to this I think will be authentication, specifically how we get access to the request and response object and how the passport middleware works with Koa, etc.

At a quick glance it doesn't look too bad with Koa. We'd just need to use https://github.com/rkusa/koa-passport-example and ctx instead of req and res. With Hapi I'm not so sure but I'm not convinced there is a lot of value in supporting Hapi, it doesn't really provide anything different than Express. At least with Koa you have generator support.

IMHO, if we are looking to be more flexible and not tied to a framework we're probably better just using standalone modules for routing, content negotiation, etc.

Member

ekryski commented Mar 12, 2016

The biggest barrier to this I think will be authentication, specifically how we get access to the request and response object and how the passport middleware works with Koa, etc.

At a quick glance it doesn't look too bad with Koa. We'd just need to use https://github.com/rkusa/koa-passport-example and ctx instead of req and res. With Hapi I'm not so sure but I'm not convinced there is a lot of value in supporting Hapi, it doesn't really provide anything different than Express. At least with Koa you have generator support.

IMHO, if we are looking to be more flexible and not tied to a framework we're probably better just using standalone modules for routing, content negotiation, etc.

@ekryski

This comment has been minimized.

Show comment
Hide comment
@ekryski

ekryski Mar 12, 2016

Member

For socket support with Koa we use this for inspiration: https://github.com/koajs/koa.io.

Member

ekryski commented Mar 12, 2016

For socket support with Koa we use this for inspiration: https://github.com/koajs/koa.io.

@ahdinosaur

This comment has been minimized.

Show comment
Hide comment
@ahdinosaur

ahdinosaur Mar 15, 2016

IMHO, if we are looking to be more flexible and not tied to a framework we're probably better just using standalone modules for routing, content negotiation, etc.

http-framework! 😉

ahdinosaur commented Mar 15, 2016

IMHO, if we are looking to be more flexible and not tied to a framework we're probably better just using standalone modules for routing, content negotiation, etc.

http-framework! 😉

@ekryski

This comment has been minimized.

Show comment
Hide comment
@ekryski

ekryski Mar 15, 2016

Member

@ahdinosaur ya that looks like more the right way to go IMHO.

Member

ekryski commented Mar 15, 2016

@ahdinosaur ya that looks like more the right way to go IMHO.

@jeffijoe

This comment has been minimized.

Show comment
Hide comment
@jeffijoe

jeffijoe May 20, 2016

I think we should make Feathers a library instead of a framework - that would make it more independent on the transport as well.

Examples:

Common code
const feathersApp = feathers().configure(rest());
feathersApp.service('todos', new NeDB('todos'));
export default feathersApp;
Framework-specific

Koa

import feathersApp from './feathersApp';
import Koa from 'koa';
import adapter from 'feathers-koa';

const app = new Koa();
app.use(adapter(feathersApp));

Express

import feathersApp from './feathersApp';
import express from 'express';
import adapter from 'feathers-express';

const app = express();
app.use(adapter(feathersApp));

Basically, the adapters will create the middleware for their particular framework.

jeffijoe commented May 20, 2016

I think we should make Feathers a library instead of a framework - that would make it more independent on the transport as well.

Examples:

Common code
const feathersApp = feathers().configure(rest());
feathersApp.service('todos', new NeDB('todos'));
export default feathersApp;
Framework-specific

Koa

import feathersApp from './feathersApp';
import Koa from 'koa';
import adapter from 'feathers-koa';

const app = new Koa();
app.use(adapter(feathersApp));

Express

import feathersApp from './feathersApp';
import express from 'express';
import adapter from 'feathers-express';

const app = express();
app.use(adapter(feathersApp));

Basically, the adapters will create the middleware for their particular framework.

@daffl

This comment has been minimized.

Show comment
Hide comment
Member

daffl commented Aug 12, 2016

@rickmed

This comment has been minimized.

Show comment
Hide comment
@rickmed

rickmed Aug 17, 2016

@daffl I just checked spirit and it looks very neat. It would be amazing to decouple feathers from express. With the up rise of so many new implementations different from express it would make feathers more robust in the future. Any decisions on this?

rickmed commented Aug 17, 2016

@daffl I just checked spirit and it looks very neat. It would be amazing to decouple feathers from express. With the up rise of so many new implementations different from express it would make feathers more robust in the future. Any decisions on this?

@ekryski

This comment has been minimized.

Show comment
Hide comment
@ekryski

ekryski Oct 31, 2016

Member

This Authentication PR feathersjs/authentication#336 should be the only major piece we need in order to be able to support to different frameworks underneath.

Member

ekryski commented Oct 31, 2016

This Authentication PR feathersjs/authentication#336 should be the only major piece we need in order to be able to support to different frameworks underneath.

@ekryski

This comment has been minimized.

Show comment
Hide comment
@ekryski

ekryski Oct 31, 2016

Member

I've been giving this a lot more thought over the last few days and now that we are getting close to winding up Auk this is going to be the major goal for the Buzzard release. As nice as it is to be modular looking at usage numbers Express isn't going anywhere. It has 70 million downloads this last year, Koa is right up there with 1.4 million and Hapi with ~2.4 million.

I think these numbers can be artificially inflated by build systems, deploy frequency, size of deployments, etc. but this gives a general idea of how popular things are.

Being completely honest, looking at the numbers there really isn't a lot of incentive to support anything but express. The major reasons I see would be:

  • http2 support (which looks to be coming in express5 eventually....)
  • reducing dependencies. Being able to use raw node http(s) or something more minimal when you are building a stateless API or microservice (primary feathers use case)
  • being future proof 👍
  • becoming more modular, so that you can swap your router, template engines, etc. Feathers really is just an architectural pattern + utility lib on top of core tech.

In my mind the first "engine" to support other than Express would be Koa. It's the most similar in design to Express and provides nice support for future ES6/ES7 language functions. It also seems to be our most requested. Personally I'd rather have support for raw node libs but that might be a lot of work.

What needs to happen

  • Identify the common top level Feathers API. I don't think we can identify this fully until we try at least one other engine underneath. However I'd ❤️ for it to be as easy as:

    const feathers = require('feathers');
    const app = feathers();
    
    // use by string name
    app.engine('express');
    
    // or pass the engine with string
    const koa = require('koa');
    app.engine('koa', koa);
    
    // or simply pass the engine. I like this best
    const koa = require('koa');
    app.engine(koa);

    This is conceptually similar to what @jeffijoe was suggesting however, I'd prefer to have people feel that they are interacting with a Feathers app directly. I think it will make for a cleaner API. However, that being said it's a bit early to tell as we might then have to shim a ton of methods. Further investigation will be required before we can determine the top level API.

  • Make sure all express methods that are used are either polyfilled, aliased or we remove dependency on them. There may be more but the ones I can think of off the top of my head are:

    • app.get
    • app.set
    • app.render
    • app.send
    • app.json
    • req.accepts
    • req.xhr
  • Make auth engine/transport agnostic (feathersjs/authentication#336)

  • Alias/alter how we register REST routes.

  • Alias/alter how we set up sockets

There might be more things. @daffl would have a better idea, specifically around socket/rest setup.

Other Considerations

  • Are we going to support all of Express's HTTP verb methods? There are a lot. Wouldn't that mean implementing a lot of Express?
  • What Express specific things are we currently relying on?
  • What helper methods on the app object do you want to keep as part of the core Feathers API.
  • What is Express 5 looking like for proposal? Been a while since I looked at it. Would be good to connect with @dougwilson and see if we can lend a hand (if it makes sense, there might already be enough cooks in that kitchen).
  • Are there modules like what @dougwilson was working on or spiritjs, http-framework, etc. that would give us the modularity without having to rewrite those abstractions over top of core node functionality.
Member

ekryski commented Oct 31, 2016

I've been giving this a lot more thought over the last few days and now that we are getting close to winding up Auk this is going to be the major goal for the Buzzard release. As nice as it is to be modular looking at usage numbers Express isn't going anywhere. It has 70 million downloads this last year, Koa is right up there with 1.4 million and Hapi with ~2.4 million.

I think these numbers can be artificially inflated by build systems, deploy frequency, size of deployments, etc. but this gives a general idea of how popular things are.

Being completely honest, looking at the numbers there really isn't a lot of incentive to support anything but express. The major reasons I see would be:

  • http2 support (which looks to be coming in express5 eventually....)
  • reducing dependencies. Being able to use raw node http(s) or something more minimal when you are building a stateless API or microservice (primary feathers use case)
  • being future proof 👍
  • becoming more modular, so that you can swap your router, template engines, etc. Feathers really is just an architectural pattern + utility lib on top of core tech.

In my mind the first "engine" to support other than Express would be Koa. It's the most similar in design to Express and provides nice support for future ES6/ES7 language functions. It also seems to be our most requested. Personally I'd rather have support for raw node libs but that might be a lot of work.

What needs to happen

  • Identify the common top level Feathers API. I don't think we can identify this fully until we try at least one other engine underneath. However I'd ❤️ for it to be as easy as:

    const feathers = require('feathers');
    const app = feathers();
    
    // use by string name
    app.engine('express');
    
    // or pass the engine with string
    const koa = require('koa');
    app.engine('koa', koa);
    
    // or simply pass the engine. I like this best
    const koa = require('koa');
    app.engine(koa);

    This is conceptually similar to what @jeffijoe was suggesting however, I'd prefer to have people feel that they are interacting with a Feathers app directly. I think it will make for a cleaner API. However, that being said it's a bit early to tell as we might then have to shim a ton of methods. Further investigation will be required before we can determine the top level API.

  • Make sure all express methods that are used are either polyfilled, aliased or we remove dependency on them. There may be more but the ones I can think of off the top of my head are:

    • app.get
    • app.set
    • app.render
    • app.send
    • app.json
    • req.accepts
    • req.xhr
  • Make auth engine/transport agnostic (feathersjs/authentication#336)

  • Alias/alter how we register REST routes.

  • Alias/alter how we set up sockets

There might be more things. @daffl would have a better idea, specifically around socket/rest setup.

Other Considerations

  • Are we going to support all of Express's HTTP verb methods? There are a lot. Wouldn't that mean implementing a lot of Express?
  • What Express specific things are we currently relying on?
  • What helper methods on the app object do you want to keep as part of the core Feathers API.
  • What is Express 5 looking like for proposal? Been a while since I looked at it. Would be good to connect with @dougwilson and see if we can lend a hand (if it makes sense, there might already be enough cooks in that kitchen).
  • Are there modules like what @dougwilson was working on or spiritjs, http-framework, etc. that would give us the modularity without having to rewrite those abstractions over top of core node functionality.
@Download

This comment has been minimized.

Show comment
Hide comment
@Download

Download Oct 31, 2016

// or simply pass the engine. I like this best
const koa = require('koa');
app.engine(koa);

Agreed! This way, people can learn Feathers once and deploy on any server that has an adapter. Great idea!

Download commented Oct 31, 2016

// or simply pass the engine. I like this best
const koa = require('koa');
app.engine(koa);

Agreed! This way, people can learn Feathers once and deploy on any server that has an adapter. Great idea!

@jeffijoe

This comment has been minimized.

Show comment
Hide comment
@jeffijoe

jeffijoe Nov 1, 2016

The challenge lies in converting libraries that rely on connection-specific things like headers.

About the popularity contest, my primary reason for wanting to use Koa is not because it's popular (not as much as express), but because it's more stable in terms of handling middleware errors.

jeffijoe commented Nov 1, 2016

The challenge lies in converting libraries that rely on connection-specific things like headers.

About the popularity contest, my primary reason for wanting to use Koa is not because it's popular (not as much as express), but because it's more stable in terms of handling middleware errors.

@kristianmandrup

This comment has been minimized.

Show comment
Hide comment
@kristianmandrup

kristianmandrup Mar 8, 2017

Please let's move Feathers to an architecture more suitable to a thin API gateway (classic web server) and stupid/simple web services that can be deployed stand-alone and are protocol independant, just listening to messages of interest (ie. best practice Micro Service pattern). Then we can start to integrate smoothly with Seneca and other popular Node.js Micro Services frameworks.

And yes, FeathersJS should be agnostic to Express, Koa, Hapi, whatever...
I'd love to see it on Nginx with HTTP2/Push as well :)

Happy days!

kristianmandrup commented Mar 8, 2017

Please let's move Feathers to an architecture more suitable to a thin API gateway (classic web server) and stupid/simple web services that can be deployed stand-alone and are protocol independant, just listening to messages of interest (ie. best practice Micro Service pattern). Then we can start to integrate smoothly with Seneca and other popular Node.js Micro Services frameworks.

And yes, FeathersJS should be agnostic to Express, Koa, Hapi, whatever...
I'd love to see it on Nginx with HTTP2/Push as well :)

Happy days!

@daffl daffl added this to Prerelease in Feathers Buzzard Jul 26, 2017

@andreafalzetti

This comment has been minimized.

Show comment
Hide comment
@andreafalzetti

andreafalzetti Oct 5, 2017

Have you guys seen this https://github.com/fastify/fastify?

I'd love to use it with FeathersJS, what's the status of this issue?

andreafalzetti commented Oct 5, 2017

Have you guys seen this https://github.com/fastify/fastify?

I'd love to use it with FeathersJS, what's the status of this issue?

@marshallswain

This comment has been minimized.

Show comment
Hide comment
@marshallswain

marshallswain Oct 9, 2017

Member

@andreafalzetti still moving forward. You can see some progress going on here: feathersjs/express#3

Member

marshallswain commented Oct 9, 2017

@andreafalzetti still moving forward. You can see some progress going on here: feathersjs/express#3

@kristianmandrup

This comment has been minimized.

Show comment
Hide comment
@kristianmandrup

kristianmandrup Oct 12, 2017

Yeah, would be super sweet to integrate feathers with fastify! Let's do it :)

kristianmandrup commented Oct 12, 2017

Yeah, would be super sweet to integrate feathers with fastify! Let's do it :)

@daffl

This comment has been minimized.

Show comment
Hide comment
@daffl

daffl Oct 12, 2017

Member

The basic integration should be fairly straightforward, however, it's authentication (specifically passport and oAuth) where things get hairy.

Our plan was to remove the hard dependency on Express and after v3 investigate what integrations make sense. I saw a talk on Fastify last week and while it was interesting, it might make even more sense for Feathers to just use Nodes HTTP (and HTTP2!) with the router Fastify is using as the main integration.

Member

daffl commented Oct 12, 2017

The basic integration should be fairly straightforward, however, it's authentication (specifically passport and oAuth) where things get hairy.

Our plan was to remove the hard dependency on Express and after v3 investigate what integrations make sense. I saw a talk on Fastify last week and while it was interesting, it might make even more sense for Feathers to just use Nodes HTTP (and HTTP2!) with the router Fastify is using as the main integration.

@kristianmandrup

This comment has been minimized.

Show comment
Hide comment
@kristianmandrup

kristianmandrup Oct 12, 2017

FYI, I started work on feathers-koa REST integration in feathers-rest-koa

I think it would make sense to extract the REST client into a separate module/package and repo ;)

kristianmandrup commented Oct 12, 2017

FYI, I started work on feathers-koa REST integration in feathers-rest-koa

I think it would make sense to extract the REST client into a separate module/package and repo ;)

@daffl

This comment has been minimized.

Show comment
Hide comment
Member

daffl commented Oct 12, 2017

@bitflower

This comment has been minimized.

Show comment
Hide comment
@bitflower

bitflower Jan 24, 2018

As a newbie to Feathers: By 2018 is Feathers completely independent from Express?

EDIT: Or in other words: Which other frameworks are supported. Is KOA fully supported?

Thanks! Love the framework and thanks for the hard work!

bitflower commented Jan 24, 2018

As a newbie to Feathers: By 2018 is Feathers completely independent from Express?

EDIT: Or in other words: Which other frameworks are supported. Is KOA fully supported?

Thanks! Love the framework and thanks for the hard work!

@kristianmandrup

This comment has been minimized.

Show comment
Hide comment
@kristianmandrup

kristianmandrup Jan 24, 2018

Ask @daffl, he's been working on it... Not sure about the current state of affairs though.

kristianmandrup commented Jan 24, 2018

Ask @daffl, he's been working on it... Not sure about the current state of affairs though.

@daffl

This comment has been minimized.

Show comment
Hide comment
@daffl

daffl Jan 24, 2018

Member

Feathers is framework independent (as in you can e.g. only use it with @feathersjs/socketio or as a standalone client to talk to other services) but it only has HTTP API bindings for Express (in @feathersjs/express).

Since the whole point of Feathers is abstracting the protocol specific things, the HTTP framework it is using ultimately shouldn't really matter that much and abstracting things like authentication away from Express is a pretty big task (all of Passport is built for Express and even the current Koa integrations just hack around that fact by mucking around with its request object to make it look like Express). Highest priority for new framework bindings for me would be plain Node HTTP which with a new service lookup mechanism would yield similar performance to Fastify and make websocket connections even faster.

Member

daffl commented Jan 24, 2018

Feathers is framework independent (as in you can e.g. only use it with @feathersjs/socketio or as a standalone client to talk to other services) but it only has HTTP API bindings for Express (in @feathersjs/express).

Since the whole point of Feathers is abstracting the protocol specific things, the HTTP framework it is using ultimately shouldn't really matter that much and abstracting things like authentication away from Express is a pretty big task (all of Passport is built for Express and even the current Koa integrations just hack around that fact by mucking around with its request object to make it look like Express). Highest priority for new framework bindings for me would be plain Node HTTP which with a new service lookup mechanism would yield similar performance to Fastify and make websocket connections even faster.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment