Mirage as express middleware #1014

mfazekas · 2017-02-16T13:53:58Z

This PR adds the option to use express instead of pretender.

Usecases:

use real HTTP traffic, use browser's network inspector
let multiple instances talk to mirage backend that survives reload
get fastboot apps running without a real backend, fastboot testing

Implementation:

We compile ember-cli-mirage and app's mirage dir to node.js using babel and borccoli. Since ember-cli-mirage uses ember some places we have ember-compat that provides the used parts of Ember in node.js

this is piggybacks on Fastboot, the main issue with is that current implementation of mirage runs in browser and we need it to run it on node.js. Fastboot already does that for the whole ember app. So we install an express middleware so that on first network request we can lazily create a Fastbooted (sandboxed) app instance, and we pass an express router to it. Then the sandboxed app can use the passed in express router to respond to real HTTP requests.

~~It also depends on universal serving intruduced in ember 2.12 apps needs to include ember-cli-fastboot, so .js assets are compiled to node.js env.~~

TODO

tests
reload

How to use

set useExpress: true in your ENV['ember-cli-mirage'] in config/environment.js

For example app see: https://github.com/mfazekas/fastboot-mirage-test/

samselikoff · 2017-02-17T02:32:02Z

I see. Could you reach out to the FastBoot team and ask if there's a way to get babel-transpiled code into the Node environment?

mfazekas · 2017-03-01T22:01:14Z

@samselikoff pls review - i think this is complete now

samselikoff · 2017-03-09T04:23:26Z

Ok so - this depends on FastBoot? Can you walk me through it a bit. I wasn't thinking we'd need FB for this

mfazekas · 2017-03-09T05:35:55Z

@samselikoff so issue we need to solve is loading ember code which normally runs in browser to Node.js env. Fastboot already solves this, so in a way it makes sense to build on that. These are the dependencies on fastboot:

ember-cli-mirage itself depends on fastboot which is just code used by ember-cli-fastboot, including it will not change the apps behaviour in any way
apps wanting to use mirage via express needs to use on ember-cli-fastboot

Which dependency are you having issue with? I'd say 1.) is not an issue, 2.) is more problematic, but it still means one can use the express backed for testing fastboot apps, and others not using ember-cli-fastboot can decide if the express backed worth enough to add that or not.

We could try to do without fastboot, but that requires locating the vendor.js and <appname>.js then loading them to node.js and reload them on change. I'd assume just using require is probably not going to work we need a lot of what fastboot does which loads the code using eval https://github.com/ember-fastboot/fastboot/blob/master/src/ember-app.js#L130

Removing the fastboot dependency would not change the way this PR works it would only change serverMiddleware and postBuild parts.

samselikoff · 2017-03-09T13:51:23Z

Thanks for the response, I've been pressed for time but hoping to look at this in more detail soon.

Conceptually my thinking was to read in the mirage code from disk and require it in node, and use it to define those route handlers within Ember CLI's existing express server.

The reason to avoid adding fastboot or ember-cli-fastboot is just because they are more dependencies and deps can cause issues. Again I'll need to go through and see how they are used.

What if we started with getting it working in Ember CLI's express but without automatic reloading?

mfazekas · 2017-03-09T14:17:34Z

Thanks for the response, I've been pressed for time but hoping to look at this in more detail soon.

Conceptually my thinking was to read in the mirage code from disk and require it in node, and use it to define those route handlers within Ember CLI's existing express server.

I've tried that route but mirage code depends on parts of ember which is not something you can just require and run inside node asis as they were meant to run in the browser. As i recall such examples are string operations, promises.

Otherwise we define the routes on Ember CLI's existing express server. Actually we define the routes on an express.Router instance object and we add that to the express server started by ember-cli.

The reason to avoid adding fastboot or ember-cli-fastboot is just because they are more dependencies and deps can cause issues. Again I'll need to go through and see how they are used.
What if we started with getting it working in Ember CLI's express but without automatic reloading?

I hear you i'll look into getting it working without fastboot by doing the same eval stuff done by fastboot, but that change should affect small part of this PR.

samselikoff · 2017-03-09T14:24:08Z

@mfazekas Ah I see - well Mirage was designed explicitly not to rely on Ember (for precisely this feature and future features related to node-land only stuff) so if there are Ember dependencies we should remove them.

At the most I expect some utilities, no Ember.Object is used anywhere. Perhaps that should be a first step.

samselikoff · 2017-03-20T14:00:31Z

addon/server.js

@@ -167,7 +130,7 @@ export default class Server {
    let hasFactories = this._hasModulesOfType(config, 'factories');
    let hasDefaultScenario = config.scenarios && config.scenarios.hasOwnProperty('default');

-    this.pretender = this.pretender || createPretender(this);
+    this.interceptor = this.interceptor || createInterceptor(this);


this.pretender was basically a public API, it let folks get access to the underlying pretender server. I'm thinking we should add code here to to create a this.pretender alias if this.intercepter instanceof Pretender.

Fixed: createPretender is now setting pretender on Server.
We'll need a test for it though

https://github.com/mfazekas/ember-cli-mirage/blob/express/addon/interceptors/create-pretender.js#L48

samselikoff · 2017-03-20T14:02:22Z

app/initializers/ember-cli-mirage.js

@@ -23,7 +25,7 @@ export function startMirage(env = ENV) {
  let modules = readModules(env.modulePrefix);
  let options = _assign(modules, {environment, baseConfig, testConfig});

-  return new Server(options);
+  return new Server(Object.assign({createInterceptor: createPretender}, options));


Let's use Lodash assign, since Object.assign doesn't work in Phantomjs

import _assign from 'lodash/object/assign'; _assign({ createInterceptor: createPretender }, options));

samselikoff · 2017-03-20T14:03:10Z

index.js

@@ -112,9 +176,6 @@ module.exports = {
  },

  _shouldIncludeFiles: function() {
-    if (process.env.EMBER_CLI_FASTBOOT) {


I believe this was here so Mirage doesn't blow up when using fastboot. Does this no longer happen?

samselikoff · 2017-03-20T14:10:17Z

tests/integration/database-test.js

@@ -1,5 +1,5 @@
 import {module, test} from 'qunit';
-import Server from 'ember-cli-mirage/server';
+import Server from 'ember-cli-mirage/pretender-server';


Shouldn't these be able to use ember-cli-mirage/server and by not passing useExpress they'll default to using Pretender?

samselikoff · 2017-03-20T14:11:36Z

index.js

+
+  let emberCliMiragePath = __dirname;
+  let mirageAddon = Funnel(emberCliMiragePath, {srcDir: 'addon', destDir: 'ember-cli-mirage'});
+  let appMirageConfig = Funnel(projectRoot, {srcDir:'mirage', destDir: 'mirage'});


Mirage's dir can be configurable, we might need to look it up here... I kind of wish it wasn't though. shrug

samselikoff · 2017-03-20T14:12:26Z

Some comments reading through it. Excellent work on this, I'm so stoked for it to land!!

Was wondering if we should ship it with 0.3 (which I was about to cut), or get it in for the last 0.2.x series.

mfazekas · 2017-03-21T17:10:12Z

I think we should target 0.3 i'll update my branch

mfazekas · 2017-04-18T19:04:01Z

The bottom pane is from running npm ls lodash, looks like ember-cli-mirage is using 3.x which is causing issues?

That is very strange are we using lodash 3.x?? What's strange that using yarn it installs 4.x and i don't see issue on ember serve

ember new ember-cli-mirage-e1-demo
cd ember-cli-mirage-e1-demo
ember install mfazekas/ember-cli-mirage#express
# ./node_modules/lodash is 3.x
yarn
# ./node_modules/lodash is 4.x

samselikoff · 2017-04-19T12:45:34Z

I get the build error when setting useExpress: true in my config, you don't?

samselikoff · 2017-04-19T12:46:06Z

I think the problem is the lodash module is supplied by ember-lodash which isn't available in node.

mfazekas · 2017-04-19T13:07:44Z

I was not getting the error when i used yarn vs npm to install modules. yarn seems to have chosen 4.0 and not 3.0 of lodash not sure why.

samselikoff · 2017-04-19T15:36:45Z

Gotchya. We'll need to get it working with npm as well.

samselikoff · 2017-05-03T13:09:20Z

ping @mfazekas. we're so close!

mfazekas · 2017-05-07T15:02:26Z

@samselikoff sorry i was busy lately with other stuff.

So about lodash version, i'm a bit stuck it seems that with ember 2.12 projects are getting lodash 3.1* by default, and even if you add a package requiring lodash-4.* it will not be upgraded.

With ember 2.13 a new project seems to get "4.17.4" even with npm.

mfazekas · 2017-05-07T16:22:52Z

I'm also working on rebasing this to current version, and we'll need to figure out some automated way to test. Since we're compling to different env (node.js) it will be something easy to brake esp if new emberjs or other dependency is introduced.

kellyselden · 2017-05-18T20:38:53Z

Not sure if it is known, but I just tried out this branch and got:

Error during build for node.js: TypeError: (0 , _utilsInflector.singularize) is not a function
    at GetShorthandRouteHandler.getModelClassFromPath (/Users/kelly.selden/code/jackrabbit/tmp/broccoli_persistent_filterbabel-output_path-hxNMndi8.tmp/ember-cli-mirage/route-handlers/base.js:40:116)
    at GetShorthandRouteHandler.BaseShorthandRouteHandler (/Users/kelly.selden/code/jackrabbit/tmp/broccoli_persistent_filterbabel-output_path-hxNMndi8.tmp/ember-cli-mirage/route-handlers/shorthands/base.js:32:35)
    at new GetShorthandRouteHandler (/Users/kelly.selden/code/jackrabbit/tmp/broccoli_persistent_filterbabel-output_path-hxNMndi8.tmp/ember-cli-mirage/route-handlers/shorthands/get.js:35:90)
    at createHandler (/Users/kelly.selden/code/jackrabbit/tmp/broccoli_persistent_filterbabel-output_path-hxNMndi8.tmp/ember-cli-mirage/route-handler.js:79:15)
    at new RouteHandler (/Users/kelly.selden/code/jackrabbit/tmp/broccoli_persistent_filterbabel-output_path-hxNMndi8.tmp/ember-cli-mirage/route-handler.js:107:20)
    at Server._registerRouteHandler (/Users/kelly.selden/code/jackrabbit/tmp/broccoli_persistent_filterbabel-output_path-hxNMndi8.tmp/ember-cli-mirage/server.js:602:26)
    at Server._this5.(anonymous function) [as get] (/Users/kelly.selden/code/jackrabbit/tmp/broccoli_persistent_filterbabel-output_path-hxNMndi8.tmp/ember-cli-mirage/server.js:578:18)
    at Server.exports.default (/Users/kelly.selden/code/jackrabbit/tmp/broccoli_persistent_filterbabel-output_path-hxNMndi8.tmp/mirage/config.js:51:8)
    at Server.loadConfig (/Users/kelly.selden/code/jackrabbit/tmp/broccoli_persistent_filterbabel-output_path-hxNMndi8.tmp/ember-cli-mirage/server.js:269:14)
    at Server.config (/Users/kelly.selden/code/jackrabbit/tmp/broccoli_persistent_filterbabel-output_path-hxNMndi8.tmp/ember-cli-mirage/server.js:206:14)
Build for node.js finished!

Hope it helps.

samselikoff · 2017-05-19T12:31:25Z

@mfazekas there was just a lodash PR merged, perhaps it addresses the issue you were running into?

I've been trying to focus mirage time on landing polymorphic associations so I haven't been able to dive into this. Also stanley was hacking on getting mirage running in a Service Worker, not sure if there are any implications for that here

mfazekas · 2017-05-19T13:58:08Z

@samselikoff sorry i try to get some time to work in on the weekend.

There is one issue that sometimes lodash-3.* is installed by npm. I'm not that concerned about that as it works with never ember versions, i'll also try with latest mirage.
Other thing is that i need proper emulate stuff like singularize, etc. under node.js i assume mirage is using ember-inflector, i'd probably need to use lodash-inflection on the node.js side.

rtablada · 2017-07-14T12:28:03Z

a potential issue with using lodash inflector is that custom inflection rules would not be properly inflected.

mfazekas · 2017-07-26T18:43:45Z

@samselikoff The real reason this PR is stuck at the moment that ember-fastboot do have nontrivial ember depencencies. One example is inflector to pluralize, etc strings. While we can do some emulation with lodash-inflection it will not be 100% compatible, like custom rules mentioned above.

Another alternative is going back to drawing board and piggyback on fastboot, as fastboot let's Ember code run on the npm side.

kratiahuja · 2017-07-29T18:25:53Z

@samselikoff what's the direction you want to take with this PR? Or what needs to be done to have mirage supported for FastBoot? I am bit confused reading the entire thread. Happy to help with it.

mfazekas · 2017-07-29T19:22:40Z

@kratiahuja this PR would make mirage work with Fastboot. But it's not tied to fastboot. There was a version where i did use fastboot to run mirage code on node.js, but in this version we're using broccoli directly to transpile mirage code to node.js. Ideally mirage would not have ember dependency at all, but it does have some use of Ember's String for example which makes this approach less than ideal.

xcambar · 2017-08-27T19:13:27Z

[...] less than ideal.

Less than ideal is better than none. As far as I'm concerned, I'm really looking forward to seeing this PR land in Mirage. The benefits of this PR are fantastic! Being able to debug "as if" we were hitting the proper API and doing analyses with the Network Panel are really worth any tradeoff.

Please land this 🙏

izelnakri · 2017-10-21T18:55:37Z

Hi @mfazekas, I tried to run this branch on a fastboot project(up-to-date) and got this error:
TypeError: Cannot read property '_passthroughCheck' of undefined.

e00dan · 2017-11-06T14:39:19Z

Hello, is this work here going to be continued?

jkeen · 2018-11-07T01:39:10Z

Is there a new strategy that's being looked into for mirage + fastboot, @samselikoff and @mfazekas? I picked up this branch and rebased it with master and am working on getting it going with a project I'm using ember-cli-fastboot-testing on, but I'm running into some rough broccoli transformation errors.

It looks like mirage now has fastboot as a dependency, should we revive the initial idea of using fastboot to do all that transforming?

samselikoff · 2018-11-07T12:28:20Z

@jkeen I am going to be working on this soon (next month or tw) to help with the Fastboot testing story. Don't believe it's going to be Fastboot doing the transforming.

ming-codes · 2019-02-14T00:51:29Z

I think Mirage as Service Worker sounds great. Would it be a better approach to modify ember-cli-mirage to be a plugin for ember-service-worker? It may even be possible to build a separate addon to bridge the gap without modifying mirage itself.

samselikoff · 2019-02-14T18:59:31Z

@ming-codes good question. I don't know much about service workers yet. I think the APIs I'll be adding to get Mirage working in node will also make it work in a SW.

samselikoff · 2019-04-20T18:34:38Z

I apologize for not communicating more around this issue & letting it linger. The reality was that this expands the problem space Mirage must support to a degree we weren't able to sustain with the resources we had on the project. Maybe this was my fault for not recruiting and empowering more folks to work on the project.

In any case, we've picked this up for some related work, and FastBoot & Node support is a priority of ours at the moment. I'm going to close this PR for now since we're taking a slightly different path, relying onesm to do the heavy lifting instead of custom Broccoli code we have to maintain.

@mfazekas Thank you so much for kicking off this work and providing a place for folks to chime in, it helped us understand the need much more! Sorry again about letting this languish but hopefully we will end up with something that works for everyone soon.

Anyone interested in the status of the feature can keep an eye on the FastBoot support issue.

mfazekas force-pushed the express branch 5 times, most recently from 44ff700 to 9139201 Compare February 16, 2017 16:31

mfazekas mentioned this pull request Feb 27, 2017

[POC] Mirage on FastBoot with express #975

Closed

mfazekas force-pushed the express branch from 38b9f9f to f58781d Compare March 1, 2017 20:20

mfazekas changed the title ~~[WIP] Mirage as express middleware~~ Mirage as express middleware Mar 1, 2017

mfazekas force-pushed the express branch from f58781d to e8963d9 Compare March 1, 2017 21:49

mfazekas force-pushed the express branch from e8963d9 to d766a46 Compare March 6, 2017 07:31

samselikoff reviewed Mar 20, 2017

View reviewed changes

mfazekas force-pushed the express branch 10 times, most recently from ced991c to 8faa142 Compare March 22, 2017 09:36

mfazekas force-pushed the express branch from 4b92fe0 to e5e8c86 Compare April 18, 2017 18:51

Mirage as express middleware

4ca0353

mfazekas force-pushed the express branch from e5e8c86 to 4ca0353 Compare April 18, 2017 19:01

simonihmig mentioned this pull request Oct 27, 2017

Fix FastBoot 1.0 #1098

Merged

samselikoff added the Feature / Enhancement label May 6, 2018

jkeen mentioned this pull request Nov 14, 2018

Request Proxy embermap/ember-cli-fastboot-testing#17

Closed

samselikoff closed this Apr 20, 2019

Mirage as express middleware #1014

Mirage as express middleware #1014

Conversation

mfazekas commented Feb 16, 2017 • edited Loading

Usecases:

Implementation:

TODO

How to use

samselikoff commented Feb 17, 2017

mfazekas commented Mar 1, 2017

samselikoff commented Mar 9, 2017

mfazekas commented Mar 9, 2017

samselikoff commented Mar 9, 2017

mfazekas commented Mar 9, 2017

samselikoff commented Mar 9, 2017

samselikoff Mar 20, 2017

Choose a reason for hiding this comment

mfazekas Mar 21, 2017 • edited Loading

Choose a reason for hiding this comment

samselikoff Mar 20, 2017

Choose a reason for hiding this comment

samselikoff Mar 20, 2017

Choose a reason for hiding this comment

samselikoff Mar 20, 2017

Choose a reason for hiding this comment

samselikoff Mar 20, 2017

Choose a reason for hiding this comment

samselikoff commented Mar 20, 2017

mfazekas commented Mar 21, 2017

mfazekas commented Apr 18, 2017

samselikoff commented Apr 19, 2017

samselikoff commented Apr 19, 2017

mfazekas commented Apr 19, 2017

samselikoff commented Apr 19, 2017

samselikoff commented May 3, 2017

mfazekas commented May 7, 2017

mfazekas commented May 7, 2017

kellyselden commented May 18, 2017

samselikoff commented May 19, 2017

mfazekas commented May 19, 2017

rtablada commented Jul 14, 2017

mfazekas commented Jul 26, 2017

kratiahuja commented Jul 29, 2017 • edited Loading

mfazekas commented Jul 29, 2017 • edited Loading

xcambar commented Aug 27, 2017

izelnakri commented Oct 21, 2017 • edited Loading

e00dan commented Nov 6, 2017

jkeen commented Nov 7, 2018 • edited Loading

samselikoff commented Nov 7, 2018

ming-codes commented Feb 14, 2019

samselikoff commented Feb 14, 2019

samselikoff commented Apr 20, 2019

mfazekas commented Feb 16, 2017 •

edited

Loading

mfazekas Mar 21, 2017 •

edited

Loading

kratiahuja commented Jul 29, 2017 •

edited

Loading

mfazekas commented Jul 29, 2017 •

edited

Loading

izelnakri commented Oct 21, 2017 •

edited

Loading

jkeen commented Nov 7, 2018 •

edited

Loading