Support Lazy Loading engines

[knownasilya]

So in the readme it's mentioned that this is coming soon. I'm wondering how this will work, since it seems like you have some ideas already.

[rwjblue]

Yeah, we do. I'll try to take some time in the next day or two and comment with some of the ideas...

[stefanpenner]

Their have been several issues opened in the past, I believe the RFC has some relevant comments.

From memory the following needs to be fixed:

1. Router.js needs to support routing to handlers that are not yet loaded
2. Likely something about query params
3. slight addition to concat-phase of ember-cli + some config to enable a quick path for this. (Although more intelligent packaging aka linker/packager work will likely be able to solve other relevant issues, like splitting vendor etc but in a more automatic way)
4. in the past, the route serialization stuff was a concern but i believe right now we don't allow link-to between engines, is that correct?
5. be sure to add support for lazy-loading fingerprinted bundles

[rwjblue]

thanks @stefanpenner, that summarizes the path pretty well

[knownasilya]

Will this feature have to be used with fastboot, or will the dist contain everything necessary? Like will the paren't app append engine script tags to the DOM when the user triggers an engine for the first time?

[rwjblue]

When we have written the code, we will let you know. 😺

In all seriousness though, we do not plan to require fastboot for anything in this addon.

[knownasilya]

Excellent 👍

[stefanpenner]

or will the dist contain everything necessary?

likely this

[bcardarella]

The last time I explored this for ember-admin I believe it could be possible by modifying how the resolver does look ups. If a module is not found locally the resolver would defer to fetching the resource remotely then try the lookup again. At that point the resolver has the resource cached.

Its been a while since I mucked around inside the resolver code but it would be ideal if however the remote fetch is done that the resolving itself is promise-based the consuming application can give some state indication that the engine's loading is pending.

[gossi]

Looks like ember engines may be what I'm looking for especially lazy-loading. In my modular structure, several engines will be distributed in different directories. Will it be possible to load engines that are not explicitely installed with ember install command but by telling ember (maybe in the html or via rest-api) where to find an engine at which mount point?

[mike183]

Looks like ember engines may be what I'm looking for especially lazy-loading. In my modular structure, several engines will be distributed in different directories. Will it be possible to load engines that are not explicitely installed with ember install command but by telling ember (maybe in the html or via rest-api) where to find an engine at which mount point?

This is something that I am also interested in.

For my use-case, a majority of engines wouldn't actually be installed into the main application at build time. They would instead be built independently and the application would be required to locate, configure and mount them at runtime.

[stefanpenner]

Its been a while since I mucked around inside the resolver code

I don't believe this should be a resolver concern, or atleast shouldn't be at the per-module granularity as they would force all possible module loads to be async. I believe that would result in much more complexity then we want to deal with. Instead larger slabs, like per engine async'ness is likely the best balance. This keeps async to explicit points where async can be correctly handled, not at every possible resolve step.

[MiguelMadero]

@stefanpenner and @dgeb I spent time working on lazy-loading for an app. Some of this work could help engines and I have some time to work on it. I solve some of the points that you mentioned above:

Router.js needs to support routing to handlers that are not yet loaded

I solved this by using a catch-all route, abort the transition, load the bundle with the route and then resume the transition. You can see an example on routes/catch-all#redirect.
I wanted to do it in a way that I didn't have to patch Ember, but I could do something similar in router.js

Likely something about query params

I've been ignoring it for now, but it might just work with the approach above.

slight addition to concat-phase of ember-cli + some config to enable a quick path for this. (Although more intelligent packaging aka linker/packager work will likely be able to solve other relevant issues, like splitting vendor etc but in a more automatic way)

This is a bit hacky and I still plan to extract it. Likely the packager will be a better long-term solution, for now, I'm basically using a different EmberApp for each "package" (you can think of them as engines, but I wanted to use a different term to make it clear that this isn't the same thing). You can see an example on ember-cli-build, but this could be encapsulated in something that looks like:

var emberApp = EmberAppWithPackages({
  sharedConfig: {},
  bootAppConfig: {},
  packagesConfig: {}
});

Alternatively, we could provide the configuration for each engine's index.js file, but still let the consuming app provide overrides. Although this works, I don't think it's they way to go, it is just the way I solved it without tweaking CLI internal's or re-doing this work directly in broccoli and breaking add-ons.

in the past, the route serialization stuff was a concern but i believe right now we don't allow link-to between engines, is that correct?

For now, I'm avoiding that or simply using a new {{link-to}} helper that uses the default serialize behavior until we get a way to override it in Router.map.

be sure to add support for lazy-loading fingerprinted bundles

Still WIP, but the idea is to define the list of asset URLs based on the names of the bundles in a map {engine-name: /assets/engine-name.js} and let AssetRev tweak it. You can see an example on index.html. The map isn't used yet at runtime, I want to move away from the global, so that's why it still needs some work.

I have an example of all of this working on https://github.com/MiguelMadero/ember-cli-packages-demo/. I would love your feedback on this and as I mentioned, I have some time to see help push this forward.

[knownasilya]

Just wanted to get this out there, but if the query params API was a service, it would be simple to integrate between different engines.

[peStoney]

For my use-case, a majority of engines wouldn't actually be installed into the main application at build time. They would instead be built independently and the application would be required to locate, configure and mount them at runtime.

@mike183 - that's exactly what I'm looking for as well. @dgeb - will this be possible with ember-engines in the future? That would allow users to customize an app by selecting the engines they actually need. At the moment it looks like every engine to be consumed needs to be defined at build time...

[dgeb]

@mike183 @peStoney Sorry to delay with a response. It is theoretically possible that routeless engines could be lazy-loaded by a parent that doesn't have any foreknowledge of the engine. However, routable engines need to eagerly provide their route maps to their parents to support routing into them that triggers lazy-loading.

[dgeb]

This issue has been superceded by #154