Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pre-RFC: Generate and serve ES2015 assets #383

Open
tomdale opened this Issue Oct 3, 2018 · 20 comments

Comments

Projects
None yet
9 participants
@tomdale
Copy link
Member

tomdale commented Oct 3, 2018

This is a pre-RFC because I don't have the bandwidth to write up a full RFC, so if someone wants to pick this up that would be awesome. 馃榿

Generate and serve ES2015 assets

Summary

Produce both modern and legacy (ES5) versions of JavaScript assets, and intelligently feature detect which version to load using <script type="module"> and <script nomodule> tags.

Motivation

Ember CLI currently produces a single version of an application's JavaScript assets, transpiled to the "lowest common denominator" browser target. However, transpiled output can be quite a bit larger and slower than when using native syntax. Even though a majority of an application's users may be using evergreen browsers with ES2015 support, we still serve them the bigger, slower ES5 version in order to not break older browsers like Internet Explorer.

The good news is that we can take advantage of two new browser features to detect and load an ES2015-compatible version of JavaScript assets in evergreen browsers, while falling back to the ES5 build in browsers that don't support ES2015.

We rely on the combination of two platform features:

  1. <script type="module">, which fetches and evaluates JavaScript files in "module" mode, and is ignored by browsers that don't support modules, i.e., browsers that also don't support other ES2015 features.
  2. <script nomodule>, which tells the browser not to fetch the linked resources if it supports ES modules. (Similar in concept to the <noscript> tag, which tells browsers to only show content if JavaScript is disabled.)

In essence, we use the existence of module support (as detected by properly-annotated script tags) as a proxy for overall ES2015 support. In practice, modules were one of the last ES2015 features to be implemented by browsers, so every browser that supports modules also reliably supports the remainder of the ES2015 spec.

Detailed design

A few open questions here.

First, how does this interact with the build targets feature? We could say that Ember CLI produces two versions of JS assets, a "legacy" version and a "modern" version, and allow users to configure the browser targets for each. But this feels bad, because it's hardcoded to the current situation and is not amenable to change in the future.

A better option is probably to extend the build targets API to support multiple targets. Here's a strawman for what this API could look like:

module.exports = [{
  output: {
    dir: 'es2015'
  },
  browsers: [
    'last 1 edge versions',
    'last 1 Chrome versions',
    'last 1 Firefox versions',
    'last 1 Safari versions'
  ]
}, {
  output: {
    dir: 'es5'
  },
  browsers: [
    'ie 11'
  ]
}];

In this example, we now support exporting an array of build targets. We also add an output section to the configuration, with a dir property that specifies which directory this target should be compiled into.

This directory is relative to dist, so the es2015 example above would be compiled into e.g. dist/es2015/app.js. The proposed output syntax is consistent Rollup's configuration format, although as proposed we only support the dir option.

Assuming that the application blueprint defaults to something like the above-listed config/targets.js, we can then update the index.html blueprint to the following:

<script type="module" src="{{rootURL}}assets/es2015/vendor.js"></script>
<script type="module" src="{{rootURL}}assets/es2015/<%= name %>.js"></script>
<script nomodule src="{{rootURL}}assets/es5/vendor.js"></script>
<script nomodule src="{{rootURL}}assets/es5/<%= name %>.js"></script>

The default defer semantics of <script type="module"> may not work for us here as I believe the app will break if vendor.js is not evaluated before app.js, so that is something we would want to investigate.

That said, implementing the above two steps I think is enough to let us start serving smaller, faster assets to the majority of users, and most importantly, without requiring any invasive changes to server configurations.

Next Steps

After writing all of this out, it may be that this should be broken into at least two standalone RFCs:

  1. Producing multiple build targets in a single build.
  2. Changing the default index.html as described above.

Anyone who is interested in working on this should feel free to pick it up, and ping me on Discord if there are questions or problems with this proposal. 馃槃

Shoutout to @kristoferbaxter from the Google AMP team for persuading me this was a good idea when I ran into him at 4am in the terminal at Norman Y. Mineta San Jose International Airport. 馃槀

@ctcpip

This comment has been minimized.

Copy link

ctcpip commented Oct 3, 2018

safe to say IE11 is the only mainstream browser that wouldn't support? also, where can we now find the current supported list of browsers (for ember)?

screen shot 2018-10-03 at 4 58 04 pm

@tomdale

This comment has been minimized.

Copy link
Member Author

tomdale commented Oct 3, 2018

@ctcpip IIRC our currently supported browser matrix can be found in our .travis.yml. At least, that鈥檚 the most honest statement of what we support. 馃槀

@kristoferbaxter

This comment has been minimized.

Copy link

kristoferbaxter commented Oct 3, 2018

Something that might help is using the recently added esmodules target in Babel.

Here's an example: https://github.com/kristoferbaxter/preset-env-modules

Edit: https://github.com/kristoferbaxter/preset-env-modules/blob/master/.babelrc.js#L4 is an example.

@chadhietala

This comment has been minimized.

Copy link
Member

chadhietala commented Oct 3, 2018

How does the registry work for the DI container?

@tomdale

This comment has been minimized.

Copy link
Member Author

tomdale commented Oct 3, 2018

@chadhietala I don鈥檛 understand the question.

@chadhietala

This comment has been minimized.

Copy link
Member

chadhietala commented Oct 4, 2018

This should clearly outline that is just a proxy for ES2015 and we would continue to use the AMD loader.

@NullVoxPopuli

This comment has been minimized.

Copy link
Contributor

NullVoxPopuli commented Oct 5, 2018

when are we dropping support for IE11?

@ctcpip

This comment has been minimized.

Copy link

ctcpip commented Oct 5, 2018

@NullVoxPopuli it is supported by MS until 2025. I reckon support could be dropped ahead of that, but market share is still high for IE11. For comparison, Angular still supports IE9 (!)
@tomdale I can't find that browser matrix you referenced. We really need to be explicit about browser support again. (IIRC, this used to be discoverable, and easily so.)

@Turbo87

This comment has been minimized.

Copy link
Member

Turbo87 commented Oct 6, 2018

I've played around with a similar feature in the vue-cli and it works quite well. I would love to get this into ember-cli but I'm a little worried about the impact it would have on build performance right now.

@tomdale

This comment has been minimized.

Copy link
Member Author

tomdale commented Oct 10, 2018

@Turbo87 Correct me if I'm wrong, but wouldn't producing two builds be relatively easy to parallelize? In general, I'm concerned that Broccoli is still missing primitives that allow it to detect and parallelize isolated sub-graphs that don't share dependencies鈥擨 think that, plus better pruning of transforms "downstream" from pure transforms that don't detect changes, would make it easier for us to experiment with things like this.

Do we have a plan for getting build performance to a place where we could feel comfortable introducing something like this?

@Turbo87

This comment has been minimized.

Copy link
Member

Turbo87 commented Oct 10, 2018

Correct me if I'm wrong, but wouldn't producing two builds be relatively easy to parallelize?

AFAIK we already try to parallelize some of the work (e.g. Babel transpilation). If all of your CPUs are already busy with Babel, then building a second asset tree in parallel will likely not be as fast as just building a single asset tree.

Do we have a plan for getting build performance to a place where we could feel comfortable introducing something like this?

I don't know the exact details, but I think the packager work was moving in a direction where most files were kept as ESlatest as long as possible and the transpilation to lower browser targets happens relatively late in the pipeline. If the two pipelines ("modern" and "legacy" builds) can share most of the work and only the transpilation step at the end is duplicated this might be fine. But as far as I'm aware that is not quite the case right now as the current pipeline (pre-packager) is quite convoluted...

@simonihmig

This comment has been minimized.

Copy link
Contributor

simonihmig commented Oct 13, 2018

Talked about the dual build performance issue briefly with @Turbo87 at Emberfest yesterday, will try to sum up my thoughts here as well:

  • I believe you very rarely would want to have the legacy build in the ember serve scenario, where build performance is most critical, as developers will use browsers that will never use the "legacy" bundle. Only in the specific case, where someone wants to test things in IE there could be an opt in to activate the legacy build (say via a CLI flag).
  • For ember test it might make sense to also disable the legacy build by default, as enabling is only useful if your test setup is able to run IE (i.e. something like Browserstack/Saucelabs), which might not be the case for many apps I would guess.
  • For ember build -prod you would want to have it included of course. But that's mostly running in CI, so not that big of a performance concern here probably?
@NullVoxPopuli

This comment has been minimized.

Copy link
Contributor

NullVoxPopuli commented Oct 13, 2018

So, ember uses babel, and we can configure babel to our heart's content...

Has anyone configured a module build?
I'd like to see how happens to my bundle size :)

@rwjblue

This comment has been minimized.

Copy link
Member

rwjblue commented Oct 13, 2018

By default development builds are already mostly evergreen browsers only (check out your config/targets.js). We only build for lowest common denominator in CI and for production.

The feature here would be to enable building two sets of assets for production and using the technique described above (script tag with modules attribute + an array in config/targets.js).

鈥-

It seems that folks here are forgetting the most straightforward answer here though. The absolute simplest way to do this would be dropping IE11 support completely 馃樅...

@NullVoxPopuli

This comment has been minimized.

Copy link
Contributor

NullVoxPopuli commented Oct 13, 2018

I just mention it because major browsers support async/await, but the transpiled code is (or was, need to check) runtime generator stuff.

@rwjblue

This comment has been minimized.

Copy link
Member

rwjblue commented Oct 13, 2018

Ya, but thats just based on what your config/targets.js has. If you only have evergreen browsers listed there already (aka don't support IE11), you will be shipping native async/await (and everything else that modern browsers support out of the box).

@NullVoxPopuli

This comment has been minimized.

Copy link
Contributor

NullVoxPopuli commented Oct 13, 2018

well, nevermind then! my bundle does indeed have async/await in it :)
馃憤

@astronomersiva

This comment has been minimized.

Copy link

astronomersiva commented Feb 28, 2019

@tomdale, we have been experimenting with this in our apps.

Initially, we based our technique on the Lightning strategy, using two different index.htmls to serve assets.

We tried using the module/nomodule approach but like you pointed out, the defer behaviour of modules makes this impossible in our apps. I am curious as to how Vue makes this work even if vendor.js loads much after app.js finishes loading.

One more issue might be handling the app/config/asset-manifest meta in index.html that is needed for engines to work.

Currently, we are changing our approach to simultaneously generate two different builds with
ember build --environment=production --output-path=legacy & MODERN=true ember build --environment=production --output-path=modern and then merging them into a dist folder. Using posthtml, we extract script tags from modern's index.html and insert them in legacy's index.html. We wrap these tags with some markers and replace them at our server before serving to the user.

<MODERN>
<script src="{{rootURL}}assets/vendor-checksum.js"></script>
<script src="{{rootURL}}assets/<%= name %>-checksum.js"></script>
</MODERN>

<LEGACY>
<script src="{{rootURL}}assets/vendor-checksum.js"></script>
<script src="{{rootURL}}assets/<%= name %>-checksum.js"></script>
</LEGACY>
@Turbo87

This comment has been minimized.

Copy link
Member

Turbo87 commented Feb 28, 2019

The default defer semantics of <script type="module"> may not work for us here as I believe the app will break if vendor.js is not evaluated before app.js, so that is something we would want to investigate.

@tomdale @astronomersiva I'm not sure I understand this part. defer is still supposed to evaluate the scripts in the order that they appear in the file. async might be what you are talking about?

somewhat related: ember-cli/ember-cli#7632

/cc @NullVoxPopuli

@astronomersiva

This comment has been minimized.

Copy link

astronomersiva commented Mar 8, 2019

@Turbo87, you are right. There was another script tag(that stays the same in both legacy and modern versions because it just does window.something = { something else }) we had that depended on vendor.js and this was the actual reason behind the errors in our end. So, if we decide to go down the module/nomodule path, we have to ensure that all script tags are converted to contain a defer attribute.

However, since variables in modules do not get automatically declared globally, there are various errors caused in vendor.js like:

  • define is not defined
  • mainContext is not defined
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can鈥檛 perform that action at this time.
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session.