Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Avoid require cache busting whenever possible. #130

Closed
wants to merge 4 commits into from

Conversation

rwjblue
Copy link
Member

@rwjblue rwjblue commented Oct 11, 2018

Generally speaking, ember-cli allows each "addon layer" in the app that is transpiling its own templates. Each layer is supposed to be able to provide its own AST plugins without affecting the others. Unfortunately, the implementation of Ember.HTMLBars.registerPlugin mutates a single shared module scope array to track which plugins should be ran.

In order to allow the layered approach mentioned above, ember-cli-htmlbars was forced to invalidate the require.cache and delete various things off of the global object. Requiring ember-template-compiler.js (~1.2mb) many times (once for the project and again for each addon that depends on ember-cli-htmlbars) is a non-trivial percentage of every app's build time and overall memory.

This PR begins the process of removing that manual cache invalidation, in favor of better APIs provided by Ember (as of Ember 1.13 and higher).

In order to stop the manual cache invalidation but still provide the guarantees that each layer's AST plugins do not affect the others' we must still continue to invalidate the caches cache busting system if we detect that anyone in the system is using the global state mutation.


Specific changes in this PR:

  • Add a test that shows the failure mode of "leaking" AST plugins in global module state.
  • Update the compilation code to avoid leveraging global state mutation (e.g. don't use Ember.HTMLBars.registerPlugin), instead pass the plugins directly to Ember.HTMLBars.precompile for each module.
  • Fallback to manual cache invalidation if we detect that any addon in the system is using an older version of ember-cli-htmlbars or ember-cli-htmlbars-inline-precompile that still uses global state mutation.
  • Avoid purging the require.cache when the requirements are met.

TODOs:

  • PR review
  • detect Ember version and properly shim the AST plugins into the correct format (we need to prevent Ember from doing manual shimming/wrapping for each template compilation)

Future PR TODO:

  • Make similar changes to ember-cli-htmlbars-inline-precompile

This test demonstrates how each "layer" in an ember app is transpiling
its own templates. Each is _supposed_ to be able to provide its own AST
plugins, **but** this test shows how this falls down and fails.

The current mechanism for registering AST plugins is fundamentally
flawed. This forced us (long ago) to add some shenanigans to clear the
require cache (to avoid this exact state leakage).
This avoids calling `registerPlugin` (and therefore mutating global
state) but maintains support for custom AST plugins by passing for each
compilation.

This works for Ember 1.13 and higher.
@rwjblue rwjblue changed the title Failing test showing AST plugins "bleed" through all instances. Avoid require cache busting whenever possible. Oct 11, 2018
In order to stop the cache busting system, we need to:

* Update the actual compilation code to pass in our custom options (done
in a prior commit)
* Ensure that **no one** is mutating the global list of plugins (if they
are, then we will always get their plugins and therefore our compilation
is "spoiled" by their AST plugins)
* Avoid purging the `require.cache` when the requirements are met
Copy link
Contributor

@stefanpenner stefanpenner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM - this brings an end to an era. I still remember us debugging this originally, thanks for tackling this!

@@ -109,12 +187,24 @@ module.exports = {
pluginCacheKey: pluginInfo.cacheKeys
};

if (this.legacyPluginRegistrationCacheBustingRequired !== false) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the !== false here makes it hard for my brain to read this

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

kk, ya I can fix that up

@rwjblue
Copy link
Member Author

rwjblue commented Feb 19, 2021

Ugh, this has bit rot a bit. Need to circle back and try to get this landed...

@rwjblue
Copy link
Member Author

rwjblue commented Mar 7, 2021

The changes that landed in #660 + #661 (and emberjs/ember.js#19429) make this largely unneeded.

At this point, we no longer purge the cache and can even remove the vm.createSandbox work (or do it conditionally) once we only support Ember 4.0+.

@rwjblue rwjblue closed this Mar 7, 2021
@rwjblue rwjblue deleted the global-state-is-bad branch March 7, 2021 20:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants