Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RFC: Rethink polyfilling story #10008

Open
nicolo-ribaudo opened this issue May 21, 2019 · 8 comments

Comments

Projects
None yet
4 participants
@nicolo-ribaudo
Copy link
Member

commented May 21, 2019

In the last two years and a half, @babel/preset-env has shown its full potential in reducing bundle sizes not only by not transpiling supported syntax features, but also by not including unnecessary core-js polyfills.
Currently Babel has three different ways to inject core-js polyfills in the source code:

  • By using @babel/preset-env's useBuiltIns: "entry" option, it is possible to inject polyfills for every ECMAScript functionality not natively supported by the target browsers;
  • By using useBuiltIns: "usage", Babel will only inject polyfills for unsupported ECMAScript features but only if they are actually used in the input souce code;
  • By using @babel/plugin-transform-runtime, Babel will inject ponyfills (which are "pure" and don't pollute the global scope) for every used ECMAScript feature supported by core-js. This is usually used by library authors.

Our position in the JavaScript ecosystem allows us to push these optimization even further. @babel/plugin-transform-runtime has big advantages for some users over useBuiltIns, but it doesn't consider target environments: it's 2019 and probably very few people need to load an Array.prototype.forEach polyfill.
Additionally, why should we limit this ability to automatically inject only the necessary polyfill to core-js? There are also DOM polyfills, Intl polyfills, and polyfills for a myriad of other web platform APIs. Additionally, not everyone wants to use core-js: there are many other valid ECMAScript polyfills, which have different tradeoffs (e.g. source size vs spec compliancy) and may work better for some users.

What if the logic to inject them was not related to the actual data about the available or required polyfills, so that they can be used and developed independently?

Concepts

  • Polyfill provider is a Babel plugin which, by itself, doesn't have any observable behavior. It is used to specify which JavaScript expressions need to be polyfilled, where to load that polyfill from and how to apply it. Multiple polyfill providers can be used at the same time, so that users can load, for example, both an ECMAScript polyfill provider and a DOM-related one.
    These plugins don't actually transform the code, and don't expose any AST visitor.

    Polyfill providers can expose three different methods of injecting the polyfills:

    • entry-global, which reflects the current useBuiltIns: "entry" option of @babel/preset-env;
    • usage-global, which reflects the current useBuiltIns: "usage" option of @babel/preset-env;
    • usage-pure, which reflects the current polyfilling behavior of @babel/plugin-transform-runtime.

    Every interested project should have their own polyfill provider: for example, babel-plugin-corejs3-provider or @ungap/babel-plugin-provider. Since they are normal plugins, it's not enforced that they contain provider but it will be a reccomended best practice.

  • Polyfill injector is a Babel plugin which analyzes the program AST, finds any expression which might need a polyfill (i.e. global references or property accesses) and checks if there is any polyfill provider which can handle them.

    The ecosystem will only need a single polyfill injector, part of the "official" Babel packages: it could either be included in @babel/preset-env, or a new @babel/plugin-inject-polyfills package.

New user experience

Suppose a user is testing some ECMAScript proposals, they are localizing their application using Intl and they are using fetch. To avoid loading too many bytes of polyfills, they are ok with supporting only commonly used browsers, and they don't want to load unused polyfills.

How would their new config look like?

{
  "presets": [
    ["@babel/preset-env", { "targets": [">1%"] }]
  ],
  "plugins": [
    ["@babel/inject-polyfills", { "method": "usage-global" }],
    "intl-polyfill-provider",
    "fetch-polyfill-provider",
    ["corejs3-provider", { "proposals": true }]
  ]
}

New developer experience

In order to provide a new cross-plugin communication protocol which can be safely used by polyfill providers developers, we must provide a few utilities to abstract away how this communication happens: we will probably use a shared entry in the File's class Map (like we already do for @babel/helper-create-class-features-plugin), but we should not expose it to the developers to restrict the possible sources of bugs and implementation complexity.

We can either create a new @babel/helper-polyfill-provider package, or export these helpers from @babel/plugin-inject-polyfills. This new API will look like this:

import createPolyfillProvider, { injectImport } from "@babel/helper-polyfill-provider";

export default createPolyfillProvider(({ types: t }, options) => {
  return {
    name: "object-entries-polyfill-provider",
    
    entryGlobal(name, targets, path) {
      if (name !== "object-entries-polyfill") return false;
      if (!supportsEntries(targets)) {
        injectImport("object-entries-polyfill/global", path);
      }
      path.remove();
      return true;
    },
    
    usageGlobal(name, targets, path) {
      if (name !== "Object.entries") return false;
      if (!supportsEntries(targets)) {
        injectImport("object-entries-polyfill/global", path);
      }
      return true;
    },
    
    usagePure(name, targets, path) {
      if (name !== "Object.entries") return false;
      if (!supportsEntries(targets)) {
        const id = path.scope.generateUidIdentifier("Object.entries");
        injectImport("object-entries-polyfill/local", path, id);
        path.parentPath.replaceWith(id);
      }
      return true;
    }
  }
});

The createPolyfillProvider function will take a polyfll plugin factory, and wrap it to create a proper Babel plugin. The factory function takes the same arguments as any other plugin: an instance of Babel's API and the plugin options.
It's return value is different from the object returned by normal plugins: it will have an optional method for each of the possible polyfill implementations. We won't disallow other keys in that object, so that we will be able to easily introduce new kind of polyfills, like "inline".

Every polyfilling method will take three parameters:

  • The name of the built-in to be polyfilled.
    • In case of global primitive variables, it will be the function name (e.g. Promise);
    • in case of static methods of globals, it will be the name of the object followed by the name of the method (e.g. Object.entries);
    • if we can't determined what is the object owning the method or property (e.g. instance methods), it will be an asterisk followed by the name of the method (e.g. *.includes).
  • The engines the users want to support. This could be a falsy value, in case the users didn't specify it.
  • The NodePath of the expression which triggered the polyfill provider call. It could be an ImportDeclaration (for entry-global), an identifier (or computed expression) representing the method name, or a BinaryExpression for "foo" in Bar checks.

Polyfill providers can either return true or false. The polyfill injector will call every polyfill provider, in the order they are specified in the user's configuration, until one of them returns ŧrue. That way, polyfill providers will be able to ignore polyfills they are not able to handle.
Note that, if a polyfill could be injected but it isn't because it is already supported by the target engines, the polyfill provider should return true: it has been handled, by explicitly choosing not to act.

Polyfill providers will be able to specify custom visitors (like normal plugins): for exapmle, core-js needs to inject some polyfills for yield* expressions.

How does this affect the current plugins?

Implementing this RFC won't require changing any of the existing plugins.
We can start working on it as an experiment (like it was done for @babel/preset-env), and wait to see how the community reacts to it.

If it will then success, we should integrate it in the main Babel project. It should be possible to do this without breaking changes:

  • Remove the custom useBuiltIns implementation from @babel/preset-env, and delegate to this plugin:
    • If useBuiltIns is enabled, @babel/preset-env will enable @babel/inject-polyfills
    • Depending on the value of the corejs option, it will inject the correct polyfill provider plugin.
  • Deprecate the regenerator and corejs options from @babel/plugin-transform-runtime: both should be implemented in their own polyfill providers.

Open questions

  • Should the polyfill injector be part of @babel/preset-env?
    • Pro: it's easier to share the targets. On the other hand, it would be more complex but feasible even if it was a separate plugin.
    • Con: it wouldn't be possible to inject polyfills without using @babel/preset-env. Currently @babel/plugin-transform-runtime has this capability.
    • If it will be only available in the preset, @babel/helper-polyfill-provider should be a separate package. Otherwise, we can just export the needed hepers from the plugin package.
    • It could be a separate plugin, but @babel/preset-env could include it.
  • Who should maintain polyfill providers?
    • Probably not us, but we shouldn't completely ignore them. It's more important to know the details of the polyfill rather than Babel's internals.
    • Currently the core-js polyfilling logic in @babel/preset-env and @babel/plugin-transform-runtime has been mostly maintained by Denis.
    • It could be a way of attracting new contributors; both for us and for the polyfills maintainers.
    • Both the regenerator and core-js providers should probably be in the Babel org, since we have been officially supporting them so far.
  • Are Babel helpers similar to a polyfill, or we should not reuse the same logic for both?
  • (Proposed by @danez) Why are polyfill providers plugins? Should they be passed as an option to the plugin injector?
    {
      "presets": [
        ["@babel/preset-env", { "targets": [">1%"] }]
      ],
      "plugins": [
        ["@babel/inject-polyfills", { 
          "method": "usage-global",
          "providers": [
            "intl",
            "fetch",
            ["corejs3", { "proposals": true }]
          ]
        }]
      ]
    }
    • Advantage: it would simplify the polyfill injector implementation, since there won't be need to implement cross-plugin communication.
    • Advantage: having all the polyfill-related info might make it easier to understand.
    • Advantage: since we aren't tied to the plugins naming schema, we could easily introduce a new one (e.g. babel-polyfill-*)
    • Disadvantage: we would need to duplicate Babel's plugins/presets resolution algorithm
    • Disadvantage: it's harder to apply a polyfill only for a specific subfolder. For example,
      module.exports = {
        plugins: [
          ["@babel/inject-polyfills", { "method": "usage-global" }]
          
          // Add core-js polyfills to the wole project
          "corejs3-provider",
        ],
        overrides: [{
          plugins: [
            include: "./web",
          
            // Only add dom plugins in the web/ folder
            "dom-provider"
          ]
        }]
      };
      

Related issues

@schmod

This comment has been minimized.

Copy link

commented May 21, 2019

Is there a short-term possibility of just creating a fork of transform-runtime that's exclusively used within preset-env?

In that scenario, we wouldn't need to rethink Babel's entire plugin architecture, but could offer significant bundle-size reductions to most library authors with a (considerably) lower level of effort.

Then, we could expose an API that looks something like:

{
  "presets": [
    ["@babel/preset-env", { "targets": [">1%"], "corejs": 3, "transformRuntime": { 
      "helpers": true, "regenerator": false 
    } }]
  ]
}

I think this proposal has considerable long-term merits, but it also seems like we can achieve 95% of what people need in a much shorter timeframe.

@rjgotten

This comment has been minimized.

Copy link

commented May 22, 2019

Had pretty much the same idea as @schmod but he beat me to it.

Never quite understood why several parts of transform-runtime were not brought over to preset-env before. E.g. the deduplication of helpers like class construction into separate modules; that seems like a very beneficial general purpose optimization you would want people to take. And thus seems quite a sane default. There will be people that need to compile one self-contained module with the helpers inlined. And they can turn that off.

Now, with env's browserslist becoming a bigger thing with conditional polyfilling, it just seems like it makes more sense to integrate the transform-runtime plugin wholesale.

"useBuiltIns" : "pure" maybe?

And then when that's all fine and working -- and tested in practice, work on adding the layer of indirection and extensibility with polyfill providers.

@nicolo-ribaudo

This comment has been minimized.

Copy link
Member Author

commented May 23, 2019

"useBuiltIns": "pure" would definetly work in the short term. Given our current release cadence (a minor version every about 2 months), I fear that the whole @babel/plugin-inject-polyfills would come in the minor release right after useBuiltIns: "pure"*, and I'd prefer to avoid introducing an option which will be soon be legacy.

@schmod If @babel/transform-runtime would magically get the targets from @babel/preset-env, how would that config be better than how it would be currently done? Personally, I feel that helpers and regenerator don't really fit in @babel/preset-env since they are about an optimization ortogonal to the one introduced by the preset.

{
  "presets": [
    ["@babel/preset-env", { "targets": [">1%"] }],
  ],
  "plugins": [
    ["@babel/transform-runtime", {
      "corejs": 3,
      "helpers": true,
      "regenerator": false,
    }]
  ]
}
@devinus

This comment has been minimized.

Copy link

commented Jun 9, 2019

I'd love to see a simple useBuiltIns: 'pure' option while still retaining the functionality of @babel/transform-runtime for people using it today. I was honestly surprised when I learned that it wasn't a thing already.

It would be relatively easy to implement and provide an immediate upgrade path without worrying about deprecating anything people are already using.

That doesn't mean keeping it around forever, but it's a lot easier to steer people already using pure to the new hotness after it's ready rather than quietly introducing a whole new thing.

@devinus

This comment has been minimized.

Copy link

commented Jun 9, 2019

Personally, I feel that helpers and regenerator don't really fit in @babel/preset-env since they are about an optimization ortogonal to the one introduced by the preset.

Really, helpers is the only one that doesn't make sense in preset-env since there are environments that legitimately support generators today.

@rjgotten

This comment has been minimized.

Copy link

commented Jun 9, 2019

That doesn't mean keeping it around forever, but it's a lot easier to steer people already using pure to the new hotness after it's ready rather than quietly introducing a whole new thing.

Well put. For something as big as an entire injection architecture around polyfills, easing people in with an easy-to-adopt upgrade path first has some benefits that should be considered, imho.

Basically; it prevents people from running into a brick wall; or from slamming on the brakes full-stop because they think they're going to.

Speaking from my own experience, I think the 'full' solution would be kind of a hard sell for some of the development teams I'm involved with, whereas the outwardly deceptively simple API of just another config switch value can do a lot to mitigate concern.

@nicolo-ribaudo

This comment has been minimized.

Copy link
Member Author

commented Jun 9, 2019

Thank you all for your feedback.


@devinus

Really, helpers is the only one that doesn't make sense in preset-env since there are environments that legitimately support generators today.

You are right, regenerator is just another polyfill. Actually, when I started working on this RFC babel-polyfill-provider-regenerator is the first one I implemented to test it 😅


@rjgotten
If useBuiltIns: "pure" and this RFC would land at the same time, do you think that useBuiltIns: "pure" would still be that valuable?
To ease the migration, adding it to the preset once this RFC is implemented would probably be trivial, but it wouldn't make it possible to add other polyfills (e.g. node APIs or web APIs) without using two parallel polyfill injectors.

@rjgotten

This comment has been minimized.

Copy link

commented Jun 10, 2019

@nicolo-ribaudo

If they would land simultaneously, I still think there's value in there being an easy adoption mode that gets users going and can ween them off of babel-runtime based polyfills. For the full package -- i.e. not just ES polyfills but also Node; Web platform; etc. -- you'd still need to go 'all in' afterwards, but that can still be a 'stage 2' of a migration process. (And one that'll be considerably easier to sell, also to a non-technical management tier, imho.)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.