Babel Roadmap (T7154) #4130

Closed
babel-bot opened this Issue Feb 27, 2016 · 10 comments

Projects

None yet

3 participants

@babel-bot
Collaborator
babel-bot commented Feb 27, 2016 edited

Issue originally made by @hzoo

Description

Roadmap/Ideas

Previous 6.0 Discussion: https://phabricator.babeljs.io/T2168

Just wrote this up real quick a week ago so for some ideas we should definetely do and others :shrug:

Ease of use

Internal Development

Community

  • Can always work on..
    • Develop in the open more (#development slack)
    • Making it easier to contribute: better docs, easier setup, etc (do we want to use github issues again)
    • Reaching out for help with issue triage, more contributors, etc
    • Users could submit PRs with reproducible test cases instead of just posting snippets (ex: #3253)
    • Watch open source projects using Babel
  • Roadmap (something like this)
  • How to write plugins (https://github.com/thejameskyle/babel-handbook)
  • Post a simple newsletter for what's been going on in the babel community (new plugins, blogs, videos, ideas, etc)?
  • Bring back a FAQ, add common stackoverflow/slack questions, etc.
  • Documentation of APIs

Wish List

  • Automating generating a config of presets/plugins based on target environments (caniuse/autoprefixer, or dynamically)
  • Syntax Plugins
    • Move babylon out of babel repo
    • Move jsx/flow out of babylon
    • relevant PRs #3376
  • Cross-file transformations, knowing the dependency graph
  • Other tooling intergrations (Minifying, Linting, Flow/Typescript)
  • Async API
@babel-bot
Collaborator

Comment originally made by Jess Telford (jesstelford)

+1 on using GitHub issues instead of Phabricator for discussions. Go where the community is.

Also; I'd like to see a roll-back of the lock-step versioning of all the babel plugins/modules. It causes bugs like [[ https://phabricator.babeljs.io/T6930 | the Browserified Babylon bundle ]] to take way too long to be fixed.

@babel-bot
Collaborator

Comment originally made by @hzoo

@jesstelford Ah yeah. We do need to figure out the versioning and all that better, but in the meantime, releasing more often especially for bug fixes (and be ready to revert/hotfix if issues come up) should help somewhat. Or we figure out how lerna/lerna#65 will work so we can do a npm publish per PR.

@babel-bot
Collaborator

Comment originally made by @AlicanC

Automating generating a config of presets/plugins based on target environments (caniuse/autoprefixer, or dynamically)

I think targets need their own place in .babelrc:

{
  // Declare the needs of my project
  "presets": ["react", "es2015"],

  // Declare the targets of my project
  "targets": {
    "some-browser": {
      "presets": ["!react"], // Target browser supports Flow and JSX, wow!
      "plugins": [
        "!transform-es2015-modules-commonjs", // CommonJS won't work on browsers
        "transform-es2015-modules-amd" // But we need AMD
      ],
    }
  }
}

So what we do here is:

  1. We introduce negation in ".babelrc" so plugins can be turned off. (This can also help solving the "Preset options" issue.)
  2. We introduce "targets" in ".babelrc" so we can turn plugins on and off depending on the target.

So when you run babel script.js you get react + es2015 and when you run babel script.js --target some-browser you get es2015 - commonjs + amd.

Targets should also be able to shipped like presets so we can do this:

{
  "presets": [...],
  "targets": ["node-lts", "node-stable", "browser-legacy", "browser-evergreen"]
}

This could introduce a whole new flow to the ecosystem:

  1. Someone creates the "browser-evergreen" package.
  2. AwesomeFeature becomes available on Edge X, Firefox Y, Chrome Z, ...
  3. Adoption rates of those versions (and above) become 95%. (99%? 100%? Whatever is safe.)
  4. Owner of the "browser-evergreen" package marks AwesomeFeature as supported in preset.
  5. Users of "browser-evergreen" automatically gets reduced code size and increased performance.
  6. We have a better web.

All these years we blamed IE for not being evergreen and now we aren't even taking advantage of features in evergreen browsers since we recklessly transpile to ES5. If we can say that ES5 is highly adopted today, a day will come that we will say arrow functions or some other feature is highly adopted. Since all major browsers are now evergreen, adoption of new features will be faster than ever. Our build tools should help us move forward and that's why I think targets and target presets is one of the most important items in the list.

@babel-bot
Collaborator

Comment originally made by @wmertens

! In T7154#75111, @dtryon wrote:
Fixing semver would help a lot.

For example, if babel-core has a version number of 6.5.2:
https://github.com/babel/babel/blob/v6.5.2/packages/babel-core/package.json

Then, its dependencies should not have semver punctuation with the ^ symbol (Caret Ranges):
example: "babel-code-frame": "^6.3.13"

Caret Ranges will only preserve the left-most version digit, in this case "6.x.x".

This means a change to the minor version (which could be a breaking change), will break all previous versions.

The semver strategy should be fixed so that previous versions continue to work allowing projects to opt-in to upgrades.

For reference: https://github.com/npm/node-semver

I thought the whole point of semver was that minor versions are only for adding functionality? So that should never break the API.

Major versions are what can break APIs, and that's what ^ is for…

@babel-bot
Collaborator

Comment originally made by @dtryon

@vmertens

Yes, you are correct. I guess my comment was a bit misdirected.
This was mainly coming out of yesterdays update that caused a lot of regressions.

Perhaps, a different release / testing process would be a better suggestion. :)
Thanks.

@babel-bot
Collaborator

Comment originally made by @hzoo

All good points, and why I brought it up in the roadmap

Better infrastructure release process with lerna

However, @loganfsmyth brought up to me the last release could of been a patch version (fixing the es3 transform) and that still would of caused a regression due to how babel-traverse was changed so ~ wouldn't of helped in that case (meaning a patch version can still cause a regression).

@babel-bot
Collaborator

Comment originally made by @amasad

Here are some thoughts on what I'd like the Babel architecture to look like in two major areas in one of our next major releases. This is not concrete enough to make them action points on the roadmap. Once I start prototyping and exploring these ideas further I'll formulate them into action items or tasks.

State

We currently keep much of the cache and state on the AST which is troubling because that means it travels around wherever the AST goes. Here are some potential bugs (most of which I ran into in some capacity):

  1. A common pattern that people use in plugins is that they build AST nodes in the module scope (as "constants") and use them in multiple places. This leads to shared-memory-like-bugs where if we are operating on multiple files in the same process we reuse the cache on these nodes which leads to using NodePaths that are invalid.
  2. When passing an AST around across multiple tools or multiple babel versions the information/objects on the AST may not match the version of babel on the receiving end leading to tricky bugs.
  3. Updating state is a currently really hard. Many of our core plugins doesn't even do it right. Anytime we change the AST the scope, binding, and references info needs to be somehow recalculated and currently we rely on doing this manually. Coupled with AST-based caching this leads to outdated information. (see [[ https://github.com/babel/babel/pull/3389 | this pull request for an example of having to update scope info manually ]])

A potential solution which I don't claim I have thought through enough is to have state saved as a graph of weak-reference associations between nodes and analysis could make this much easier. For example to get the references for a certain binding you just access the binding graph by the var declar node as the key which will return all the references. If the node is replaced or removed from the AST, it will be garbage collected and will automatically be removed from the graph. Additionally, if the AST is immutable then for any given node the information should always be up-to-date

Mutations and perf

As evident from the previous section, we rely heavily on mutating the AST for transformations and state-keeping. I think that if we moved the AST to be a persistent data structure we could unlock some performance wins:

  1. The first thing this unlocks is AST-caching to save on parsing the exact same source text.
  2. In constructing transformation pipelines you sometimes need to generate multiple versions of the same file (say one for production build and one for development). The common way of doing this now is to reparse and retransform the file for different configuration which means a lot of duplicate work. Keeping a copy of the data-structure before a certain transformation means that we can fork the pipeline and send it down a different path without repeating the work.
  3. This can speed up generation by reusing part of the original source that was untouched after transformation. The generator "diffing" operation can be merely reference equality checks.
@babel-bot
Collaborator

Comment originally made by Jess Telford (jesstelford)

@hzoo:

releasing more often especially for bug fixes (and be ready to revert/hotfix if issues come up) should help somewhat.

What about when thinking on CDN / cachability? Ie; will having multiple versions with exactly the same content mean people's cache's aren't utilized to their maximum capability?

Looking at babel-polyfill. Its last [[ https://github.com/babel/babel/commit/79214b3b05f1c46c6c1176f1c1d05080c284849e#diff-3a66980cf45934765253153cf92550c8 | actual code change ]] was 27 days ago, published in v6.6.0, and later popped onto cdnjs at v6.6.1. However, as of writing, the latest version of the monorepo is actually v6.6.5 (which is [[ https://github.com/babel/babel/blob/master/packages/babel-polyfill/package.json#L3 | not reflected ]] in babel-pollyfill's package.json).

This seems confusing (to me) and counter to the ideals behind a mono-repo. In this case; I'd argue in favour of a separate repo for babel-polyfill which is tagged, versioned, published, and CDN'd independently of the greater babel.

@babel-bot
Collaborator

Comment originally made by @hzoo

The specific reason why babel-polyfill is at v6.6.1 was because I forgot to run the script which changes babel-polyfill and thus had to rerun it (I added it to the build script so that won't happen again) and publish v6.6.1 (and it was 27 days ago because we merged the PR but didn't do a release until then). And yeah it's not 6.6.5 because it hasn't been changed since then

@danez danez added the discussion label Oct 17, 2016
@hzoo
Member
hzoo commented Nov 24, 2016

Going to make a new issue for this - we did some of the things ^

@hzoo hzoo closed this Nov 24, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment