How do we want to handle .babelrc lookup in 7.x

[loganfsmyth]

Edit 2

Most recent status: #6766 (comment)

Edit 1

All below still applies, but also see more explanation in #6766 (comment)

Original:

Not a fleshed out proposal, but filing for tracking and discussion.

Currently if you have:

.babelrc
src/
  index.js
  utils/
    .babelrc
    other.js

compiling index.js will use the root-level .babelrc, and other.js will use the src/utils/.babelrc. I assume this pattern was pulled from other utilities like ESLint, but not 100% sure.

We're at the point now where this approach has become a problem for us because:

1. Very few people seem to expect this behavior to begin with.
2. People expect their project-level .babelrc file to also apply to things that live in node_modules. Babel 6 was terrible at figuring out how to handle node_modules because it always assumes things are strict mode, and always rewrites this, but a lot of those issues aren't as big in Babel 7, or at least ideally won't be, because we plan to have better handling of the sourceType: script vs sourceType: module distinction. This kind of works now, but see point (3).
3. If people publish packages to npm and include .babelrc files in their packages, or more troublingly, use the babel key in package.json (which you can't really get around publishing), users trying to use Webpack or anything like it to compile node_modules get extremely confusing errors about not being able to find plugins and presets referenced in those files. See Enable babel-preset-env for node_modules that target newer Node versions facebook/create-react-app#1125 for a massive list of users that want to compile node_modules, but would absolutely be bitten by this issue.

So where does that leave us? The simplest thing I can think for us to do would be to change up Babel's API to, by default, accept a root directory as an argument where it would search for the config, defaulting to the working directory.

We're already at a point where I've thought about adding an argument for root directory to resolve the filename relative to, so that should be uncontroversial. Actually changing config resolution is a bigger question.

It would be pretty easy for us to allow users to opt for the old behavior, so I think my proposal would probably be to add this new root-dir config resolution logic and make it the default, but keep the old behavior around as an opt in. So where now we have

babelrc: boolean

to toggle config resolution off and on, I could imagine us doing

babelrc: true | "root" => Look for config in root directory
babelrc: "relative" => Look for config, starting from compiled file, working upward
babelrc: false => Skip search entirely.

instead.

The primary loss of functionality here is that if users do want to use .babelrc files inside nested folders to change Babel configs based on path, they don't have an easy way to do it anymore. I don't however have a good grasp of how many users that might be. It seems like we may want to consider #5451 as something to be included in this effort if we do want to change, which might end up blocked on #6765.

[hzoo]

Seems totally reasonable to me

• I haven't ever seen a nested babelrc file, not sure how we can find out
• I assume everyone would want to transform all their code in the same way (sounds hard to debug/understand)
• Most people have 1 babelrc so they would assume Babel only would look at that config. I would be surprised it would try to read a babelrc from node_modules
• Default to the intuitive behavior that helps us support new use cases while providing an opt-out sounds good to me 👍 👍 let's do it.
• Given we ourselves liked using overrides for ESLint yeah we can consider [Feature Request] feature similar to ESLint "overrides" (config based on glob) #5451

[kentcdodds]

Thanks for all your work on babel folks! 👏

Here are some thoughts:

• I really like ESLint's model. I like the whole thing, in particular how the root: true functionality works and just minutes ago was looking to see if babel had support for something like this. I'm not even joking.
• I don't ever want to transpile node_modules, but I think that it should be possible and I would expect that the hierarchy should apply there as well. I can't speak too much to this though because I'm not big on the idea of transpiling node_modules. I just feel like there's probably a use case for it. But making it a little difficult to do isn't a bad thing IMO.

I hope that's helpful! Thanks again!

[hzoo]

@kentcdodds

There is a babelrc: false option which stops lookup as mentioned ^

Compiling node_modules is for when you want to consume a package from npm that is ES2015+. Maybe you support a different environment so you need to compile it down to a different level than ES5. Example: maybe a react package uses async but you don't want to use the "main" file which uses regenerator, so you want the package to output async. It may or may not be transpiled to async (no change), generators, or regenerator depending on the users preset-env targets

Ya it's interesting that eslint would never have this issue since there's no need to look at node_modules there 😛

[kentcdodds]

There is a babelrc: false option which stops lookup as mentioned ^

👍

Solid use case example 👍 So long as it's possible that'd be good with me. And if I could disable the lookup using webpack/babel-cli etc. then that'd be sufficient I think

[niieani]

How about non-precompiled/non-distributable monorepos? This is currently what we do at @futuresimple, so e.g. we have packages/pkg-name, each with its own version of .babelrc that can vary across the packages in the monorepo. On the other hand, the root folder (repository root) doesn't have any babel config.

To handle such a case well, I think the default behavior should be to use the .babelrc next to the closest package.json, rather than closest to the file (as is now).

[sokra]

I've seen nested babelrc files in repos which contain server and client build.

[hzoo]

I'm sure not everyone here will have read everything but we could possibly do #5451 which is an "overrides" option similar to eslint which gives you glob config + different config so you don't need nested .babelrcs.

[nicolo-ribaudo]

I'm strongly in favor of only using the root babelrc, since config files in nested directories are really easy to miss.
If people want more control on their builds, they can use tools like gulp.

+1 for #5451

[niieani]

@hzoo #5451 won't help for monorepos (when used without a build step). You'd have to re-declare all the babelrc's in the root of the monorepo, which doesn't sound fun.

When you install packages in a monorepo, they end up symlinked as node_modules/MONOREPO_PACKAGE.
Since each package can have its own .babelrc, we depend on babel resolving those .babelrc's to transpile the code from those modules using the settings they declare.

I am all for this change, as long as it's possible to declare in the root .babelrc something like deepResolveBabelRc: true. Having this as a programmatic option in the API (as proposed in the issue) won't work for cases where we'd like to directly run: node -r babel-runtime node_modules/MONOREPO_PACKAGE/script.

[loganfsmyth]

@niieani Do you have example where you require different babel configs for different packages in a monorepo? I'd have thought they'd be very similar, and thus something like #5451 might be enough for your case.

[Andarist]

@loganfsmyth I imagine when having both front end and back end code in your monorepo you can have different babel configs. Also personally I like to be explicit about my deps, even used transforms, and I do not include a jsx transform when transpiling utility library even though in other directory of the monorepo I have a React UI library

[niieani]

@loganfsmyth exactly as @Andarist says. We have both CLI and Frontend packages in the monorepo, at the same time we'd like to be able to open source some of the packages, so each package in the monorepo should be able to be run without the dependency of the root directory of the monorepo. #5451 introduces such a dependency, i.e. the package won't work when it's moved out of the monorepo into it's own repository via a git subtree.

[loganfsmyth]

Sounds good, thanks for the clarification. So it is not that there is are package-level .babelrcs that might merge with the one at the monorepo root, it's essentially that each inner package could be thought of as having its own root.

[Andarist]

As for me - that would be desired. If I'd like to extend some other config I might do that easily with .babelrc.js, but in most cases I just want to keep specific babel config per package, even if they all live in monorepo at the moment.

[benjamn]

Meteor supports nested .babelrc files, for essentially the same purpose that @sokra cited above: #6766 (comment)

We do not attempt to merge multiple .babelrc files in ancestor directories. The first one found in an ancestor directory within the project root takes precedence.

I'm strongly in favor of only using the root babelrc, since config files in nested directories are really easy to miss.

@nicolo-ribaudo So don't use nested .babelrc files in your own projects. Taking that flexibility away from developers who want it is paternalistic and not helpful.

[loganfsmyth]

@benjamn Would #5451 address any of your concerns?

That would potentially allow you to do something like

{
  overrides: [
    {
      test: "./client",
      extends: "./client/.babelrc.js",
    },
    {
      test: "./server",
      extends: "./server/.babelrc.js",
    },
  ]
}

so the root config could still easily delegate to another config.

I could also imagine the monorepo case being handled like

{
  overrides: [
    {
      test: "./packages/*",
      extends: "<TEST>/.babelrc.js",
    },
  ]
}

for instance, so the root config would control everything, but still allow for flexibility.

Alternatively we could have the root config allow a relativeSearch: boolean flag, so if set, it'd load the root config, but then do search relative to the file.

Maybe we could even do overrideConfigs: "./packages/*" to automatically search in the given path for configs, and then essentially allow overrideConfigs: "**/*" to essentially say "search everywhere" like we do now.

I feel like I'd want the file to merge on top of the root config by default, but maybe we allow config files to say overwrite: true to overwrite existing config values when they load?

My point is, there are tons of options that would allow essentially the same flexibility. I just want it to be explicit instead of implicit in Babel's core behavior, because the implicit relative lookup is causing problems, as stated in the original issue.

[novemberborn]

AVA, which uses Babel to compile test files, can be directed to inherit from a user's Babel config. We only look for .babelrc files in the project directory (next to the package.json file), or in the package.json file itself. This was done so the Babel options only need to be resolved once, rather than for each test file.

In other words I'm 👍 on this proposal (as stated in @loganfsmyth's opening comment).

[ljharb]

(I also agree with #6766 (comment))

I occasionally create a nested .babelrc file in my tests directory, so that certain transforms only apply to test files (not the same as "only transform in the test NODE_ENV"). Similarly, I might want certain transforms in a dev-only directory, and different ones in a production directory.

How would I do this with this proposal?

[xtuc]

Just my 2 cents. I would like to explicitly pass the configuration file location and in addition I would disable completely the config lookup (not a BC since it's a new option).

This would solve many issues with "how can I use Babel from this dir? can I store my config there? ..." and probably some .babelrc config inheritance issues.

[loganfsmyth]

@ljharb What are your thoughts on my comments in #6766 (comment) I'm definitely not looking to break that usecase, though I am promoting changing the implementation and mental model behind how they would fit in.

If I had to summarize the goal of this issue, it's to try to clarify that Babel configs are configuring a project as a whole with compilation logic, rather than defining configs for specific files. That means files in node_modules shouldn't have their configs processed unless the root project asks for that, and project-root configs should be able to influence how node_modules are compiled. As Node gets more and more ES6 features, people publish more and more modules to npm with ES6 modules. We can tell people to compile to ES5 before publishing all we want, but modules containing ES6 are going to happen either way, and I think it is important that we position Babel in a place where it can address those concerns without it being extremely painful for users.

Additionally, I'm very concerned about the pain users will feel if they run Babel 7 and some of their node_modules have published .babelrc files meant for Babel 6. Things will fail quite badly and they'll have no way to fix it.

[ljharb]

Certainly, eslint 4+'s glob overrides approach would address the use case, but then it doesn't allow me to "hide" the overridden settings inside the directories that have different approaches. Perhaps this isn't a big deal.

It's a fair point that people who try to transpile node_modules (which I hope are still ignored by default), would run afoul of this issue simply by a module using different plugins/transforms than they have installed, and even more so with babel 6 vs 7.

Perhaps - tangential to this issue - babel configs for 7+ should be required to explicitly declare their version number?

Additionally, perhaps those who choose to transpile code they didn't write should be required to explicitly specify the location for the config (whether that's a specific .babelrc, or a specific package.json's "babel" section)?

[jdalton]

Specifying parent config and override paths seems like heavy config for answering the question of should this config be treated as done or rolled into a parent config. I think something like a "root": true prop could be used to answer yes or no to whether to roll-up the config to the parent or not. So folks wanting separate configs for tests can just add a "root": true to those configs and the default behavior would be that it rolls-up to the parent.

[loganfsmyth]

I think something like a "root": true prop could be used to answer yes or no to whether to roll-up the config to the parent or not.

We already stop at the first .babelrc that is encountered, so root: true wouldn't have any effect. Users currently instead have to opt in with a parent like extends: "../../.babelrc" if they wanted to have a parent config continue applying.

I should clarify my main goal on this.

Primary Issue

The primary issue at the core of this is that people often expect their project's .babelrc to be that, a project-level configuration. By having the files resolved relative to each individual file, it means we pick up entirely unrelated .babelrc files in node_modules. In Babel 6, we occasionally got reports early on because people had published Babel 5 .babelrc files. When we publish Babel 7, that's 100% going to happen again, and as we start publishing new versions of Babel more often, it's going to get even more confusing for users, especially if we decide to make more breaking changes in the future.

Even in the case of dependency modules with .babelrc for Babel 7, even if it is for the same version, the config file could easily have been one that already ran to create the code that is already published to npm anyway. The only place that actually knows what config a node module should actually use is the root application itself, not the node module itself.

An alternative to my original proposal would be that we check if a file is in node_modules, e.g.

project/
  node_modules/mod/
    .babelrc
    src.js
  .babelrc

If this project tries to compile project/node_modules/mod/src.js, we could immediately jump upward and start resolving above the node_modules folder, thus finding the project-level .babelrc

The primary edge case there is that I guess technically your project itself could for some reason be inside a folder called node_modules, but we could certainly accept that as something that will just not be supported by users.

To expand, right now we do:

const filepath = ...
let dirname = path.dirname(filepath);
while (true) {
  if (exists(path.join(dirname), ".babelrc")) // ...
  const parent = path.dirname(dirname);
  if (parent === dirname) break;
  dirname = parent;
}

but you could imagine us changing this to (as non-production code anyway)

const filepath = ...
let dirname = path.dirname(filepath);

// Chop off everything after the first "node_modules" in the path
let parts = dirname.split(path.sep);
let modIndex = parts.indexOf("node_modules");
if (modIndex !== -1) {
  dirname = parts.slice(0, modIndex).join(path.sep);
}

// Resolve as usual.
while (true) {
  if (exists(path.join(dirname), ".babelrc")) // ...
  const parent = path.dirname(dirname);
  if (parent === dirname) break;
  dirname = parent;
}

[jdalton]

but you could imagine us changing this to (as non-production code anyway)

I think that's how I thought it would work. Here's how I resolve config for std/esm:

I walk backwards in the directory structure, stopping at node_modules with a return of null if a config is not found. The mechanism is called getting package info (PkgInfo). So it's gated to packages. This means if I roll up config it stops at a package boundary. This also means I could add support for a "root":true to indicate not rolling up to the package boundary and stopping a that given config.

For folks that want to force a set of options across package boundaries we have a programmatic convention for that.

Update:
Clarified things a bit.

[loganfsmyth]

I walk backwards in the directory structure and stop at "nod_modules". The mechanism is called getting package info. So it's gated to packages.

Could you clarify that or point me at particular code?

In my description the config inside node_modules be be explicitly ignored, which I think is the opposite of what you're describing? Though that may make more sense for std/esm.

[jdalton]

I treat node_modules as the boundary (so ignored). If it walks back and hits node_modules it returns null to indicate not finding a config since it's gone outside its package boundary.

[loganfsmyth]

Gotcha. So for us I'm saying we'd explicitly never load configs a dependency package's own config, and then (maybe as an opt-in), the project-level .babelrc could apply to dependency packages.

[loganfsmyth]

Having thought about this more over the last 2 weeks, I think where I've settled on the following:

• Add a root option that defaults to cwd. Babel will only search for .babelrc during compilation if the file being compiled is (a) inside the root (b) not inside a node_modules. This is already essentially the same logic we have in babel-register's automatic ignore. If we ended up defaulting sourceType to script we could even consider removing that from babel-register entirely, since core might be able to bail out fast enough to just run on every single file that is required.
• Automatically look for a babel.config.js file inside the root directory, which will apply to any file being compiled by Babel, just like programmatic options would.
• Add a configFile option for users who want to set an explicit config file.
• (Optionally) If a config file is found in the root, or passed explicitly, automatically set babelrc: false. If users want local babelrc files, require them to set babelrc: true in their config file or programmatic options. This would for the current behavior while not doing a ton of extra work unless users opt in.

I think this would address most of the current major concerns while keeping the existing behavior relatively consistent for the average case, at least inside your own package.

For other raised points:

• For symlinked node_modules, we can add a new symlinks option that would be allowed in config files and programmatic options, which Babel could resolve. If the file being compiled matches a symlink, it could essentially remap the passed-in filename to see if it matches any of the overrides from [Feature Request] feature similar to ESLint "overrides" (config based on glob) #5451.
• For babel-eslint we probably want to find the closest package.json and use that as root, since you're almost never going to be linting something in node_modules, and users could always use a .eslintrc.js to explicitly pass in a root option in parserOptions.
• We could also absolutely expand the babelrc option to have a way to opt into the full current behavior of looking for .babelrc files, even in node_modules and outside root, but I'm not totally convinced it is a usecase that people actually want.

[damianobarbati]

1. Is there any existing documentation about the .babelrc.js file?
2. Is there any alternative to have specific env configuration overrides without external configuration files (but babel section in package.json)?

I'm currently using this (package.json):

"babel": {
    "presets": [
        [
            "@babel/preset-env",
            {
                "modules": false,
                "useBuiltIns": "usage",
                "debug": true
            }
        ],
        "@babel/preset-stage-0",
        "@babel/preset-react"
    ],
    "plugins": [
        "lodash",
        "@babel/plugin-proposal-decorators",
        [
            "@babel/plugin-proposal-class-properties",
            {
                "loose": true
            }
        ]
    ]
},

All I'd like to do is having "debug: false when NODE_ENV=production.
I just want to override @babel/preset-env configuration.
But I don't want to copy over everything again!

[rtsao]

IMHO, compilation of node_modules is not a good pattern.

1. Publishing a module to npm assuming the consumer is using Babel is not good for the ecosystem, especially since Babel is used for more than just ECMAScript spec features; people do all kinds of stuff with Babel plugins
2. Can't use things like https://unpkg.com/ and <script type="module"/> if you need Babel compilation of unreleased or non-spec JS
3. It's slower for end-developers. Why shift the compilation burden to module consumers? It's better to do compilation once at publish-time so then everyone can skip running Babel in node_modules.

Instead, library authors should just pick an arbitrary minimum supported Node.js version and compile down to that with the env preset and bundle as (CJS and MJS) with Rollup. When Node 10 is released, it will automatically pick up the .mjs version.

If they support browser usage, authors should pick which ECMAScript specs they support (or multiple specs).

Note: To do this, we need #6731 in babel-preset-env

Then the package.json looks like this:

{
  "files": [
    "dist"
  ],
  "main": "./dist/index",
  "module": "./dist/index.mjs",
  "browser": {
    "./dist/index.js": "./dist/browser.es5.js",
    "./dist/index.mjs": "./dist/browser.es5.mjs"
  },
  "es2015": {
    "./dist/browser.es5.mjs": "./dist/browser.es2015.mjs"
  },
  "es2017": {
    "./dist/browser.es5.mjs": "./dist/browser.es2017.mjs",
    "./dist/browser.es2015.mjs": "./dist/browser.es2017.mjs"
  },
  "engines": {
    "node": ">= 8"
   }
}

dist/
├─ index.js   // CJS for Node.js 8+
├─ index.mjs  // MJS for Node.js 10+
├─ browser.es5.js       // ES5 CJS for browserify and legacy consumers
├─ browser.es5.mjs      // ES5 MJS for webpack and legacy browsers
├─ browser.es2015.mjs   // ES2015 MJS for weppack and modern browsers
└─ browser.es2017.mjs   // ES2017 MJS for webpack and evergreen

Then folks using webpack can just add extra enhanced-resolve fields and opt-into newer ECMAScript versions. Everything is backwards compatible with progressive enhancement.

It's not a big burden for authors to do this (I've made a helper CLI that does this out of the box)

[gaearon]

FYI, we're starting the work to compile deps with babel-preset-env in Create React App: facebook/create-react-app#3776

Let us know if you have feedback about how this should work. Note we intentionally don’t plan to respect .babelrc, and will only compile “real” JS features without JSX/Flow/etc.

[guybedford]

@loganfsmyth I wonder if it might be worth allowing the babelrc option to be a string if the API user wants to specify the exact file to use. I found myself wanting this feature today on a project that somehow wasn't picking up the right babelrc, and it might reduce cognitive load as well if the string value could also take the place of your suggested configFile property.

[loganfsmyth]

@guybedford I think that may just confuse things because I generally think users expect .babelrc files to behave strictly hierarchically. It also means people need to know what a babelrc is to search for the concept, and it's not really an obvious name for new users.

[devongovett]

FYI there is a discussion about this in parcel as well. parcel-bundler/parcel#13

We will also be supporting compilation of node_modules with babel-preset-env. We will use the package.json engines field, or browserslist field to determine the language of the source module, and the browserslist of the target app to determine what to compile to. Then we do a diff to include only the necessary plugins. That way modules will not be compiled all the way to ES5, but to the app’s browser target instead. parcel-bundler/parcel#559

Most of the discussion now revolves around custom Babel plugins in node_modules, which is convenient for local development with linked modules. I proposed adding a field to package.json to selectively turn on custom babelrc support in node_modules. Would be interested to hear what you all think of that. I’d like to come up with something that can be used across tools, not just something parcel specific.

[damianobarbati]

@devongovett how are you going to get the entrypoint of the untranspiled sources of your dependencies in node_modules?
Both main and module links to transpiled code today.
webpack/webpack#5935

[devongovett]

Just made an alternative suggestion to enable distributing both transpiled code and source code together. parcel-bundler/parcel#13 (comment)

[loganfsmyth]

Alright everyone, I've created #7358. Please look over what I've posted there and let me know what you think.

[phawxby]

@sokra and others. For this of us searching for a solution to the problem now using Babel 6 how is this usually solved?

I've got a client side repo included in a server side repo via git submodules. I need to import some of the client code into the server side but the babelrc for the client application is screwing up the import due to the different configurations.

[loganfsmyth]

@phawxby Feel free to join Slack or post on our recommended support channels, but pinging everyone on this thread about unrelated stuff will distract from the goals of this issue so I'd rather not help here.

[robwierzbowski]

A little late to the party, just wanted to add some background for transforming/transpiling packages in node_modules (cc @ljharb).

For maximum performance of front-end bundles, some people (e.g., Phil Walton, myself and the team I work with) want to pull as many dependencies as possible in as uncompiled ES modules. This:

• improves tree shaking
• gives us control of the top and bottom end of browser support (we can keep more JS minimally transpiled for our modern browsers, improving performance)
• helps prevent multiple inclusions of global polyfills and helper functions (if dependency A's pre-transpiled version bundles Bluebird promise and our app polyfills at runtime with Polyfill.io we're downloading unused bytes)

I got bit by the nested babelrc's this week, and would love a root: true or similar feature in Babel 7.

[hzoo]

Hey @robwierzbowski thanks for your input! And yeah we have moved discussion a while back to #7358 and are going to be moving forward with that. And yes we understand compiling node_modules is a use-case we all want to shoot for, not really the reasoning but how it would work!