New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How do we want to handle `.babelrc` lookup in 7.x #6766

Closed
loganfsmyth opened this Issue Nov 7, 2017 · 73 comments

Comments

Projects
None yet
@loganfsmyth
Member

loganfsmyth commented Nov 7, 2017

Edit 2

Most recent status: #6766 (comment)

Edit 1

All below still applies, but also see more explanation in #6766 (comment)

Original:

Not a fleshed out proposal, but filing for tracking and discussion.

Currently if you have:

.babelrc
src/
  index.js
  utils/
    .babelrc
    other.js

compiling index.js will use the root-level .babelrc, and other.js will use the src/utils/.babelrc. I assume this pattern was pulled from other utilities like ESLint, but not 100% sure.

We're at the point now where this approach has become a problem for us because:

  1. Very few people seem to expect this behavior to begin with.
  2. People expect their project-level .babelrc file to also apply to things that live in node_modules. Babel 6 was terrible at figuring out how to handle node_modules because it always assumes things are strict mode, and always rewrites this, but a lot of those issues aren't as big in Babel 7, or at least ideally won't be, because we plan to have better handling of the sourceType: script vs sourceType: module distinction. This kind of works now, but see point (3).
  3. If people publish packages to npm and include .babelrc files in their packages, or more troublingly, use the babel key in package.json (which you can't really get around publishing), users trying to use Webpack or anything like it to compile node_modules get extremely confusing errors about not being able to find plugins and presets referenced in those files. See facebook/create-react-app#1125 for a massive list of users that want to compile node_modules, but would absolutely be bitten by this issue.

So where does that leave us? The simplest thing I can think for us to do would be to change up Babel's API to, by default, accept a root directory as an argument where it would search for the config, defaulting to the working directory.

We're already at a point where I've thought about adding an argument for root directory to resolve the filename relative to, so that should be uncontroversial. Actually changing config resolution is a bigger question.

It would be pretty easy for us to allow users to opt for the old behavior, so I think my proposal would probably be to add this new root-dir config resolution logic and make it the default, but keep the old behavior around as an opt in. So where now we have

babelrc: boolean

to toggle config resolution off and on, I could imagine us doing

babelrc: true | "root" => Look for config in root directory
babelrc: "relative" => Look for config, starting from compiled file, working upward
babelrc: false => Skip search entirely.

instead.

The primary loss of functionality here is that if users do want to use .babelrc files inside nested folders to change Babel configs based on path, they don't have an easy way to do it anymore. I don't however have a good grasp of how many users that might be. It seems like we may want to consider #5451 as something to be included in this effort if we do want to change, which might end up blocked on #6765.

@hzoo hzoo added this to the Babel 7.next milestone Nov 7, 2017

@hzoo

This comment has been minimized.

Show comment
Hide comment
@hzoo

hzoo Nov 7, 2017

Member

Seems totally reasonable to me

  • I haven't ever seen a nested babelrc file, not sure how we can find out
  • I assume everyone would want to transform all their code in the same way (sounds hard to debug/understand)
  • Most people have 1 babelrc so they would assume Babel only would look at that config. I would be surprised it would try to read a babelrc from node_modules
  • Default to the intuitive behavior that helps us support new use cases while providing an opt-out sounds good to me 👍 👍 let's do it.
  • Given we ourselves liked using overrides for ESLint yeah we can consider #5451
Member

hzoo commented Nov 7, 2017

Seems totally reasonable to me

  • I haven't ever seen a nested babelrc file, not sure how we can find out
  • I assume everyone would want to transform all their code in the same way (sounds hard to debug/understand)
  • Most people have 1 babelrc so they would assume Babel only would look at that config. I would be surprised it would try to read a babelrc from node_modules
  • Default to the intuitive behavior that helps us support new use cases while providing an opt-out sounds good to me 👍 👍 let's do it.
  • Given we ourselves liked using overrides for ESLint yeah we can consider #5451
@kentcdodds

This comment has been minimized.

Show comment
Hide comment
@kentcdodds

kentcdodds Nov 7, 2017

Member

Thanks for all your work on babel folks! 👏

Here are some thoughts:

  • I really like ESLint's model. I like the whole thing, in particular how the root: true functionality works and just minutes ago was looking to see if babel had support for something like this. I'm not even joking.
  • I don't ever want to transpile node_modules, but I think that it should be possible and I would expect that the hierarchy should apply there as well. I can't speak too much to this though because I'm not big on the idea of transpiling node_modules. I just feel like there's probably a use case for it. But making it a little difficult to do isn't a bad thing IMO.

I hope that's helpful! Thanks again!

Member

kentcdodds commented Nov 7, 2017

Thanks for all your work on babel folks! 👏

Here are some thoughts:

  • I really like ESLint's model. I like the whole thing, in particular how the root: true functionality works and just minutes ago was looking to see if babel had support for something like this. I'm not even joking.
  • I don't ever want to transpile node_modules, but I think that it should be possible and I would expect that the hierarchy should apply there as well. I can't speak too much to this though because I'm not big on the idea of transpiling node_modules. I just feel like there's probably a use case for it. But making it a little difficult to do isn't a bad thing IMO.

I hope that's helpful! Thanks again!

@hzoo

This comment has been minimized.

Show comment
Hide comment
@hzoo

hzoo Nov 7, 2017

Member

@kentcdodds

There is a babelrc: false option which stops lookup as mentioned ^

Compiling node_modules is for when you want to consume a package from npm that is ES2015+. Maybe you support a different environment so you need to compile it down to a different level than ES5. Example: maybe a react package uses async but you don't want to use the "main" file which uses regenerator, so you want the package to output async. It may or may not be transpiled to async (no change), generators, or regenerator depending on the users preset-env targets

Ya it's interesting that eslint would never have this issue since there's no need to look at node_modules there 😛

Member

hzoo commented Nov 7, 2017

@kentcdodds

There is a babelrc: false option which stops lookup as mentioned ^

Compiling node_modules is for when you want to consume a package from npm that is ES2015+. Maybe you support a different environment so you need to compile it down to a different level than ES5. Example: maybe a react package uses async but you don't want to use the "main" file which uses regenerator, so you want the package to output async. It may or may not be transpiled to async (no change), generators, or regenerator depending on the users preset-env targets

Ya it's interesting that eslint would never have this issue since there's no need to look at node_modules there 😛

@kentcdodds

This comment has been minimized.

Show comment
Hide comment
@kentcdodds

kentcdodds Nov 7, 2017

Member

There is a babelrc: false option which stops lookup as mentioned ^

👍

Solid use case example 👍 So long as it's possible that'd be good with me. And if I could disable the lookup using webpack/babel-cli etc. then that'd be sufficient I think

Member

kentcdodds commented Nov 7, 2017

There is a babelrc: false option which stops lookup as mentioned ^

👍

Solid use case example 👍 So long as it's possible that'd be good with me. And if I could disable the lookup using webpack/babel-cli etc. then that'd be sufficient I think

@niieani

This comment has been minimized.

Show comment
Hide comment
@niieani

niieani Nov 7, 2017

How about non-precompiled/non-distributable monorepos? This is currently what we do at @futuresimple, so e.g. we have packages/pkg-name, each with its own version of .babelrc that can vary across the packages in the monorepo. On the other hand, the root folder (repository root) doesn't have any babel config.

To handle such a case well, I think the default behavior should be to use the .babelrc next to the closest package.json, rather than closest to the file (as is now).

niieani commented Nov 7, 2017

How about non-precompiled/non-distributable monorepos? This is currently what we do at @futuresimple, so e.g. we have packages/pkg-name, each with its own version of .babelrc that can vary across the packages in the monorepo. On the other hand, the root folder (repository root) doesn't have any babel config.

To handle such a case well, I think the default behavior should be to use the .babelrc next to the closest package.json, rather than closest to the file (as is now).

@sokra

This comment has been minimized.

Show comment
Hide comment
@sokra

sokra Nov 8, 2017

I've seen nested babelrc files in repos which contain server and client build.

sokra commented Nov 8, 2017

I've seen nested babelrc files in repos which contain server and client build.

@hzoo

This comment has been minimized.

Show comment
Hide comment
@hzoo

hzoo Nov 8, 2017

Member

I'm sure not everyone here will have read everything but we could possibly do #5451 which is an "overrides" option similar to eslint which gives you glob config + different config so you don't need nested .babelrcs.

Member

hzoo commented Nov 8, 2017

I'm sure not everyone here will have read everything but we could possibly do #5451 which is an "overrides" option similar to eslint which gives you glob config + different config so you don't need nested .babelrcs.

@nicolo-ribaudo

This comment has been minimized.

Show comment
Hide comment
@nicolo-ribaudo

nicolo-ribaudo Nov 8, 2017

Member

I'm strongly in favor of only using the root babelrc, since config files in nested directories are really easy to miss.
If people want more control on their builds, they can use tools like gulp.

+1 for #5451

Member

nicolo-ribaudo commented Nov 8, 2017

I'm strongly in favor of only using the root babelrc, since config files in nested directories are really easy to miss.
If people want more control on their builds, they can use tools like gulp.

+1 for #5451

@niieani

This comment has been minimized.

Show comment
Hide comment
@niieani

niieani Nov 8, 2017

@hzoo #5451 won't help for monorepos (when used without a build step). You'd have to re-declare all the babelrc's in the root of the monorepo, which doesn't sound fun.

When you install packages in a monorepo, they end up symlinked as node_modules/MONOREPO_PACKAGE.
Since each package can have its own .babelrc, we depend on babel resolving those .babelrc's to transpile the code from those modules using the settings they declare.

I am all for this change, as long as it's possible to declare in the root .babelrc something like deepResolveBabelRc: true. Having this as a programmatic option in the API (as proposed in the issue) won't work for cases where we'd like to directly run: node -r babel-runtime node_modules/MONOREPO_PACKAGE/script.

niieani commented Nov 8, 2017

@hzoo #5451 won't help for monorepos (when used without a build step). You'd have to re-declare all the babelrc's in the root of the monorepo, which doesn't sound fun.

When you install packages in a monorepo, they end up symlinked as node_modules/MONOREPO_PACKAGE.
Since each package can have its own .babelrc, we depend on babel resolving those .babelrc's to transpile the code from those modules using the settings they declare.

I am all for this change, as long as it's possible to declare in the root .babelrc something like deepResolveBabelRc: true. Having this as a programmatic option in the API (as proposed in the issue) won't work for cases where we'd like to directly run: node -r babel-runtime node_modules/MONOREPO_PACKAGE/script.

@loganfsmyth

This comment has been minimized.

Show comment
Hide comment
@loganfsmyth

loganfsmyth Nov 14, 2017

Member

@niieani Do you have example where you require different babel configs for different packages in a monorepo? I'd have thought they'd be very similar, and thus something like #5451 might be enough for your case.

Member

loganfsmyth commented Nov 14, 2017

@niieani Do you have example where you require different babel configs for different packages in a monorepo? I'd have thought they'd be very similar, and thus something like #5451 might be enough for your case.

@Andarist

This comment has been minimized.

Show comment
Hide comment
@Andarist

Andarist Nov 15, 2017

Member

@loganfsmyth I imagine when having both front end and back end code in your monorepo you can have different babel configs. Also personally I like to be explicit about my deps, even used transforms, and I do not include a jsx transform when transpiling utility library even though in other directory of the monorepo I have a React UI library

Member

Andarist commented Nov 15, 2017

@loganfsmyth I imagine when having both front end and back end code in your monorepo you can have different babel configs. Also personally I like to be explicit about my deps, even used transforms, and I do not include a jsx transform when transpiling utility library even though in other directory of the monorepo I have a React UI library

@niieani

This comment has been minimized.

Show comment
Hide comment
@niieani

niieani Nov 15, 2017

@loganfsmyth exactly as @Andarist says. We have both CLI and Frontend packages in the monorepo, at the same time we'd like to be able to open source some of the packages, so each package in the monorepo should be able to be run without the dependency of the root directory of the monorepo. #5451 introduces such a dependency, i.e. the package won't work when it's moved out of the monorepo into it's own repository via a git subtree.

niieani commented Nov 15, 2017

@loganfsmyth exactly as @Andarist says. We have both CLI and Frontend packages in the monorepo, at the same time we'd like to be able to open source some of the packages, so each package in the monorepo should be able to be run without the dependency of the root directory of the monorepo. #5451 introduces such a dependency, i.e. the package won't work when it's moved out of the monorepo into it's own repository via a git subtree.

@loganfsmyth

This comment has been minimized.

Show comment
Hide comment
@loganfsmyth

loganfsmyth Nov 15, 2017

Member

Sounds good, thanks for the clarification. So it is not that there is are package-level .babelrcs that might merge with the one at the monorepo root, it's essentially that each inner package could be thought of as having its own root.

Member

loganfsmyth commented Nov 15, 2017

Sounds good, thanks for the clarification. So it is not that there is are package-level .babelrcs that might merge with the one at the monorepo root, it's essentially that each inner package could be thought of as having its own root.

@Andarist

This comment has been minimized.

Show comment
Hide comment
@Andarist

Andarist Nov 15, 2017

Member

As for me - that would be desired. If I'd like to extend some other config I might do that easily with .babelrc.js, but in most cases I just want to keep specific babel config per package, even if they all live in monorepo at the moment.

Member

Andarist commented Nov 15, 2017

As for me - that would be desired. If I'd like to extend some other config I might do that easily with .babelrc.js, but in most cases I just want to keep specific babel config per package, even if they all live in monorepo at the moment.

@benjamn

This comment has been minimized.

Show comment
Hide comment
@benjamn

benjamn Nov 23, 2017

Contributor

Meteor supports nested .babelrc files, for essentially the same purpose that @sokra cited above: #6766 (comment)

We do not attempt to merge multiple .babelrc files in ancestor directories. The first one found in an ancestor directory within the project root takes precedence.

I'm strongly in favor of only using the root babelrc, since config files in nested directories are really easy to miss.

@nicolo-ribaudo So don't use nested .babelrc files in your own projects. Taking that flexibility away from developers who want it is paternalistic and not helpful.

Contributor

benjamn commented Nov 23, 2017

Meteor supports nested .babelrc files, for essentially the same purpose that @sokra cited above: #6766 (comment)

We do not attempt to merge multiple .babelrc files in ancestor directories. The first one found in an ancestor directory within the project root takes precedence.

I'm strongly in favor of only using the root babelrc, since config files in nested directories are really easy to miss.

@nicolo-ribaudo So don't use nested .babelrc files in your own projects. Taking that flexibility away from developers who want it is paternalistic and not helpful.

@loganfsmyth

This comment has been minimized.

Show comment
Hide comment
@loganfsmyth

loganfsmyth Nov 23, 2017

Member

@benjamn Would #5451 address any of your concerns?

That would potentially allow you to do something like

{
  overrides: [
    {
      test: "./client",
      extends: "./client/.babelrc.js",
    },
    {
      test: "./server",
      extends: "./server/.babelrc.js",
    },
  ]
}

so the root config could still easily delegate to another config.

I could also imagine the monorepo case being handled like

{
  overrides: [
    {
      test: "./packages/*",
      extends: "<TEST>/.babelrc.js",
    },
  ]
}

for instance, so the root config would control everything, but still allow for flexibility.

Alternatively we could have the root config allow a relativeSearch: boolean flag, so if set, it'd load the root config, but then do search relative to the file.

Maybe we could even do overrideConfigs: "./packages/*" to automatically search in the given path for configs, and then essentially allow overrideConfigs: "**/*" to essentially say "search everywhere" like we do now.

I feel like I'd want the file to merge on top of the root config by default, but maybe we allow config files to say overwrite: true to overwrite existing config values when they load?

My point is, there are tons of options that would allow essentially the same flexibility. I just want it to be explicit instead of implicit in Babel's core behavior, because the implicit relative lookup is causing problems, as stated in the original issue.

Member

loganfsmyth commented Nov 23, 2017

@benjamn Would #5451 address any of your concerns?

That would potentially allow you to do something like

{
  overrides: [
    {
      test: "./client",
      extends: "./client/.babelrc.js",
    },
    {
      test: "./server",
      extends: "./server/.babelrc.js",
    },
  ]
}

so the root config could still easily delegate to another config.

I could also imagine the monorepo case being handled like

{
  overrides: [
    {
      test: "./packages/*",
      extends: "<TEST>/.babelrc.js",
    },
  ]
}

for instance, so the root config would control everything, but still allow for flexibility.

Alternatively we could have the root config allow a relativeSearch: boolean flag, so if set, it'd load the root config, but then do search relative to the file.

Maybe we could even do overrideConfigs: "./packages/*" to automatically search in the given path for configs, and then essentially allow overrideConfigs: "**/*" to essentially say "search everywhere" like we do now.

I feel like I'd want the file to merge on top of the root config by default, but maybe we allow config files to say overwrite: true to overwrite existing config values when they load?

My point is, there are tons of options that would allow essentially the same flexibility. I just want it to be explicit instead of implicit in Babel's core behavior, because the implicit relative lookup is causing problems, as stated in the original issue.

@novemberborn

This comment has been minimized.

Show comment
Hide comment
@novemberborn

novemberborn Nov 26, 2017

Contributor

AVA, which uses Babel to compile test files, can be directed to inherit from a user's Babel config. We only look for .babelrc files in the project directory (next to the package.json file), or in the package.json file itself. This was done so the Babel options only need to be resolved once, rather than for each test file.

In other words I'm 👍 on this proposal (as stated in @loganfsmyth's opening comment).

Contributor

novemberborn commented Nov 26, 2017

AVA, which uses Babel to compile test files, can be directed to inherit from a user's Babel config. We only look for .babelrc files in the project directory (next to the package.json file), or in the package.json file itself. This was done so the Babel options only need to be resolved once, rather than for each test file.

In other words I'm 👍 on this proposal (as stated in @loganfsmyth's opening comment).

@ljharb

This comment has been minimized.

Show comment
Hide comment
@ljharb

ljharb Nov 27, 2017

(I also agree with #6766 (comment))

I occasionally create a nested .babelrc file in my tests directory, so that certain transforms only apply to test files (not the same as "only transform in the test NODE_ENV"). Similarly, I might want certain transforms in a dev-only directory, and different ones in a production directory.

How would I do this with this proposal?

ljharb commented Nov 27, 2017

(I also agree with #6766 (comment))

I occasionally create a nested .babelrc file in my tests directory, so that certain transforms only apply to test files (not the same as "only transform in the test NODE_ENV"). Similarly, I might want certain transforms in a dev-only directory, and different ones in a production directory.

How would I do this with this proposal?

@xtuc

This comment has been minimized.

Show comment
Hide comment
@xtuc

xtuc Nov 27, 2017

Member

Just my 2 cents. I would like to explicitly pass the configuration file location and in addition I would disable completely the config lookup (not a BC since it's a new option).

This would solve many issues with "how can I use Babel from this dir? can I store my config there? ..." and probably some .babelrc config inheritance issues.

Member

xtuc commented Nov 27, 2017

Just my 2 cents. I would like to explicitly pass the configuration file location and in addition I would disable completely the config lookup (not a BC since it's a new option).

This would solve many issues with "how can I use Babel from this dir? can I store my config there? ..." and probably some .babelrc config inheritance issues.

@loganfsmyth

This comment has been minimized.

Show comment
Hide comment
@loganfsmyth

loganfsmyth Nov 27, 2017

Member

@ljharb What are your thoughts on my comments in #6766 (comment) I'm definitely not looking to break that usecase, though I am promoting changing the implementation and mental model behind how they would fit in.

If I had to summarize the goal of this issue, it's to try to clarify that Babel configs are configuring a project as a whole with compilation logic, rather than defining configs for specific files. That means files in node_modules shouldn't have their configs processed unless the root project asks for that, and project-root configs should be able to influence how node_modules are compiled. As Node gets more and more ES6 features, people publish more and more modules to npm with ES6 modules. We can tell people to compile to ES5 before publishing all we want, but modules containing ES6 are going to happen either way, and I think it is important that we position Babel in a place where it can address those concerns without it being extremely painful for users.

Additionally, I'm very concerned about the pain users will feel if they run Babel 7 and some of their node_modules have published .babelrc files meant for Babel 6. Things will fail quite badly and they'll have no way to fix it.

Member

loganfsmyth commented Nov 27, 2017

@ljharb What are your thoughts on my comments in #6766 (comment) I'm definitely not looking to break that usecase, though I am promoting changing the implementation and mental model behind how they would fit in.

If I had to summarize the goal of this issue, it's to try to clarify that Babel configs are configuring a project as a whole with compilation logic, rather than defining configs for specific files. That means files in node_modules shouldn't have their configs processed unless the root project asks for that, and project-root configs should be able to influence how node_modules are compiled. As Node gets more and more ES6 features, people publish more and more modules to npm with ES6 modules. We can tell people to compile to ES5 before publishing all we want, but modules containing ES6 are going to happen either way, and I think it is important that we position Babel in a place where it can address those concerns without it being extremely painful for users.

Additionally, I'm very concerned about the pain users will feel if they run Babel 7 and some of their node_modules have published .babelrc files meant for Babel 6. Things will fail quite badly and they'll have no way to fix it.

@ljharb

This comment has been minimized.

Show comment
Hide comment
@ljharb

ljharb Nov 28, 2017

Certainly, eslint 4+'s glob overrides approach would address the use case, but then it doesn't allow me to "hide" the overridden settings inside the directories that have different approaches. Perhaps this isn't a big deal.

It's a fair point that people who try to transpile node_modules (which I hope are still ignored by default), would run afoul of this issue simply by a module using different plugins/transforms than they have installed, and even more so with babel 6 vs 7.

Perhaps - tangential to this issue - babel configs for 7+ should be required to explicitly declare their version number?

Additionally, perhaps those who choose to transpile code they didn't write should be required to explicitly specify the location for the config (whether that's a specific .babelrc, or a specific package.json's "babel" section)?

ljharb commented Nov 28, 2017

Certainly, eslint 4+'s glob overrides approach would address the use case, but then it doesn't allow me to "hide" the overridden settings inside the directories that have different approaches. Perhaps this isn't a big deal.

It's a fair point that people who try to transpile node_modules (which I hope are still ignored by default), would run afoul of this issue simply by a module using different plugins/transforms than they have installed, and even more so with babel 6 vs 7.

Perhaps - tangential to this issue - babel configs for 7+ should be required to explicitly declare their version number?

Additionally, perhaps those who choose to transpile code they didn't write should be required to explicitly specify the location for the config (whether that's a specific .babelrc, or a specific package.json's "babel" section)?

@jdalton

This comment has been minimized.

Show comment
Hide comment
@jdalton

jdalton Nov 29, 2017

Member

Specifying parent config and override paths seems like heavy config for answering the question of should this config be treated as done or rolled into a parent config. I think something like a "root": true prop could be used to answer yes or no to whether to roll-up the config to the parent or not. So folks wanting separate configs for tests can just add a "root": true to those configs and the default behavior would be that it rolls-up to the parent.

Member

jdalton commented Nov 29, 2017

Specifying parent config and override paths seems like heavy config for answering the question of should this config be treated as done or rolled into a parent config. I think something like a "root": true prop could be used to answer yes or no to whether to roll-up the config to the parent or not. So folks wanting separate configs for tests can just add a "root": true to those configs and the default behavior would be that it rolls-up to the parent.

@loganfsmyth

This comment has been minimized.

Show comment
Hide comment
@loganfsmyth

loganfsmyth Nov 29, 2017

Member

I think something like a "root": true prop could be used to answer yes or no to whether to roll-up the config to the parent or not.

We already stop at the first .babelrc that is encountered, so root: true wouldn't have any effect. Users currently instead have to opt in with a parent like extends: "../../.babelrc" if they wanted to have a parent config continue applying.

I should clarify my main goal on this.

Primary Issue

The primary issue at the core of this is that people often expect their project's .babelrc to be that, a project-level configuration. By having the files resolved relative to each individual file, it means we pick up entirely unrelated .babelrc files in node_modules. In Babel 6, we occasionally got reports early on because people had published Babel 5 .babelrc files. When we publish Babel 7, that's 100% going to happen again, and as we start publishing new versions of Babel more often, it's going to get even more confusing for users, especially if we decide to make more breaking changes in the future.

Even in the case of dependency modules with .babelrc for Babel 7, even if it is for the same version, the config file could easily have been one that already ran to create the code that is already published to npm anyway. The only place that actually knows what config a node module should actually use is the root application itself, not the node module itself.

An alternative to my original proposal would be that we check if a file is in node_modules, e.g.

project/
  node_modules/mod/
    .babelrc
    src.js
  .babelrc

If this project tries to compile project/node_modules/mod/src.js, we could immediately jump upward and start resolving above the node_modules folder, thus finding the project-level .babelrc

The primary edge case there is that I guess technically your project itself could for some reason be inside a folder called node_modules, but we could certainly accept that as something that will just not be supported by users.

To expand, right now we do:

const filepath = ...
let dirname = path.dirname(filepath);
while (true) {
  if (exists(path.join(dirname), ".babelrc")) // ...
  const parent = path.dirname(dirname);
  if (parent === dirname) break;
  dirname = parent;
}

but you could imagine us changing this to (as non-production code anyway)

const filepath = ...
let dirname = path.dirname(filepath);

// Chop off everything after the first "node_modules" in the path
let parts = dirname.split(path.sep);
let modIndex = parts.indexOf("node_modules");
if (modIndex !== -1) {
  dirname = parts.slice(0, modIndex).join(path.sep);
}

// Resolve as usual.
while (true) {
  if (exists(path.join(dirname), ".babelrc")) // ...
  const parent = path.dirname(dirname);
  if (parent === dirname) break;
  dirname = parent;
}
Member

loganfsmyth commented Nov 29, 2017

I think something like a "root": true prop could be used to answer yes or no to whether to roll-up the config to the parent or not.

We already stop at the first .babelrc that is encountered, so root: true wouldn't have any effect. Users currently instead have to opt in with a parent like extends: "../../.babelrc" if they wanted to have a parent config continue applying.

I should clarify my main goal on this.

Primary Issue

The primary issue at the core of this is that people often expect their project's .babelrc to be that, a project-level configuration. By having the files resolved relative to each individual file, it means we pick up entirely unrelated .babelrc files in node_modules. In Babel 6, we occasionally got reports early on because people had published Babel 5 .babelrc files. When we publish Babel 7, that's 100% going to happen again, and as we start publishing new versions of Babel more often, it's going to get even more confusing for users, especially if we decide to make more breaking changes in the future.

Even in the case of dependency modules with .babelrc for Babel 7, even if it is for the same version, the config file could easily have been one that already ran to create the code that is already published to npm anyway. The only place that actually knows what config a node module should actually use is the root application itself, not the node module itself.

An alternative to my original proposal would be that we check if a file is in node_modules, e.g.

project/
  node_modules/mod/
    .babelrc
    src.js
  .babelrc

If this project tries to compile project/node_modules/mod/src.js, we could immediately jump upward and start resolving above the node_modules folder, thus finding the project-level .babelrc

The primary edge case there is that I guess technically your project itself could for some reason be inside a folder called node_modules, but we could certainly accept that as something that will just not be supported by users.

To expand, right now we do:

const filepath = ...
let dirname = path.dirname(filepath);
while (true) {
  if (exists(path.join(dirname), ".babelrc")) // ...
  const parent = path.dirname(dirname);
  if (parent === dirname) break;
  dirname = parent;
}

but you could imagine us changing this to (as non-production code anyway)

const filepath = ...
let dirname = path.dirname(filepath);

// Chop off everything after the first "node_modules" in the path
let parts = dirname.split(path.sep);
let modIndex = parts.indexOf("node_modules");
if (modIndex !== -1) {
  dirname = parts.slice(0, modIndex).join(path.sep);
}

// Resolve as usual.
while (true) {
  if (exists(path.join(dirname), ".babelrc")) // ...
  const parent = path.dirname(dirname);
  if (parent === dirname) break;
  dirname = parent;
}
@jdalton

This comment has been minimized.

Show comment
Hide comment
@jdalton

jdalton Nov 29, 2017

Member

but you could imagine us changing this to (as non-production code anyway)

I think that's how I thought it would work. Here's how I resolve config for std/esm:

I walk backwards in the directory structure, stopping at node_modules with a return of null if a config is not found. The mechanism is called getting package info (PkgInfo). So it's gated to packages. This means if I roll up config it stops at a package boundary. This also means I could add support for a "root":true to indicate not rolling up to the package boundary and stopping a that given config.

For folks that want to force a set of options across package boundaries we have a programmatic convention for that.

Update:
Clarified things a bit.

Member

jdalton commented Nov 29, 2017

but you could imagine us changing this to (as non-production code anyway)

I think that's how I thought it would work. Here's how I resolve config for std/esm:

I walk backwards in the directory structure, stopping at node_modules with a return of null if a config is not found. The mechanism is called getting package info (PkgInfo). So it's gated to packages. This means if I roll up config it stops at a package boundary. This also means I could add support for a "root":true to indicate not rolling up to the package boundary and stopping a that given config.

For folks that want to force a set of options across package boundaries we have a programmatic convention for that.

Update:
Clarified things a bit.

@loganfsmyth

This comment has been minimized.

Show comment
Hide comment
@loganfsmyth

loganfsmyth Nov 29, 2017

Member

I walk backwards in the directory structure and stop at "nod_modules". The mechanism is called getting package info. So it's gated to packages.

Could you clarify that or point me at particular code?

In my description the config inside node_modules be be explicitly ignored, which I think is the opposite of what you're describing? Though that may make more sense for std/esm.

Member

loganfsmyth commented Nov 29, 2017

I walk backwards in the directory structure and stop at "nod_modules". The mechanism is called getting package info. So it's gated to packages.

Could you clarify that or point me at particular code?

In my description the config inside node_modules be be explicitly ignored, which I think is the opposite of what you're describing? Though that may make more sense for std/esm.

@jdalton

This comment has been minimized.

Show comment
Hide comment
@jdalton

jdalton Nov 29, 2017

Member

I treat node_modules as the boundary (so ignored). If it walks back and hits node_modules it returns null to indicate not finding a config since it's gone outside its package boundary.

Member

jdalton commented Nov 29, 2017

I treat node_modules as the boundary (so ignored). If it walks back and hits node_modules it returns null to indicate not finding a config since it's gone outside its package boundary.

@loganfsmyth

This comment has been minimized.

Show comment
Hide comment
@loganfsmyth

loganfsmyth Nov 29, 2017

Member

Gotcha. So for us I'm saying we'd explicitly never load configs a dependency package's own config, and then (maybe as an opt-in), the project-level .babelrc could apply to dependency packages.

Member

loganfsmyth commented Nov 29, 2017

Gotcha. So for us I'm saying we'd explicitly never load configs a dependency package's own config, and then (maybe as an opt-in), the project-level .babelrc could apply to dependency packages.

@rtsao

This comment has been minimized.

Show comment
Hide comment
@rtsao

rtsao Jan 5, 2018

Contributor

IMHO, compilation of node_modules is not a good pattern.

  1. Publishing a module to npm assuming the consumer is using Babel is not good for the ecosystem, especially since Babel is used for more than just ECMAScript spec features; people do all kinds of stuff with Babel plugins
  2. Can't use things like https://unpkg.com/ and <script type="module"/> if you need Babel compilation of unreleased or non-spec JS
  3. It's slower for end-developers. Why shift the compilation burden to module consumers? It's better to do compilation once at publish-time so then everyone can skip running Babel in node_modules.

Instead, library authors should just pick an arbitrary minimum supported Node.js version and compile down to that with the env preset and bundle as (CJS and MJS) with Rollup. When Node 10 is released, it will automatically pick up the .mjs version.

If they support browser usage, authors should pick which ECMAScript specs they support (or multiple specs).

Note: To do this, we need #6731 in babel-preset-env

Then the package.json looks like this:

{
  "files": [
    "dist"
  ],
  "main": "./dist/index",
  "module": "./dist/index.mjs",
  "browser": {
    "./dist/index.js": "./dist/browser.es5.js",
    "./dist/index.mjs": "./dist/browser.es5.mjs"
  },
  "es2015": {
    "./dist/browser.es5.mjs": "./dist/browser.es2015.mjs"
  },
  "es2017": {
    "./dist/browser.es5.mjs": "./dist/browser.es2017.mjs",
    "./dist/browser.es2015.mjs": "./dist/browser.es2017.mjs"
  },
  "engines": {
    "node": ">= 8"
   }
}
dist/
├─ index.js   // CJS for Node.js 8+
├─ index.mjs  // MJS for Node.js 10+
├─ browser.es5.js       // ES5 CJS for browserify and legacy consumers
├─ browser.es5.mjs      // ES5 MJS for webpack and legacy browsers
├─ browser.es2015.mjs   // ES2015 MJS for weppack and modern browsers
└─ browser.es2017.mjs   // ES2017 MJS for webpack and evergreen

Then folks using webpack can just add extra enhanced-resolve fields and opt-into newer ECMAScript versions. Everything is backwards compatible with progressive enhancement.

It's not a big burden for authors to do this (I've made a helper CLI that does this out of the box)

Contributor

rtsao commented Jan 5, 2018

IMHO, compilation of node_modules is not a good pattern.

  1. Publishing a module to npm assuming the consumer is using Babel is not good for the ecosystem, especially since Babel is used for more than just ECMAScript spec features; people do all kinds of stuff with Babel plugins
  2. Can't use things like https://unpkg.com/ and <script type="module"/> if you need Babel compilation of unreleased or non-spec JS
  3. It's slower for end-developers. Why shift the compilation burden to module consumers? It's better to do compilation once at publish-time so then everyone can skip running Babel in node_modules.

Instead, library authors should just pick an arbitrary minimum supported Node.js version and compile down to that with the env preset and bundle as (CJS and MJS) with Rollup. When Node 10 is released, it will automatically pick up the .mjs version.

If they support browser usage, authors should pick which ECMAScript specs they support (or multiple specs).

Note: To do this, we need #6731 in babel-preset-env

Then the package.json looks like this:

{
  "files": [
    "dist"
  ],
  "main": "./dist/index",
  "module": "./dist/index.mjs",
  "browser": {
    "./dist/index.js": "./dist/browser.es5.js",
    "./dist/index.mjs": "./dist/browser.es5.mjs"
  },
  "es2015": {
    "./dist/browser.es5.mjs": "./dist/browser.es2015.mjs"
  },
  "es2017": {
    "./dist/browser.es5.mjs": "./dist/browser.es2017.mjs",
    "./dist/browser.es2015.mjs": "./dist/browser.es2017.mjs"
  },
  "engines": {
    "node": ">= 8"
   }
}
dist/
├─ index.js   // CJS for Node.js 8+
├─ index.mjs  // MJS for Node.js 10+
├─ browser.es5.js       // ES5 CJS for browserify and legacy consumers
├─ browser.es5.mjs      // ES5 MJS for webpack and legacy browsers
├─ browser.es2015.mjs   // ES2015 MJS for weppack and modern browsers
└─ browser.es2017.mjs   // ES2017 MJS for webpack and evergreen

Then folks using webpack can just add extra enhanced-resolve fields and opt-into newer ECMAScript versions. Everything is backwards compatible with progressive enhancement.

It's not a big burden for authors to do this (I've made a helper CLI that does this out of the box)

@stasm stasm referenced this issue Jan 12, 2018

Closed

Upgrade to Babel 7 #126

4 of 4 tasks complete
@gaearon

This comment has been minimized.

Show comment
Hide comment
@gaearon

gaearon Jan 13, 2018

Member

FYI, we're starting the work to compile deps with babel-preset-env in Create React App: facebook/create-react-app#3776

Let us know if you have feedback about how this should work. Note we intentionally don’t plan to respect .babelrc, and will only compile “real” JS features without JSX/Flow/etc.

Member

gaearon commented Jan 13, 2018

FYI, we're starting the work to compile deps with babel-preset-env in Create React App: facebook/create-react-app#3776

Let us know if you have feedback about how this should work. Note we intentionally don’t plan to respect .babelrc, and will only compile “real” JS features without JSX/Flow/etc.

@guybedford

This comment has been minimized.

Show comment
Hide comment
@guybedford

guybedford Jan 28, 2018

Contributor

@loganfsmyth I wonder if it might be worth allowing the babelrc option to be a string if the API user wants to specify the exact file to use. I found myself wanting this feature today on a project that somehow wasn't picking up the right babelrc, and it might reduce cognitive load as well if the string value could also take the place of your suggested configFile property.

Contributor

guybedford commented Jan 28, 2018

@loganfsmyth I wonder if it might be worth allowing the babelrc option to be a string if the API user wants to specify the exact file to use. I found myself wanting this feature today on a project that somehow wasn't picking up the right babelrc, and it might reduce cognitive load as well if the string value could also take the place of your suggested configFile property.

@loganfsmyth

This comment has been minimized.

Show comment
Hide comment
@loganfsmyth

loganfsmyth Jan 30, 2018

Member

@guybedford I think that may just confuse things because I generally think users expect .babelrc files to behave strictly hierarchically. It also means people need to know what a babelrc is to search for the concept, and it's not really an obvious name for new users.

Member

loganfsmyth commented Jan 30, 2018

@guybedford I think that may just confuse things because I generally think users expect .babelrc files to behave strictly hierarchically. It also means people need to know what a babelrc is to search for the concept, and it's not really an obvious name for new users.

@devongovett

This comment has been minimized.

Show comment
Hide comment
@devongovett

devongovett Jan 31, 2018

Contributor

FYI there is a discussion about this in parcel as well. parcel-bundler/parcel#13

We will also be supporting compilation of node_modules with babel-preset-env. We will use the package.json engines field, or browserslist field to determine the language of the source module, and the browserslist of the target app to determine what to compile to. Then we do a diff to include only the necessary plugins. That way modules will not be compiled all the way to ES5, but to the app’s browser target instead. parcel-bundler/parcel#559

Most of the discussion now revolves around custom Babel plugins in node_modules, which is convenient for local development with linked modules. I proposed adding a field to package.json to selectively turn on custom babelrc support in node_modules. Would be interested to hear what you all think of that. I’d like to come up with something that can be used across tools, not just something parcel specific.

Contributor

devongovett commented Jan 31, 2018

FYI there is a discussion about this in parcel as well. parcel-bundler/parcel#13

We will also be supporting compilation of node_modules with babel-preset-env. We will use the package.json engines field, or browserslist field to determine the language of the source module, and the browserslist of the target app to determine what to compile to. Then we do a diff to include only the necessary plugins. That way modules will not be compiled all the way to ES5, but to the app’s browser target instead. parcel-bundler/parcel#559

Most of the discussion now revolves around custom Babel plugins in node_modules, which is convenient for local development with linked modules. I proposed adding a field to package.json to selectively turn on custom babelrc support in node_modules. Would be interested to hear what you all think of that. I’d like to come up with something that can be used across tools, not just something parcel specific.

@damianobarbati

This comment has been minimized.

Show comment
Hide comment
@damianobarbati

damianobarbati Jan 31, 2018

@devongovett how are you going to get the entrypoint of the untranspiled sources of your dependencies in node_modules?
Both main and module links to transpiled code today.
webpack/webpack#5935

damianobarbati commented Jan 31, 2018

@devongovett how are you going to get the entrypoint of the untranspiled sources of your dependencies in node_modules?
Both main and module links to transpiled code today.
webpack/webpack#5935

@devongovett

This comment has been minimized.

Show comment
Hide comment
@devongovett

devongovett Jan 31, 2018

Contributor

Just made an alternative suggestion to enable distributing both transpiled code and source code together. parcel-bundler/parcel#13 (comment)

Contributor

devongovett commented Jan 31, 2018

Just made an alternative suggestion to enable distributing both transpiled code and source code together. parcel-bundler/parcel#13 (comment)

@loganfsmyth

This comment has been minimized.

Show comment
Hide comment
@loganfsmyth

loganfsmyth Feb 8, 2018

Member

Alright everyone, I've created #7358. Please look over what I've posted there and let me know what you think.

Member

loganfsmyth commented Feb 8, 2018

Alright everyone, I've created #7358. Please look over what I've posted there and let me know what you think.

@phawxby

This comment has been minimized.

Show comment
Hide comment
@phawxby

phawxby Mar 2, 2018

@sokra and others. For this of us searching for a solution to the problem now using Babel 6 how is this usually solved?

I've got a client side repo included in a server side repo via git submodules. I need to import some of the client code into the server side but the babelrc for the client application is screwing up the import due to the different configurations.

phawxby commented Mar 2, 2018

@sokra and others. For this of us searching for a solution to the problem now using Babel 6 how is this usually solved?

I've got a client side repo included in a server side repo via git submodules. I need to import some of the client code into the server side but the babelrc for the client application is screwing up the import due to the different configurations.

@loganfsmyth

This comment has been minimized.

Show comment
Hide comment
@loganfsmyth

loganfsmyth Mar 2, 2018

Member

@phawxby Feel free to join Slack or post on our recommended support channels, but pinging everyone on this thread about unrelated stuff will distract from the goals of this issue so I'd rather not help here.

Member

loganfsmyth commented Mar 2, 2018

@phawxby Feel free to join Slack or post on our recommended support channels, but pinging everyone on this thread about unrelated stuff will distract from the goals of this issue so I'd rather not help here.

@robwierzbowski

This comment has been minimized.

Show comment
Hide comment
@robwierzbowski

robwierzbowski Apr 19, 2018

A little late to the party, just wanted to add some background for transforming/transpiling packages in node_modules (cc @ljharb).

For maximum performance of front-end bundles, some people (e.g., Phil Walton, myself and the team I work with) want to pull as many dependencies as possible in as uncompiled ES modules. This:

  • improves tree shaking
  • gives us control of the top and bottom end of browser support (we can keep more JS minimally transpiled for our modern browsers, improving performance)
  • helps prevent multiple inclusions of global polyfills and helper functions (if dependency A's pre-transpiled version bundles Bluebird promise and our app polyfills at runtime with Polyfill.io we're downloading unused bytes)

I got bit by the nested babelrc's this week, and would love a root: true or similar feature in Babel 7.

robwierzbowski commented Apr 19, 2018

A little late to the party, just wanted to add some background for transforming/transpiling packages in node_modules (cc @ljharb).

For maximum performance of front-end bundles, some people (e.g., Phil Walton, myself and the team I work with) want to pull as many dependencies as possible in as uncompiled ES modules. This:

  • improves tree shaking
  • gives us control of the top and bottom end of browser support (we can keep more JS minimally transpiled for our modern browsers, improving performance)
  • helps prevent multiple inclusions of global polyfills and helper functions (if dependency A's pre-transpiled version bundles Bluebird promise and our app polyfills at runtime with Polyfill.io we're downloading unused bytes)

I got bit by the nested babelrc's this week, and would love a root: true or similar feature in Babel 7.

@hzoo

This comment has been minimized.

Show comment
Hide comment
@hzoo

hzoo Apr 19, 2018

Member

Hey @robwierzbowski thanks for your input! And yeah we have moved discussion a while back to #7358 and are going to be moving forward with that. And yes we understand compiling node_modules is a use-case we all want to shoot for, not really the reasoning but how it would work!

Member

hzoo commented Apr 19, 2018

Hey @robwierzbowski thanks for your input! And yeah we have moved discussion a while back to #7358 and are going to be moving forward with that. And yes we understand compiling node_modules is a use-case we all want to shoot for, not really the reasoning but how it would work!

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.