Allow more flexible file-based configuration while preventing .babelrcs from breaking things

[loganfsmyth]

Q	A
Fixed Issues?	Fixes #6766
Patch: Bug Fix?
Major: Breaking Change?	Y
Minor: New Feature?	Y
Tests Added + Pass?
Documentation PR
Any Dependency Changes?
License	MIT

Alright! Let's do this. This is my proposal for fixing the issues spelled out in #6766. This PR does the following:

When searching for .babelrc files for a given file, stop at package.json

Currently, Babel will look all the way up the directory hierarchy. This change prevents config files outside of a given package from potentially breaking your project's builds, which has been a problem occasionally. Also potentially frustrating for projects that integrate Babel, because they often allow users can optionally create .babelrc files, and if they haven't created one, but have one in a parent folder, it might find a bad config.

The downside here is that monorepos like

.babelrc
packages/
  one/
    package.json
    index.js
  two/
    package.json
    index.js

will fail to find the .babelrc. I think that is acceptable because there are alternatives:

1. Have each module have their own .babelrc doing { extends: "../../.babelrc" } to explicitly load the config from a parent directory.
2. See the next point.

Adds support for a babel.config.js file along the lines of what Webpack does

This PR adds a root option (that defaults to the working directory), that defines the conceptual "root" of your project. This location is used to automatically search for the config file babel.config.js. Optionally instead the user can specify configFile: "./foo.js" or whatever they'd like.

The file itself behaves just like a .babelrc.js file would, but is not loaded relative to the individual file being compiled, and thus has more flexibility in what files it applies to. For instance, if users are symlinking in node_modules or something, it is currently impossible for a .babelrc to configure their builds any way other than by hard-coding their config into webpack.config.js, which is not great.

With our new support for overrides it's now easy for users to have a single babel.config.js which could define the full build process for their project, when used alongside env.

Only search for a .babelrc for files inside the root (which defaults to cwd) (unless the user opts out)

Specifically excluding node_modules folders. Given the path of the file being compiled, if the path relative to the root contains node_modules, do not attempt to search for .babelrc files. This essentially has the effect of making .babelrc files only useful by default in a "root" package, which is essentially how most people used them.

This (potentially combined with the previous babel.config.js) allows users to safely do

test: /\.js$/,
loader: "babel-loader",

without risking loading a .babelrc from a node_modules that was not intented to apply to the current project. There are still other edge cases to compiling node_modules, but this was a particularly dangerous edge case, especially for things like babel-register that don't have a nice rules list like Webpack does.

To allow people to opt into .babelrc usage, potentially for local development, this PR allows Babel's babelrc option to be an array, so users could do

test: /\.js$/,
loader: "babel-loader",
options: {
  babelrc: [
    ".",
    "../some-linked-package"
  ]
}

which would also attempt to load .babelrc files for anything within that package. Alternatively

options: {
  babelrc: [
    ".",
    "node_modules/foo"
  ]
}

if there was a specific package you wanted to load a config for, and were confident in its installation location.

What this PR does not do

Part of the reason I personally had so much back and forth on this issue is because it wasn't necessarily obvious what root should actually be responsible for. For instance, if Babel is passed a file that is not in the root, should it compile it at all? As is hopefully clear, my answer there was yes.

One core thing that this PR does is that it leaves it up to the individual callers of Babel to decide what files should actually be passed to Babel in the first place. Alongside that, if you configure Babel with a custom root or configFile or babelrc array, the user will have to decide if those need to be replicated between Babel callers, because they are top-level Babel options, not ones that can be included inside a config file (for obvious reasons).

What is left

This PR has no tests, and an unoptimized implementation of a lot of stuff. It will need more work before it can land, but I want to post it now to get a feel for what everyone thinks.

[nicolo-ribaudo]

I like this implementation. Just one thought: should Babel accept babel.config.json? Most of the babelrcs I use are just JSON files, so from a user's point of view it seems unnecessary to require to use JavaScript.

[billyjanitsch]

It might be confusing for babel.config.js to behave differently than .babelrc.js, since the widely-used cosmiconfig treats foorc.js and foo.config.js identically. For example, postcss (which is often used alongside Babel) allows postcss.config.js and .postcssrc.js to be used interchangeably, with no difference in lookup semantics.

Maybe there's a better name for this file?

[loganfsmyth]

should Babel accept babel.config.json? Most of the babelrcs I use are just JSON files, so from a user's point of view it seems unnecessary to require to use JavaScript.

That's a fair point. I didn't bother because while I'm happy to run JSON5 on .babelrc, I don't love the idea of running it on a real .json file, and I know people like using comments and such, but we might be able to find a middle-ground. The main downside is that it's one more file that Babel has to search for, which means more calls to fs.stat and slightly slower config loading times.

It might be confusing for babel.config.js to behave differently than .babelrc.js

Maybe, but at the end of the day I feel like there are only names that we'd want to encourage. In this case I was definitely mimicing Webpack, which I don't believe does hierarchical lookups. This PR also makes .babelrc stop at the package boundary, which that also doesn't appear to do by default. Not sure if postcss adds that though. If people do have name suggestions we can explore it. My goal is very explicitly to make this file not hierarchical though, so I'm not sure anything but documentations can really make that clear.

[hzoo]

Is there a way to limit the number of config files we support? Or is that better than having another option in configs, or we are ok with this?

Options:

• cli for basic stuff like babel --presets env
• .babelrc as majority
• package.json "babel" key
• pass config babel-core/loader directly
• new .babelrc.js
• new babel.config.js - I think it's going to be confusing some of these act differently unless its like babel.root.js or something but that's probably not great either? Or can we remove the new .babelrc.js

also cc @babel/eslint regarding eslintrc lookup and root and this idea for babel

[loganfsmyth]

Is there a way to limit the number of config files we support?

Good question. At the end of the day, the hierarchical behavior of .babelrc is the problem here, so we at least need some way around that. That said, we've also heard from plenty of people over the years that aren't a fan of using dotfiles for build configuration, which I think is a fair complaint.

.babelrc as majority

Personally I think the majority of people don't need or care about configuring via hierarchy, so it'd be just as easy to recommend babel.config.js as the default.

Or can we remove the new .babelrc.js

I don't know, I feel like the logic there is identical to .babelrc, it just happens to be a .js file, so I think most people will get it.

I see Babel's config as having a few pieces

• Config passed programmatically (wether babel-loader, or CLI or whatever, they are all the same)
• Config from a babel.config.js file that works pretty much like programmatic args, but allows you to load it from a file centrally.
• Config passed based on directory hierarchy, like .babelrc/.babelrc.js/package.json#babel (now limited to within a given package only)

[danez]

I'm not sure exactly but this might break react native, because they transpile node_modules, although I'm not sure how they handle the babelconfig for this packages. We should maybe ask someone who is more into react native.

[loganfsmyth]

@danez Fair point. The only question would be stopping .babelrc resolution at package boundaries. This PR still allows people to do babelrc: true to enable resolving babelrc config for every single file (rather than those in the root package), it's just that now project-level babelrcs won't be affect things in node_modules.

[hzoo]

cc @zertosh to ping someone on react-native then?

Whatever we land, we'll really want to have good docs for the config behavior + and just in general (and probably some examples with folders/code)?

• basic setup
• symlinking a package that has a config in it
• monorepo setup
• integrating with another tool that will read a babelrc
• compiling node_modules (also using overrides)

[zertosh]

@vjeux who has thoughts on this on the RN side? @hzoo, internally we tend not to use babelrc configs (the ones we do have are really basic and self contained), we tend to use transpire scripts that wire up the config

[vjeux]

@yungsters spent some time in that area

[hulkish]

i feel like enhanced-resolve would really be ideal for this. Would need to write some resolver plugins to facilitate the read-up-but-stop-at logic.

Or even, grant dev user the control to compose the strategy as part of the config. As well as exposing a configurable cache predicate.

[michael-ciniawsky]

Why not just using cosmiconfig ? It has a defined set of config (names) which are supported, including package.json and it also stops at the first 'valid' config found, which is quite convenient if config lookup starts relative to the file being processed by default (Config Cascading). While specifying a custom path is easy (to start from/or exact) if needed. Stoping at apackage.json is not always really 1:1 to where a .babelrc maybe located and tangles to 'independent' files together e.g the monorepo example with cosmiconfig could work like this (via Config Cascading)

|– packages
| |– 1
| | |– index.js (start here)
| | |– package.json (with 'babel' field => 'local' config for this package (if needed, stops here))
| |– 2
| | |– index.js (start here)
| | |– package.json (no 'babel' field => use 'global' .babelrc)
|– .babelrc ('Global')

ℹ️ Cascading works with any config 'format'

[loganfsmyth]

@michael-ciniawsky

Stoping at a package.json is not always really 1:1 to where a .babelrc maybe located and tangles to 'independent' files together

I agree. That's essentially the downside I called out in the PR description. The issue is that if we don't stop there, where do we stop? We could had yet another option to Babel, but that starts getting extreme. I explicitly want to stop doing "search all the way up" because that has caused users pain in the past, and the monorepo case seems to be the only one where it's potentially painful, and this PR presents an alternative.

Why not just using cosmiconfig?

That's essentially what we've already had until now, just implemented internally. I'm trying here to add a config file that doesn't cascade because cascading breaks for anyone who wants to symlink packages during local development, and for anyone looking to specific compilation configuration for packages that live inside node_modules.

[michael-ciniawsky]

I agree. That's essentially the downside I called out in the PR description. The issue is that if we don't stop there, where do we stop?

process.cwd() < ... < file.dirname (default lookup range) or set an absolute config path (no lookup)

I explicitly want to stop doing "search all the way up" because that has caused users pain in the past,

What pains ? :)

That's essentially what we've already had until now, just implemented internally.

postcss, posthtml, eslint etc already use cosmiconfig => only one package dependency, no direct maintenance, common config format(s)

I'm trying here to add a config file that doesn't cascade because cascading breaks for anyone who wants to symlink packages during local development, and for anyone looking to specific compilation configuration for packages that live inside node_modules

symlink => specify the absolute path to the config one intend to use ?
node_modules (not sure if I completely follow, but in case it goes in the direction of a 'shareable' config (e.g eslint-config-*), =>require it in the projects .babelrc.js or set the absolute config path ? Or the package name as 'path' and babel require()'s it

[loganfsmyth]

process.cwd() > ... > file.dirname (default lookup range) or an absolute config path (no lookup)

What if the file being compiled isn't even in the working directory? e.g.

my-projects/
  some-library/
    index.js
  some-app/
    node_modules/
      some-library -> ../../some-library/
    index.js

assuming some-app is the working directory. Even if you exposed an option to pass Babel the root, you could have multiple libraries and each one would have its own root, making the ergonomics extremely painful.

What pains ? :)

Most often its new/inexperienced users who create a .babelrc in their home directory or something. Then some time later they create a new project and get babel errors because it is reading the config from outside the project (since they either haven't created one in their new project yet, or are passing the options programmatically instead).

It's note insanely common, but it exposes what I consider to be a pretty big flaw in hierarchical configs.

postcss, posthtml eslint etc already use cosmiconfig => only one package dependency, no direct maintenance, common config format(s)

What I'm saying with that comment is that cosmicconfig doesn't add anything new functionality-wise. Adding it would not materially affect the objectives that this PR is trying to address, from what I can tell anyway. I'm also not familiar with how any of those projects handle cross-package compilation.

symlink => specify the absolute path to the config you intend to use

That's pretty much what this PR is adding with babel.config.js except that it loads from the working directory automatically and lets you pass an explicit name as an override.

node_modules (not sure if I completely follow, but in case it goes in the direction of a 'shareable' config (e.g eslint-config-*), =>require it in the projects .babelrc.js or set the absolute config path

To clarify, for ESLint a configuration is by-definition associated with a package itself. A package defines its own linting rules. Where that breaks down is that for Babel, in the case of node_modules, it is often some root application that wishes to configure the compilation rules. The fact that a given node_modules has a .babelrc with a set of rules isn't relevant for the root-level application.

Say the user has

project/
  .babelrc
  src/
    index.js
  node_modules/
    random-third-party-module/
      .babelrc // random config that was published because the author of the module didn't think to put it in npmignore
      index.js

Because the node_modules has a .babelrc, hierarchical resolution will not only block the correct config (the project/.babelrc one) from loading, it will load a .babelrc that is totally unrelated to the project being compiled and will probably cause Babel to throw an exception.

[hulkish]

I sort of feel like the find-up and hierarchical babelrc logic may be the source of this challenge.

What if the file being compiled isn't even in the working directory? e.g.

Couldn't it be argued that this is not a case we can reliably support with auto find up behavior?

Maybe, instead of a hierarchical babelrc we allow assignment of paths/patterns to a known babelrc location? Something like webpack's rules config structure:

{
  
  configs: [
    { 
       include: [path.resolve('src')],
       use: path.resolve('.babelrc') // or inline config
    }
  ]
}

Then, for something like babel cli - we would throw an error to use this strategy if we failed to find a babelrc up to or a particular level?

[loganfsmyth]

I sort of feel like the find-up and hierarchical babelrc logic may be the source of this challenge.

Yeah, to me it feels like find-up only makes sense on an individual-package basis.

Maybe, instead of a hierarchical babelrc we allow assignment of paths/patterns to a known babelrc location? Something like webpack's rules config structure:

With this PR, you can get that by putting those rules into an overrides block inside the babel.config.js or passed programmatically to Babel.

Then, for something like babel cli - we would throw an error to use this strategy if we failed to find a babelrc up to or a particular level?

I'm not sure I follow what you mean by that.

[michael-ciniawsky]

What if the file being compiled isn't even in the working directory? e.g.

my-projects/
  some-library/
    index.js
  some-app/
    node_modules/
      some-library -> ../../some-library/
    index.js

Dependens where the .babelrc is located in this example :)

my-projects/
  .babelrc (works for some-library && some-app)
  some-library/
    .bablerc (works for some-library only, if not set as absolute path)
    index.js
  some-app/
    .babelrc (works for some-app only, if not set as absolute path)
    node_modules/
      some-library -> ../../some-library/
    index.js

assuming some-app is the working directory. Even if you exposed an option to pass Babel the root, you could have multiple libraries and each one would have its own root, making the ergonomics extremely painful.

If I would need to work with the file tree given in the example I would assume process.cwd() (Root) is my-projects as the stop-dir. If some-library is needed in some-app => node_modules (as dependency) and otherwise each 'package/lib/app' needs their own .babelrc

Where would the package.json's be located in this example ?

Most often its new/inexperienced users who create a .babelrc in their home directory or something. Then some time later they create a new project and get babel errors because it is reading the config from outside the project (since they either haven't created one in their new project yet, or are passing the options programmatically instead).

Yeah, it's something one needs to learn and it can be annoying at first, but that's something one at some point definitely needs to understand imho :). Is babel merging .babelrc's or looking up the configs when options are already applied programmatically ? Why ? programmatically (no lookup) > .babelrc (look up) > Error ?

That's pretty much what this PR is adding with babel.config.js except that it loads from the working directory automatically and lets you pass an explicit name as an override.

👍 That's what I meant

To clarify, for ESLint a configuration is by-definition associated with a package itself. A package defines its own linting rules.

This is/can imho also true for transpiling in case the package author uses a prepare (prebuild) hook/script or provides a .babelrc with preset-env (as a 'babel convention') etc etc. to publish his/her stuff. Still eslint <=> babel definitely differ in scope

Where that breaks down is that for Babel, in the case of node_modules, it is often some root application that wishes to configure the compilation rules.

1. An author should publish the packages already transpiled (especially if additional stuff is used (plugins, marcos etc))
2. Untranspiled and babel-preset-env (presets), without an .babelrc published by the package itself (without additional plugins, marcos etc, 'just' ES201X => ES201Y)
3. 'root' and package .babelrc use babel-preset-env => .babelrc (bottom-up lookup) && babel-preset-env (.browserlistrc) (top-down lookup), while babel is also able to (auto) install eventual missing additional plugins/marcos for dependencies, if missing in the root applications package.json
4. Your/other 'unique/exotic' combos might simply break at some point :) => No Support => Refactor

|– src
| |– index.js
|
|– node_modules
| |– dep
| | |– .babelrc (preset-env + additional plugins/marcos*)
| | |– .browserslistrc (=> I'm not used in this example :) )
| | |– package.json
| | |– index.js
|
|– .babelrc (preset-env)
|– .browserslistrc
|– package.json

{
  'src/index.js': {
     babelrc: '../'
     browserslist: '../'
  },
  'node_modules/dep/index.js'. {
     babelrc: './node_modules/dep'
     browserslist: '../../'
  }
}

*

try {
  plugin = require('babel-plugin-*')
} catch (err) {
  const installer = cp.spawn('npm', [ 'i', '-D', 'babel-plugin-*' ])

  installer.on('close', (code) => {
     if (code !== 0)  throw new Error(`... ${code}`)
    
     plugin = require('babel-plugin-*')
  })
}

But yeah... 😛 the latter is 💯 learn how, follow the convention, but be done and it's idealistic/naive in the current ecosystem shape, but ¯_(ツ)_/¯

[loganfsmyth]

@michael-ciniawsky

An author should publish the packages already transpiled (especially if additional stuff is used (plugins, marcos etc))

I think we all agree that non-standard language functionality should be compiled before publishing, but it is quite clear at this point that the community as a whole is going to adopt language features as they land in Node (LTS anyway). That means over time users looking to compile for browsers are going to need to run babel-preset-env on things in node_modules more often.

Untranspiled and babel-preset-env (presets), without an .babelrc published by the package itself (without additional plugins, marcos etc, 'just' ES201X => ES201Y)

You mean the package in node_modules should be published with a .babelrc? To me that would be a massive foot-gun for users. Most package developers are never going to stop and think "oh I shouldn't publish this .babelrc to npm unless others can use it". A .babelrc is an extremely common file and if we went this route and it is published accidentially, it would break the build of every user using that package or that had it as a subdependency. Not to mention, a .babelrc and the plugins within it are highly tied to the version of Babel that is trying to use it. Existing packages that have published .babelrc files are all going to reference Babel 6 plugins that may well not work on 7.x. If we consider published .babelrc files as part of the public API of a module from Babel's perspective, it would essentially require us to never make a breaking change to Babel's APIs ever again, and Babel's API is huge.

'root' and package .babelrc use babel-preset-env => .babelrc (bottom-up lookup) && babel-preset-env (.browserlistrc) (top-down lookup), while babel is also able to (auto) install eventual missing additional plugins/marcos for dependencies, if missing in the root applications package.json

Automatic installation of packages isn't something I can see us ever wanting. If a config is so complicated that a user can't install it themselves, to me that points to an architectural problem in the config loading system itself. Avoiding that is a big part of why I'm doing this.

Doing .browserlistrc in this way would also break once again in the symlink case because the .browserlistrc wouldn't be in the parent folder of dep at all.

Going back to the example structure

my-projects/
  .babelrc
  some-library/
    .bablerc
    package.json
    index.js
  some-app/
    .babelrc
    webpack.config.js
    package.json
    index.js
    node_modules/
      some-library -> ../../some-library/
      other-library/
        .babelrc
        package.json
        index.js

First off, to clarify, what I'm talking about here is a user configuration where some-app has a webpack.config.js with something like

rules: [{
    test: /\.js$/,
    loader: 'babel-loader',
}]

Webpack will attempt to pass all JS files in a project. In this world, a user is conceptually trying to compile "some-app" with a set of dependencies. As far as they are concerned, they don't particularly care what configuration is in individual node_modules. They just want to say "compile all standards-compliant code to ESX" based on their own config in some-app/. Because of hierarchical lookup, there is literally no way for config options inside my-projects/some-app/.babelrc to affect my-projects/some-library/index.js. It is entirely unrealistic to expect users to create a my-projects/.babelrc file since that would likely be outside of a version controlled folder, making it a maintenance nightmare. This PR allows creating my-projects/some-app/babel.config.js that would be loaded and apply to all files passed to babel-loader.

The core tenants of .babelrc files after this PR are that

• Users generally don't want individual node_modules dependencies to be able to configure their own compilation process in the first place.
• When they do, they can opt into it specifically, and accept that they are then required to install whatever plugins that .babelrc reference (which may or may not even be possible depending on directory structure).
• If they do opt into it and the module doesn't have a .babelrc, picking one up from a random parent directory is never what they'd expect.

But yeah... 😛 the latter is 💯 learn how, follow the convention, but be done and it's idealistic/naive in the current ecosystem shape, but ¯_(ツ)_/¯

I'm not sure I follow 100%, but I'd say being idealistic/naive is just what Babel has been doing up until now, and this is my attempt to fix that so our users have options for usecases that are quite painful currently, but are also becoming more and more common.

[loganfsmyth]

Woops, missed these:

Yeah, it's something one needs to learn and it can be annoying at first, but that's something one at some point definitely needs to understand imho :). Is babel merging .babelrc's or looking up the configs when options are already applied programmatically ? Why ? programmatically (no lookup) > .babelrc (look up) > Error ?

Babel does not merge multiple .babelrc unless the first .babelrc explicitly extends: "../.babelrc" (or something) but it does merge programmatic options with .babelrc options. Users can opt for only programmatic args by passing babelrc: false.

That's pretty much what this PR is adding with babel.config.js except that it loads from the working directory automatically and lets you pass an explicit name as an override.

👍 That's what I meant

I thought you wanted babel.config.js to be search for hierarchically too? If we did that, it would require users to pass an explicit path for the config to apply consistently to dependencies, which doesn't seem ideal to me.

[michael-ciniawsky]

Could node_modules/ not better be skipped anyways (by default), instead of two different configs and match common naming conventions --config, pkg.babel, .babelrc.json, babelrc.js, babel.config.js (similiar to cosmiconfig) but besides JSON/JS behave the same ? Just read everything again, makes perfectly sense now. Final questions :)

1. Two different config loading systems ? (This is really confusing)
2. Merging programmatically and config file by default ?
3. Still support config lookup (excluding node_modules) relative to the file, if nothing else is setup (stops at root (process.cwd()) by default) ?

I thought you wanted babel.config.js to be search for hierarchically too? If we did that, it would require users to pass an explicit path for the config to apply consistently to dependencies, which doesn't seem ideal to me.

Yep :), for the project side of affairs (as mentioned I initially thought dependencies should be individually 'transpilable' with their respective configs (as a requirement), but that's rightfully going in the opposite direction as it seems)

|– src
| |– index.js
| |– directory
| | |– .babelrc (still works, not that it's recommended or often needed)
| | |– index.js
|
|– .babelrc (recommended to place the config in './')
|
|– node_modules ('lookup' for deps files 'starts' here)
| |– dependency (no lookup)
| | | – index.js  (no lookup)
|
|– package.json

[hulkish]

@loganfsmyth

Then, for something like babel cli - we would throw an error to use this strategy if we failed to find a babelrc up to or a particular level?

I'm not sure I follow what you mean by that.

Meaning; if babel cli fails to find .babelrc in the cwd/wherever (assuming you intend to keep that logic) - then we throw an error recommending dev user explicitly defines where config file is located.

eslint approaches this with a concept of "overrides" - allowing you to assign linting rules to a particular list of paths/patterns. I feel like that hits close to home for this.

They also have a concept of "root" - definitely something that could apply here as well.

[CH-RhyMoore]

I'm not opted into the beta version I don't think and presume this is released now? I think this has broken my build, I don't find the documentation of very helpful so far (particularly not in terms of what I need to change in my monorepo to get it working the way it did yesterday), and I didn't read super closely, but I think I'd find the behavior described here and in the documentation thoroughly undesirable and would vastly prefer cascading config that applies to sibling files like eslint's.

[lukescott]

@CH-RhyMoore add a babel.config.js file to the root of the monorepo with the following:

module.exports = {
	babelrcRoots: ["./packages/*"],
}

Assuming you run commands from the root. If you run a command from within a package, add a babel.config.js file within the package you are running the command from:

module.exports = {
	babelrcRoots: ["../*"],
}

babel.config.js is picked up in the current working directory. It also gets picked up by build tools like webpack. Each package will use the .babelrc file that belongs to that package as before.

The only thing that doesn't work is it won't transform files in node_modules. For that you have to target the module in webpack and specify your babel options. Something along these lines:

{
	test: /\.jsx?$/,
	include: /node_modules\/module-name/,
	use: [
		{
			loader: "babel-loader",
			options: {
				babelrc: false,
				// customize your babel options below
				presets: ["@babel/env", "@babel/react"],
				plugins: [
					"@babel/plugin-proposal-class-properties",
					"@babel/plugin-proposal-object-rest-spread",
				],
			},
		},
	],
}

The reason for this is a given node module may use a different version of babel. Most node modules usually pre-compile, so it's not a common problem.

[CH-RhyMoore]

Okay, that is simple enough. Thank you very, very much!

Still, my 2 cents--eslint really has always had my favorite configuration experience. Personal preference plays a part here, but I've always found it very intuitive.

[CH-RhyMoore]

I might have another issue going on at the same time, potentially with Webpack 4? Because that alone isn't fixing it. Scenario:

|_ project/
   |- babel.config.js
   |- package.json
   |_ packages/
      |_ package-a/
         |- .babelrc
         |- babel.config.js
         |- package.json
         |- webpack.config.babel.js

/project/packages/package-a/webpack.config.babel.js:1
(function (exports, require, module, __filename, __dirname) { import path from 'path';
                                                                     ^^^^
SyntaxError: Unexpected identifier

But searching that sent me back here. Oh well, I'll keep looking.

[lukescott]

It may have something to do with the modules on the@babel/env preset. Webpack 4 can handle ES6 module syntax. Hard to say what you need to do exactly, but that's likely where the problem is.

Babel has an excellent Slack community. Someone there would likely know better than me.

[CH-RhyMoore]

Thing is, we use es2015 for some of the ones that are failing. Our monorepo also currently mixes Babel 6 and Babel 7.

project root

|_ project/
   |- babel.config.js

const lernaConfig = require('./lerna.json');

module.exports = {
  babelrcRoots: ['.'].concat(lernaConfig.packages),
  rootMode: 'upward-optional'
};

package with build script

.babelrc

|_ project/
   |_ packages/
      |_ package-a/
         |- .babelrc

{
  "presets": ["es2015"]
}

|_ project/
   |_ packages/
      |_ package-b/
         |- .babelrc

{
  "presets": [["env"]],
  "plugins": ["lodash", "transform-runtime"]
}

babel.config.js

|_ project/
   |_ packages/
      |_ package-a/
         |- babel.config.js
      |_ package-b/
         |- babel.config.js

const lernaConfig = require('./../../lerna.json');

module.exports = {
  babelrcRoots: ['.', '../*'].concat(lernaConfig.packages),
  rootMode: 'upward-optional'
};

Tried a few different combinations of this. With and without '.' in babelrcRoots. With and without the project root babel.config.js. With and without the package-level babel.config.js. With and without 'rootMode'. With and without .concat(lernaConfig.packages) in the package-level babel.config.js.

The symptoms started yesterday in Jest with our source code and now are happening on the Webpack configuration file (though I didn't initially connect that error to this one). Everything worked last week and we didn't actually update anything on purpose in the interim. Same .babelrc file position and contents, same version of Webpack, same versions of Babels...within the constraints of Lerna's use of package-lock.json with hoisting, which I believe I've previously observed as being looser than one might prefer.

Not convinced it's entirely Babel. Ugh, Slack. Talk about a waste of everyone's energy. Solutions hidden where no one else can see or learn from them? No wonder I can't find an answer.

[CH-RhyMoore]

FYI, getting a 523 at that Slack link. Was trying to peek at it in case I felt brave.

[lukescott]

The es2015 preset is deprecated. I think even in Babel 6. As far as Jest is concerned, I'm not sure if they've updated to Babel 7 or not, but if they are pointing to the old babel-core package, it's just a peer dependency so you can install 7.0.0-bridge.0. I'm not sure you really can mix babel 6 and 7.

The Slack link: Looks like the host that does the redirect is down. Was working earlier today. I don't have the direct link. I copied it from the project README. EDIT: It's back up.

[CH-RhyMoore]

In a monorepo where each package manages its own build, it should absolutely be possible. Why wouldn't it be? npm is quite capable of installing multiple versions of the same package when their semver ranges aren't compatible, Lerna is quite capable of not getting in the way of npm doing so, and Node.js is quite capable of resolving the version needed by files in a particular package.

(We've been doing so without any issues for ~2 months. Our packages produce sufficiently different things that their builds aren't consistent enough to raise up to the project level at this point. We'll do that at some later point when we see which consistent patterns emerge.)

My biggest issue wasn't Babel related directly at all, I don't believe. For some reason, the update to Webpack 4 we did 15 days ago suddenly began requiring us to install @babel/register|babel-register and pass it as an argument (webpack -r @babel/register|webpack -r babel-register) in order to use ES import/export in our webpack.config.babel.js file. Previously, simply using the file extension babel.js was sufficient. This was also necessary for webpack-dev-server. While the -r option is documented, I couldn't find any documentation of this change in functionality.

I think that issue is entirely about how Webpack (CLI only? *shrug*) does or doesn't run Babel on its own configuration file. So FYI for the future, if someone mentions the errors coming from their Webpack config file, it could be this issue.

But by the time I finally found that, I'd already upgraded a half dozen or more dependencies in each package (times 20) to use Babel 7 in everything (and update plugins and then the tools that use the plugins to versions capable of handling it). We have updating on our roadmap for this month, so I here I am working late, because I thought I might as well continue, and I hope to god I'm close to finishing.

I've had to change a multitude of other things to get this far in migrating from Babel 6 to 7 today, but, strangely, adding babel.config.js configuration for .babelrc was not one. I deleted all of them and everything that's not broken for another reason is still working. I should have guessed sooner that it was a red herring for the first Webpack issue, since the couple of packages that used Babel 7 already were fine (albeit they didn't use .babelrc, I have two working examples now that do and don't need babel.config.js anywhere in the project).

So it seems if you set up your monorepo like ours, you may not need to do anything about the new configuration behavior. Though it's possible it may yet arise as an issue. I haven't even gotten to Jest yet today since I'm still trying to get everything in the project building.

...later...

Jest was fine as well with babel-core@7.0.0-bridge.0. (Thanks for that tip and your help in general.)

[sxt1992]

Sometimes we need to parse the files in the node_modules folder. So, I want you to provide a configurable place, not a fixed one.

[nicolo-ribaudo]

You just need to use babel.config.js and it will transpile node modules.

[sxt1992]

According to your method, I compiled successfully. Thank you.