Symlinks in project - loader not working when using include?

[elado]

Can webpack read through symlinks? I have a simple symlink under /src with .jsx files. I have { test: /\.jsx?$/, loader: 'babel-loader?...', include: path.resolve(__dirname, 'src') } but it won't load files with babel-loader, I get ES5 compilation error.

If I replace the symlink with the actual folder and files - it works.
If I replace the include with exclude: /node_modules/ it also works.

Bug, or am I doing it wrong?

[skipjack]

Having a similar issue with symlinks... I have all my modules under a /src folder in the root. Inside that /src folder I created a symlink to a directory /globals which is outside the root. Importing modules through the symlink works fine, however the loaders (in this case babel) don't transform these modules properly which causes:

1:61 error Parsing error: Illegal export declaration
...
You may need an appropriate loader to handle this file type.

Tried a few things in the config such adding the actual path of the directory I symlinked to in resolveLoader.modulesDirectories and in loader.include.

This issue seems similar but none of the solutions there solved the issue. Feel like I'm missing something obvious...

[arunabha]

+1, I have this exact problem, the loader (babel) does not seem to be getting applied to files under a symlink. Replacing the symlink with an actual directory works.

[s-panferov]

I think that the main issue here is that webpack resolves symlinks and it actually 'sees' this files by their absolute path. And this absolute path doesn't match your include directive or test regexp.

Could you please check this?

[s-panferov]

Some info about this behavior can be found here #554

[jraede]

I was able to fix this by doing the following in webpack.config.js (this is for importing our components library using the jsx files rather than the compiled umd):

var includePaths = [
  fs.realpathSync(__dirname + '/app'),
  fs.realpathSync(__dirname + '/node_modules/my-repo/lib'),
];

And then later:

module: {
    loaders: [{
      test: /\.js?$/,
      include: includePaths,
      loader: 'babel'
    }, {
      test: /\.jsx?$/,
      include: includePaths,
      loader: 'babel'
    },{
      test: /\.json?$/,
      loader: 'json'
    }, {
      test: /\.css$/,
      loader: 'style!css?modules&localIdentName=[name]---[local]---[hash:base64:5]'
    }]
  },

[eagsalazar]

I got this to work by setting the includePaths but also by setting both resolve.root and resolveLoader.root to path.resolve(__dirname, 'node_modules') (or wherever you want those external sources to be able to resolve modules). @jraede I think that worked for you only because your include paths both had node_modules in an ancestor directory. You need to set both so files external to your project that you want to import can themselves (a) import other modules from your project's node_modules, and (b) so your loaders can be resolved (otherwise you get cannot resolve module 'babel', etc).

  resolve: {
    alias: { src: srcPath, common: commonPath },
    extensions: ['', '.js', '.scss', '.json'],
    root: [path.resolve(__dirname, 'node_modules')]
  },

  resolveLoader: {
    root: [path.resolve(__dirname, 'node_modules')]
  },

[BarukhOr]

@jraede would it be possible for you to provide your webpack.config I'm running into a similar problem and even after implementing your suggested solution i get the following error

ERROR in multi main
Module not found: Error: Cannot resolve 'file' or 'directory' /LifeInYourWay/iGem/src/index in /home/utibe/Desktop/GroundControl
 @ multi main

Note: /LifeInYourWay/ is a symlink pointing to a folder on a mounted drive

[Elijen]

I tried the fs.realpathSync trick, but it seems that symlinked modules are still not transpiled. For some reason they are included in the build as ES6 (build passes, but browser complains).

I used yarn link and yarn link module-name to link the module.

[soluml]

@eagsalazar's suggestion of using resolveLoader worked for my predicament. I didn't need to set resolve or use fs.realPathSync.

[Undistraction]

For anyone trying the resolveLoader approach with Webpack 2, the Webpack 2 equivalent of root seems to be modules, however this doesn't work. The only solution that works for me with Webpack 2 was to use fs.realpathSync.

Anyway, @eagsalazar's suggestion of using an absolute path for the value of root (or now modules might be problematic - see docs).

[DenysVuika]

The modules approach does not work with Webpack 2. Sometimes it works when there's only 1 symlink, but once there are many of them (based on my scenario - 15) it stops working completely. For now, have to copy files to node_modules instead of using symlinks. but this is far from what I've expected to have with Webpack 2 to be honest.

[Undistraction]

@DenisVuyka Yep. Trying to work with linked modules in Webpack 2 is proving to be nightmare. I've wasted a horrible amount of time on related issues.

[DenysVuika]

@Undistraction It is much better than Webpack 1 based on my experiments. Setting up 1 project with 1 symlink it usually a breeze. The problem is that it seems nobody tested the flow beyond 1 symlink.

[ghost]

Trying to remove the emotion here, but this really needs to be fixed: Ticket still unresolved for 68 weeks...

[elado]

Updating followers of this ticket:

I ditched the symlink approach; instead, I created another node module which compiles on its own with babel that is locally linked. That package has this in package.json:

"name": "awesome-module",
"main": "lib/index.js",
"scripts": {
  "build:clean": "rimraf dist lib",
  "build:ensure": "mkdir -p lib",
  "build:js": "npm run build:ensure && babel -d lib/ src/ -s inline",
  "build": "npm run build:clean && npm run build:ensure && npm run build:js",
  "watch:js": "npm run build:js -- -w",
  "watch": "npm run build && npm run watch:js",
  "postinstall": "npm run build",
  // ...
}

And src/index.js etc.

Then execute npm link in that dir, which registers the module globally on my machine, and npm link awesome-module in my initial module. That way, webpack isn't responsible for that module.

Also running npm run watch to have files updated on save.

[Undistraction]

@elado Thanks for the info, but using npm link or yarn link is just using symlinks under the hood isn't it? npm link creates a symlink in your user dir, then npm link x creates another symlink from the dir in node_modules to this symlink.

[elado]

@Undistraction correct, but the idea is to have webpack only include the other module without transpiling it with babel-loader. It's already compiled by babel on its own, and looks like any other ES5 module.

[Undistraction]

@elado Sorry. Was getting confused with one of other bugs related to symlinks I've been hitting - #985 - which is just a general issue with symlinked files, not specifically related to compilation. Can see how your solution negates this issue.

[mikecfisher]

Still can't quite get this to work with Webpack 2.2.1 on node 6.10.0 even using the fs.realpathsync approach.

var includePaths = [
	fs.realpathSync(__dirname + '/src'),
	fs.realpathSync(__dirname + '/node_modules/my-core-services'),
	fs.realpathSync(__dirname + '/node_modules/my-view-modules'),
];

module: {
   {test: /\.js$/, include: includePaths, use: ['babel-loader']},
}

I have external javascript modules in a separate package that need to be compiled.

When I console log the includePaths I see

'/Users/mike/Dev/my-core-services',     <==== npm linked
 '/Users/mike/Dev/myapp/node_modules/my-view-modules' <==== not npm linked

However when I run webpack all the javascript in the npm linked module throws compile errors as it doesn't recognize the syntax I'm using. Not sure what I'm missing here. Any ideas?

[zandroid]

Not the same problem but similar to this is resolved for me by --preserve-symlinks node option. https://nodejs.org/dist/latest-v6.x/docs/api/cli.html#cli_preserve_symlinks

I needed to load with babel-register symlinked ES6 script (with export & import) that is outside from project root. Without node option this script was not compiled by babel. But with option --preserve-symlinks it works fine.

[httpete]

I am on the latest webpack 2.11.1 and latest wds. These lines allow me to use my file:.. linked packages in package.json for local dev symlinked and have them update when I change it. The key is the path.resolve you want to make sure you point it to the TOP and only the top node_modules.

           modules: [
                path.resolve(path.join(__dirname, 'node_modules'))
            ],
            symlinks: true

[dilas12345]

hi,

i have been working on a web app using reactjs but when i run "npm start" it
always print "Failed to compile.
Error in Cannot find module 'commondir'
@ multi main"

is there anything i can do to solve this please.

i tried running npm install commondir but still did not work.
and also remove node_modules and install again nothing
seems to change.

thanks

[HyperCed]

I was able to fix that problem modifying webpack.dev.conf.js which is in build folder.
As you see, in this file, you'll find something like
new CopyWebpackPlugin([
{
from: path.resolve(__dirname, '../static'),
to: config.dev.assetsSubDirectory,
ignore: ['.*']
}

I added some files to copy which are symlinks (lib for example). So I have static/lib --> ../../../lib
And I added this code to CopyWebpackPlugin
new CopyWebpackPlugin([
{
from: path.resolve(__dirname, '../static'),
to: config.dev.assetsSubDirectory,
ignore: ['.']
},
{
from: path.resolve(__dirname, '../static/lib'),
to: config.dev.assetsSubDirectory+'/lib',
ignore: ['.']
}

[DjRAST]

Try setting resolve.symlinks: false (https://webpack.js.org/configuration/resolve/#resolve-symlinks), worked for me, YMMV.

Also worked for us! If you have a symlink to a folder outside of your project root, this seems to be the way. Because otherwise the symlinked folder is resolved in an absolute path which is outside the project root and is then being ignored

[lizhengnacl]

node v10.15.3 webpack 4.35.3

the following works for me.

include: [
  path.resolve(__dirname, 'src'),
],
exclude: [
  path.resolve(__dirname, 'node_modules')
],

[matzeeable]

I figured out that babel-loader can not resolve modules outside of the project. For example I am using lerna@3 and webpack@4.41.2. After debugging I ended up that @babel/parser is responsible for the "issue" (I do not think it's an issue, more a CWD problem) because the plugins and loaders could not be resolved to node_modules. There is also an interesting article about that in the babel configuration: https://babeljs.io/docs/en/config-files#monorepos

However, after moving all my babel configs (originally from package.json#babel) to my webpack.config.js it works without any problems:

// [...]
{
    test: /\.tsx$/,
    exclude: /(disposables)/,
    use: {
        loader: "babel-loader?cacheDirectory",
        options: require("./package.json")).babel
    }
}
// [...]

References:

• babel-loader fails to transpile imported packages.  lerna/lerna#1656

[TotooriaHyperion]

Try setting resolve.symlinks: false (https://webpack.js.org/configuration/resolve/#resolve-symlinks), worked for me, YMMV.

This work for me.

Try this solution when using lerna(monorepo with symlink) with webpack, you might find the loader trying to load what should be excluded.
It's because webpack resolve it to nearest relative path, so the path don't have a node_modules in it, thus not excluded.

Keywords for search engine: typescript ts-loader symlink lerna webpack
Keywords: export was not found in
Hope people in trouble can find this solution.

[vankop]

Should work with webpack@latest. Feel free to report new issue with reproducible repo.

[AmirTugi]

For me (webpack 4.46.0) I needed to add both resolve.symlinks: false and add an include path to the rule.

{
  module: {
    rules: [
      {
        oneOf: [
          {
            //The rule I want to fix
            // This is to also include monorepo packages
            include: [/node_modules\/@organization/]
          }
        ]
      }
    ],
  },
  resolve: {
    symlinks: false
  }
}

[baltsevicha]

For me (webpack 4.46.0) I needed to add both resolve.symlinks: false and add an include path to the rule.

{
  module: {
    rules: [
      {
        oneOf: [
          {
            //The rule I want to fix
            // This is to also include monorepo packages
            include: [/node_modules\/@organization/]
          }
        ]
      }
    ],
  },
  resolve: {
    symlinks: false
  }
}

It works for me, thank you!

[Ark-kun]

Should work with webpack@latest.

It does not work for me with v5.71.0 on Windows.

npm linked absolute_root_path\app\node_modules\some-package to absolute_root_path\some-package

Using absolute_root_path\app\node_modules\some-package\src in includes does not work.
Using absolute_root_path\some-package\src in includes works.

The hack from https://medium.com/capriza-engineering/sharing-source-code-and-libraries-in-react-bd30926df312 step 5.1/5 is still required.

@vankop Are you sure webpack checks the paths against the include patterns both before and after resolving the symlinks?

[vankop]

there is nothing todo with module.rules here.. only resolve option make sense. make sure resolve.symlinks=true https://webpack.js.org/configuration/resolve/#resolvesymlinks .

Otherwise please add infrastructureLogging.debug=true https://webpack.js.org/configuration/other-options/#debug and post stdout here ( there should be log from resolver )

[talesAtTilt]

resolve: {
    symlinks: false
}

That was enough for me. Thank you folks.