Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Running lerna bootstrap does not install all dependencies #1457

Closed
scottmcpherson opened this issue Jun 7, 2018 · 66 comments
Closed

Running lerna bootstrap does not install all dependencies #1457

scottmcpherson opened this issue Jun 7, 2018 · 66 comments
Labels
scope: package management Issues with the bootstrap/add/link commands that relate to package management

Comments

@scottmcpherson
Copy link

scottmcpherson commented Jun 7, 2018

Expected Behavior

When running lerna bootstrap all dependencies should be install in packages.

Current Behavior

If I run lerna bootstrap and then run lerna start to start all my clients and packages, I get the following error:

@raisalholdings/raisal-ui:    2263 modules
@raisalholdings/raisal-ui: ERROR in ../raisal-utils/src/reduxUtils.js
@raisalholdings/raisal-ui: Module not found: Error: Can't resolve 'redux-actions' in '/Users/scott/Develop/Raisal/2.0/raisal-clients/packages/raisal-utils/src'
@raisalholdings/raisal-ui: ERROR in ../raisal-utils/src/reduxUtils.js
@raisalholdings/raisal-ui: Module not found: Error: Can't resolve 'redux-optimistic-ui' in '/Users/scott/Develop/Raisal/2.0/raisal-clients/packages/raisal-utils/src'

If I cd into packages/raisal-ui, and run yarn, I get the following error:

error An unexpected error occurred: "ENOENT: no such file or directory, lstat '/Users/scott/Develop/Raisal/2.0/raisal-clients/packages/raisal-ui/node_modules/@raisalholdings/raisal-utils/node_modules/history/node_modules'".
...

But if I run yarn one more time, everything installs in raisal-ui and I can then run lerna run start and everything starts fine.

Possible Solution

Install all deps or provide logs to debug

I have the following setup:

root/
  clients/
  packages/
  lerna.json
  package.json

lerna.json

{
  "lerna": "2.5.1",
  "packages": [
    "clients/*",
    "packages/*"
  ],
  "npmClient": "yarn",
  "version": "2.1.13"
}

Your Environment

Executable Version
lerna --version 2.5.1
npm --version 6.0.0
yarn --version 1.6.0
node --version v8.9.3
OS Version
Sierra 10.13.1
@xpepermint
Copy link

This is a serious issue which happens on Node 8, 9 and 10. It's not fixed in lerna@3.0.0-beta.21. Is there a quick workaround?

@scottmcpherson
Copy link
Author

scottmcpherson commented Jun 23, 2018

@xpepermint What ended up working for us was creating a custom js bootstrap script and executing that using yarn bootstrap.

The script is scripts/bootstrap.js:

const spawnSync = require('child_process').spawnSync
const fs = require('fs-extra')
const path = require('path')
const chalk = require('chalk')

const RAISAL_UI_DIR = path.join(process.cwd(), 'packages', 'raisal-ui')

if (!fs.existsSync(RAISAL_UI_DIR)) {
  console.log(chalk.red(`Trying to find: ${RAISAL_UI_DIR}`))
  return console.log(
    chalk.red('Yarn bootstrap must be run from root of project')
  )
}
spawnSync('lerna', ['bootstrap'], {
  stdio: 'inherit'
})
spawnSync(/^win/.test(process.platform) ? 'yarn.cmd' : 'yarn', ['install'], {
  cwd: RAISAL_UI_DIR,
  stdio: 'inherit'
})
spawnSync(/^win/.test(process.platform) ? 'yarn.cmd' : 'yarn', ['install'], {
  cwd: RAISAL_UI_DIR,
  stdio: 'inherit'
})

Replace const RAISAL_UI_DIR = path.join(process.cwd(), 'packages', 'raisal-ui') with the dir that needs the packages reinstalled.

And then in my root package.json I added an npm script:

"scripts": {
    "start": "lerna run start --stream --parallel",
    "bootstrap": "node scripts/bootstrap.js",
    ...

@xpepermint
Copy link

Thanks @scottmcpherson! I also came to this simple workaround:

// package.json
{
  "scripts": {
    "bootstrap": "lerna exec npm install",
    "publish": "lerna publish --force-publish",
    "test": "lerna run test"
  },
  "devDependencies": {
    "lerna": "3.0.0-beta.21"
  }
}

So the lerna exec npm install command does the trick.

@lucasowens
Copy link

I am also having this issue, but I'm having trouble determining exactly which package is the issue.

@scottmcpherson
Copy link
Author

@lucasowens I would suggest posting your terminal output along with your comment. Maybe there's something there that could help?

@lucasowens
Copy link

Thanks for the reply @scottmcpherson!

Here is the output when I run lerna bootstrap

λ lerna bootstrap
lerna info version 2.11.0
lerna info versioning independent
lerna info Bootstrapping 19 packages
lerna ERR! execute callback with error
lerna ERR! Error: Command failed: yarn install --mutex network:42424 --non-interactive
lerna ERR! warning "eslint-config-kentcdodds > eslint-import-resolver-babel-module@3.0.0" has unmet peer dependency "babel-core@^6.0.0".
lerna ERR! warning "workspace-aggregator-ee9a09ea-1ce3-403a-a7a8-d1b456360123 > @ournamespace/es-client > request-promise-native@1.0.5" has unmet peer dependency "request@^2.34".
lerna ERR! error An unexpected error occurred: "ENOENT: no such file or directory, lstat 'C:\\Users\\me\\Source\\my-project\\packages\\weartime-domain\\node_modules\\@ournamespace'".
lerna ERR! yarn install v1.7.0
lerna ERR! [1/4] Resolving packages...
lerna ERR! [2/4] Fetching packages...
lerna ERR! info fsevents@1.2.4: The platform "win32" is incompatible with this module.
lerna ERR! info "fsevents@1.2.4" is an optional dependency and failed compatibility check. Excluding it from installation.
lerna ERR! [3/4] Linking dependencies...
lerna ERR! info If you think this is a bug, please open a bug report with the information provided in "C:\\Users\\me\\Source\\my-project\\yarn-error.log".
lerna ERR! info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.
lerna ERR!
lerna ERR!     at Promise.all.then.arr (C:\Users\me\AppData\Local\Yarn\Data\global\node_modules\execa\index.js:236:11)
lerna ERR!     at <anonymous>
{ Error: Command failed: yarn install --mutex network:42424 --non-interactive
warning "eslint-config-kentcdodds > eslint-import-resolver-babel-module@3.0.0" has unmet peer dependency "babel-core@^6.0.0".
warning "workspace-aggregator-ee9a09ea-1ce3-403a-a7a8-d1b456360123 > @ournamespace/es-client > request-promise-native@1.0.5" has unmet peer dependency "request@^2.34".
error An unexpected error occurred: "ENOENT: no such file or directory, lstat 'C:\\Users\\me\\Source\\my-project\\packages\\weartime-domain\\node_modules\\@ournamespace'".
yarn install v1.7.0
[1/4] Resolving packages...
[2/4] Fetching packages...
info fsevents@1.2.4: The platform "win32" is incompatible with this module.
info "fsevents@1.2.4" is an optional dependency and failed compatibility check. Excluding it from installation.
[3/4] Linking dependencies...
info If you think this is a bug, please open a bug report with the information provided in "C:\\Users\\me\\Source\\my-project\\yarn-error.log".
info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.

    at Promise.all.then.arr (C:\Users\me\AppData\Local\Yarn\Data\global\node_modules\execa\index.js:236:11)
    at <anonymous>
    at process._tickCallback (internal/process/next_tick.js:188:7)
  code: 1,
  killed: false,
  stdout: 'yarn install v1.7.0\n[1/4] Resolving packages...\n[2/4] Fetching packages...\ninfo fsevents@1.2.4: The platform "win32" is incompatible with this module.\ninfo "fsevents@1.2.4" is an optional dependency and failed compatibility check. Excluding it from installation.\n[3/4] Linking dependencies...\ninfo If you think this is a bug, please open a bug report with the information provided in "C:\\\\Users\\\\me\\\\Source\\\\Repos\\\\GitLab\\\\my-project\\\\yarn-error.log".\ninfo Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.\n',
  stderr: 'warning "eslint-config-kentcdodds > eslint-import-resolver-babel-module@3.0.0" has unmet peer dependency "babel-core@^6.0.0".\nwarning "workspace-aggregator-ee9a09ea-1ce3-403a-a7a8-d1b456360123 > @ournamespace/es-client > request-promise-native@1.0.5" has unmet peer dependency "request@^2.34".\nerror An unexpected error occurred: "ENOENT: no such file or directory, lstat \'C:\\\\Users\\\\me\\\\Source\\\\Repos\\\\GitLab\\\\my-project\\\\packages\\\\weartime-domain\\\\node_modules\\\\@ournamespace\'".\n',
  failed: true,
  signal: null,
  cmd: 'yarn install --mutex network:42424 --non-interactive',
  timedOut: false,
  exitCode: 1 }

The second time I run it, it works as expected.

When I run yarn directly I get:

λ yarn
yarn install v1.7.0
[1/4] Resolving packages...
[2/4] Fetching packages...
info fsevents@1.2.4: The platform "win32" is incompatible with this module.
info "fsevents@1.2.4" is an optional dependency and failed compatibility check. Excluding it from installation.
[3/4] Linking dependencies...
warning "eslint-config-kentcdodds > eslint-import-resolver-babel-module@3.0.0" has unmet peer dependency "babel-core@^6.0.0".
warning "workspace-aggregator-602dc8d5-75c6-4990-b45c-8e79f4bb2e3d > @ournamespace/es-client > request-promise-native@1.0.5" has unmet peer dependency "request@^2.34".
error An unexpected error occurred: "ENOENT: no such file or directory, lstat 'C:\\Users\\me\\Source\\my-project\\packages\\weartime-domain\\node_modules\\@ournamespace'".
info If you think this is a bug, please open a bug report with the information provided in "C:\\Users\\me\\Source\\my-project\\yarn-error.log".
info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.

Again it works successfully the second time I run it.

It seems that this is somehow related to winston but I cant be sure, I'm getting inconsistent results.

Any help appreciated!

@evocateur
Copy link
Member

I literally can't debug without an actual reproduction. Telling me what your tree looks like is worthless without the actual contents of the package.json files.

@StrahilKazlachev
Copy link

StrahilKazlachev commented Jul 3, 2018

My issue seems the same, quite hard to share.
B depends on A.

  • packages
    • A - all is fine
    • B - not fine, A dependencies are not installed

Doing lerna exec -- npm i && lerna link does what I want.
From bootstrap description

  1. npm install all external dependencies of each package.

@evocateur does this mean that the external dependencies of the linked packages will not be installed?
Edit: I'm not hoisting.

@evocateur
Copy link
Member

evocateur commented Jul 3, 2018 via email

@StrahilKazlachev
Copy link

@evocateur Do I understand correctly that when using bootstrap a local sibling external dependencies will stay "under" it(local-pkg-B/node_modules/local-pkg-A/node_modules/ext-dep-1) - loosing the flat structure in local-pkg-B/node_modules I would normally get with npm install?

@evocateur
Copy link
Member

evocateur commented Jul 3, 2018 via email

@lucasowens
Copy link

I just wanted to mention that this error disappeared for me and I'm not sure why.

@scottmcpherson
Copy link
Author

scottmcpherson commented Jul 15, 2018

@evocateur The problem that I ran into while using file specifiers when designing our front-end was that we have multiple core packages that are reused across four clients. They are all in the same mono repo. And the core packages are all started and built with webpack. When we use the file specifiers, we lost live reload, because symlinks were no longer pointing to the original packages. That made file specifiers unusable for our use case.

Have you ran into this issue?

@evocateur
Copy link
Member

evocateur commented Jul 15, 2018 via email

@DesignByOnyx
Copy link

DesignByOnyx commented Sep 10, 2018

This problem was fixed for our project by changing our "packages" path to include two asterisks instead of one. I just happened to get lucky on this one:

{
  "packages": [
    "modules/**"
  ],
  "version": "1.0.0"
}

UPDATE: spoke a little too soon I guess, I get different results with successive runs of lerna bootstrap. The first run will install all deps for packages A and B, but not C and D. The second run will install all packages for A and D, but not B and C. It changes on every run. Running lerna link && lerna exec npm install seems to do the trick for now. Still investigating.

@silvenon
Copy link

silvenon commented Oct 7, 2018

Repro

I created a Docker container with instructions how to reproduce the issue: silvenon/lerna-bootstrap-issue. You can check out the failing (lerna bootstrap) vs successful (lerna exec yarn && lerna link) script.

@TheAifam5
Copy link

TheAifam5 commented Nov 16, 2018

Hey ho,
I have a similar issue but with webpack and yarn workspaces.
I'm using:

  • Node: 11.2.0
  • Webpack: 4.25.1
  • Yarn: 1.12.3
  • lerna: 3.4.3

Workspace config:

  "workspaces": [
    "shared",
    "microservices/*"
  ]

Directory layout:

shared/package.json
microservices/<name>/package.json

Webpack config:

import * as webpack from 'webpack';
import * as path from 'path';
import * as nodeExternals from 'webpack-node-externals';
import tsConfigPathsPlugin from 'tsconfig-paths-webpack-plugin';
import EntryRunnerWebpackPlugin from 'entry-runner-webpack-plugin';

const config: webpack.Configuration = {
  entry: ['webpack/hot/poll?1000', './main.hmr.ts'],
  target: 'node',
  context: path.resolve('./src'),
  output: {
    path: path.join(__dirname, 'dist'),
    filename: 'server.js',
  },
  externals: [
    nodeExternals({
      whitelist: ['webpack/hot/poll?1000'],
    }),
  ],
  module: {
    rules: [
      {
        test: /\.tsx?$/,
        use: 'ts-loader',
        exclude: /node_modules/,
      },
    ],
  },
  mode: 'development',
  resolve: {
    extensions: ['.tsx', '.ts', '.js'],
    plugins: [new tsConfigPathsPlugin()],
  },
  plugins: [
    new webpack.HotModuleReplacementPlugin(),
    new EntryRunnerWebpackPlugin(path.resolve(__dirname), 'server.js'),
  ],
};

export default config;

Log:

lerna ERR! yarn run build exited 1 in 'microservice-database'
lerna ERR! yarn run build stdout:
$ webpack
Hash: 11947ae751359d9ff957
Version: webpack 4.25.1
Time: 11689ms
Built at: 11/17/2018 12:05:48 AM
    Asset      Size  Chunks             Chunk Names
server.js  8.86 MiB    main  [emitted]  main
Entrypoint main = server.js
[0] multi webpack/hot/poll?1000 ./main.hmr.ts 40 bytes {main} [built]
[../../../node_modules/@nestjs/common/utils sync recursive] /home/theaifam5/Documents/Projects/TheAifam5/nestjs-docker-microservices/node_modules/@nestjs/common/utils sync 160 bytes {main} [optional] [built]
[../../../node_modules/webpack/hot/log-apply-result.js] (webpack)/hot/log-apply-result.js 1.27 KiB {main} [built]
[../../../node_modules/webpack/hot/log.js] (webpack)/hot/log.js 1.11 KiB {main} [built]
[../../../node_modules/webpack/hot/poll.js?1000] (webpack)/hot/poll.js?1000 1.15 KiB {main} [built]
[./app.module.ts] 933 bytes {main} [built]
[./database/database.module.ts] 1.02 KiB {main} [built]
[./main.hmr.ts] 1.22 KiB {main} [built]
[./main.options.ts] 262 bytes {main} [built]
[./users/users.controller.ts] 2.85 KiB {main} [built]
[./users/users.module.ts] 1.24 KiB {main} [built]
[./users/users.providers.ts] 242 bytes {main} [built]
[./users/users.service.ts] 2.44 KiB {main} [built]
[http] external "http" 42 bytes {main} [built]
[https] external "https" 42 bytes {main} [built]
    + 1477 hidden modules

WARNING in /home/theaifam5/Documents/Projects/TheAifam5/nestjs-docker-microservices/node_modules/@nestjs/common/utils/load-package.util.js 8:15-35
Critical dependency: the request of a dependency is an expression
 @ /home/theaifam5/Documents/Projects/TheAifam5/nestjs-docker-microservices/node_modules/@nestjs/core/nest-factory.js
 @ /home/theaifam5/Documents/Projects/TheAifam5/nestjs-docker-microservices/node_modules/@nestjs/core/index.js
 @ ./main.hmr.ts
 @ multi webpack/hot/poll?1000 ./main.hmr.ts

WARNING in /home/theaifam5/Documents/Projects/TheAifam5/nestjs-docker-microservices/node_modules/optional/optional.js 6:11-26
Critical dependency: the request of a dependency is an expression
 @ /home/theaifam5/Documents/Projects/TheAifam5/nestjs-docker-microservices/node_modules/@nestjs/core/nest-application.js
 @ /home/theaifam5/Documents/Projects/TheAifam5/nestjs-docker-microservices/node_modules/@nestjs/core/index.js
 @ ./main.hmr.ts
 @ multi webpack/hot/poll?1000 ./main.hmr.ts

WARNING in /home/theaifam5/Documents/Projects/TheAifam5/nestjs-docker-microservices/node_modules/sequelize-typescript/lib/services/models.js 175:29-46
Critical dependency: the request of a dependency is an expression
 @ /home/theaifam5/Documents/Projects/TheAifam5/nestjs-docker-microservices/node_modules/sequelize-typescript/lib/models/BaseModel.js
 @ /home/theaifam5/Documents/Projects/TheAifam5/nestjs-docker-microservices/node_modules/sequelize-typescript/lib/models/Model.js
 @ /home/theaifam5/Documents/Projects/TheAifam5/nestjs-docker-microservices/node_modules/sequelize-typescript/index.js
 @ ./users/user.entity.ts
 @ ./users/users.service.ts
 @ ./users/users.module.ts
 @ ./app.module.ts
 @ ./main.hmr.ts
 @ multi webpack/hot/poll?1000 ./main.hmr.ts

WARNING in /home/theaifam5/Documents/Projects/TheAifam5/nestjs-docker-microservices/node_modules/sequelize/lib/sequelize.js 393:61-74
Critical dependency: the request of a dependency is an expression
 @ /home/theaifam5/Documents/Projects/TheAifam5/nestjs-docker-microservices/node_modules/sequelize/index.js
 @ /home/theaifam5/Documents/Projects/TheAifam5/nestjs-docker-microservices/node_modules/sequelize-typescript/lib/enums/DataType.js
 @ /home/theaifam5/Documents/Projects/TheAifam5/nestjs-docker-microservices/node_modules/sequelize-typescript/index.js
 @ ./users/user.entity.ts
 @ ./users/users.service.ts
 @ ./users/users.module.ts
 @ ./app.module.ts
 @ ./main.hmr.ts
 @ multi webpack/hot/poll?1000 ./main.hmr.ts

WARNING in /home/theaifam5/Documents/Projects/TheAifam5/nestjs-docker-microservices/node_modules/sequelize/lib/dialects/mssql/connection-manager.js 21:19-62
Critical dependency: the request of a dependency is an expression
 @ /home/theaifam5/Documents/Projects/TheAifam5/nestjs-docker-microservices/node_modules/sequelize/lib/dialects/mssql/index.js
 @ /home/theaifam5/Documents/Projects/TheAifam5/nestjs-docker-microservices/node_modules/sequelize/lib/sequelize.js
 @ /home/theaifam5/Documents/Projects/TheAifam5/nestjs-docker-microservices/node_modules/sequelize/index.js
 @ /home/theaifam5/Documents/Projects/TheAifam5/nestjs-docker-microservices/node_modules/sequelize-typescript/lib/enums/DataType.js
 @ /home/theaifam5/Documents/Projects/TheAifam5/nestjs-docker-microservices/node_modules/sequelize-typescript/index.js
 @ ./users/user.entity.ts
 @ ./users/users.service.ts
 @ ./users/users.module.ts
 @ ./app.module.ts
 @ ./main.hmr.ts
 @ multi webpack/hot/poll?1000 ./main.hmr.ts

WARNING in /home/theaifam5/Documents/Projects/TheAifam5/nestjs-docker-microservices/node_modules/sequelize/lib/dialects/mysql/connection-manager.js 32:19-62
Critical dependency: the request of a dependency is an expression
 @ /home/theaifam5/Documents/Projects/TheAifam5/nestjs-docker-microservices/node_modules/sequelize/lib/dialects/mysql/index.js
 @ /home/theaifam5/Documents/Projects/TheAifam5/nestjs-docker-microservices/node_modules/sequelize/lib/sequelize.js
 @ /home/theaifam5/Documents/Projects/TheAifam5/nestjs-docker-microservices/node_modules/sequelize/index.js
 @ /home/theaifam5/Documents/Projects/TheAifam5/nestjs-docker-microservices/node_modules/sequelize-typescript/lib/enums/DataType.js
 @ /home/theaifam5/Documents/Projects/TheAifam5/nestjs-docker-microservices/node_modules/sequelize-typescript/index.js
 @ ./users/user.entity.ts
 @ ./users/users.service.ts
 @ ./users/users.module.ts
 @ ./app.module.ts
 @ ./main.hmr.ts
 @ multi webpack/hot/poll?1000 ./main.hmr.ts

WARNING in /home/theaifam5/Documents/Projects/TheAifam5/nestjs-docker-microservices/node_modules/sequelize/lib/dialects/postgres/connection-manager.js 22:16-59
Critical dependency: the request of a dependency is an expression
 @ /home/theaifam5/Documents/Projects/TheAifam5/nestjs-docker-microservices/node_modules/sequelize/lib/dialects/postgres/index.js
 @ /home/theaifam5/Documents/Projects/TheAifam5/nestjs-docker-microservices/node_modules/sequelize/lib/sequelize.js
 @ /home/theaifam5/Documents/Projects/TheAifam5/nestjs-docker-microservices/node_modules/sequelize/index.js
 @ /home/theaifam5/Documents/Projects/TheAifam5/nestjs-docker-microservices/node_modules/sequelize-typescript/lib/enums/DataType.js
 @ /home/theaifam5/Documents/Projects/TheAifam5/nestjs-docker-microservices/node_modules/sequelize-typescript/index.js
 @ ./users/user.entity.ts
 @ ./users/users.service.ts
 @ ./users/users.module.ts
 @ ./app.module.ts
 @ ./main.hmr.ts
 @ multi webpack/hot/poll?1000 ./main.hmr.ts

WARNING in /home/theaifam5/Documents/Projects/TheAifam5/nestjs-docker-microservices/node_modules/sequelize/lib/dialects/sqlite/connection-manager.js 25:19-62
Critical dependency: the request of a dependency is an expression
 @ /home/theaifam5/Documents/Projects/TheAifam5/nestjs-docker-microservices/node_modules/sequelize/lib/dialects/sqlite/index.js
 @ /home/theaifam5/Documents/Projects/TheAifam5/nestjs-docker-microservices/node_modules/sequelize/lib/sequelize.js
 @ /home/theaifam5/Documents/Projects/TheAifam5/nestjs-docker-microservices/node_modules/sequelize/index.js
 @ /home/theaifam5/Documents/Projects/TheAifam5/nestjs-docker-microservices/node_modules/sequelize-typescript/lib/enums/DataType.js
 @ /home/theaifam5/Documents/Projects/TheAifam5/nestjs-docker-microservices/node_modules/sequelize-typescript/index.js
 @ ./users/user.entity.ts
 @ ./users/users.service.ts
 @ ./users/users.module.ts
 @ ./app.module.ts
 @ ./main.hmr.ts
 @ multi webpack/hot/poll?1000 ./main.hmr.ts

WARNING in /home/theaifam5/Documents/Projects/TheAifam5/nestjs-docker-microservices/node_modules/express/lib/view.js 81:13-25
Critical dependency: the request of a dependency is an expression
 @ /home/theaifam5/Documents/Projects/TheAifam5/nestjs-docker-microservices/node_modules/express/lib/application.js
 @ /home/theaifam5/Documents/Projects/TheAifam5/nestjs-docker-microservices/node_modules/express/lib/express.js
 @ /home/theaifam5/Documents/Projects/TheAifam5/nestjs-docker-microservices/node_modules/express/index.js
 @ /home/theaifam5/Documents/Projects/TheAifam5/nestjs-docker-microservices/node_modules/@nestjs/core/adapters/express-adapter.js
 @ /home/theaifam5/Documents/Projects/TheAifam5/nestjs-docker-microservices/node_modules/@nestjs/core/nest-factory.js
 @ /home/theaifam5/Documents/Projects/TheAifam5/nestjs-docker-microservices/node_modules/@nestjs/core/index.js
 @ ./main.hmr.ts
 @ multi webpack/hot/poll?1000 ./main.hmr.ts

WARNING in /home/theaifam5/Documents/Projects/TheAifam5/nestjs-docker-microservices/node_modules/mysql2/lib/helpers.js
Module not found: Error: Can't resolve 'cardinal' in '/home/theaifam5/Documents/Projects/TheAifam5/nestjs-docker-microservices/node_modules/mysql2/lib'
 @ /home/theaifam5/Documents/Projects/TheAifam5/nestjs-docker-microservices/node_modules/mysql2/lib/helpers.js
 @ /home/theaifam5/Documents/Projects/TheAifam5/nestjs-docker-microservices/node_modules/mysql2/lib/parsers/binary_parser.js
 @ /home/theaifam5/Documents/Projects/TheAifam5/nestjs-docker-microservices/node_modules/mysql2/lib/commands/execute.js
 @ /home/theaifam5/Documents/Projects/TheAifam5/nestjs-docker-microservices/node_modules/mysql2/lib/commands sync ^\.\/.*\.js$
 @ /home/theaifam5/Documents/Projects/TheAifam5/nestjs-docker-microservices/node_modules/mysql2/lib/commands/index.js
 @ /home/theaifam5/Documents/Projects/TheAifam5/nestjs-docker-microservices/node_modules/mysql2/lib/connection.js
 @ /home/theaifam5/Documents/Projects/TheAifam5/nestjs-docker-microservices/node_modules/mysql2/index.js
 @ /home/theaifam5/Documents/Projects/TheAifam5/nestjs-docker-microservices/node_modules/sequelize/lib/dialects/mysql/connection-manager.js
 @ /home/theaifam5/Documents/Projects/TheAifam5/nestjs-docker-microservices/node_modules/sequelize/lib/dialects/mysql/index.js
 @ /home/theaifam5/Documents/Projects/TheAifam5/nestjs-docker-microservices/node_modules/sequelize/lib/sequelize.js
 @ /home/theaifam5/Documents/Projects/TheAifam5/nestjs-docker-microservices/node_modules/sequelize/index.js
 @ /home/theaifam5/Documents/Projects/TheAifam5/nestjs-docker-microservices/node_modules/sequelize-typescript/lib/enums/DataType.js
 @ /home/theaifam5/Documents/Projects/TheAifam5/nestjs-docker-microservices/node_modules/sequelize-typescript/index.js
 @ ./users/user.entity.ts
 @ ./users/users.service.ts
 @ ./users/users.module.ts
 @ ./app.module.ts
 @ ./main.hmr.ts
 @ multi webpack/hot/poll?1000 ./main.hmr.ts

WARNING in /home/theaifam5/Documents/Projects/TheAifam5/nestjs-docker-microservices/node_modules/sequelize/lib/dialects/postgres/connection-manager.js
Module not found: Error: Can't resolve 'pg' in '/home/theaifam5/Documents/Projects/TheAifam5/nestjs-docker-microservices/node_modules/sequelize/lib/dialects/postgres'
 @ /home/theaifam5/Documents/Projects/TheAifam5/nestjs-docker-microservices/node_modules/sequelize/lib/dialects/postgres/connection-manager.js
 @ /home/theaifam5/Documents/Projects/TheAifam5/nestjs-docker-microservices/node_modules/sequelize/lib/dialects/postgres/index.js
 @ /home/theaifam5/Documents/Projects/TheAifam5/nestjs-docker-microservices/node_modules/sequelize/lib/sequelize.js
 @ /home/theaifam5/Documents/Projects/TheAifam5/nestjs-docker-microservices/node_modules/sequelize/index.js
 @ /home/theaifam5/Documents/Projects/TheAifam5/nestjs-docker-microservices/node_modules/sequelize-typescript/lib/enums/DataType.js
 @ /home/theaifam5/Documents/Projects/TheAifam5/nestjs-docker-microservices/node_modules/sequelize-typescript/index.js
 @ ./users/user.entity.ts
 @ ./users/users.service.ts
 @ ./users/users.module.ts
 @ ./app.module.ts
 @ ./main.hmr.ts
 @ multi webpack/hot/poll?1000 ./main.hmr.ts

@elie222
Copy link

elie222 commented Nov 18, 2018

Same issues for us. Banging my head against a wall. Works fine on my Mac. Causing me issues on remote Ubuntu server or in a docker container.

@silvenon
Copy link

@TheAifam5 @elie222 what is your Lerna configuration (lerna.json)?

@TheAifam5
Copy link

@silvenon

{
  "packages": ["shared/*", "apps/*", "services/*"],
  "npmClient": "yarn",
  "useWorkspaces": true,
  "version": "independent",
  "stream": true,
  "parallel": true
}

I think I should open a new issue, for webpack-only related stuff.

@silvenon
Copy link

Maybe, but it seems like the shared part of your packages field doesn't match workspaces configuration. I think it should be shared rather than shared/*.

@TheAifam5
Copy link

TheAifam5 commented Nov 18, 2018

@silvenon shared contains also subfolders with packages which I use everywhere. That's not a problem here. The problem on my side, is about that webpack does not work properly.

but what I seen, webpack transpiles to the "server.js" and is usable, but the error messages about missing dependencies is annoying.

services/webpack.common.ts

import * as webpack from 'webpack';

import * as webpackMerge from 'webpack-merge';
import * as nodeExternals from 'webpack-node-externals';
import { TsconfigPathsPlugin } from 'tsconfig-paths-webpack-plugin';

type WebpackMode = 'development' | 'production';

export default (
  configs: webpack.Configuration[],
  _: any,
  argv: { [name: string]: any },
): webpack.Configuration => {
  let mode: WebpackMode = 'development';
  let isHot = false;

  if (process.env.NODE_ENV) {
    mode = process.env.NODE_ENV as WebpackMode;
  } else if ('p' in argv) {
    mode = 'production';
  } else if ('mode' in argv) {
    mode = argv.mode;
  }

  if ('hmr' in argv) {
    isHot = argv.hmr;
  }

  if (mode === 'production' && isHot) {
    throw new Error('Unable to use HMR in production mode!');
  }

  return webpackMerge.smartStrategy({
    'module.rules.use': 'prepend',
  })(
    {
      mode,
      entry: ['webpack/hot/poll?100', `./src/main${isHot ? '.hmr' : ''}.ts`],
      target: 'node',
      output: {
        filename: 'server.js',
        publicPath: '/',
        libraryTarget: 'commonjs',
      },
      module: {
        rules: [
          {
            test: /\.(js|ts)x?$/,
            loader: 'babel-loader',
            options: {
              presets: [
                ['@babel/env', { modules: false }],
                '@babel/typescript',
              ],
              plugins: [
                [
                  '@babel/plugin-proposal-decorators',
                  {
                    decoratorsBeforeExport: true,
                  },
                ],
                '@babel/plugin-proposal-class-properties',
              ],
            },
            include: __dirname,
            exclude: /node_modules/,
          },
        ],
      },
      plugins: [
        new webpack.WatchIgnorePlugin([/\.js$/, /\.d\.ts$/]),
        new webpack.DefinePlugin({
          'process.env.NODE_ENV': JSON.stringify(mode),
        }),
        /*new webpack.NormalModuleReplacementPlugin(
          /\.\.\/migrate/,
          '../util/noop.js',
        ),
        new webpack.NormalModuleReplacementPlugin(
          /\.\.\/seed/,
          '../util/noop.js',
        ),
        new webpack.IgnorePlugin(/mariasql/),
        new webpack.IgnorePlugin(/mssql/),
        new webpack.IgnorePlugin(/mongodb/),
        new webpack.IgnorePlugin(/sql\.js/),
        new webpack.IgnorePlugin(/redis/),
        new webpack.IgnorePlugin(/react-native-sqlite-storage/),
        new webpack.IgnorePlugin(/mysql/),
        new webpack.IgnorePlugin(/mysql2/),
        new webpack.IgnorePlugin(/oracle/),
        new webpack.IgnorePlugin(/oracledb/),
        new webpack.IgnorePlugin(/pg/),
        new webpack.IgnorePlugin(/pg-query-stream/),
        new webpack.IgnorePlugin(/sqlite3/),
        new webpack.IgnorePlugin(/strong-oracle/),
        new webpack.IgnorePlugin(/pg-native/),*/
      ],
      optimization: {
        noEmitOnErrors: true,
        usedExports: !isHot,
        minimize: !isHot,
        removeEmptyChunks: !isHot,
      },
      externals: [nodeExternals()],
      resolve: {
        extensions: ['.ts', '.tsx', '.js', '.json'],
        plugins: [new TsconfigPathsPlugin()],
        modules: ['node_modules'],
        symlinks: false,
      },
      stats: 'verbose',
      /*stats: {
        warningsFilter: /export .* was not found in/,
      },*/
    },
    ...configs,
  );
};

and every subproject in services has:

import * as path from 'path';
import webpack from 'webpack';
import webpackCommon from '../webpack.common';

export default webpackCommon.bind(undefined, [
  {
    context: path.resolve(__dirname),
    output: {
      path: path.join(__dirname, 'dist'),
    },
    resolve: {
      modules: [path.join(__dirname, 'node_modules')],
    },
  },
] as webpack.Configuration[]);

@TheAifam5
Copy link

TheAifam5 commented Nov 18, 2018

Nevermind, I found the problem, for my case:

services/webpack.common.ts

externals: [
        nodeExternals(),
        nodeExternals({
          modulesFromFile: true,
        }),
        nodeExternals({
          modulesDir: path.resolve(__dirname, '../node_modules'),
        }),
      ],

@stale
Copy link

stale bot commented Jan 17, 2019

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

@stale stale bot added the stale label Jan 17, 2019
@stale stale bot closed this as completed Jan 24, 2019
@veeramarni
Copy link

@marschhuynh I set to below, but still devDependencies from packages are not getting installed.

    "lerna": "cross-env NODE_ENV=development lerna bootstrap",

@TomaszG
Copy link

TomaszG commented Apr 16, 2021

I had the same issue and the root cause was that some packages were not listed in the root package.json file (probably because they were not added using npx lerna add but manually). All packages in a monorepo should be listed in root package.json file:

// root package.json
{
  (...)
  "dependencies": {
    "@repo/package-name": "file:packages/package-name",
    (...)
  }
}

@GiancarlosIO
Copy link

GiancarlosIO commented Apr 28, 2021

so.. is there any solution for this problem? 😢

@hamzahalilovic
Copy link

yarn policies set-version 1.18.0 Running this line before adding dependencies solved the problem for me

@mig8447
Copy link

mig8447 commented Jun 27, 2021

I ended up adding a postinstall bootstrap script to the root's package.json as follows:

"scripts": {
    ...
    "bootstrap": "lerna bootstrap && lerna exec npm install",
    ...
}

This way, I get to do my normal npm install and once it finishes, I will execute npm run bootstrap, which in turn will execute the lerna bootstrapping along with installing all my dependencies in all of my packages.

NOTE: Making this to be in the postinstall lifecycle script was not a good idea because when someone installs the dependency it will execute the postinstall script, and will cause errors

@gondar00
Copy link

gondar00 commented Dec 27, 2021

@evocateur I face this issue when lerna is part of my dependancies instead of devDependancies in package.json, also faced it if I have "dependancies" key mentioned mentioned in the root package.json (that is beside lerna.json), If i do not have any dependancies mentioned in the root package json things work fine on running lerna bootrap --hoist

@fesaza
Copy link

fesaza commented Mar 4, 2022

This fixed the issue for me. I have a devDependecy that Lerna was ignoring because I had the NODE_ENV in production

In case you want devDependencies to be installed, you have to make sure that NODE_ENV is not "production"

@whatwg6
Copy link

whatwg6 commented Mar 14, 2022

best solution?
prisma/language-tools#272

@RioChndr
Copy link

In my way, update root package.json like this

"scripts": {
    "install": "lerna exec -- yarn install",
    "build": "lerna exec -- yarn build"
  },

So, I just run yarn install or yarn build

I don't know is this the right way to do this.

@igortas
Copy link

igortas commented Apr 4, 2022

Have the same issue in ci/cd. It can't find dependencies in on folder that have dependencies from other folder in packages/* :/
In the dependecies key, I have set the package as "my-package": "*"

@josteph
Copy link

josteph commented Apr 18, 2022

This is actually the exact issue: #1457 (comment)

I globally set NODE_ENV as production in the CI runner and it literally messed up everything

@JamesHenry JamesHenry added the scope: package management Issues with the bootstrap/add/link commands that relate to package management label Jun 14, 2022
@erdemgonul
Copy link

Problem still exist on Lerna 5.1.8.

I tried the workaround butlerna exec -- npm installignores .npmrc file which is at root level

We can't believe in the continuation of such a bug

@ericsvendsen
Copy link

For me, lerna bootstrap --hoist worked fine on my dev machine, but in my client's facility it would only install top-level dependencies. Upgrading that system from Node 14.x to 16.x solved the problem.

@erdemgonul
Copy link

@ericsvendsen solution worked, thank you!

For me, lerna bootstrap --hoist worked fine on my dev machine, but in my client's facility it would only install top-level dependencies. Upgrading that system from Node 14.x to 16.x solved the problem.

@itayganor
Copy link

For us, it was specifically Heroku that failed our builds. That's because they set NODE_ENV to production, which prevents the installation of devDependencies.

We ended up changing our postinstall script from:

"postinstall": "npx lerna bootstrap",

to:

"postinstall": "npx lerna bootstrap -- --include=dev",

@toddpla
Copy link

toddpla commented Oct 7, 2022

In my particular instance after migrating to node version v16.15.0 with npm v8.10.0 this started working as expected.

@muratozgul
Copy link

muratozgul commented Oct 28, 2022

After a lot of trial & error, this worked for me:

lerna.json

{
  "$schema": "node_modules/lerna/schemas/lerna-schema.json",
  "useNx": true,
  "version": "independent",
  "packages": ["packages/*"]
}

top level package.json

{
  "scripts": {
    "postinstall": "lerna bootstrap --no-ci",
  }
}

Run:
npx lerna clean -y
npm install

(lerna v5.5.4, node v16.17.0, npm 8.15.0)

@JamesHenry
Copy link
Member

Hi Folks 👋

Please take a look at our published roadmap for Lerna v7 here: #3410

One of the key items covered at length on there (please do read it for full context) is that now that we find ourselves in late 2022, it no longer makes sense for lerna to supplement package management concerns (such as installation, boostrapping, linking etc) which are covered reliably for monorepo workspaces by the three main package managers: npm, yarn and pnpm. lerna bootstrap et al were developed in completely different era of the JavaScript ecosystem.

If you have any specific concerns please do join in on that discussion, and provide as much context as possible.

Many thanks 🙏

@JamesHenry JamesHenry closed this as not planned Won't fix, can't repro, duplicate, stale Nov 29, 2022
@lerna lerna locked and limited conversation to collaborators Nov 29, 2022
@lerna lerna locked and limited conversation to collaborators Nov 29, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
scope: package management Issues with the bootstrap/add/link commands that relate to package management
Projects
None yet
Development

No branches or pull requests