Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

skip applications install when generating workspaces. #18869

Merged
merged 2 commits into from
Jun 10, 2022

Conversation

mshima
Copy link
Member

@mshima mshima commented Jun 9, 2022

Current jdl generation makes npm workspaces unsuitable for auto install.
So disable it by default.

npm 8. changed the workspaces behavior by deduping dependencies to the root.
We generate applications in parallel, so install will be executed 3 times in same npm tree causing lots of problems.
We need to execute npm install only once.


Please make sure the below checklist is followed for Pull Requests.

When you are still working on the PR, consider converting it to Draft (below reviewers) and adding skip-ci label, you can still see CI build result at your branch.

cli/import-jdl.js Outdated Show resolved Hide resolved
@mraible
Copy link
Contributor

mraible commented Jun 9, 2022

I don't know if I'm testing this branch correctly. Here's what I tried:

jhipster jdl reactive-mf --workspaces --skip-commit-hook

I saw the 'npm install' was skipped due to workspaces message, but it's a little buried so I didn't run it at first. It seems necessary to run it so maybe we need to colorize this so it sticks out more?

I tried npm run ci:e2e:prepare in the top-level directory and since Docker images are missing, it gave me an error. This seems to be expected.

> ci:e2e:prepare
> docker-compose -f docker-compose/docker-compose.yml up -d

[+] Running 0/3
 ⠿ store Error                                                                                                     1.6s
 ⠿ gateway Error                                                                                                   1.6s
 ⠿ blog Error                                                                                                      1.6s
Error response from daemon: pull access denied for store, repository does not exist or may require 'docker login': denied: requested access to the resource is denied

Then, I tried building Docker images in each app. They all fail.

gateway
npm WARN cleanup Failed to remove some directories [
npm WARN cleanup   [
npm WARN cleanup     undefined,
npm WARN cleanup     AssertionError [ERR_ASSERTION]: rimraf: missing path
npm WARN cleanup         at rimraf (/opt/homebrew/lib/node_modules/npm/node_modules/rimraf/rimraf.js:54:3)
npm WARN cleanup         at node:internal/util:360:7
npm WARN cleanup         at new Promise (<anonymous>)
npm WARN cleanup         at rimraf (node:internal/util:346:12)
npm WARN cleanup         at /opt/homebrew/lib/node_modules/npm/node_modules/@npmcli/arborist/lib/arborist/reify.js:532:20
npm WARN cleanup         at Array.map (<anonymous>)
npm WARN cleanup         at Arborist.[rollbackCreateSparseTree] (/opt/homebrew/lib/node_modules/npm/node_modules/@npmcli/arborist/lib/arborist/reify.js:532:8)
npm WARN cleanup         at /opt/homebrew/lib/node_modules/npm/node_modules/@npmcli/arborist/lib/arborist/reify.js:1078:50
npm WARN cleanup         at processTicksAndRejections (node:internal/process/task_queues:96:5)
npm WARN cleanup         at async Arborist.[reifyPackages] (/opt/homebrew/lib/node_modules/npm/node_modules/@npmcli/arborist/lib/arborist/reify.js:235:11) {
npm WARN cleanup       generatedMessage: false,
npm WARN cleanup       code: 'ERR_ASSERTION',
npm WARN cleanup       actual: undefined,
npm WARN cleanup       expected: true,
npm WARN cleanup       operator: '=='
npm WARN cleanup     }
npm WARN cleanup   ]
npm WARN cleanup ]
npm ERR! code 1
npm ERR! path /Users/mraible/Downloads/mfr/node_modules/puppeteer
npm ERR! command failed
npm ERR! command sh -c node install.js
npm ERR! ERROR: Failed to set up Chromium r869685! Set "PUPPETEER_SKIP_DOWNLOAD" env variable to skip download.
npm ERR! [Error: EEXIST: file already exists, symlink 'Versions/Current/Resources' -> '/Users/mraible/Downloads/mfr/node_modules/puppeteer/.local-chromium/mac-869685/chrome-mac/Chromium.app/Contents/Frameworks/Chromium Framework.framework/Resources'] {
npm ERR!   errno: -17,
npm ERR!   code: 'EEXIST',
npm ERR!   syscall: 'symlink',
npm ERR!   path: 'Versions/Current/Resources',
npm ERR!   dest: '/Users/mraible/Downloads/mfr/node_modules/puppeteer/.local-chromium/mac-869685/chrome-mac/Chromium.app/Contents/Frameworks/Chromium Framework.framework/Resources'
npm ERR! }
blog
npm WARN deprecated puppeteer@9.1.1: Version no longer supported. Upgrade to @latest
npm ERR! code 1
npm ERR! path /Users/mraible/Downloads/mfr/node_modules/puppeteer
npm ERR! command failed
npm ERR! command sh -c node install.js
npm ERR! ERROR: Failed to set up Chromium r869685! Set "PUPPETEER_SKIP_DOWNLOAD" env variable to skip download.
npm ERR! [Error: ENOENT: no such file or directory, open '/Users/mraible/Downloads/mfr/node_modules/puppeteer/.local-chromium/chrome-mac.zip'] {
npm ERR!   errno: -2,
npm ERR!   code: 'ENOENT',
npm ERR!   syscall: 'open',
npm ERR!   path: '/Users/mraible/Downloads/mfr/node_modules/puppeteer/.local-chromium/chrome-mac.zip'
npm ERR! }
store
<s> [webpack.Progress] 92% sealing asset processing RealContentHashPlugin

node:internal/event_target:912
  process.nextTick(() => { throw err; });
                           ^
Error: Cannot find module '../types'
Require stack:
- /Users/mraible/Downloads/mfr/node_modules/jest-worker/build/workers/threadChild.js
    at Function.Module._resolveFilename (node:internal/modules/cjs/loader:933:15)
    at Function.Module._load (node:internal/modules/cjs/loader:778:27)
    at Module.require (node:internal/modules/cjs/loader:1005:19)
    at require (node:internal/modules/cjs/helpers:102:18)
    at Object.<anonymous> (/Users/mraible/Downloads/mfr/node_modules/jest-worker/build/workers/threadChild.js:13:14)
    at Module._compile (node:internal/modules/cjs/loader:1105:14)
    at Object.Module._extensions..js (node:internal/modules/cjs/loader:1159:10)
    at Module.load (node:internal/modules/cjs/loader:981:32)
    at Function.Module._load (node:internal/modules/cjs/loader:822:12)
    at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:77:12)
Emitted 'error' event on Worker instance at:
    at Worker.[kOnErrorMessage] (node:internal/worker:289:10)
    at Worker.[kOnMessage] (node:internal/worker:300:37)
    at MessagePort.<anonymous> (node:internal/worker:201:57)
    at MessagePort.[nodejs.internal.kHybridDispatch] (node:internal/event_target:643:20)
    at MessagePort.exports.emitMessage (node:internal/per_context/messageport:23:28) {
  code: 'MODULE_NOT_FOUND',
  requireStack: [
    '/Users/mraible/Downloads/mfr/node_modules/jest-worker/build/workers/threadChild.js'
  ]
}
npm ERR! Lifecycle script `webapp:build:prod` failed with error:
npm ERR! Error: command failed
npm ERR!   in workspace: store@0.0.1-SNAPSHOT
npm ERR!   at location: /Users/mraible/Downloads/mfr/store
npm ERR! Lifecycle script `webapp:prod` failed with error:
npm ERR! Error: command failed
npm ERR!   in workspace: store@0.0.1-SNAPSHOT
npm ERR!   at location: /Users/mraible/Downloads/mfr/store
Error: write EPIPE
    at process.target._send (node:internal/child_process:866:20)
    at process.target.send (node:internal/child_process:739:19)
    at /Users/mraible/Downloads/mfr/node_modules/fork-ts-checker-webpack-plugin/lib/rpc/expose-rpc.js:27:31
    at new Promise (<anonymous>)
    at sendMessage (/Users/mraible/Downloads/mfr/node_modules/fork-ts-checker-webpack-plugin/lib/rpc/expose-rpc.js:19:38)
    at /Users/mraible/Downloads/mfr/node_modules/fork-ts-checker-webpack-plugin/lib/rpc/expose-rpc.js:60:27
    at Generator.next (<anonymous>)
    at fulfilled (/Users/mraible/Downloads/mfr/node_modules/fork-ts-checker-webpack-plugin/lib/rpc/expose-rpc.js:5:58)
    at processTicksAndRejections (node:internal/process/task_queues:96:5) {
  errno: -32,
  code: 'EPIPE',
  syscall: 'write'
}

> Task :webapp FAILED

If I run npm install from the top-level and try again, it seems to work. The errors I receive seem to be related to Docker and my local setup.

Task :jibDockerBuild FAILED
Containerizing application to Docker daemon as gateway...
Base image 'eclipse-temurin:11-jre-focal' does not use a specific image digest - build may not be reproducible
The base image requires auth. Trying again for eclipse-temurin:11-jre-focal...
The credential helper (docker-credential-desktop) has nothing for server URL: registry-1.docker.io

Got output:

credentials not found in native keychain

The credential helper (docker-credential-desktop) has nothing for server URL: registry.hub.docker.com

Got output:

credentials not found in native keychain

Using credentials from Docker config (/Users/mraible/.docker/config.json) for eclipse-temurin:11-jre-focal
Executing tasks:
[===========                   ] 36.7% complete
> building dependencies layer


FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':jibDockerBuild'.
> com.google.cloud.tools.jib.plugins.common.BuildStepsExecutionException: Build to Docker daemon failed, perhaps you should make sure your credentials for 'registry-1.docker.io/library/eclipse-temurin' are set up correctly. See https://github.com/GoogleContainerTools/jib/blob/master/docs/faq.md#what-should-i-do-when-the-registry-responds-with-unauthorized for help

docker login results in the following so I'm not sure how to fix this:

Authenticating with existing credentials...
Login Succeeded

@mraible
Copy link
Contributor

mraible commented Jun 9, 2022

I was able to fix my Docker issues using this answer on Stack Overflow.

After fixing Docker, I built each image and ran npm run ci:e2e:prepare at the top level. After everything was started, I tried:

npm run e2e --workspaces

This worked!

Screen Shot 2022-06-09 at 10 20 08

mraible
mraible previously approved these changes Jun 9, 2022
@mshima mshima changed the title skip install for workspaces. skip applications install when generating workspaces. Jun 9, 2022
@mshima mshima force-pushed the skip_ci-workspaces_install branch from 73152c1 to af823ab Compare June 9, 2022 21:21
@mshima
Copy link
Member Author

mshima commented Jun 10, 2022

@mraible now it skips install at applications, installs at the root.

Tested using:

jhipster jdl reactive-mf --workspaces
npm run java:docker:arm64
npm run ci:e2e:prepare

Then access localhost:8080

@mraible
Copy link
Contributor

mraible commented Jun 10, 2022

We might want to add some information about workspaces and how it works in our microfrontends documentation. Linking to an outside explanation is fine, but I'd love to see this example provided and briefly explained.

Both of these pages show up when I search for "jhipster microservices" and "jhipster microfrontends".

@mraible
Copy link
Contributor

mraible commented Jun 10, 2022

I love how fast the M1 is for e2e tests!

$ npm run e2e --workspaces
...
Execution time: 42 s.

Here's my build-cypress.sh script:

#/bin/zsh

VERSION=9.7.0 # set version here

cd /tmp
git clone git@github.com:cypress-io/cypress.git
cd cypress
git checkout tags/v$VERSION
yarn install
yarn binary-build --platform darwin --version $VERSION
mkdir -p ~/Library/Caches/Cypress/$VERSION
rm -rf ~/Library/Caches/Cypress/$VERSION
cp -R build/build/mac-arm64 ~/Library/Caches/Cypress/$VERSION

@mshima mshima merged commit 4d85cb3 into jhipster:main Jun 10, 2022
@mshima mshima deleted the skip_ci-workspaces_install branch June 10, 2022 02:50
@DanielFran DanielFran added this to the 7.9.0 milestone Jun 22, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants