Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

invoke markdownlint does not run in the docker environment #4061

Closed
gsnider2195 opened this issue Jul 5, 2023 · 2 comments · Fixed by #4082
Closed

invoke markdownlint does not run in the docker environment #4061

gsnider2195 opened this issue Jul 5, 2023 · 2 comments · Fixed by #4082
Assignees
Labels
type: bug Something isn't working as expected

Comments

@gsnider2195
Copy link
Contributor

gsnider2195 commented Jul 5, 2023

Environment

  • Nautobot version (Docker tag too if applicable): 2.0.0-b1
  • Python version: 3.8.17
  • Database platform, version:
  • Middleware(s):

Steps to Reproduce

  1. Run invoke markdownlint in the docker dev environment

Expected Behavior

Markdownlint runs

Observed Behavior

The command errors out with:

npm ERR! could not determine executable to run

npm ERR! A complete log of this run can be found in:
npm ERR!     /root/.npm/_logs/2023-07-05T21_28_35_823Z-debug-0.log

The contents of the log:

0 verbose cli /usr/bin/node /usr/bin/npm
1 info using npm@9.5.1
2 info using node@v18.16.1
3 timing npm:load:whichnode Completed in 2ms
4 timing config:load:defaults Completed in 2ms
5 timing config:load:file:/usr/lib/node_modules/npm/npmrc Completed in 0ms
6 timing config:load:builtin Completed in 0ms
7 timing config:load:cli Completed in 4ms
8 timing config:load:env Completed in 0ms
9 timing config:load:file:/source/.npmrc Completed in 0ms
10 timing config:load:project Completed in 1ms
11 timing config:load:file:/root/.npmrc Completed in 0ms
12 timing config:load:user Completed in 0ms
13 timing config:load:file:/usr/etc/npmrc Completed in 0ms
14 timing config:load:global Completed in 0ms
15 timing config:load:setEnvs Completed in 1ms
16 timing config:load Completed in 9ms
17 timing npm:load:configload Completed in 9ms
18 timing npm:load:mkdirpcache Completed in 0ms
19 timing npm:load:mkdirplogs Completed in 1ms
20 verbose title npm exec markdownlint --ignore nautobot/project-static --ignore nautobot/ui/node_modules --config .markdownlint.yml --rules scripts/use-relative-md-links.js nautobot examples CHANGELOG.md CODE_OF_CONDUCT.md CONTRIBUTING.md README.md SECURITY.md
21 verbose argv "exec" "--" "markdownlint" "--ignore" "nautobot/project-static" "--ignore" "nautobot/ui/node_modules" "--config" ".markdownlint.yml" "--rules" "scripts/use-relative-md-links.js" "nautobot" "examples" "CHANGELOG.md" "CODE_OF_CONDUCT.md" "CONTRIBUTING.md" "README.md" "SECURITY.md"
22 timing npm:load:setTitle Completed in 1ms
23 timing config:load:flatten Completed in 3ms
24 timing npm:load:display Completed in 3ms
25 verbose logfile logs-max:10 dir:/root/.npm/_logs/2023-07-05T21_28_35_823Z-
26 verbose logfile /root/.npm/_logs/2023-07-05T21_28_35_823Z-debug-0.log
27 timing npm:load:logFile Completed in 4ms
28 timing npm:load:timers Completed in 0ms
29 timing npm:load:configScope Completed in 0ms
30 timing npm:load Completed in 21ms
31 silly logfile done cleaning log files
32 timing arborist:ctor Completed in 1ms
33 http fetch GET 200 https://registry.npmjs.org/markdownlint 222ms (cache miss)
34 timing command:exec Completed in 243ms
35 verbose stack Error: could not determine executable to run
35 verbose stack     at getBinFromManifest (/usr/lib/node_modules/npm/node_modules/libnpmexec/lib/get-bin-from-manifest.js:17:23)
35 verbose stack     at exec (/usr/lib/node_modules/npm/node_modules/libnpmexec/lib/index.js:185:15)
35 verbose stack     at async module.exports (/usr/lib/node_modules/npm/lib/cli.js:134:5)
36 verbose pkgid markdownlint@0.29.0
37 verbose cwd /source
38 verbose Linux 6.1.0-9-amd64
39 verbose node v18.16.1
40 verbose npm  v9.5.1
41 error could not determine executable to run
42 verbose exit 1
43 timing npm Completed in 275ms
44 verbose code 1
45 error A complete log of this run can be found in:
45 error     /root/.npm/_logs/2023-07-05T21_28_35_823Z-debug-0.log

The npm dependencies are installed in /source/ in the dockerfile dependencies-dev stage but:

  1. Immediately after they're installed /source/ is overwritten
  2. /source/ gets bound from the host filesystem so anything that was in that directory in the container image is lost

Workaround until the bug is fixed:

inv cli
npm install from within the container will install the dependencies to the locally bound filesystem

@gsnider2195 gsnider2195 added the type: bug Something isn't working as expected label Jul 5, 2023
@glennmatthews glennmatthews self-assigned this Jul 6, 2023
@glennmatthews
Copy link
Contributor

Thanks for filing this Gary! I've noticed the same symptom.

glennmatthews added a commit that referenced this issue Jul 10, 2023
Dockerfile and Docker Compose changes:
  - Split python and node installation and compilation into separate parallel build stages
  - Add cache mounts for `apt` to hopefully improve rebuild performance
  - Change `dev` and `final-dev` targets to install to `/opt/nautobot` virtualenv and run as the `nautobot` user
  - Build Nautobot 2.0 UI as part of Docker image build
  - `nodejs` container now uses the nautobot `dev` image instead of pulling `node` image
  - Removed `node_modules` shared volume between `nautobot` and `nodejs` containers as no longer needed

Other changes:
  - Fix tasks.py definition for `invoke markdownlint` in docker workflow (fixes #4061)
  - Update installed Poetry version to 1.5.1
  - Update `markdownlint` to 0.35.0 and fix some new warnings it raises
  - Remove separate package.json for markdownlint/eslint/prettier, roll them into the main package.json
@bryanculver bryanculver linked a pull request Jul 10, 2023 that will close this issue
5 tasks
glennmatthews added a commit that referenced this issue Jul 13, 2023
* Refactor Dockerfile, fix #4061, etc.

Dockerfile and Docker Compose changes:
  - Split python and node installation and compilation into separate parallel build stages
  - Add cache mounts for `apt` to hopefully improve rebuild performance
  - Change `dev` and `final-dev` targets to install to `/opt/nautobot` virtualenv and run as the `nautobot` user
  - Build Nautobot 2.0 UI as part of Docker image build
  - `nodejs` container now uses the nautobot `dev` image instead of pulling `node` image
  - Removed `node_modules` shared volume between `nautobot` and `nodejs` containers as no longer needed

Other changes:
  - Fix tasks.py definition for `invoke markdownlint` in docker workflow (fixes #4061)
  - Update installed Poetry version to 1.5.1
  - Update `markdownlint` to 0.35.0 and fix some new warnings it raises
  - Remove separate package.json for markdownlint/eslint/prettier, roll them into the main package.json

* More markdownlint fixes

* Make a stab at fixing tasks and PR CI

* NPM/linting updates here too

* Add temporary build stage for PRs

* Simplify test docker tags

* Lock cache mounts to avoid problems with parallel builds

* Add generated files to .dockerignore, remove apt cache mounts, rework Dockerfile

* Black/flake8

* Remove temporary build
@glennmatthews
Copy link
Contributor

Fixed by #4082.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Oct 12, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
type: bug Something isn't working as expected
Projects
No open projects
Archived in project
Development

Successfully merging a pull request may close this issue.

2 participants