Docker on Debian 11 fails on version 1.1.15 and 1.1.16 giving NPM errors

[gogglespisano]

Mesh Central on Debian Docker fails with NPM errors
This also happened on version 1.1.11 (I think) but started working again a version later. Now the problem is back.
The only setting that I've been changing is the docker image tag.
Debian (Docker Host) packages all up to date.

Using version 1.1.15 and 1.1.16 give this error

meshcentral    | Installing modules...
meshcentral    | ERROR: Unable to install required modules. MeshCentral may not have access to npm, or npm may not have suffisent rights to load the new module. To manualy install this module try:
meshcentral    |
meshcentral    |    cd "/opt/meshcentral/meshcentral"
meshcentral    |    npm install --no-audit --no-package-lock --no-optional --omit=optional --no-save meshcentral@1.1.16 archiver@5.3.2 body-parser@1.20.2 cbor@5.2.0 compression@1.7.4 cookie-session@2.0.0 express@4.18.2 express-handlebars@5.3.5 express-ws@4.0.0 ipcheck@0.1.0 minimist@1.2.8 multiparty@4.2.3 @yetzt/nedb node-forge@1.3.1 ua-parser-js@1.0.36 ws@8.14.2 yauzl@2.10.0 ssh2@1.14.0 semver@7.5.4 nodemailer@6.9.6 otplib@10.2.3
meshcentral    |    node node_modules/meshcentral
meshcentral    | Installing modules...

Using version 1.1.14 (since 15 and 16 don’t run) to run the suggested commands in bash inside container

1d8de0f7d6a1:/opt/meshcentral$ cd "/opt/meshcentral/meshcentral"
1d8de0f7d6a1:/opt/meshcentral/meshcentral$  npm install --no-audit --no-package-lock --no-optional --omit=optional --no-save meshcentral@1.1.16 archiver@5.3.2 body-parser@1.20.2 cbor@5.2.0 compression@1.7.4 cookie-session@2.0.0 express@4.18.2 express-handlebars@5.3.5 express-ws@4.0.0 ipcheck@0.1.0 minimist@1.2.8 multiparty@4.2.3 @yetzt/nedb node-forge@1.3.1 ua-parser-js@1.0.36 ws@8.14.2 yauzl@2.10.0 ssh2@1.14.0 semver@7.5.4 nodemailer@6.9.6 otplib@10.2.3
npm WARN config optional Use `--omit=optional` to exclude optional dependencies, or
npm WARN config `--include=optional` to include them.
npm WARN config
npm WARN config     Default value does install optional deps unless otherwise omitted.
npm ERR! code EACCES
npm ERR! syscall mkdir
npm ERR! path /.npm
npm ERR! errno -13
npm ERR!
npm ERR! Your cache folder contains root-owned files, due to a bug in
npm ERR! previous versions of npm which has since been addressed.
npm ERR!
npm ERR! To permanently fix this problem, please run:
npm ERR!   sudo chown -R 1003:0 "/.npm"

npm ERR! Log files were not written due to an error writing to the directory: /.npm/_logs
npm ERR! You can rerun the command with `--loglevel=verbose` to see the logs in your terminal

docker-compose.yaml

version: '3'

services:
  meshcentral:
    restart: always
    container_name: meshcentral
    user: "1003"
    # use the official meshcentral container
    image: ghcr.io/ylianst/meshcentral:1.1.16
    ports:
      # MeshCentral will moan and try everything not to use port 80, but you can also use it if you so desire, just change the config.json according to your needs
      - 4430:4430
      - 4435:4435
    env_file:
      - .env
    volumes:
      # config.json and other important files live here. A must for data persistence
      - ./meshcentral/data:/opt/meshcentral/meshcentral-data
      # where file uploads for users live
      - ./meshcentral/user_files:/opt/meshcentral/meshcentral-files
      # location for the meshcentral-backups - this should be mounted to an external storage
      - ./meshcentral/backup:/opt/meshcentral/meshcentral-backup
      # location for site customization files
      - ./meshcentral/web:/opt/meshcentral/meshcentral-web

.env

NODE_ENV=production
# initial mongodb-variables
MONGO_INITDB_ROOT_USERNAME=mongodbadmin
MONGO_INITDB_ROOT_PASSWORD=xxxxx
# initial meshcentral-variables
# the following options are only used if no config.json exists in the data-folder
# your hostname
HOSTNAME=mesh.mydomain
USE_MONGODB=false
# set to your reverse proxy IP if you want to put meshcentral behind a reverse proxy 
REVERSE_PROXY=false
REVERSE_PROXY_TLS_PORT=
# set to true if you wish to enable iframe support
IFRAME=false
# set to false if you want disable self-service creation of new accounts besides the first (admin)
ALLOW_NEW_ACCOUNTS=false
# set to true to enable WebRTC - per documentation it is not officially released with meshcentral and currently experimental. Use with caution
WEBRTC=false
# set to true to allow plugins
ALLOWPLUGINS=false
# set to true to allow session recording
LOCALSESSIONRECORDING=false
# set to enable or disable minification of json, reduces traffic
MINIFY=true

config.json

{
  "$schema": http://info.meshcentral.com/downloads/meshcentral-config-schema.json,
  "settings": {
    "plugins":{"enabled": false},
    "_mongoDb": null,
    "Cert": "mesh.mydomain",
    "sessionKey": "xxxx",
    "_WANonly": true,
    "_LANonly": true,
    "ExactPorts": true,
    "port": 4430,
    "aliasPort": 443,
    "RelayPort": 4435,
    "RelayAliasPort": 453,
    "TLSOffload": "172.16.1.1",
    "AgentPong": 100,
    "AgentPing": 100,
    "redirPort": 0,
    "MpsPort": 0,
    "agentLogDump": true,
    "agentCoreDump": true,
    "_ignoreAgentHashCheck": true,
    "SelfUpdate": false,
    "AllowFraming": false,
    "WebRTC": false
  },
  "domains": {
    "": {
      "title": " Mesh Central",
      "_title2": "Servername",
      "minify": true,
      "NewAccounts": false,
      "localSessionRecording": false,
      "userNameIsEmail": false,
      "certUrl": https://mesh.mydomain:443/,
      "novnc": false,
      "mstsc": true,
      "ssh": true
    },
    "migval": {
      "title": "migval Mesh Central",
      "_title2": "Servername",
      "minify": true,
      "NewAccounts": false,
      "localSessionRecording": false,
      "userNameIsEmail": false,
      "certUrl": https://mesh.mydomain:443/,
      "novnc": false,
      "mstsc": true,
      "ssh": true
    }
  },
  "smtp": {
    "host": "mailer.mydomain",
    "port": 25,
    "from": Mesh@mydomain,
    "tls": false
    }
}

[si458]

PLZ FOLLOW THE BUG TEMPLATE/FORMATTING IN FUTURE AS IT HAS INFORMATION THAT IS REQUIRED TO HELP YOU!!!

[si458]

looking into it but there are a few issues you have

1. your scheme value is wrong, the $schema should read:
   "$schema": "https://raw.githubusercontent.com/Ylianst/MeshCentral/master/meshcentral-config-schema.json"

2. any values thats is a string should be enclosed by speech marks "https://mesh.mydomain:443/" "Mesh@mydomain"

3. huh? why is it installing meshcentral via npm on startup?

EDIT: if you add ARGS=--debug to your environment variables, it will spit out alot more info to help you debug stuff

[si458]

OK I CAN REPLICATE THE ISSUE!
the issue is because you have set a user id in docker-compose (im guessing ur local users id is 1003)
if you remove user:1003 and let node run as root in the docker, it works no problem!
but then your files will be written as root and not your user!

[gogglespisano]

Thanks Simon!

I've updated the schema. The missing quotes were just in what was posted after removing personal info. The files had the quotes correctly.

I removed the user: "1003" statement and npm installed the modules and loaded meshcentral.

Is there a way to give npm permission to the non-root user? Other than this npm issue, meshcentral has been working with the non-root user since I started 6-12 months ago.

[gogglespisano]

Welll, it worked once.
Without the user entry, so running as root:

Removing the container and recreating it does npm "installing modules" on both 1.1.15 and 1.1.16 and eventually fails.

meshcentral    | Installing modules...
meshcentral    | ERROR: Unable to install required modules. MeshCentral may not have access to npm, or npm may not have suffisent rights to load the new module. To manualy install this module try:
meshcentral    |
meshcentral    |    cd "/opt/meshcentral/meshcentral"
meshcentral    |    npm install --no-audit --no-package-lock --no-optional --omit=optional --no-save meshcentral@1.1.15 archiver@5.3.2 body-parser@1.20.2 cbor@5.2.0 compression@1.7.4 cookie-session@2.0.0 express@4.18.2 express-handlebars@5.3.5 express-ws@4.0.0 ipcheck@0.1.0 minimist@1.2.8 multiparty@4.2.3 @yetzt/nedb node-forge@1.3.1 ua-parser-js@1.0.36 ws@8.14.2 yauzl@2.10.0 ssh2@1.14.0 semver@7.5.4 nodemailer@6.9.6 otplib@10.2.3
meshcentral    |    node node_modules/meshcentral
meshcentral    | Installing modules...

1.1.14 starts up quickly without any npm step.

meshcentral    | MeshCentral v1.1.14, Hybrid (LAN + WAN) mode, Production mode.
meshcentral    | MeshCentral HTTP relay server running on port 4435, alias port 453.
meshcentral    | MeshCentral HTTP server running on port 4430, alias port 443.
meshcentral    | Loaded web certificate from "https://mesh.mydomain:443/", host: "mesh.mydomain"
meshcentral    |   SHA384 cert hash: xxxxxxxxxxxxxxxxx
meshcentral    |   SHA384 key hash: xxxxxxxxxxxxxxxxxxxx
meshcentral    | Loaded web certificate from "https://mesh.mydomain:443/", host: "mesh.mydomain"
meshcentral    |   SHA384 cert hash: xxxxxxxxxxxxxxxxxxxxx
meshcentral    |   SHA384 key hash: xxxxxxxxxxxxxxxxxxxxxx

[gogglespisano]

This problem turned out to be a timeout caused by bug #5681. Because all modules were being reinstalled on startup and the machine is not really fast, the npm command timed out and kept looping trying to install the modules.

Once #5681 is fixed, this problem will go away and startup in general will be faster for Docker.

[gogglespisano]

Bug #5684 is the cause of all modules needing to be installed for the last 2-3 months, roughly since 1.15

[gogglespisano]

OK I CAN REPLICATE THE ISSUE! the issue is because you have set a user id in docker-compose (im guessing ur local users id is 1003) if you remove user:1003 and let node run as root in the docker, it works no problem! but then your files will be written as root and not your user!

Coming back to the user issue now that the NPM problem has been figured out.

Using user: "1003" or "1003:1003" still works. It doesn't need to be run under root (at least with my config). All the files in the data folder are owned by "1003:1003"