Extracted repo missing node_modules folder

[FredrikLiindgren]

Describe the bug

Frogbot scan fails complaining about missing node_modules folder in the extracted repo.
npm install has been run beforehand.
ls -la after failure shows node_modules in the project folder
If npm install is not run beforehand it complains about the repo missing node_modules before extracting it.

Current behavior

GitLab CI output:

$ npm install
npm WARN old lockfile 
npm WARN old lockfile The package-lock.json file was created with an old version of npm,
npm WARN old lockfile so supplemental metadata must be fetched from the registry.
npm WARN old lockfile 
npm WARN old lockfile This is a one-time fix-up, please be patient...
npm WARN old lockfile 
added 50 packages, and audited 51 packages in 4s
2 packages are looking for funding
  run `npm fund` for details
found 0 vulnerabilities
$ frogbot ${FROGBOT_CMD}
12:39:04 [Info] Frogbot version: 2.6.1
12:39:04 [Info] Running Frogbot "scan-pull-request" command
12:39:04 [Info] Auditing project: /builds/cloud-and-container/public/demo/demo-app
12:39:04 [Info] Detected: npm.
12:39:08 [Info] JFrog Xray version is: 3.65.2
12:39:08 [Info] Scanning module helloworld-frontend:0.0.8...
12:39:09 [Info] Waiting for scan to complete...
12:39:30 [Info] Auditing  demo-app main
12:39:30 [Info] demo-app repository downloaded successfully. Starting with repository extraction...
12:39:30 [Info] Extracted repository successfully
12:39:30 [Info] Auditing project: /tmp/jfrog.cli.temp.-1680611970-18[53](https://git-tos.intern.folksam.se/cloud-and-container/public/demo/demo-app/-/jobs/626443#L53)662[57](https://git-tos.intern.folksam.se/cloud-and-container/public/demo/demo-app/-/jobs/626443#L57)5
12:39:30 [Info] Detected: npm.
12:39:32 [Info] Used npm version: 8.19.2
12:39:32 [Error] audit command in /tmp/jfrog.cli.temp.-1680[61](https://git-tos.intern.folksam.se/cloud-and-container/public/demo/demo-app/-/jobs/626443#L61)1970-18536[62](https://git-tos.intern.folksam.se/cloud-and-container/public/demo/demo-app/-/jobs/626443#L62)575 failed:
'npm' audit failed when building dependency tree:
node_modules isn't found in '/tmp/jfrog.cli.temp.-1680611970-1853[66](https://git-tos.intern.folksam.se/cloud-and-container/public/demo/demo-app/-/jobs/626443#L66)2575'. Hint: Restore node_modules folder by running npm install or npm ci.
Running after_script
00:01
Running after script...
$ ls -la
total 60
drwxrwsrwx.  6 1000830000 1000830000   186 Apr  4 12:39 .
drwxrwsrwx.  4 1000830000 1000830000    42 Apr  4 12:38 ..
drwxrwsrwx.  2 1000830000 1000830000    32 Apr  4 12:38 .frogbot
drwxrwsrwx.  6 1000830000 1000830000   128 Apr  4 12:38 .git
-rw-rw-rw-.  1 1000830000 1000830000  5979 Apr  4 12:38 .gitlab-ci.yml
-rw-rw-rw-.  1 1000830000 1000830000   277 Apr  4 12:38 .yamllint
-rw-rw-rw-.  1 1000830000 1000830000   204 Apr  4 12:38 Dockerfile
drwxrwsrwx.  3 1000830000 1000830000    60 Apr  4 12:38 chart
drwxr-sr-x. 52 1000830000 1000830000  4096 Apr  4 12:39 node_modules

Reproduction steps

No response

Expected behavior

node_modules should be extracted with the project if it is present

JFrog Frogbot version

2.6.1

Package manager info

node, npm

Git provider

GitLab

JFrog Frogbot configuration yaml file

No response

Operating system type and version

docker image based on RHEL8 (ubi8/nodejs-16:1-8)

JFrog Xray version

3.65.2

[FredrikLiindgren]

Hi @eyalbe4 - Any reason for instantly closing this? I'd be happy to provide further details if necessary.

Kind Regards,
Fredrik

[sverdlov93]

It was mistakenly closed @FredrikLiindgren
We are looking into it and will let you know very soon.

[FredrikLiindgren]

@sverdlov93 alright, thanks for taking a look!

[EyalDelarea]

Hey @FredrikLiindgren,

Thank you for using Frogbot!

The cause of the the missing node_modules folder could be because the installCommand inside
your frogbot-config.yml (line 30) file is not set.
Could you provide us with your configuration ?

And i'm just about to release a new version of Frogbot which will not require to set the installCommand for npm packages anymore, so hope you won't encounter this problem in the future.

[FredrikLiindgren]

Hey @EyalDelarea,

You are correct in that I've not set installCommand in frogbot-config.yml.
However I've set the GitLab CI/CD variable JF_INSTALL_DEPS_CMD: npm i which I thought would be enough but I might be wrong.
I've done as much config as possible via GitLab CI/CD variables but added the frogbot-config.yml the frogbot commands were not working without it. Might be missing some other mandatory config in there.

My full frogbot-config.yml:

- params:
    git:
      repoName: demo-app
      branches:
        - main

Nice to hear about the about not needing to set installCommand!

[EyalDelarea]

For a quick fix you could add the following line to your config file:

  - params:
        git:
          repoName: demo-app
          branches:
            - main
        scan:
          projects:
            - installCommand: "npm i"

Or wait for the release.
Either way waiting to hear how it worked out for you !

[FredrikLiindgren]

Hi that worked great,

However this makes me think the docs about that is quite missleading as this

    # [Mandatory if the two conditions below are met]
    # 1. The project uses npm, yarn 2, NuGet or .NET to download its dependencies
    # 2. The `installCommand` variable isn't set in your frogbot-config.yml file.
    #
    # The command that installs the project dependencies (e.g "npm i", "nuget restore" or "dotnet restore")
    JF_INSTALL_DEPS_CMD: ""

Makes me think I do not have to define the installCommand in the frogbot-config.yml file if I've defined JF_INSTALL_DEPS_CMD :)

[EyalDelarea]

Hey @FredrikLiindgren , that's great to hear!

We will definitely update the docs so it won't be missleading :)

Thank you for your feedback!

[FredrikLiindgren]

Great @EyalDelarea - Thanks for your helpful & timely responses!