Npm dependency cache never gets created

[mjgs]

Description:

When caching npm dependencies, initial cache never gets created.

Action version:

v3

Platform:

• Ubuntu
• macOS
• Windows

Runner type:

• Hosted
• Self-hosted

Tools version:

Run actions/setup-node@v3
Attempting to download 20.4.0...
Acquiring 20.4.0 - x64

Repro steps:

I have setup-node configured with npm cache:

      - name: Setup NodeJS
        uses: actions/setup-node@v3
        with:
          node-version: ${{ env.nodejs_version }}
          cache: 'npm'
          cache-dependency-path: ${{ env.repo_dir }}/package-lock.json

      - name: Install NodeJS modules
        run: | 
          cd $repo_dir && npm ci

When workflow runs, setup-node sees there is no cache:

Run actions/setup-node@v3
Attempting to download 20.4.0...
Acquiring 20.4.0 - x64 from https://github.com/actions/node-versions/releases/download/20.4.0-5484146983/node-20.4.0-linux-x64.tar.gz
Extracting ...
/usr/bin/tar xz --strip 1 --warning=no-unknown-keyword -C /home/runner/work/_temp/48b89860-06fe-4719-88d8-9b348eae167b -f /home/runner/work/_temp/9cc686a7-e143-4ead-85e4-ecfcab256528
Adding to the cache ...
Environment details
/opt/hostedtoolcache/node/20.4.0/x64/bin/npm config get cache
/home/runner/.npm
npm cache is not found

Install of packages completes:

Run cd $repo_dir && npm ci
(node:1860) [DEP0170] DeprecationWarning: The URL git+ssh://git@github.com-id_rsa_file:mjgs/file#7c95d46d827416d4dd377e14fbeda5430cec26cc is invalid. Future versions of Node.js will throw an error.
(Use `node --trace-deprecation ...` to show where the warning was created)
added 205 packages, and audited 206 packages in 3m

However looking at Actions page Caches tab afterwards, no cache got created. Same thing happens on re-running the workflow.

Expected behavior:

I expected a cache entry to get created on Actions page Caches tab, and for that to get restored on subsequent runs.

Actual behavior:

No cache ever gets created. How do I create the initial cache?

[dusan-trickovic]

Hello, @mjgs! Thank you for creating this issue, we will investigate it and see what can be done :)

[mjgs]

Thanks @dusan-trickovic In the interim the cache page, after a considerable delay of around 20 mins, eventually did display a cache entry.

On re-run of workflow, the cache is being restored:

Run actions/setup-node@v3
Attempting to download 20.4.0...
Acquiring 20.4.0 - x64 from https://github.com/actions/node-versions/releases/download/20.4.0-5484146983/node-20.4.0-linux-x64.tar.gz
Extracting ...
/usr/bin/tar xz --strip 1 --warning=no-unknown-keyword -C /home/runner/work/_temp/e6ed6750-3bb0-4d96-820c-2586453cff44 -f /home/runner/work/_temp/9d9cc450-8894-43cb-89b2-4ced7f9bcbfd
Adding to the cache ...
Environment details
/opt/hostedtoolcache/node/20.4.0/x64/bin/npm config get cache
/home/runner/.npm
Received 11175195 of 11175195 (100.0%), 60.2 MBs/sec
Cache Size: ~11 MB (11175195 B)
/usr/bin/tar -xf /home/runner/work/_temp/d0781676-7b04-4444-ab7f-974f927b342a/cache.tzst -P -C /home/runner/work/personal-website/personal-website --use-compress-program unzstd
Cache restored successfully
Cache restored from key: node-cache-Linux-npm-0c7dca0fd57a37dbdc6980a56c26be75a0269edd83feea7d44e98543a85dd5fa

However:

1. Install times are unchanged, still at around 3mins, I would expect it to be in the order of seconds if installing from cached modules
2. The cache size of 11MB seems very small for what amounts to several hundred modules

What could cause the cache to have no impact on install time?

[mjgs]

@dusan-trickovic some additional strangeness:

I removed the cache options completely, deleted all caches from Caches page. When I run the workflow a cached version of the code is clearly running.

I know this because I added some new debug logging in a private module. When private module is run on its own, the new debug logging appears in the logs. When run from the consuming project, which previously had cache configured, the new debug logging does not appear.

I notice that the setup-node step has the following in the logs:

“Adding to the cache ...”

No additional mentions of cache.

Why are cached versions of modules being used when the cache is cleared and removed?

[mjgs]

I’m effectively completely blocked on this project now because of setup-node. I can no longer update any modules I am using in my project because the updates never make it into the workflow.

The only way I can think to unblock is to literally move all code to a different repo, and be sure to never use the cache feature.

Is there another way?

[dusan-trickovic]

Hello, @mjgs! I'm looking into this now and will respond promptly. In the meantime, would it be possible for you to share (at least a snippet of) what your current workflow and/or code structure looks like after the changes?

Regarding the suspiciously small size of your cache, perhaps that's somewhat expected, as the action itself isn't caching the entire node_modules directory, hence working with smaller file sizes.

As for the rest of the issue, I can't really tell the structure of your code from this output here, but I wanted to check if your package-lock.json file is in the root directory of the project? If so, perhaps you could add a Checkout step (unless you already have and the configuration you've provided is just a snippet) and instead try using just cache-dependency-path: ./package-lock.json.

When it comes to cache deletion, perhaps you could also list the caches for your repository to see if anything may be left behind (that isn't displayed on the Caches page) that the Action is picking up? You can find more info on how to do it in this section of our docs. Although it may sound unlikely, I think it's still better to check it just to be sure.

In addition to all of that, you can also try and enable debug logging to get a better picture of what's going on with the Action in more detail at every step.

In any case, I will continue my investigation on this and let you know about my findings and potential solutions.

Please, in the meantime, feel free to reach out again and let us know if you have any more questions. Alternatively, feel free to post an update for this issue and let us know if some of these suggestions helped you solve it.

Thank you very much for your patience and contribution! :)

[mjgs]

@dusan-trickovic Thanks for the response.

Answering your questions:

would it be possible for you to share (at least a snippet of) what your current workflow and/or code structure looks like after the changes?

Here is the current setup-node relevant steps:

      - name: Setup NodeJS
        uses: actions/setup-node@v3
        with:
          node-version: ${{ env.nodejs_version }}

      - name: Install NodeJS modules
        run: | 
          cd $repo_dir && npm ci

the action itself isn't caching the entire node_modules directory

What exactly is cached then? How is the cache supposed to speed up the install if the modules are not cached?

try using just cache-dependency-path: ./package-lock.json

I already use this setting since the package-lock.json is in a subdirectory. You can see this in my original post.

perhaps you could also list the caches for your repository to see if anything may be left behind (that isn't displayed on the Caches page) that the Action is picking up?

I tried empty search field on Cache page, which I think lists all cache entries. I also tried adding branch name to search field. Both returned zero cache entries on the Caches page.

Is that what you mean or is there another way to list? Are you saying I should list via the HTTP api with something like curl?

I have a couple of questions for you:

1. If the cache is somehow still being used, why doesn’t this appear in the logs?
2. Where is setup-node getting the cached module from if, as you say, node_modules folder is not cached?

[mjgs]

@dusan-trickovic I ran the workflow with debug enabled. It prints a load of stuff including what seems to be a list of every file in the node runtime. There was some mention of the cache right at the end. Here’s a screenshot:

What does it mean?
Is there some other info I can provide that would help you with your investigation?

[mjgs]

@dusan-trickovic I ran the workflow with debug enabled. It prints a load of stuff including what seems to be a list of every file in the node runtime. There was some mention of the cache right at the end. Here’s a screenshot:

What does it mean?
Is there sone other info I can provide that would help you with your investigation?

[mjgs]

@dusan-trickovic I figured out why old code was being installed. Npm was installing from package-lock.json.

I updated install command to:

cd $repo_dir && npm install --no-package-lock

You have to specifically tell npm to not use the lock file. You might consider mentioning this gotcha in your docs or on stdout.

So now I am back to my original issue:

Why isn’t the cache speeding up installs?

Shouldn’t the install take seconds since it’s using the npm cache which is restored by the setup-node cache feature?

[HarithaVattikuti]

Hello @mjgs,

The setup-node uses global cache please refer to the documentation for reference.

For more info on global cache you can refer the documentation related to npm. In the majority cases the global cache gets impact on builds with medium or large amount of dependencies.

Please feel free to reach out again and let us know if you have any more questions.

[dsame]

Hello @mjgs

First of all the action is not supposed to cache node_modules, it is anti pattern and must not be used in the team development.

Instead it caches the ~/.npm directory where the downloaded packages are cached by npm and installed to node_modules from.

I created the demo build reproducing the problem, which shows the caching works effectively:

the 1st run:

the 2nd run:

It shows the npm does not issue the http request on the 2nd run
https://github.com/akv-demo/setup-node-test/actions/runs/7320563462/job/19939865353#step:4:24
comparing with the first run:
https://github.com/akv-demo/setup-node-test/actions/runs/7320563462/job/19939842553#step:4:24

Please pay attention how the env variables are set: https://github.com/akv-demo/setup-node-test/actions/runs/7320563462/workflow#L15

If your build still have a problem please try to clone the demo repo (branch issue-851) and modify it to reproduce the problem