github action cache not working

[enKODING1]

Description:
When I run the code below, I confirmed that the cache is created in management.

name: ESLint Check

on: push

jobs:
  lint:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - uses: actions/setup-node@v3
        with:
          node-version: 18
          cache: 'npm'
      - run: npm ci

      - name: Run ESLint
        run: npm run lint

On the second run, cache-hit returns true, but it still takes a long time because it reinstalls dependencies.

The link below says that the machine's global cache is used. How does the machine's global cache work?
global cache

Action version:
Specify the action version

Platform:

• Ubuntu
• macOS
• Windows

Runner type:

• Hosted
• Self-hosted

Tools version:

[dusan-trickovic]

Hello @enKODING1 ! Thank you for reporting the issue, we will investigate it :)

[laurence-myers]

I, too, have tried usingsetup-node's caching, but it did not reduce my build times.

I then tried using actions/cache directly, caching the path ~/.npm. This significantly sped up my build time, as expected.

      - uses: actions/cache@v3
        with:
          path: '~/.npm'
          key: ${{ runner.os }}-${{ matrix.node-version }}-9.8.0-npmcache-${{ hashFiles('**/package-lock.json') }}

Something I noticed, my successful cache was 510 MB, but the setup-node cache is only 38 MB.

[boldwade]

I too am not noticing any improvement in build times using the setup-node with cache. It might reduce it maybe ~15 seconds from a 2 minute build time... so nothing significant, and that could just be because of environmental things.

[dmitry-shibanov]

Hello @enKODING1. Thank you for your report. If you need more information regarding to global cache you can take at this. In the majority cases the global cache gets impact on builds with medium or large amount of dependencies.

Hello @laurence-myers. Could you please provide a link to the build ? Do you use hosted or self-hosted runners ?

[byron70]

Seeing the same thing, this is my step.

# Note: this is initiated by another workflow with workflow_call, don't see how this would matter.
      - uses: actions/setup-node@v3
        with:
          node-version: 18
          cache: yarn
          cache-dependency-path: 'yarn.lock'
          check-latest: true

What I noticed are 2 entries in Caches with the same name but listing with different refs, one for the PR and one for main after the merge.

node-cache-Linux-yarn-a7e52870b0f6b2d8a0a65229ca202fd9d7917d4b70c81b3567458e3d9409fd43
160 MB cached 10 minutes ago
[main]

node-cache-Linux-yarn-a7e52870b0f6b2d8a0a65229ca202fd9d7917d4b70c81b3567458e3d9409fd43
160 MB cached 3 hours ago
refs/pull/92/merge

When the second build happens after the PR merge into main, This is the main branch build output.

Run actions/setup-node@v3
Attempt to resolve the latest version from manifest...
Resolved as '18.17.1'
Attempting to download 18.17.1...
Acquiring 18.17.1 - x64 from https://github.com/actions/node-versions/releases/download/18.17.1-...
Extracting ...
/usr/bin/tar xz --strip 1 --warning=no-unknown-keyword -C /home/runner/work/_temp/579226[16]-d896-4ab3-b3c1-3b16480be856 -f /home/runner/work/_temp/1a90467b-2045-4dd5-8eeb-922785c0050b
Adding to the cache ...
Environment details
  node: v18.[17]
  npm: 9.6.7
  yarn: 3.6.1
/usr/local/bin/yarn --version
3.6.1
/usr/local/bin/yarn config get cacheFolder
/home/runner/work/ORG/REPO/.yarn/cache
/usr/local/bin/yarn config get enableGlobalCache
false
All dependencies are managed locally by yarn3, the previous cache can be used
yarn cache is not found

It seems to me in this case, the cache is being pulled based on the ref/branch as well as the yarn.lock file. Not sure if this is the intended behavior, but seems like a miss.

Apologies this is a private repo and cannot provide the link.

[laurence-myers]

Hello @enKODING1. Thank you for your report. If you need more information regarding to global cache you can take at this. In the majority cases the global cache gets impact on builds with medium or large amount of dependencies.

Hello @laurence-myers. Could you please provide a link to the build ? Do you use hosted or self-hosted runners ?

Hosted runners. I should have mentioned, we are using a monorepo & lerna. Here's some builds, they're in a private repo.

Using just setup-node:

• Initial cache creation: https://github.com/doohly/doohly/actions/runs/5721896197/job/15504159311
• Use of cache, 38 MB https://github.com/doohly/doohly/actions/runs/5721910824/job/15504198681

Then, using the cache actions directly:

• Initial cache creation (of ~/.npm): https://github.com/doohly/doohly/actions/runs/5723560287/job/15508365034
• Use of cache, 508 MB: https://github.com/doohly/doohly/actions/runs/5723560287/job/15508462138

[djthornton1212]

I too have noticed that I have 27 instances of the same cache in management. Each is tied to a specific ref (branches & pull/#/merge).

Does anyone know if these are pointers/references to a common object or are they truly duplicates? Also, is it rebuilding the cache for each of these?

[laurence-myers]

I too have noticed that I have 27 instances of the same cache in management. Each is tied to a specific ref (branches & pull/#/merge).

@byron70 @djthornton1212 This is intended behaviour, due to access restrictions: https://docs.github.com/en/actions/using-workflows/caching-dependencies-to-speed-up-workflows#restrictions-for-accessing-a-cache

Workflow runs can restore caches created in either the current branch or the default branch (usually main).
...
Workflow runs cannot restore caches created for child branches or sibling branches.

So, if you want to share a cache between branches/refs, you must create the cache on your main branch. Which greatly limits the usefulness of the cache. 🫤

To avoid the restrictions, as a workaround, you could investigate using a different caching action, uploading the cache zip to e.g. S3: https://medium.com/nerd-for-tech/implementing-a-custom-github-actions-cache-375eb844e00a

[david-molnar-oculai]

It doesn't work for use either.

[m-nathani]

my workers are not caching the modules at all.. no matter how many times i run the same build.. it always a cache miss.

jobs:
  build:
    runs-on: ubuntu-latest
    permissions: write-all
    steps:
      - uses: actions/checkout@v4
      - uses: fregante/setup-git-user@v2
      - uses: actions/setup-node@v4
        with:
          node-version: '20.x'
          registry-url: 'https://registry.npmjs.org'
          cache: 'npm'

Not sure exactly what is going wrong 😕

[aparnajyothi-y]

Hello everyone, Thank you for the issue.

The setup-node uses global cache and please refer to the documentation for reference.
For more info on global cache you can refer the documentation related to npm. Besides, We tried to reproduce the issue from our end and compared it with actions/cache with the same configuration as setup-node uses and it worked as expected.

For now I am going to close the issue. If you have any concerns feel free to ping us.

[curtdept]

@aparnajyothi-y
The miss is that caches are being created multiple times when it could be once.

In a perfect flow a branch or branch PR is merged to master, in this case the cache is built and stored twice, once for the PR and once post merge to master.

It would be awesome if there was a "global" cache config option that allowed someone to just store and use all caches at a default branch level and not at a ref level. The above example would then only generate a cache entry once and allow greater reusability, even across PRs and branches. This also would not violate the parent cache access permission structure.

I would move we reopen this issue.

[dschnare]

I have the same issue as the OP @enKODING1 and others. I feel I'm missing something here. "caching global packages data" is not clear. If it were indeed caching then when installing modules it wouldn't take several minutes anymore. Has someone figured this one out or is the workaround to use actions/cache directly?

[dschnare]

So I think I get what it's supposed to cache now. However, it don't work at all as you'd expect; enable caching then continue to install your dependencies as usual. Then on the subsequent passes the installs will be much faster since the global cache data would be restored, and not HTTP calls would be made. However that never happens.

[kwypchlo]

@aparnajyothi-y this is happening to me now too, below my configuration and my list of caches, any insight appreciated

jobs:
  lint:
    runs-on: [ubuntu-latest]
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: '20'
          cache: 'npm'
      - run: npm i -g @nestjs/cli nx
      - run: npm ci
      - ...

[jakallergis]

this works for me:

jobs:
  validate:
    name: Validate
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3

      - name: Use Node.js 18.x
        uses: actions/setup-node@v3
        with:
          node-version: "18.x"

      - uses: actions/cache@v4
        if: ${{ !env.ACT }}
        id: yarn-cache # use this to check for `cache-hit` (`steps.yarn-cache.outputs.cache-hit != 'true'`)
        with:
          path: ./node_modules
          key: ${{ runner.os }}-modules-${{ hashFiles('**/yarn.lock') }}
          restore-keys: |
            ${{ runner.os }}-yarn-

      - name: Install Node Modules
        run: yarn install --network-timeout 100000

      - name: Validate and run tests
        run: yarn validate

yarn install runs in les than 1s.
Screenshot for proof:

[curtdept]

this works for me:

jobs:
  validate:
    name: Validate
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3

      - name: Use Node.js 18.x
        uses: actions/setup-node@v3
        with:
          node-version: "18.x"

      - uses: actions/cache@v4
        if: ${{ !env.ACT }}
        id: yarn-cache # use this to check for `cache-hit` (`steps.yarn-cache.outputs.cache-hit != 'true'`)
        with:
          path: ./node_modules
          key: ${{ runner.os }}-modules-${{ hashFiles('**/yarn.lock') }}
          restore-keys: |
            ${{ runner.os }}-yarn-

      - name: Install Node Modules
        run: yarn install --network-timeout 100000

      - name: Validate and run tests
        run: yarn validate

yarn install runs in les than 1s. Screenshot for proof:

Across branches?

[SimpleCreations]

I'm having the same issue too. Multiple cached created with the same key, while cache reads always miss. Will abandon global yarn caching and use actions/cache instead.

[aparnajyothi-y]

Hello everyone,
The hash value for the cache keys remains the same because it is generated from the contents of the dependency files, which are likely identical across different environments. This hash does not include the operating system, architecture, or branch, leading to the same hash value for different cache keys. This ensures that caches are correctly isolated per branch, avoiding conflicts and ensuring that each branch's dependencies can be cached and retrieved independently. This is intended behavior in setup-node. You can check more on caches in the following links:
Caching global packages data
Caching packages data
actions/cache#cache-scopes
GitHub documentation on caching dependencies

[curtdept]

Hello everyone, The hash value for the cache keys remains the same because it is generated from the contents of the dependency files, which are likely identical across different environments. This hash does not include the operating system, architecture, or branch, leading to the same hash value for different cache keys. This ensures that caches are correctly isolated per branch, avoiding conflicts and ensuring that each branch's dependencies can be cached and retrieved independently. This is intended behavior in setup-node. You can check more on caches in the following links: Caching global packages data Caching packages data actions/cache#cache-scopes GitHub documentation on caching dependencies

"Workflow runs can restore caches created in either the current branch or the default branch (usually main)."

We want an option to store and restore all caches on the default branch only to greatly enhance cache reusability. As is now, in current structure, due to the ephemeral nature of branches, the effective rate of my caches being reused are maybe 10-30%, when they could be at least 50% and likely significantly higher across branches.

[mahabaleshwars]

Hello @curtdept, could you please help us by providing the public repro link for reproducing the issue? This will help us investigate further and resolve it more efficiently.

[mahabaleshwars]

Hi @curtdept, just checking in to see if you could provide the public repro link needed to reproduce the issue. Thanks!

[mahabaleshwars]

Hi @curtdept,Just a friendly reminder to check if you could provide the public repro link needed to reproduce the issue. Thanks!

[mahabaleshwars]

Hi @curtdept, I will be closing this issue due to inactivity for the time being. Should you need further assistance or if the issue arises again, please feel free to reopen this issue or reach out to us for any other concerns.