Error in post-job step of CacheV2 task when caching yarn packages #12892

c-eliasson · 2020-05-13T15:01:55Z

Required Information

Entering this information will route you directly to the right team and expedite traction.

Question, Bug, or Feature?
Type: Bug

Enter Task Name: CacheV2 (https://github.com/microsoft/azure-pipelines-tasks/tree/master/Tasks/CacheV2)

Environment

Server - Azure Pipelines
- Account name: markethype
- Team project name: MarketHype
- Build ID: 1563
- URL: https://dev.azure.com/markethype/MarketHype/_build/results?buildId=1563
Agent - Hosted:
- Agent Queue name: Azure Pipelines (https://dev.azure.com/markethype/MarketHype/_settings/agentqueues?queueId=17&view=jobs)

Issue Description

Using the CacheV2 task to cache Yarn packages, we're getting an error in the post-job step of the cache task.

The task is defined exactly as in the docs.

variables:
  YARN_CACHE_FOLDER: $(Pipeline.Workspace)/.yarn

steps:
- task: Cache@2
  inputs:
    key: 'yarn | "$(Agent.OS)" | yarn.lock'
    restoreKeys: |
       yarn | "$(Agent.OS)"
    path: $(YARN_CACHE_FOLDER)
  displayName: Cache Yarn packages

We're using the vs2017-win2016 image.

Error logs

Starting: Cache Yarn packages
==============================================================================
Task         : Cache
Description  : Cache files between runs
Version      : 2.0.0
Author       : Microsoft Corporation
Help         : https://aka.ms/pipeline-caching-docs
==============================================================================
Resolving key:
 - yarn                         [string]
 - "Windows_NT"                 [string]
 - common/config/rush/yarn.lock [file] --> 0551FA989D7CDCB2101F1397E8E7C5468396D4C72935548B6A74BEB126F0296F
Resolved to: yarn|"Windows_NT"|57F7K6kT3NtH/ZuSuWsxueTn4wCxuYls973WEnqCJgE=
Information, ApplicationInsightsTelemetrySender will correlate events with X-TFS-Session 06101a50-81b0-41b6-ab13-b130ee5c98ae
Information, Getting a pipeline cache artifact with one of the following fingerprints:
Information, Fingerprint: `yarn|"Windows_NT"|57F7K6kT3NtH/ZuSuWsxueTn4wCxuYls973WEnqCJgE=`
Information, There is a cache miss.
tar: D\:\\a\\1\\.yarn: Cannot open: No such file or directory
tar: Error is not recoverable: exiting now
Information, ApplicationInsightsTelemetrySender correlated 1 events with X-TFS-Session 06101a50-81b0-41b6-ab13-b130ee5c98ae
##[error]Process returned non-zero exit code: 2
Finishing: Cache Yarn packages

The text was updated successfully, but these errors were encountered:

jmaragon · 2020-06-10T11:08:08Z

Same issue here, but this time with npm

Starting: Cache npm
Task : Cache
Description : Cache files between runs
Version : 2.0.1
Author : Microsoft Corporation
Help : https://aka.ms/pipeline-caching-docs
Resolving key:

npm [string]
"Windows_NT" [string]
./product/package.json [file] --> 66FF0092057590D58252E7A1365E9520D41F83083CE0C8C1DA0E038DC4C2D9D1
Resolved to: npm|"Windows_NT"|2DZ3WcfStD108j8qipq/0v97TF5eyy6fpvtPYfpDrZM=
Information, ApplicationInsightsTelemetrySender will correlate events with X-TFS-Session xxxxxxxxxx-1282-485c-a6f4-b92a59ed4955
Information, Getting a pipeline cache artifact with one of the following fingerprints:
Information, Fingerprint: npm|"Windows_NT"|2DZ3WcfStD108j8qipq/0v97TF5eyy6fpvtPYfpDrZM=
Information, There is a cache miss.
tar: Cannot change to directory E:\Agents\3\283\s\node_modules: No such file or directory
tar: Error is not recoverable: exiting now
Information, ApplicationInsightsTelemetrySender correlated 1 events with X-TFS-Session XXXXXXXX282-485c-a6f4-b92a59ed4955
##[error]Process returned non-zero exit code: 2
Finishing: Cache npm

bradringel · 2020-06-10T13:08:12Z

Same issue here on a hosted linux agent

2020-06-09T21:30:32.6150811Z ##[section]Starting: Cache yarn packages
2020-06-09T21:30:32.6163777Z ==============================================================================
2020-06-09T21:30:32.6164193Z Task         : Cache
2020-06-09T21:30:32.6164523Z Description  : Cache files between runs
2020-06-09T21:30:32.6164828Z Version      : 2.0.1
2020-06-09T21:30:32.6165145Z Author       : Microsoft Corporation
2020-06-09T21:30:32.6165519Z Help         : https://aka.ms/pipeline-caching-docs
2020-06-09T21:30:32.6165941Z ==============================================================================
2020-06-09T21:30:32.9975064Z Resolving key:
2020-06-09T21:30:33.0154988Z  - yarn      [string]
2020-06-09T21:30:33.0162685Z  - "Linux"   [string]
2020-06-09T21:30:33.0743802Z  - yarn.lock [file] --> EA85735E476B7CE9FF55138A52CE89B7299F92856B621075773F035440C87D93
2020-06-09T21:30:33.0785039Z Resolved to: yarn|"Linux"|QlvZU5gmusF+bBuImFOlW68Qf4aNmdJ1pmhH2LChb5o=
2020-06-09T21:30:33.6622757Z Information, ApplicationInsightsTelemetrySender will correlate events with X-TFS-Session c6edd63d-1238-44dd-a471-3439ac2f402e
2020-06-09T21:30:33.6861181Z Information, Getting a pipeline cache artifact with one of the following fingerprints:
2020-06-09T21:30:33.6862267Z Information, Fingerprint: `yarn|"Linux"|QlvZU5gmusF+bBuImFOlW68Qf4aNmdJ1pmhH2LChb5o=`
2020-06-09T21:30:34.2316763Z Information, There is a cache miss.
2020-06-09T21:30:34.2834153Z tar: /home/vsts/work/1/.yarn: Cannot open: No such file or directory
2020-06-09T21:30:34.2834821Z tar: Error is not recoverable: exiting now
2020-06-09T21:30:34.5635719Z Information, ApplicationInsightsTelemetrySender correlated 1 events with X-TFS-Session c6edd63d-1238-44dd-a471-3439ac2f402e
2020-06-09T21:30:34.5689658Z ##[error]Process returned non-zero exit code: 2
2020-06-09T21:30:34.5844564Z ##[section]Finishing: Cache yarn packages```

stevewarsa · 2020-06-19T15:50:26Z

Same issue here with yarn - it is in some autogenerated task after my WAR deploy is done - the task is called "Post-job: Cache":

tar: /blah/blahgent/_work/1/s//node_modules, !/node_modules/**/node_modules: Cannot open: No such file or directory
tar: Error is not recoverable: exiting now
Information, ApplicationInsightsTelemetrySender correlated 1 events with X-TFS-Session 18de2057-e96c-4ff9-9091-595b9595f5cf
##[error]Process returned non-zero exit code: 2
Finishing: Cache

Any clue how to solve?

lesterchan · 2020-06-24T14:51:04Z

Came in here to +1 to this, below is my config:

variables:
  - name: yarnCacheFolder
    value: $(Pipeline.Workspace)/.yarn

  - task: Cache@2
    displayName: Cache Yarn Packages
    inputs:
      key: yarn | $(Agent.OS) | yarn.lock
      restoreKeys: |
        yarn | $(Agent.OS)
        yarn
      path: $(yarnCacheFolder)

Post-job: Cache Yarn Packages will throw an error

/home/vsts/work/1/.yarn: Cannot open: No such file or directory

jfheins · 2020-08-04T06:34:27Z

For me it works, but I have set the directory directly because I found no suitable pipeline variable:

      - task: Cache@2
        inputs:
          key: 'yarn | "$(Agent.OS)" | Frontend/.../yarn.lock'
          restoreKeys: yarn | "$(Agent.OS)"
          path: /home/vsts/.cache/yarn
        displayName: Cache Yarn packages

I'm using a hosted agent (ubuntu-18.04); maybe this works for you as well.
One can print the relevant directory for yarn with yarn cache dir

jotatoledo · 2020-08-05T11:37:25Z

@jfheins thanks for the hint.

Current configuration works for me in cloud-hosting:

pool:
  vmImage: 'ubuntu-latest'
variables:
 - name: YARN_CONFIG_CACHE
    value: /home/vsts/.cache/yarn/v6
...
- task: Cache@2
  displayName: Cache node modules
  inputs:
    key: 'yarn | "$(Agent.OS)" | yarn.lock'
    restoreKeys: |
      yarn | "$(Agent.OS)"
    path: $(YARN_CONFIG_CACHE)

github-actions · 2021-02-01T12:14:16Z

This issue is stale because it has been open for 180 days with no activity. Remove the stale label or comment on the issue otherwise this will be closed in 5 days

bradringel · 2021-02-02T15:47:05Z

Bumping because this is still happening and I don't think it should be closed

sheldonlynn · 2021-02-03T02:23:49Z

I was running into this exact issue and fixed it by adding YARN_CACHE_FOLDER to the pipeline variables.

I had the Cache task as part of a task group and in the pipeline that uses the task group, there is a required field that asks for the value of YARN_CACHE_FOLDER which is used as 'path' for the Cache task. Filled in the field with $(Pipeline.Workspace)/.yarn but the post-job kept failing with same error as above.

Once I also defined YARN_CACHE_FOLDER=$(Pipeline.Workspace)/.yarn in the pipeline variables, the post-job stopped failing.

rfink · 2021-02-16T22:51:00Z

Bump - this actually just started happening on a build that hasn't changed since December.

loxy · 2021-04-23T13:14:54Z

Same here for Maven:

variables:
  IS_MASTER_OR_DEVELOP: $[in(variables['Build.SourceBranch'], 'refs/heads/master', 'refs/heads/develop')]
  MAVEN_CACHE_FOLDER: $(Pipeline.Workspace)/.m2/repository
  MAVEN_OPTS: '-Dmaven.repo.local=$(MAVEN_CACHE_FOLDER)'

pool:
  vmImage: 'ubuntu-latest'

steps:
  - task: Cache@2
    displayName: "Cache Maven local repo"
    inputs:
      key: 'maven | "$(Agent.OS)" | **/pom.xml'
      restoreKeys: |
        maven | "$(Agent.OS)"
        maven
      path: $(MAVEN_CACHE_FOLDER)

  - task: Maven@3
    displayName: "Publish artifacts"
    condition: eq(variables.IS_MASTER_OR_DEVELOP, true)
    inputs:
      mavenPomFile: './pom.xml'
      mavenOptions: '-Xmx3072m $(MAVEN_OPTS)'
      mavenAuthenticateFeed: true
      goals: 'deploy'

kevinpiske · 2021-09-28T18:56:07Z

Same as @loxy

sondretj · 2021-10-13T08:23:15Z

I'm not sure why the pipeline I created started throwing the /home/vsts/work/1/.yarn: Cannot open: No such file or directory error as I use the exact same setup other places, with $(Pipeline.Workspace)/.yarn. I tried using different vmImages with no luck. Here's how I solved it for the ones interested.

          - bash: echo "##vso[task.setvariable variable=yarnCacheDir;]$(yarn cache dir)"
            displayName: Set yarn cache dir variable
          - task: Cache@2
            inputs:
              key: '"yarn" | "$(Agent.OS)" | yarn.lock'
              restoreKeys: |
                yarn | "$(Agent.OS)"
              path: $(yarnCacheDir)
            displayName: Cache Yarn packages

pradpant · 2021-10-18T14:30:14Z

Same here for Maven:

variables:
  IS_MASTER_OR_DEVELOP: $[in(variables['Build.SourceBranch'], 'refs/heads/master', 'refs/heads/develop')]
  MAVEN_CACHE_FOLDER: $(Pipeline.Workspace)/.m2/repository
  MAVEN_OPTS: '-Dmaven.repo.local=$(MAVEN_CACHE_FOLDER)'

pool:
  vmImage: 'ubuntu-latest'

steps:
  - task: Cache@2
    displayName: "Cache Maven local repo"
    inputs:
      key: 'maven | "$(Agent.OS)" | **/pom.xml'
      restoreKeys: |
        maven | "$(Agent.OS)"
        maven
      path: $(MAVEN_CACHE_FOLDER)

  - task: Maven@3
    displayName: "Publish artifacts"
    condition: eq(variables.IS_MASTER_OR_DEVELOP, true)
    inputs:
      mavenPomFile: './pom.xml'
      mavenOptions: '-Xmx3072m $(MAVEN_OPTS)'
      mavenAuthenticateFeed: true
      goals: 'deploy'

@loxy Any Suggestion ? I'm also stuck here .

loxy · 2021-10-18T15:14:25Z

@pradpant No, didn't found a solution

adriansergheev · 2021-11-08T14:29:39Z

bump!

daerogami · 2021-11-16T02:18:40Z

This was working in a single stage pipeline, once I added another stage that causes this to get called twice, I now get the error with the cache post-job
Local agent running Win10 pro

Starting: Cache Yarn packages
==============================================================================
Task         : Cache
Description  : Cache files between runs
Version      : 2.0.1
Author       : Microsoft Corporation
Help         : https://aka.ms/pipeline-caching-docs
==============================================================================
Resolving key:
 - yarn                          [string]
 - "Windows_NT"                  [string]
 - src/MyProject.Web.Host/yarn.lock [file] --> A51A47DAA3D812F7DBE1028017AC2CA60169EA036A0C8BDD2DE0B6DADB8B21FE
Resolved to: yarn|"Windows_NT"|+Z30KjYR/75Haaf6MF5eU1mX/TrlZ7I5cG6FuT0QJfE=
ApplicationInsightsTelemetrySender will correlate events with X-TFS-Session 7d701a80-cde5-40c0-abed-7f66814cc117
Getting a pipeline cache artifact with one of the following fingerprints:
Fingerprint: `yarn|"Windows_NT"|+Z30KjYR/75Haaf6MF5eU1mX/TrlZ7I5cG6FuT0QJfE=`
There is a cache miss.
tar: could not chdir to 'C:\agent\_work\5\.yarn'

ApplicationInsightsTelemetrySender correlated 1 events with X-TFS-Session 7d701a80-cde5-40c0-abed-7f66814cc117
##[error]Process returned non-zero exit code: 1
Finishing: Cache Yarn packages

nadworny · 2021-11-26T09:42:13Z

Same here. Seems the cache plugin is broken. Found another solution here: https://xebia.com/blog/caching-your-node-modules-in-azure-devops/

github-actions · 2022-05-25T12:13:02Z

This issue is stale because it has been open for 180 days with no activity. Remove the stale label or comment on the issue otherwise this will be closed in 5 days

loxy · 2022-05-25T17:57:21Z

/remove-stale

TiceWise · 2022-09-21T07:24:04Z

I'm also running into this issue, also with two consecutive @cache2 tasks (one for $(Pipeline.Workspace)/.npm
and one for $(Pipeline.Workspace)/.cache/Cypress). Also tried with CacheBeta@1, same issue.
The first post-job executed is Cache Cypress (not npm which is the first cache task), which fails with:

There is a cache miss.
tar: /agent/_work/1/.cache/Cypress: Cannot open: No such file or directory
tar: Error is not recoverable: exiting now
ApplicationInsightsTelemetrySender correlated 1 events with X-TFS-Session ***-**-**-**-***
##[error]Process returned non-zero exit code: 2
Finishing: Cache Cypress binary

Bengkel · 2022-09-29T09:05:24Z

I think caching will only work on a self hosted agent?

jfheins · 2022-10-11T11:02:18Z

I think caching will only work on a self hosted agent?

No, this task is especially good for the Azure hosted agents. In a self hosted scenario, most of the caching works without this task. (For example: yarn has a machine wide global cache that persists in a self hosted agent)

The tricky thing is to get the cache directory right.
For example, an Azure agent mighthave the yarn cache in "/home/vsts/.cache/yarn" while a self hosted might need "/home/AzDevOps/.cache/yarn"

I'd recommend to include a step that logs out the cache directories for easier troubleshooting:

      - script: |
          node -v
          npm -v
          yarn -v
          yarn cache dir
          npm config get cache
        displayName: "Log versions & paths"

esoxjem · 2023-02-08T15:52:52Z

Solution

So, the issue seems to be that $(Pipeline.Workspace) does not return the home directory and hence the step can't find the build folders that are typically in the home dir. Take the output from the command @jfheins posted and add that to the variable directly.

# example for .npm

# replace this
variables:
  CACHE_FOLDER: $(Pipeline.Workspace)/.npm

# with this
variables:
  CACHE_FOLDER: /home/vsts/.npm

Result

Here's the result of the successful run after making the change

vixero-dev · 2023-05-16T15:09:52Z

I have the same error, but with pnpm. This is the yaml from the docs:

variables:
  pnpm_config_cache: $(Pipeline.Workspace)/.pnpm-store

steps:
  - task: Cache@2
    inputs:
      key: 'pnpm | "$(Agent.OS)" | pnpm-lock.yaml'
      path: $(pnpm_config_cache)
    displayName: Cache pnpm

  - script: |
      corepack enable
      corepack prepare pnpm@latest-8 --activate
      pnpm config set store-dir $(pnpm_config_cache)
    displayName: "Setup pnpm"

  - script: |
      pnpm install
      pnpm run build
    displayName: "pnpm install and build"

johnterickson · 2023-05-16T21:01:34Z

@vixero-dev Can you share your the log from the task failure?

A lot of these issues look like folks either aren't:

Setting the correct variable (as defined by the package manager) at the pipeline-level to configure where the package manager should cache things
Actually invoking the package manager (the cache task does not do this)

vixero-dev · 2023-05-17T13:26:10Z

@johnterickson you are correct. The variable wasn't being set.

After changing this part:

  - script: |
      corepack enable
      corepack prepare pnpm@latest-8 --activate
      pnpm config set store-dir $(pnpm_config_cache)
    displayName: "Setup pnpm"

to this:

  - script: |
      corepack enable
      corepack prepare pnpm@latest-8 --activate
    displayName: "setup pnpm"

  - script: pnpm config set store-dir $(pnpm_config_cache)
    displayName: "setup pnpm config"

I stopped experiencing this issue.

dombarnes · 2023-11-03T14:32:05Z

I had some similar issues that ended up being due to agents getting different versions of yarn.
I added

- bash: |
        corepack enable
        echo "Requesting Yarn version $(YARN_VERSION)"
        yarn set version $(YARN_VERSION)
        echo "Running Yarn $(yarn --version)"

along with configuring the YARN_VERSION variable to 3.6.4 in my case. I found out some agents would give me 1.22 sometimes in some build steps and 3.6 in others. This helped me diagnose this.

yooakim · 2023-11-23T17:11:50Z

Also experience this issue when following Microsoft's documentation. It is an issue with the paths. This blog posts is a great dive into how to fix this and the Cache task.

nickshanks347 · 2024-02-14T17:06:10Z

Try adding an s between $(Pipeline.Workspace) and .yarn, so variable goes from:

YARN_CACHE_FOLDER: $(Pipeline.Workspace)/.yarn

To:

YARN_CACHE_FOLDER: $(Pipeline.Workspace)/s/.yarn

This is because when using checkout: self, it checks out the repository to $(Pipeline.Workspace)/s, as is explained here. Unless your using magic, your .yarn folder should be inside the repository.

github-actions bot assigned kmkumaran and johnterickson May 13, 2020

github-actions bot added Area: PipelineCaching Area: Release bug labels May 13, 2020

johnterickson assigned fadnavistanmay May 13, 2020

github-actions bot added the stale label Feb 1, 2021

github-actions bot removed the stale label Feb 2, 2021

kmkumaran removed their assignment Feb 15, 2021

kmkumaran removed the Area: Release label Feb 15, 2021

github-actions bot added the stale label May 25, 2022

github-actions bot removed the stale label May 25, 2022

nickshanks347 mentioned this issue Feb 14, 2024

Update Pipeline Caching doc to clarify s needed for checkout: self. MicrosoftDocs/azure-devops-docs#14001

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Error in post-job step of CacheV2 task when caching yarn packages #12892

Error in post-job step of CacheV2 task when caching yarn packages #12892

c-eliasson commented May 13, 2020

jmaragon commented Jun 10, 2020 •

edited

bradringel commented Jun 10, 2020

stevewarsa commented Jun 19, 2020 •

edited

lesterchan commented Jun 24, 2020

jfheins commented Aug 4, 2020

jotatoledo commented Aug 5, 2020 •

edited

github-actions bot commented Feb 1, 2021

bradringel commented Feb 2, 2021

sheldonlynn commented Feb 3, 2021

rfink commented Feb 16, 2021

loxy commented Apr 23, 2021 •

edited

kevinpiske commented Sep 28, 2021

sondretj commented Oct 13, 2021 •

edited

pradpant commented Oct 18, 2021

loxy commented Oct 18, 2021

adriansergheev commented Nov 8, 2021

daerogami commented Nov 16, 2021

nadworny commented Nov 26, 2021

github-actions bot commented May 25, 2022

loxy commented May 25, 2022

TiceWise commented Sep 21, 2022

Bengkel commented Sep 29, 2022

jfheins commented Oct 11, 2022 •

edited

esoxjem commented Feb 8, 2023

vixero-dev commented May 16, 2023

johnterickson commented May 16, 2023

vixero-dev commented May 17, 2023

dombarnes commented Nov 3, 2023

yooakim commented Nov 23, 2023

nickshanks347 commented Feb 14, 2024

Error in post-job step of CacheV2 task when caching yarn packages #12892

Error in post-job step of CacheV2 task when caching yarn packages #12892

Comments

c-eliasson commented May 13, 2020

Required Information

Environment

Issue Description

Error logs

jmaragon commented Jun 10, 2020 • edited

Same issue here, but this time with npm

bradringel commented Jun 10, 2020

Same issue here on a hosted linux agent

stevewarsa commented Jun 19, 2020 • edited

lesterchan commented Jun 24, 2020

jfheins commented Aug 4, 2020

jotatoledo commented Aug 5, 2020 • edited

github-actions bot commented Feb 1, 2021

bradringel commented Feb 2, 2021

sheldonlynn commented Feb 3, 2021

rfink commented Feb 16, 2021

loxy commented Apr 23, 2021 • edited

kevinpiske commented Sep 28, 2021

sondretj commented Oct 13, 2021 • edited

pradpant commented Oct 18, 2021

loxy commented Oct 18, 2021

adriansergheev commented Nov 8, 2021

daerogami commented Nov 16, 2021

nadworny commented Nov 26, 2021

github-actions bot commented May 25, 2022

loxy commented May 25, 2022

TiceWise commented Sep 21, 2022

Bengkel commented Sep 29, 2022

jfheins commented Oct 11, 2022 • edited

esoxjem commented Feb 8, 2023

Solution

Result

vixero-dev commented May 16, 2023

johnterickson commented May 16, 2023

vixero-dev commented May 17, 2023

dombarnes commented Nov 3, 2023

yooakim commented Nov 23, 2023

nickshanks347 commented Feb 14, 2024

jmaragon commented Jun 10, 2020 •

edited

stevewarsa commented Jun 19, 2020 •

edited

jotatoledo commented Aug 5, 2020 •

edited

loxy commented Apr 23, 2021 •

edited

sondretj commented Oct 13, 2021 •

edited

jfheins commented Oct 11, 2022 •

edited