CacheBeta task: "The path provided is invalid" for Maven repository

[rmetzger]

Required Information

Question, Bug, or Feature?
Type: Question

Enter Task Name: CacheBeta@0

I'm trying to use the cache in a Maven project.
These are the relevant parts of the azure-pipelines.yml file:

variables:
  MAVEN_CACHE_FOLDER: /home/user/.m2/repository
  MAVEN_OPTS: '-Dmaven.repo.local=$(MAVEN_CACHE_FOLDER)'

resources:
  containers:
  - container: flink-build-container
    image: rmetzger/flink-ci:latest

jobs:
- job: main
  steps:
  - task: CacheBeta@0
    inputs:
      key: maven | **/pom.xml
      path: /home/user/.m2/repository
    displayName: Cache Maven local repo
  - task: Maven@3
   ...

I'm getting the following unexpected error:

##[error]The path provided is invalid.

This is a build with that error: https://dev.azure.com/rmetzger/Flink/_build/results?buildId=86&view=logs&j=a6fe42f4-2b81-59e8-5c7c-1a0db487583c
This is the full azure-pipelines.yml file from that build: https://github.com/rmetzger/flink/blob/3d201d6abd020adad68e3bc842b4c4b7e0d3d260/azure-pipelines.yml

I have put the path directly, instead of using the variable as suggested in the documentation to rule out any issues with the variable system.

[rmetzger]

In this commit and build, I'm using the configuration exactly as in the documentation.

[ptr1120]

I am having the same problem: ##[error]The path provided is invalid.. The caching task crashes on post-job on other agents (osx, linux) then windows.

[reyem]

Same issue here. Tried also to set the variables on job level rather than pipeline, but did not help. Further I tried CacheBeta@1. But unfortunately not success.

[jhchill666]

Exactly the same here. Build Windows first - restores cache and updates it post-job. Then builds Mac but crashes post-job.

To confound things further, each step uses a build.yml template, so there are definitely no subtle differences between the configuration of each step.

[jhchill666]

Yml extract:

variables:
  ELECTRON_CACHE: '$(Pipeline.Workspace)/.cache/electron'

- task: CacheBeta@0
  displayName: 'cache electron binaries'
  continueOnError: true
  inputs:
    key: $(Agent.OS) | $(Build.SourcesDirectory)/build/config/.electron-version
    path: $(ELECTRON_CACHE)

Working Windows console output:

##[section]Starting: cache electron-builder binaries
==============================================================================
Task         : Cache (Beta)
Description  : Cache files between runs
Version      : 0.1.0
Author       : Microsoft Corporation
Help         : https://aka.ms/pipeline-caching-docs
==============================================================================
Resolving key `Windows_NT | D:\a\1\s/build/config/.electron-version`...
File hashes summarized as `3QR6O6WKl2Qis61SqF7vS0jD6BpGuzRAPEJHlVWUVAw=` from BASE64(SHA256(`
SHA256(s\build\config\.electron-version)=[15]94009CC45ADE54B952D549DB38773DB5F900712B437DCBB756F15A6AE5C56992`))
Resolved to `Windows_NT|3QR6O6WKl2Qis61SqF7vS0jD6BpGuzRAPEJHlVWUVAw=`.
Information, Getting a pipeline cache artifact with one of the following fingerprints:
Information, Fingerprint: `Windows_NT|3QR6O6WKl2Qis61SqF7vS0jD6BpGuzRAPEJHlVWUVAw=`
Information, There is a cache hit: `Windows_NT|3QR6O6WKl2Qis61SqF7vS0jD6BpGuzRAPEJHlVWUVAw=`
Information, Used scope: 20;9f10842b-2d58-4981-b1be-77e09026a430;refs/heads/master;pulse360/pulse360-app
Cache with fingerprint `Windows_NT|3QR6O6WKl2Qis61SqF7vS0jD6BpGuzRAPEJHlVWUVAw=` already exists.
##[section]Finishing: cache electron-builder binaries

Non-working Mac console output:

##[section]Starting: cache electron-builder binaries
==============================================================================
Task         : Cache (Beta)
Description  : Cache files between runs
Version      : 0.1.0
Author       : Microsoft Corporation
Help         : https://aka.ms/pipeline-caching-docs
==============================================================================
Resolving key `Darwin | /Users/vsts/agent/2.155.1/work/1/s/build/config/.electron-version`...
File hashes summarized as `8WTLW4tx5uP1HrnScuYIKbt6I3vvMcEih0K0L8ClrFc=` from BASE64(SHA256(`
SHA256(s/build/config/.electron-version)=[14]BED1DD13266C394F6F498FFC1D512CDCE5D852524169AAAB31EFACE7FC97445C`))
Resolved to `Darwin|8WTLW4tx5uP1HrnScuYIKbt6I3vvMcEih0K0L8ClrFc=`.
Information, Getting a pipeline cache artifact with one of the following fingerprints:
Information, Fingerprint: `Darwin|8WTLW4tx5uP1HrnScuYIKbt6I3vvMcEih0K0L8ClrFc=`
Information, There is a cache miss.
Information, DedupManifestArtifactClient will correlate http requests with X-TFS-Session e06d23a6-fb05-4fa3-825d-dec4d4cfef39
##[error]The path provided is invalid.
##[section]Finishing: cache electron-builder binaries

[ifiokjr]

I'm having the exact same issue, but following the yarn task instructions. Here's a link to the failing build.

My config looks like this

steps:
  - task: NodeTool@0
    inputs:
      versionSpec: '10.x'
    displayName: 'Install Node.js'
  - task: CacheBeta@0
    inputs:
      key: yarn | $(Agent.OS) | yarn.lock
      path: $(Pipeline.Workspace)/.yarn
    displayName: Cache Yarn packages
  - bash: yarn audit
    displayName: 'Check for vulnerabilities'
  - bash: yarn ci
    displayName: 'Install dependencies'

[jhchill666]

An update on this: When I run the build with system.debug=true it sheds further light on the subject.

##[debug]Processed: ##vso[telemetry.publish area=AzurePipelinesAgent;feature=PipelineCache]{"FileCount":"0","PlanId":"3f5e1632-9c42-4bd2-98ac-1b88bb226502","JobId":"e3fb9182-1a97-5c79-f611-2faaac30494d","TaskInstanceId":"f7976f0b-ce73-5111-8459-8ff18465114e","CacheResult":"Miss","ActionDurationMs":"1369","ActionName":"PipelineCache.RestoreCache","ActionResult":"Success","AttemptNumber":"1","ItemCount":"0","Level":"ThirdParty","CreatedUtcNow":"2019-09-25T15:14:09.2465090Z","BaseAddress":"https://vsblobprodcus3.vsblob.visualstudio.com/Ad185458f-f87a-4b6f-a504-e55f3b32d741/","X_TFS_Session":"f267cb85-f6ac-4743-ad34-29634ca93d1f","DeploymentEnvironment":"PRODUCTION","DeploymentEnvironmentIsProduction":"True","VSOAccount":"vsblobprodcus3","OSName":"Darwin","OSVersion":"root:xnu-4903.271.2~2/RELEASE_X86_64","FrameworkDescription":".NET Core ","ProcessName":"Agent.PluginHos","DotNetReleaseDword":"-1","Version":"17.156.29126.0","ExceptionCount":"0"}
Information, DedupManifestArtifactClient will correlate http requests with X-TFS-Session f267cb85-f6ac-4743-ad34-29634ca93d1f
##[error]The path provided is invalid.

I notice that FileCount is zero. Is the task simply failing as it can't find any files to cache? This would make sense in my case, as there may not always be files to be added to the cache.

[marcinzi]

Also hit this issue today.
Yarn cache on Linux hosted pipeline.
Tried @0 and @1 version of CacheBeta task.
Also tried to specify absolute path to yarn.lock file with use of Source folder variable, instead of just saying yarn.lock

[reyem]

In my case the issue could be solved. I was wrongly assuming that MAVEN_OPTS will always be picked up by the maven execution. It turned out this is not the case so I had to pass this explicitly to the options in my pipeline:

- task: Maven@3
  inputs:
      ...
      options: '$(MAVEN_OPTS)'
     ...

After this, the documented configuration works.

[danielwertheim]

Same issue when following NuGet docs.

Worked fine with a solution having one project. Added a new project to the solution and then it started to fail with the error The path provided is invalid.

2019-10-17T17:28:57.3137452Z ##[error]The path provided is invalid.
2019-10-17T17:28:57.3147261Z ##[debug]Processed: ##vso[task.logissue type=error;]The path provided is invalid.
2019-10-17T17:28:57.3162646Z ##[debug]Processed: ##vso[task.complete result=Failed;]
2019-10-17T17:28:57.3213601Z ##[debug]   at Microsoft.VisualStudio.Services.BlobStore.WebApi.DedupManifestArtifactClient.PublishAsync(String fullPath, ArtifactPublishOptions artifactPublishOptions, String manifestFileOutputPath, CancellationToken cancellationToken)
   at Agent.Plugins.PipelineCache.PipelineCacheServer.<>c__DisplayClass0_1.<<UploadAsync>b__3>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at Microsoft.VisualStudio.Services.BlobStore.Common.Telemetry.BlobStoreClientTelemetry.MeasureActionAsync[TResult](BlobStoreTelemetryRecord record, Func`1 actionAsync)
   at Agent.Plugins.PipelineCache.PipelineCacheServer.UploadAsync(AgentTaskPluginExecutionContext context, Fingerprint fingerprint, String path, CancellationToken cancellationToken, String preferredContentFormat)
   at Agent.Plugins.PipelineCache.SavePipelineCacheV0.ProcessCommandInternalAsync(AgentTaskPluginExecutionContext context, Fingerprint fingerprint, Func`1 restoreKeysGenerator, String path, CancellationToken token)
   at Agent.Plugins.PipelineCache.PipelineCacheTaskPluginBase.RunAsync(AgentTaskPluginExecutionContext context, CancellationToken token)
   at Agent.Plugins.PipelineCache.SavePipelineCacheV0.RunAsync(AgentTaskPluginExecutionContext context, CancellationToken token)
   at Agent.PluginHost.Program.Main(String[] args)

[gpad]

I have the the same issue this feature simple doesn't work ...

It's also quite difficult understand which folder the task is checking ...

##[debug]Evaluating condition for step: 'CacheBeta'
##[debug]Evaluating: AlwaysNode()
##[debug]Evaluating AlwaysNode:
##[debug]=> True
##[debug]Result: True
Starting: CacheBeta
==============================================================================
Task         : Cache (Beta)
Description  : Cache files between runs
Version      : 0.1.0
Author       : Microsoft Corporation
Help         : https://aka.ms/pipeline-caching-docs
==============================================================================
Resolving key: manpremo_cache_deps
 - manpremo_cache_deps [string]
Resolved to: manpremo_cache_deps
##[debug]Dedup parallelism: 192
Information, ApplicationInsightsTelemetrySender will correlate events with X-TFS-Session 52e4b8bb-1635-43c2-b931-cd6288f64f36
Information, Getting a pipeline cache artifact with one of the following fingerprints:
Information, Fingerprint: `manpremo_cache_deps`
Information, There is a cache miss.
##[debug]Processed: ##vso[telemetry.publish area=AzurePipelinesAgent;feature=PipelineCache]{"FileCount":"0","PlanId":"c0f85dd8-4d11-43d4-8fff-5fc73999be43","JobId":"4c57a26c-f146-51a2-2fc0-20606ae5602d","TaskInstanceId":"c1dd596c-f587-57b4-294a-32b5bb67f050","CacheResult":"Miss","ActionDurationMs":"709","ActionName":"PipelineCache.RestoreCache","ActionResult":"Success","AttemptNumber":"1","ItemCount":"0","Level":"ThirdParty","CreatedUtcNow":"2019-10-21T10:21:37.0550051Z","SentUtcNow":"2019-10-21T10:21:37.7773138Z","BaseAddress":"https://vsblobprodsu6weu.vsblob.visualstudio.com/Af22e5165-9a5c-47fe-8f6b-d0fd92abdc9d/","X_TFS_Session":"52e4b8bb-1635-43c2-b931-cd6288f64f36","DeploymentEnvironment":"PRODUCTION","DeploymentEnvironmentIsProduction":"True","VSOAccount":"vsblobprodsu6weu","OSName":"Linux","OSVersion":"2019","FrameworkDescription":".NET Core ","ProcessName":"Agent.PluginHos","DotNetReleaseDword":"-1","Version":"17.158.29305.0","ExceptionCount":"0"}
Information, DedupManifestArtifactClient will correlate http requests with X-TFS-Session 52e4b8bb-1635-43c2-b931-cd6288f64f36
Information, ApplicationInsightsTelemetrySender correlated 3 events with X-TFS-Session 52e4b8bb-1635-43c2-b931-cd6288f64f36
##[error]The path provided is invalid.
##[debug]Processed: ##vso[task.logissue type=error;]The path provided is invalid.
##[debug]Processed: ##vso[task.complete result=Failed;]
##[debug]   at Microsoft.VisualStudio.Services.BlobStore.WebApi.DedupManifestArtifactClient.PublishAsync(String fullPath, ArtifactPublishOptions artifactPublishOptions, String manifestFileOutputPath, CancellationToken cancellationToken)
   at Agent.Plugins.PipelineCache.PipelineCacheServer.<>c__DisplayClass0_1.<<UploadAsync>b__2>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at Microsoft.VisualStudio.Services.BlobStore.Common.Telemetry.BlobStoreClientTelemetry.MeasureActionAsync[TResult](BlobStoreTelemetryRecord record, Func`1 actionAsync)
   at Agent.Plugins.PipelineCache.PipelineCacheServer.UploadAsync(AgentTaskPluginExecutionContext context, Fingerprint fingerprint, String path, CancellationToken cancellationToken, String preferredContentFormat)
   at Agent.Plugins.PipelineCache.SavePipelineCacheV0.ProcessCommandInternalAsync(AgentTaskPluginExecutionContext context, Fingerprint fingerprint, Func`1 restoreKeysGenerator, String path, CancellationToken token)
   at Agent.Plugins.PipelineCache.PipelineCacheTaskPluginBase.RunAsync(AgentTaskPluginExecutionContext context, CancellationToken token)
   at Agent.Plugins.PipelineCache.SavePipelineCacheV0.RunAsync(AgentTaskPluginExecutionContext context, CancellationToken token)
   at Agent.PluginHost.Program.Main(String[] args)

[rmetzger]

@fadnavistanmay sorry for pinging you directly. It seems that quite a few different users are affected by this. Can you take a look?

[gpad]

I have also seen that the cache doesn't keep the correct mode of the file so some file that are executable lost their mod ...

[fadnavistanmay]

Thanks for reporting @rmetzger - I'll take a look at this today.

[fadnavistanmay]

Hi @rmetzger / others. - I have set up a basic yaml e.gs for some tools. - https://github.com/fadnavistanmay/azure-pipelines-caching-yaml/blob/master/templates/java-with-maven.yml Could you please check and let us know if that helps.

Plus, you can see the build definition using - https://github.com/fadnavistanmay/azure-pipelines-caching-yaml .

• @mitchdenny - who is using Pipeline Caching for maven.

[danielwertheim]

@fadnavistanmay The key you are generating: https://github.com/fadnavistanmay/azure-pipelines-caching-yaml/blob/master/templates/asp.net-core.yml#L15 is unique for each build, right? And does not take the hash of the package.lock.json file into account as described here: https://docs.microsoft.com/en-us/azure/devops/pipelines/caching/?view=azure-devops#netnuget

How will that work? Will that not defeat the purpose of a cache since the key is not "the same" across builds sharing the same dependencies?

[fadnavistanmay]

Hi @danielwertheim - that's correct. That key is just for my testing purpose, Actually I have different jobs running, so it will be a cache hit in the next job, in the same build.

We are heads down in updating the docs/ and the sample yaml repo,(setting up builds to accept PRs, etc) - and it should be done by early next week. I'll circle back to this thread after that, and we can continue the discussion then :)

Thanks.

[rmetzger]

@fadnavistanmay:
Thanks a lot for your example. It seems that CacheBeta@1 is not compatible with the Maven@3 task.
This yml config file: https://github.com/rmetzger/flink/blob/cd2c71c54189a204e2b6dc721d7e9382edcd531c/azure-pipelines.yml
causes this error: https://dev.azure.com/rmetzger/Flink/_build/results?buildId=1443&view=logs&j=96e03a1a-0ac4-5903-83b1-91e6a9dce517

Using a script: mvn ... task works.
I want to use the Maven@3 task to get the test reporting.

[rmetzger]

Is it normal that cache restores are full of exceptions? https://dev.azure.com/rmetzger/5bd3ef0a-4359-41af-abca-811b04098d2e/_apis/build/builds/1535/logs/26
All the builds I've checked so far, which use the cache are full of those (I guess this could also explain the 4 minutes cache restore time).

[fadnavistanmay]

@rmetzger - I took a quick look at the yaml, I suspect the container might be behaving weirdly in terms of host/container path mapping. But, $(Pipeline.Workspace) is mapped to container, can you please give a quick try without it and see if it works.

"Is it normal that cache restores are full of exceptions?" -> Those exceptions are harmless and coming from Azure Storage. But that doesn't explain the 4 min it took at restore cache. Could you point us to the build which took 4 mins.

[rmetzger]

I have tried various things, all without real success. It seems that the files are not restored into the specified folder.

Abbreviated config:

variables:
  MAVEN_CACHE_FOLDER: /home/user/.m2/repository

...

jobs:
- job: runOnDefaultAgentPool
  steps:
  - script: ls -lisah /home/user/.m2/repository
  - task: CacheBeta@1
    inputs:
      key: $(CACHE_KEY)
      restoreKeys: $(CACHE_FALLBACK_KEY)
      path: $(MAVEN_CACHE_FOLDER)
      cacheHitVar: CACHE_RESTORED
    displayName: Cache Maven local repo
  - script: ls -lisah $(MAVEN_CACHE_FOLDER)

Leads to:

I would expect the files to be in /home/user/.m2/repository after the cache restore.

Full file: https://github.com/rmetzger/flink/blob/1029f71132c4b92e9575aa1c2421d3027ca1ece9/azure-pipelines.yml

---

In this build, the download takes more than 4 minutes: https://dev.azure.com/rmetzger/Flink/_build/results?buildId=1519&view=logs&j=96e03a1a-0ac4-5903-83b1-91e6a9dce517

---

In general, I believe there's a way of making the caching functionality work. It is probably a matter of my custom docker container, my own build servers and me not having enough time to fully look into the issue.
Since the use of the cache does not bring as much time advantage as hoped, I won't have time to focus on debugging this for now. If we decide to go with AZ pipelines, I will look into this further.
I'm saying this because I might reduce my attention to this ticket.

[johnterickson]

I have also seen that the cache doesn't keep the correct mode of the file so some file that are executable lost their mod ...

@gpad That is rolling out per #10925

@danielwertheim and @ifiokjr - could you file separate issues for problems you are seeing with yarn/npm/dotnet/nuget? I don't want those to be lost in this Maven discussion.

[johnterickson]

@rmetzger Since you are using containers, you can also keep an eye on: #11034

[aasim]

Hi @rmetzger - We have a new task out, MavenAuthenticate@0. The design philosophy behind this task is that we will authenticate all the repositories you describe in the task, and get out of the way leaving you fully in control of running your mvn commands in the pipeline as you see fit.

You mentioned above that running the mvn command directly worked for you, can you try using the authenticate task and running your custom mvn command to see if caching works as expected? Let me know and I can help you setup this command correctly.

You also mentioned that you use the Maven@3 command only to get test reporting, can you elaborate on that? There are other test reporting tasks that you can use to achieve the same results, you can ping me and I can help you set those up as well.

[rmetzger]

Thanks for your comments.
@johnterickson The container download time is really not a concern for us at the moment. But I'll still keep an eye on this.

@aasim: After I commented here, I had the same thought and I found the PublishTestResults@2 task. I will investigate this soon (I consider it a nice to have task)
I don't think I need the MavenAuthenticate@0 task, as we are are only relying on public dependencies.

[fadnavistanmay]

Hi folks,

Just wanted to circle back and check if @aasim 's fix resolved the issue for you @rmetzger .

Thanks

[johnterickson]

It looks like ##[debug]Processed: ##vso[task.logissue type=error;]The path provided is invalid. is being thrown when there are either:

1. No files
2. directory doesn't exist
3. Bad characters in the path

We should clarify in the exception

[rmetzger]

@fadnavistanmay Yes, I was able to resolve the issue.
+1 on improving the error message.

[fadnavistanmay]

Thanks @rmetzger .
@b-barthel is already working on improving the exception message :)

[winniehell]

I was wrongly assuming that MAVEN_OPTS will always be picked up by the maven execution. It turned out this is not the case so I had to pass this explicitly to the options in my pipeline:

(from #11259 (comment))

This behavior is now at least documented: MicrosoftDocs/azure-devops-docs#8142 (though I would prefer to have the Maven task respect the global variable)