Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Consider supporting caches relative to home dir #1400

Open
lread opened this issue May 16, 2024 · 1 comment
Open

Consider supporting caches relative to home dir #1400

lread opened this issue May 16, 2024 · 1 comment

Comments

@lread
Copy link

lread commented May 16, 2024

Thanks!

First, thanks for all the hard work and for a very useful action!

Intro

My tests run on macOS, Ubuntu, and Windows GitHub runners, so I was excited to explore the enableCrossOsArchive feature.

I'm finding it great except for one thing.

The natural location of my project's downloaded dependencies is relative to the home dir and includes, for example, an .m2 repository.

Cross-OS caching files relative to the home dir turns out to be awkward.

Details

As far as I understand it, the current cache action always saves and restores relative to the workspace dir.

I did some poking around on a GitHub Ubuntu runner.
My workspace dir is /home/runner/work/my-repo-name/my-repo-name,
and my home dir is /home/runner.
So if I am creating my cache with a path of ~/.m2 on Ubuntu, files will get archived as, for an example, ../../../.m2/repository/com/jcraft/jsch.agentproxy.usocket-jna/0.0.9/jsch.agentproxy.usocket-jna-0.0.9.pom.

This is all fine and dandy on Ubuntu and macOS because they share the same basic directory structure. But on Windows, the directory layout is different.

I did some poking around on a GitHub Windows runner.
My workspace dir is D:\a\my-repo-name\my-repo-name,
and my home dir is C:\Users\runneradmin.

So if I restore a Ubuntu runner created cache on a Windows runner, my cache will be restored to ../../.. which is D:\, but I really want to restore to my Windows home dir.

My Current Hack

I can run a step after restoring the cache to fix things up, but this is brittle and awkward:

    - name: Cache fixup for Windows
      if: ${{ matrix.os == 'windows' }}
      run: |
        mv ../../../.m2 ${USERPROFILE}
      shell: bash

Other Alternatives

I could figure out how to have my project dependencies downloaded to my workspace dir.
But this is also awkward, unnatural, and prone to human error.

Your Current Docs...

Do warn:

Avoid using directory pointers such as ${{ github.workspace }} or ~ (home) which eventually evaluate to an absolute path that does not match across OSs.

But I think it would be great to be able to cross-OS cache relative to ~.

What I'd Really Like

Is a way to save and restore a cache relative to the home dir.

One way to do this might be to introduce a new base-path input which would act as an optional override for the hard-coded workspace dir.

lread added a commit to clj-commons/etaoin that referenced this issue May 16, 2024
* ci: improve deps downloading and caching

Bring down deps faster (brought over from work done on rewrite-clj).

Enable cross OS caching for GitHub Actions deps.
This should allow Windows runner to benefit from setup job caching.

Closes #577

* ci: don't convert line endings on clone on Windows

We want consistent hashes for our cross-os caches.
Our hashes are based on file contents.
We need the file contents to be the same across OSes.

* ci: experiment with fixing cross-os cache for win

Restoring the cache on Windows is awkward, I've raised an issue actions/cache#1400, I'll adapt based on response.
@ruffsl
Copy link

ruffsl commented Jun 13, 2024

Yeah, this would be great! I encounter the same issues when sharing caches between containerized and non-containerized workflow jobs, given that the cache paths in containers could be arbitrary paths in the container file system. My current work around is to also include a similar cache dancing step to move restored cache back into the same relative position. However, this can still be problematic for non-containerized jobs, as the caches could be resorted to paths external to the current workspace, and thus bleed over into the host runner's OS. That can of course cause idempotent issues when the same runner is re-used for multiple jobs.

Even having the cache paths respect the relative working directory set for the job would be great ergonomic improvement:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants