Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Select GNU tar for caching if available on hosted runners #552

Open
dhadka opened this issue Aug 10, 2020 · 1 comment
Open

Select GNU tar for caching if available on hosted runners #552

dhadka opened this issue Aug 10, 2020 · 1 comment
Labels
bug Something isn't working

Comments

@dhadka
Copy link
Member

dhadka commented Aug 10, 2020

Describe the bug

BSD tar has been problematic for the caching action for several reasons:

  1. bsdtar does not handle recursive directory structures created by junction points - Tar fails on Windows after 15min due to symlink cycle cache#315

  2. bsdtar does not work with zstd compression, causing the process to hang - zstd compression hanging on Windows with large files cache#301

  3. Changing the PATH will cause the cache to switch between BSD tar and GNU tar causing compatibility issues on Windows (due to the zstd compression bug) - Robustness bug - cache restore finds nothing even though cache was saved successfully (Windows) cache#465

  4. BSD tar on MacOS corrupting executables and.dylib files - Unable to restore C++ artifacts in some situations cache#460 and Cached data restored from Rust's Cargo build is corrupted cache#403

On hosted runners, tar.exe is available in several locations locations:

  1. BSD tar - C:\Windows\System32\tar.exe (Windows 2019 only)
  2. GNU tar - C:\Program Files\Git\usr\bin\tar.exe
  3. GNU tar - C:\msys64\usr\bin\tar.exe

Caching correctly selects GNU tar on Windows 2016 runners, since (2) shows up on the PATH. Windows 2019 runners use BSD tar because (1) shows up first on the PATH. (3) does not appear on the system PATH so it is never selected.

Given the problems with bsdtar, it seems we should either ensure GNU tar appears first on the system PATH or improve the lookup logic in the caching module to find GNU tar. Right now, a workaround is to explicitly set the system PATH:

    - name: Use GNU Tar from Msys
      if: ${{ runner.os == 'Windows' }}
      run: |
        echo "::add-path::C:\msys64\usr\bin"

Expected behavior
GNU tar is selected if it is available.

Desktop (please complete the following information):

  • OS: Windows
@dhadka dhadka added the bug Something isn't working label Aug 10, 2020
@eregon
Copy link

eregon commented Nov 12, 2020

In ruby/setup-ruby#102 we noticed that the MSYS2 tar doesn't seem to work with the given arguments.
Probably, the list of paths needs to be / paths and not \ paths.
Issue for that: #632

colinrotherham added a commit to alphagov/govuk-frontend that referenced this issue Jul 26, 2023
We’re using this option to enable the faster GNU tar for all runners. Windows runners previously used BSD tar

Historically, GitHub Actions cache used gzip on Windows and zstd on Linux and macOS

See: actions/toolkit#552
colinrotherham added a commit to alphagov/govuk-frontend that referenced this issue Jul 26, 2023
Better explain how we’re using the `enableCrossOsArchive` option to enable the faster GNU tar for all runners

Windows runners use BSD tar without this option

* actions/toolkit#552
ManasJayanth added a commit to esy/esy that referenced this issue Jul 13, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants