Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support multiple compression algorithms when restoring cache #543

Closed
Vampire opened this issue Aug 5, 2020 · 5 comments
Closed

Support multiple compression algorithms when restoring cache #543

Vampire opened this issue Aug 5, 2020 · 5 comments
Assignees
Labels
cache enhancement New feature or request wontfix This will not be worked on

Comments

@Vampire
Copy link

Vampire commented Aug 5, 2020

Currently if you create a cache entry on a system where gzip is used and then restore the cache on a system where zstd is used, you get a cache miss.
It would be nice if instead mutliple compression algorithms were supported.
Something like this:

on cache creation:

  • zstd with --long present
    • use zstd with --long
    • end
  • zstd present
    • use zstd
    • end
  • use gzip

on cache restoring:

  • zstd with --long present
    • if according cache entry is present
      • use that
      • end
    • if according cache entry is not present
      • continue
  • zstd present
    • if according cache entry is present
      • use that
      • end
    • if according cache entry is not present
      • continue
  • use gzip
    • if according cache entry is present
      • use that
      • end

This way, for example a cache entry created on a gzip system can be used even on a zstd system.

And to be even fancier, when restoring a "lower" level (gzip or zstd on zstd with --long, or gzip on zstd), the file could be recompressed with the "higher" level and a new cache entry for that higher level created, so that next time it can be used more efficiently.

@lvpx
Copy link
Contributor

lvpx commented Nov 16, 2022

Hi @Vampire , while we do support multiple compression algorithms and have implemented optimizations since you posted this, the idea of having the customer choose their own algorithm does have merit and can be looked into. For now, closing this issue as this feature is not planned currently. Feel free to provide feedback on #984 for the current proposal on changes to tar compression.

@lvpx lvpx closed this as completed Nov 16, 2022
@Vampire
Copy link
Author

Vampire commented Nov 16, 2022

@pdotl I nowhere suggested to let the customer choose the algorithm.
It was an algorithm that should automatically work, so that for example a build on windows-2019 could be used on windows-2016. (I'm aware that 2016 is not there anymore, but that was the original problem that made me write this suggestion)

@lvpx
Copy link
Contributor

lvpx commented Nov 16, 2022

@Vampire apologies for the misunderstanding. We have now proposed defaulting to the same tar and compression algorithm wherever possible to ensure cross-os compatibility. Also we have ensured to default to the tools available across all hosted runners so this should not be a problem anymore. Feel free to drop by #984 to provide feedback. Many thanks!

@Vampire
Copy link
Author

Vampire commented Nov 16, 2022

Yeah, sure, if all use the same (also in the future) this is of course obsolete :-)

@Phantsure
Copy link
Contributor

@Vampire We have released a new beta release which should fix this issue. Try tag: actions/cache@3.1.0-beta. Head over to discussion for feedback: actions/cache#1019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cache enhancement New feature or request wontfix This will not be worked on
Projects
None yet
Development

No branches or pull requests

4 participants