New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Introduce support for OS-specific standard cache directories #676

Merged
merged 6 commits into from Oct 27, 2017

Conversation

Projects
None yet
2 participants
@soc
Contributor

soc commented Oct 23, 2017

This cleans up the home directories of users and helps making sure that
Coursier's cache is not accidentically backed up by applications or OS
functionality.

The precedence is as follows:

  • existing $COURSIER_CACHE environment variable
  • existing coursier.cache Java property
  • existing ~/.coursier/
  • existing operating system specific standards:
    • Linux: $XDG_CACHE_HOME/coursier, with fallback to ~/.cache/coursier
    • Windows: {SpecialFolder.LocalApplicationData}/cache/Coursier
    • macOS: $HOME/Library/Caches/Coursier

Simon Ochsenreither and others added some commits Oct 23, 2017

Simon Ochsenreither
Introduce support for OS-specific standard cache directories
This cleans up the home directories of users and helps making sure that
Coursier's cache is not accidentically backed up by applications or OS
functionality.

The precedence is as follows:

- existing $COURSIER_CACHE environment variable
- existing coursier.cache Java property
- existing ~/.coursier/
- existing operating system specific standards:
  - Linux:   $XDG_CACHE_HOME/coursier, with fallback to ~/.cache/coursier
  - Windows: {SpecialFolder.LocalApplicationData}/cache/Coursier
  - macOS:   $HOME/Library/Caches/Coursier
Depend on directories via its sources
This is required for the bootstrap module, whose JAR is launched as is
by the launchers (scripts/generate-launcher.sh).
@alexarchambault

This comment has been minimized.

Show comment
Hide comment
@alexarchambault

alexarchambault Oct 24, 2017

Member

@soc Shouldn't dataDir used here, instead of cacheDir? XDG base directory spec says that the cache dir is for "non-essential data files", whereas dataDir is for "user specific data files". I feel like the latter would be a better match (the "cache" of coursier consists of data files, and I wouldn't say they're "non-essential").

I mean the coursier cache would be in a sub-directory of dataDir.

Member

alexarchambault commented Oct 24, 2017

@soc Shouldn't dataDir used here, instead of cacheDir? XDG base directory spec says that the cache dir is for "non-essential data files", whereas dataDir is for "user specific data files". I feel like the latter would be a better match (the "cache" of coursier consists of data files, and I wouldn't say they're "non-essential").

I mean the coursier cache would be in a sub-directory of dataDir.

@alexarchambault

This comment has been minimized.

Show comment
Hide comment
@alexarchambault

alexarchambault Oct 24, 2017

Member

About the precedence, I think the last two points should be permuted, like

  • if OS specific dir for coursier exists, use it,
  • else if ~/.coursier exists, use it,
  • else, if none exists, fallback to OS specific dir.

That way, running a previous version of coursier (which would create a ~/.coursier) wouldn't change the cache directory for the later versions (which would pick the OS specific dir if it's there, no matter the ~/.coursier).

Member

alexarchambault commented Oct 24, 2017

About the precedence, I think the last two points should be permuted, like

  • if OS specific dir for coursier exists, use it,
  • else if ~/.coursier exists, use it,
  • else, if none exists, fallback to OS specific dir.

That way, running a previous version of coursier (which would create a ~/.coursier) wouldn't change the cache directory for the later versions (which would pick the OS specific dir if it's there, no matter the ~/.coursier).

@soc

This comment has been minimized.

Show comment
Hide comment
@soc

soc Oct 24, 2017

Contributor

@alexarchambault

Shouldn't dataDir used here, instead of cacheDir?

No, not in this case. There is no platform-independent way to avoid moving GBs of cached files across the network if you put them into the data dir (only Windows supports the distinction with roaming vs. local application data).

Cache is the right folder, because it perfectly fits the distinction of "the only thing users see after deleting the cache is that some things take more time".

On the other hand, packages that are published locally should go into the dataDir, as they are likely not recoverable from the online repositories.

This way, users have the huge benefit of being able to delete their caches without losing their locally published artifacts, and have their locally published artifacts synced across machines without dragging GBs of cached artifacts along with them.

About the precedence

Makes sense, I can change that.

Compile main sources with Java 6

Ooops, sorry I ported directories to Java 6 in soc/directories-jvm@ed193b4 already. I didn't see your PR. :-/ Please let me know if there are any required changes remaining.

Contributor

soc commented Oct 24, 2017

@alexarchambault

Shouldn't dataDir used here, instead of cacheDir?

No, not in this case. There is no platform-independent way to avoid moving GBs of cached files across the network if you put them into the data dir (only Windows supports the distinction with roaming vs. local application data).

Cache is the right folder, because it perfectly fits the distinction of "the only thing users see after deleting the cache is that some things take more time".

On the other hand, packages that are published locally should go into the dataDir, as they are likely not recoverable from the online repositories.

This way, users have the huge benefit of being able to delete their caches without losing their locally published artifacts, and have their locally published artifacts synced across machines without dragging GBs of cached artifacts along with them.

About the precedence

Makes sense, I can change that.

Compile main sources with Java 6

Ooops, sorry I ported directories to Java 6 in soc/directories-jvm@ed193b4 already. I didn't see your PR. :-/ Please let me know if there are any required changes remaining.

@alexarchambault

This comment has been minimized.

Show comment
Hide comment
@alexarchambault

alexarchambault Oct 26, 2017

Member

Ok, let's go for cacheDir then. (That's right that it makes no sense to somehow sync / save the cache dir of coursier, so it's better there.)

And, yep, I'm going to revert my commit about v1. It's better to put the cache in a sub dir of the cacheDir returned by directories (to separate it from other things that could be kept in cacheDir), and I can't think of a better name (and v1 is shorter than e.g. downloads).

Member

alexarchambault commented Oct 26, 2017

Ok, let's go for cacheDir then. (That's right that it makes no sense to somehow sync / save the cache dir of coursier, so it's better there.)

And, yep, I'm going to revert my commit about v1. It's better to put the cache in a sub dir of the cacheDir returned by directories (to separate it from other things that could be kept in cacheDir), and I can't think of a better name (and v1 is shorter than e.g. downloads).

alexarchambault added some commits Oct 26, 2017

@alexarchambault

This comment has been minimized.

Show comment
Hide comment
@alexarchambault

alexarchambault Oct 27, 2017

Member

@soc Merging, thanks for having pushed for that!

Member

alexarchambault commented Oct 27, 2017

@soc Merging, thanks for having pushed for that!

@alexarchambault alexarchambault merged commit 1871610 into coursier:master Oct 27, 2017

2 checks passed

continuous-integration/appveyor/pr AppVeyor build succeeded
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details

alexarchambault added a commit that referenced this pull request Oct 27, 2017

Tweak v1 component handling in cache path
Up to coursier 1.0.0-RC12, setting COURSIER_CACHE=foo makes files land
in e.g. foo/https/repo1.maven.org/….
#676 changed that to foo/v1/https/….
This commit reverts things back to what they were before that.

In the long term, it would be better to keep the v1 component there, but
I'd prefer not to change that behavior right now.

@alexarchambault alexarchambault referenced this pull request Oct 27, 2017

Merged

Tweaking #679

@soc

This comment has been minimized.

Show comment
Hide comment
@soc

soc Oct 27, 2017

Contributor

@alexarchambault It has been a pleasure!

Did you have any chance yet to verify that the paths chosen by directories work fine on Windows? (Mac, too, but I have less concerns there, because the solution to derive the directories is substantially less complex than Window's "let's tell the JVM to tell the OS to run a new process, run PowerShell in that process and then call functions from .NET").

Contributor

soc commented Oct 27, 2017

@alexarchambault It has been a pleasure!

Did you have any chance yet to verify that the paths chosen by directories work fine on Windows? (Mac, too, but I have less concerns there, because the solution to derive the directories is substantially less complex than Window's "let's tell the JVM to tell the OS to run a new process, run PowerShell in that process and then call functions from .NET").

alexarchambault added a commit that referenced this pull request Oct 27, 2017

Tweak v1 component handling in cache path
Up to coursier 1.0.0-RC12, setting COURSIER_CACHE=foo makes files land
in e.g. foo/https/repo1.maven.org/….
#676 changed that to foo/v1/https/….
This commit reverts things back to what they were before that.

In the long term, it would be better to keep the v1 component there, but
I'd prefer not to change that behavior right now.
@alexarchambault

This comment has been minimized.

Show comment
Hide comment
@alexarchambault

alexarchambault Oct 27, 2017

Member

@soc The Windows CI didn't complain, I'm guessing things work fine there.

Member

alexarchambault commented Oct 27, 2017

@soc The Windows CI didn't complain, I'm guessing things work fine there.

alexarchambault added a commit that referenced this pull request Oct 27, 2017

Tweak v1 component handling in cache path
Up to coursier 1.0.0-RC12, setting COURSIER_CACHE=foo makes files land
in e.g. foo/https/repo1.maven.org/….
#676 changed that to foo/v1/https/….
This commit reverts things back to what they were before that.

In the long term, it would be better to keep the v1 component there, but
I'd prefer not to change that behavior right now.
@alexarchambault

This comment has been minimized.

Show comment
Hide comment
@alexarchambault

alexarchambault Oct 29, 2017

Member

@soc Where cache files land on Windows 10 (in a free VM):
coursier-cache-win10

Member

alexarchambault commented Oct 29, 2017

@soc Where cache files land on Windows 10 (in a free VM):
coursier-cache-win10

@soc

This comment has been minimized.

Show comment
Hide comment
@soc

soc Oct 29, 2017

Contributor

@alexarchambault This agrees with http://doc.qt.io/qt-5/qstandardpaths.html, but I would love to know how they came up with this path in the first place. Can't find anything in the Microsoft documentation.

Contributor

soc commented Oct 29, 2017

@alexarchambault This agrees with http://doc.qt.io/qt-5/qstandardpaths.html, but I would love to know how they came up with this path in the first place. Can't find anything in the Microsoft documentation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment