Join GitHub today
GitHub is home to over 31 million developers working together to host and review code, manage projects, and build software together.
Sign upLet Cargo put data into platform-specific directories #1615
Conversation
This comment has been minimized.
This comment has been minimized.
|
There's a stale pull request to do something like this: rust-lang/cargo#2127, I'll gladly update it if it is desired. I figured that it might be big enough of a change to justify an RFC. |
steveklabnik
reviewed
May 14, 2016
|
|
||
| * This increases the complexity of where to find the files Cargo uses. This can | ||
| be alleviated by providing library functions for Cargo related tools and a | ||
| command line to tell which paths are in use. |
This comment has been minimized.
This comment has been minimized.
steveklabnik
May 14, 2016
Member
It's not just this though; it's also things like documentation, explaining how cargo works to the user, etc
This comment has been minimized.
This comment has been minimized.
plietar
reviewed
May 16, 2016
|
|
||
| ``` | ||
| cache: .cache/cargo | ||
| config .config/cargo |
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
tbu-
May 16, 2016
Author
Contributor
They are, perhaps I should clarify that a little. We're following the XDG spec, and by default it's these directories.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
withoutboats
reviewed
May 16, 2016
|
|
||
| ``` | ||
| cache: Library/Caches/Cargo | ||
| config: Library/Cargo |
This comment has been minimized.
This comment has been minimized.
withoutboats
May 16, 2016
Contributor
I'm currently using OS X, and the config for all of the CLI tools I have any interest in configuring are stored somewhere under my home directory. I've never touched a config file under Library.
This comment has been minimized.
This comment has been minimized.
tbu-
May 16, 2016
Author
Contributor
I don't know a lot about OS X. Isn't there a Library folder in the home directory as well?
This comment has been minimized.
This comment has been minimized.
Sean1708
May 16, 2016
Others may disagree but in my experience the unwritten rule of OSX is that GUI apps use ~/Library and CLI apps use unixy locations.
This comment has been minimized.
This comment has been minimized.
withoutboats
May 16, 2016
Contributor
My CLI tools configs and caches are all at the same location as they'd be on Linux, including some that are following XDG.
This comment has been minimized.
This comment has been minimized.
plietar
May 16, 2016
Homebrew uses /Library/Caches/Homebrew/ and pip ~/Library/Caches/pip/, so I guess both could be valid.
On the other hand, configuration should definitely use XDG.
This comment has been minimized.
This comment has been minimized.
|
I personally think all CLI tools should adopt XDG on all *nix platforms as the only reasonable attempt at a standard that anyone's produced. There's no actual standard on any *nix platform as far as I know, and I don't see a way that storing the files in the way specified by XDG is worse than storing them in I don't think that Rust tools should do something different on OS X from what they do on other *nixen, for the reasons cited in the issue. |
This comment has been minimized.
This comment has been minimized.
|
@withoutboats Indeed, and that's also what |
randomPoison
reviewed
May 16, 2016
| * Putting caches in designated cache directories allows backup tools to ignore | ||
| them. | ||
| * Using a `.cargo` directory on Windows is not idiomatic at all, no programs | ||
| for Windows would such a directory. |
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
tbu-
added some commits
May 16, 2016
This comment has been minimized.
This comment has been minimized.
|
@withoutboats I exchanged the plan for OS X with one of the alternatives. Now everything is the same across non-Windows platforms and only Windows differs. |
alexcrichton
added
the
T-dev-tools
label
May 16, 2016
This comment has been minimized.
This comment has been minimized.
fenhl
commented
May 18, 2016
|
Another huge upside to this is that it removes clutter from |
This comment has been minimized.
This comment has been minimized.
Could you elabrate? What tools will interacts well (better) with Cargo, and why? Edit: cache or disk cleanup tools will benifit at least, as the motivation section said. |
This comment has been minimized.
This comment has been minimized.
|
One drawback that may want to be added to this is that there are a good number of other apps and such which don't follow the XDG spec on Linux, and likely for their own "good reasons". This in turn means that if one day the spec becomes widely adopted it will likely need to be changed in one form or another. Put another way, many other major projects aren't following this spec, and to follow the spec it's quite likely that it'll need some sort of breaking change in one way or another. It's seems like it may be risky to hitch ourselves to this bandwagon? Also, are there any other apps which follow XDG on OSX? Is the spec even defined for OSX? I'd be quite hesitant to follow a Linux spec on OSX as it seems like we'd just be winding up in the same situation of OSX users thinking that we're not doing the "native thing" |
This comment has been minimized.
This comment has been minimized.
|
@alexcrichton On my MacBook, I have "configstore", "fish", "git', "gtk-2.0", "htop", and "nvim" directories in my |
This comment has been minimized.
This comment has been minimized.
|
On Linux, my impression is that all per-app dotfiles/dotdirs either predate or are done in ignorance of the XDG spec. Separating configuration from caching is quite useful and I cannot envision any downsides. |
This comment has been minimized.
This comment has been minimized.
reddraggone9
commented
May 24, 2016
That's been my experience as well. @alexcrichton Are you aware of any projects that would follow the spec if some sort breaking change were implemented? Most cases I've seen where it was brought up and rejected were due to inertia. |
This comment has been minimized.
This comment has been minimized.
jmesmon
commented
May 24, 2016
•
|
@alexcrichton on linux:
Is there a specific program with issues with xdg that you're thinking of? |
This comment has been minimized.
This comment has been minimized.
|
From the RFC text:
[1]: the installer can/already do this (one off job). not a benefit. So, there is only one benefit: to make 3rd-party tools happy, to help them find/clean cargo's cache data. And this is not a benefit for us, the users and developers of cargo. We get little benefit on this. But it will make difficulty to our life: we must remember more dirs, if we want to find them; we must do more os-specific work to implement it and maintain it; we can't easily create a portable cargo that all in one directory. One more thing: it's not clear what will be put into cache directory. (e.g. Will (XDG spec is nice. But what's the benefit if we follow it?) |
This comment has been minimized.
This comment has been minimized.
fenhl
commented
Jun 21, 2016
|
@liigo It is not the only benefit. See my comment from earlier. This should also be added to the RFC text. |
This comment has been minimized.
This comment has been minimized.
[1] is a benefit because that way, you don't end up with a path in I think it's pretty clear that all of these are benefits. Whether they're worth the trade-off is what the RFC discussion should be about, but claiming that they aren't benefits isn't really helping. EDIT: Regarding making 3rd party tools happy vs interacting with them: I as a user would be happy if the backup tool would know to ignore the Cargo cache. This is not for the benefit of the 3rd party tool, but for the benefit of me, the user of Cargo. |
This comment has been minimized.
This comment has been minimized.
|
@liigo as a cargo user who sometimes uses other tools also, making cargo interoperate with them better definitely makes my life better. :-) |
This comment has been minimized.
This comment has been minimized.
To me it seems like everyone would want to implement the change but may not be able to. If no one actually implements it then no one's following the spec, which seems to defeat the purpose of having it in the first place? It seems to me that if updates can't happen, then the spec is frozen as-is today. Then if it's frozen as-is there are major tools which have good reasons for not following it, and there's little hope for those reasons to get resolved?
Looking in my home folder, programs which don't follow xdg include:
Notably including many other languages' package managers. I forget if this has been brought up before, but I've at least discussed it with others. The point about operating with third-party programs is also an interesting one. There are programs which don't follow XDG, so any robust third-party program would already have to handle non-conforming programs with some heuristics. In that sense you may not actually get any benefit at all if Cargo follows XDG because the heuristics are already correctly classifying Cargo. If Cargo isn't falling in the right bucket but could easily do so for some popular programs, that would also be a reasonable course of action I think. |
This comment has been minimized.
This comment has been minimized.
|
Both pip and git support XDG.
I use rsnapshot for backups. The heuristic is the user adding exceptions on their own. It would be much nicer if Cargo didn't need an exception. I mentioned advantages and disadvantages in the RFC. It seems that at least the advantages are real for some users. Maybe the discussion should focus on how to weigh these arguments against each other. Especially on Windows, there seems little resistance against using the Windows-specific directories. |
This comment has been minimized.
This comment has been minimized.
reddraggone9
commented
Jul 5, 2016
Not everyone. For some programs which have been around longer than I've been alive, following the XDG spec as a fallback would be a change in functionality that has worked for decades. Inertia gets in the way there. "If it ain't broke, don't fix it." I'm not aware of any project that has been told about the spec and decided not to follow it but would have followed it if the spec had X changed where X wasn't just keep doing what already works.
People do implement it: $ ls .config
abrt goa-1.0 Pinta
anjuta google-chrome ppsspp
autostart gtk-2.0 pulse
beets gtk-3.0 QtProject.conf
cef_user_data gvbam ranger
Clementine hexchat retroarch
CuteMarkEd Project htop Rygel
dconf ibus rygel.conf
dleyna-renderer-service.conf imsettings sealert.conf
dleyna-server-service.conf inkscape sonata
easytag insta-site syncthing
enchant jstest-gtk systemd
eog keepnote tilda
epiphany libreoffice tomboy
evince libvirt totem
evolution markdownlint tracker
filezilla menus Trolltech.conf
gambatte mimeapps.list unity3d
gconf monitors.xml user-dirs.dirs
gedit mono.addins user-dirs.locale
gespeaker MonoGame VirtualBox
git mopidy vlc
gnome-boxes mpv watch-later
gnome-builder mupen64plus yelp
gnome-control-center myapp youtube-dl
gnome-initial-setup-done nautilus Zeal
gnome-session nvim
gnote PCSX2Just not everyone: $ ls -d .?*
.. .git-credential-cache .lastpass .rbtools-cookies
.adobe .gitk .lesshst .sdkman
.bash_aliases .gnome .local .serverauth.22085
.bash_history .gnupg .m2 .serverauth.4384
.bash_logout .gradle .macromedia .speech-dispatcher
.bashrc .grails .mozilla .ssh
.cache .grails_history .multirust .subversion
.cargo .groovy .netrc .Xauthority
.config .gstreamer-0.10 .npm .xinitrc
.emacs.d .ICEauthority .nvimlog .xsel.log
.esd_auth .idlerc .pki .zshrc
.exercism.json .inputrc .profile
.gimp-2.8 .java .python_history
If you're going to make that argument, having at least one example would be nice. You have a point about most other languages' package managers though. You can add sdkman to the list. |
This comment has been minimized.
This comment has been minimized.
jmesmon
commented
Jul 5, 2016
|
I asked about programs that have issues with XDG, not ones simply that don't impliment it. I'm looking for a downside, a complication, that would cause a program to actively avoid XDG. Right now one hasn't been presented.
I don't believe that is the case. At the least, no one has noted a reason (other than a program being old or inertia) that things don't impliment it. Please let us know if there is some reason you're aware of. |
This comment has been minimized.
This comment has been minimized.
|
My opinions on this subject (not commenting directly on the content of the RFC, which I've only skimmed - sorry). I think we should change where It's important for consistency that users can find Rust tools in expected places. There is no universal agreement on Unixes about the future of the XDG spec, and we should not use it by default. For those that want XDG we should consider making it optional. |
This comment has been minimized.
This comment has been minimized.
|
Whatever this ends up being, please add a way to invoke |
This comment has been minimized.
This comment has been minimized.
flying-sheep
commented
Apr 15, 2018
|
@mitsuhiko Don’t worry: the PR for implementing this in cargo (rust-lang/cargo#5183) does this already by adding a |
This comment has been minimized.
This comment has been minimized.
|
@flying-sheep on phone atm so I can't check the commit but last time I saw that there was no way to get that output in parsable format that can be used in a shell script. Just wanted to clarify that I meant it needs something like |
This comment has been minimized.
This comment has been minimized.
soc
commented
Apr 15, 2018
•
|
@brson Sorry, I had trouble understanding parts of your comments, I hope I got the gist of them:
No, the code checks whether .cargo exists and uses the legacy dir structure.
I agree that the RFC is underspecified. I believe that the best way forward is to write some code to fix the issues, so that people can discuss specific details and allow changes to be proposed as code. This makes sure that everyone is on the same page, and avoids that people waste time talking past each other.
I think it is important to consider all possible use cases and setups and make sure all of them work. That was my motivation when I wrote some code, and asked for feedback and ways how things could fail. But I think it's important to also not over-complicate things: I think it's counter-productive to come up with complex migration or symlinking schemes that are not absolutely required, while there is no full understanding of the changes and their effects. @mitsuhiko Yes, that's intended to work. I think the current PR doesn't implement this at the moment, but what you propose is exactly the idea how it should work. |
This comment has been minimized.
This comment has been minimized.
|
I would like this functionality (finding the relevant directories) to be implemented in a separate crate and published to crates.io. This avoids the issue of different implementations between cargo/rustup and allows talking about a specific version (I imagine there will be changes as issues are found). Then rustup/cargo could add experimental support (enabled via an env variable) allowing some time to iterate on the design and make sure it works on all the systems rust supports before enabling it by default. |
This comment has been minimized.
This comment has been minimized.
soc
commented
Apr 15, 2018
|
@Diggsey It is worthwhile? Isn't it simply checking |
This comment has been minimized.
This comment has been minimized.
I believe |
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
|
One thing that is under-specified is how rustup will determine whether the symlinks need to exist for cargo? What springs to mind is to add a new subcommand to cargo at the same time as these changes are made - old cargo versions will error when passed that subcommand. However, this kind of check is expensive and so cannot be done on every invocation. Presumably the symlinks will be created somewhere unobtrusive, and then rustup will set |
This comment has been minimized.
This comment has been minimized.
soc
commented
Apr 15, 2018
But that functionality is already living in a distinct crate.
I'm not planning to use symlinks.
I'm fine with adding the command earlier. |
This comment has been minimized.
This comment has been minimized.
Sorry, I was typing drunk on a phone! Happens. @Diggsey @soc @tbu-, fwiw I recognize the tide is firmly against me here, (and also my last post here is hilariously incomprehensible). I have though, in the meantime, dropped a massive comment-bomb in @soc's cargo PR, detailing in extreme detail the impact I think this has on rustup. I suggest you give it a read if you have an entire afternoon to spare. |
This comment has been minimized.
This comment has been minimized.
soc
commented
Apr 16, 2018
•
|
@brson Nevermind! Your help is really appreciated! |
This comment has been minimized.
This comment has been minimized.
The RFC differs from the cargo PR in some ways: The PR adds a CARGO_DATA_DIR that is not in this RFC. The PR's prioritization algorithm is different from that described in this PR (and is currently buggy, in ways I described in my review of it). In this RFC, "If any of the new variables CARGO_BIN_DIR, CARGO_CACHE_DIR, CARGO_CONFIG_DIR are set and nonempty, use the new directory structure." is unclear. I believe it is saying if any of those are set, use those values, and also use the default platform-specific values for those that aren't specified. (Note that the posted Cargo PR differs dramatically from this interpretation - for new-env-vars that are not set it falls back to the FWIW I agree with the algorithm in this PR, and it supports the rustup design I laid out here, specifically because it allows rustup to set The PR uses mac-specific directories not XDG. Also, this RFC says two sentences about rustup: "Rustup will replicate Cargo's priorisation algorithm. If the results differ from what the executed version of Cargo will do, Rustup will add environment variables CARGO_BIN_DIR, CARGO_CACHE_DIR, CARGO_CONFIG_DIR for the new versions of Cargo, and add symlinks for the old versions of Cargo.". I honestly don't even know what this is saying. Anyway I wrote in vast detail how to make rustup work with this scheme. Edit: After thinking about this for a while, I think part of what this is saying is something similar to my linked proposal, where In this RFC: "Cargo (and Rustup) are going to gain a new subcommand, cargo dirs" is unclear. Is there going to be a This RFC makes no explicit backwards-compatibility affordances for tools that expect to be able to find cargo by preserving a I think you should go ahead and change rustup's directory structure at the same time - I kinda don't understand why there's such an uproar about cargo not obeying XDG, but not rustup. Some of the description in my previous link about modifying rustup for this alludes to how to change rustup in a similar way, but doesn't go into detail. I'm not privy at this point to your plans for rustup/cargo integration, but it would be pretty trivial to just throw rustup's data into cargo's directories; or alternate create another set of RUSTUP_CACHE_DIR, etc variables that all behave the same way for rustup. I mentioned this in the cargo PR, but I recommend feature-gating this (probably with an environment variable) in cargo until rustup is also similarly modified and they are extensively tested to work correctly together, with past and future toolchains, and through upgrades. |
This comment has been minimized.
This comment has been minimized.
jtojnar
commented
Apr 16, 2018
I would say people who care about XDG specs probably also care about rest of their system being organized, so they would be more likely to use their distribution’s package manager to install Rust, rather than rustup. |
This comment has been minimized.
This comment has been minimized.
soc
commented
Apr 17, 2018
I think that's only because people might use cargo everyday, but rustup only occasionally, so cargo is the more "prominent" offender in this regard. |
soc
added a commit
to soc/cargo
that referenced
this pull request
Apr 28, 2018
This comment has been minimized.
This comment has been minimized.
macOS is not LinuxPlease don't group macOS and Linux as "Unix" platforms. File system layout on macOS is very different, and it isn't any more similar to Linux than it is to Windows. While some Unix-like directories exist on macOS for compatibility, they are not a good choice for well-behaved programs. macOS doesn't care about XDG or anything like that. Using Linux layout on macOS is as jarring and bizarre as if this RFC proposed Linux cache in macOS has its own custom filesystem layout and Apple-designed conventions. On macOS While an occasional Having said that, I'd advise against putting Cargo's config files in |
This comment has been minimized.
This comment has been minimized.
soc
commented
May 6, 2018
|
@kornelski Are you commenting on the text of the RFC or on the code of the implementation? |
This comment has been minimized.
This comment has been minimized.
|
Text of the RFC |
This comment has been minimized.
This comment has been minimized.
soc
commented
May 6, 2018
|
@kornelski I think the RFC doesn't reflect the state of the art. Apart from that, deciding on the directories is pretty much the least important thing, it can pretty much be adapted in the last minute. I'd be more interested in help and advice on making the necessary changes and migration mechanisms to cargo and rustup. |
retep998
referenced this pull request
Nov 14, 2018
Open
Windows: Cargo should put files in ~/appdata, not directly in ~/ #1976
Centril
added
the
A-platform
label
Nov 22, 2018
This comment has been minimized.
This comment has been minimized.
FranklinYu
commented
Dec 8, 2018
•
@kornelski Any comment/objection about placing configuration in And what about
Does macOS really clean up this directory? It does this on iOS, but similar behavior on macOS isn’t mentioned by the official documentation (or I didn’t find it). |
This comment has been minimized.
This comment has been minimized.
ghost
commented
Feb 4, 2019
|
FYI, I should point out that at least according to the opinion of a Debian developer, that people are misusing the XDG Base spec:
In other words, using the spec to enforce application-specific configuration on Linux systems is already a bit of a stretch, let alone extending it to Mac OS and Windows (which already have their own "FHS" so to speak). If we are following the Linux FHS, section 3.8.2 states:
So most applications are already following the actual standard and not the one that is causing a lot of controversy in many issue trackers. Just my 2 cents. |
This comment has been minimized.
This comment has been minimized.
|
On Mon, Feb 04, 2019 at 05:15:28AM -0800, remyabel wrote:
FYI, I should point out that at least according to the [opinion](https://lists.debian.org/debian-user/2014/11/msg01817.html) of a Debian developer, that people are misusing the XDG Base spec:
That's a post by a Debian user, not a Debian developer, and it's a
misunderstanding of the XDG base directory standard. Please weight it
accordingly and don't give it undue credence; anyone can post an
opinion.
|
This comment has been minimized.
This comment has been minimized.
|
Since this RFC is effectively stalled for two years already and the current solution works, is there a big reason for not just closing this RFC down? |
This comment has been minimized.
This comment has been minimized.
flying-sheep
commented
Feb 4, 2019
•
|
@remyabel Serge cherry-picked one of the motivations for XDG and ignored the others. the spec notably does not limit itself to a specific subset of applications. Serge is wrong. You also left out the FHS section 3.8.3, which comes directly after the one you quoted:
I can’t help but see that omission as intellectually dishonest, no way you missed that.
It doesn’t for me. I don’t want caches on my home partition, I have Also the general idea has been approved. IDK if closing and opening a new one is the right way or updating this one, but closing this without replacement is definitely not the right way. |
This comment has been minimized.
This comment has been minimized.
@FranklinYu If However, beware of just mapping So there's no single place in macOS that is valid for If you can't separate them properly, then it's better to use |
This comment has been minimized.
This comment has been minimized.
|
On Mon, Feb 04, 2019 at 06:28:53AM -0800, Kornel wrote:
> Any comment/objection about placing configuration in $XDG_CONFIG_HOME/cargo (falling back to ~/.config/cargo) on macOS? Git is doing that as well.
If `$XDG_CONFIG_HOME` is `~/Library/Preferences` then `$XDG_CONFIG_HOME/cargo/config` is OK. Better would be `$XDG_CONFIG_HOME/org.rust-lang.cargo/config`.
However, beware of just mapping `CARGO_HOME` there, because `$XDG_CONFIG_HOME/cargo/registry` is *very* inappropriate. It'd have to be in `$XDG_CACHE_HOME/cargo/registry` or better in `$XDG_CACHE_HOME/org.rust-lang.cargo/registry`.
Right, and many bits of ~/.cargo *should* move to `XDG_CACHE_HOME`.
That's also where we'd want to place any future cache of compiled
crates.
|
tbu- commentedMay 14, 2016
(rendered)