Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Let Cargo put data into platform-specific directories #1615

Closed
wants to merge 8 commits into from
170 changes: 170 additions & 0 deletions text/1615-cargo-platform-standards.md
@@ -0,0 +1,170 @@
- Feature Name: N/A
- Start Date: 2016-05-14
- RFC PR:
[rust-lang/rfcs#1615](https://github.com/rust-lang/rfcs/pull/1615)
- Cargo Issues:
[rust-lang/cargo#1734](https://github.com/rust-lang/cargo/issues/1734)
[rust-lang/cargo#1976](https://github.com/rust-lang/cargo/issues/1976)
[rust-lang/cargo#2127](https://github.com/rust-lang/cargo/pull/2127)
- Rustup Issues:
[rust-lang-nursery/rustup.rs#247](https://github.com/rust-lang-nursery/rustup.rs/issues/247)
[rust-lang-nursery/rustup.rs#473](https://github.com/rust-lang-nursery/rustup.rs/issues/473)

# Summary

Improve Cargo's integration into the host operating system by using
platform-specific paths for config files, cache files and executables,
so it interacts well with other tools on that platform.

# Motivation

Currently, Cargo puts all its files in a directory named `.cargo` below the
home directory of the user. Using this proposal, it would reuse existing
standard directories of the host platform. This allows other tools oblivious to
Cargo to function in the best way.

Benefits include:

* Using a `.cargo` directory is violating the recommendations/rules on most
operating systems. _(Linux, Windows, macOS. Especially painful on Windows,
where dotfiles are not hidden.)_

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The hidden attribute is user settable: rustup and cargo are entirely able to set it when creating .cargo and .rustup, making this 'Especially painful' comment a small matter of a bugfix in rustup. rust-lang/rustup#2418

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have hidden files to always be displayed, so whether the folder is hidden or not is irrelevant. In fact, I want it to not be hidden, because any application that is installed normally is not hidden, so why does rustup/cargo need to hide where it is installed?

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I too have hidden always shown; I'm mainly responding to the 'especially painful on Windows' narrative in this document, which seems to expect Windows to have the same norms as Linux, but Windows users often don't have the same norms or expectations.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well it is especially painful on Windows, not because it isn't hidden, but because it's a dotfile on Windows! It's a blatant sign of software that was written for linux and poorly ported over to Windows, and it leaves a very bad impression of Rust as not supporting Windows that well.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree with you on that, but that sentence gives an entirely different context :).

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds like

, where dotfiles are not hidden

should be removed, then.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@retep998 would "Especially painful on Windows, where the use of a dot-prefixed directories and unusual locations gives a strong impression of poor platform support." be a good replacement?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure.

* Putting caches in designated cache directories allows backup tools to ignore
them. _(Linux, Windows, macOS. Example: Time Machine ignores the cache
directory on macOS.)_
* It makes it easier for users to manage, share and version-control their
configuraton files, as configuration files from different applications end up

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Typo in the first configuration - missing the second i.

in the same place, instead of being intermingled with cache files. _(Linux
and macOS.)_

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this not relevant for Windows users?

* Cargo contributes to the slow cleanup of the `$HOME` directory by stopping to
add its application-private clutter to it. _(Linux.)_
* Using a standard directory for binary outputs can allow the user to execute
Cargo-installed binaries without modifying their `PATH` variable. _(Linux)_

Solving this problem will likely also solve the same problem in Cargo-related
tools such as `rustup` as their strategy is "do what Cargo does".

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ish. Rustup did move away from Appdata\Local back in 2016 to line up with Cargo, but rustup has its own .rustup directory, and will need to migrate independently - and any migration of cargo bin directories will need to be coordinated with rustup as well. I don't think it makes it easier for rustup to follow XDG for rustups own files, but it would make it easier for the files that live in CARGO_HOME/bin, which is effectively the only intersection point.


This seems to be implemented in pip, the Python package manager, already.

# Detailed design

We are going to introduce new environment variables:
```
CARGO_BIN_DIR
CARGO_CACHE_DIR
CARGO_CONFIG_DIR
```

For the default values of these variables if they are set to the empty string
or not set (which will be the common case), see below.

These will be used to split the current `.cargo` (`CARGO_HOME`) directory up:
The cached packages (`.cargo/git`, `.cargo/registry`) will go into
`CARGO_CACHE_DIR`, binaries (`.cargo/bin`) installed by Cargo will go into
`CARGO_BIN_DIR` and the config (`.cargo/config`) will go into
`CARGO_CONFIG_DIR`.

In order to maintain backward compatibility, the old directory locations will
be checked if the new ones don't exist. In detail, this means:

1. If any of the new variables `CARGO_BIN_DIR`, `CARGO_CACHE_DIR`,
`CARGO_CONFIG_DIR` are set and nonempty, use the new directory structure.
2. Else, if there is an override for the legacy Cargo directory, using
`CARGO_HOME`, the directories for cache, configuration and executables are
placed inside this directory.
3. Otherwise, if the Cargo-specfic platform-specific directories exist, use
them. What constitutes a Cargo-specific directory is laid out below, for
each platform.
4. If that's not the case, check whether the legacy directory exists (`.cargo`)
and use it in that case.
5. If everything else fails, create the platform-specific directories and use
them.

This makes Cargo use platform-specific directories for new installs while
retaining compatibility for the old directory layout. It also allows one to
keep all Cargo related data in one place if one wishes to.

## Windows

We'll obtain each of the following directories using the correct API.

```
cache: AppData\Local\Temp\Cargo
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this should be %LOCALAPPDATA%\Cargo as I think %TEMP% is more for things that will be deleted by the time the program closes but the Cargo cache is useful for subsequent executions.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Using %TEMP% has the advantage that disk cleaners know about this directory.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's true but I would hope that Cargo can manage it's own cache. Do you have any examples of other software using %TEMP% like this? For example Firefox and pip both store their caches in %LOCALAPPDATA%.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Dunno, maybe look at Internet Explorer or Edge.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IE's cache is in %LOCALAPPDATA%\Microsoft\Windows\INetCache and Edge's is in %LOCALAPPDATA%\Packages\Microsoft.MicrosoftEdge_8wekyb3d8bbwe.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So it sounds like LOCALAPPDATA is the directory of choice here? Should the RFC be updated?

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Using a .cargo directory is violating the recommendations/rules on most operating systems.

This isn't true, at least not on Linux, per the FHS. See my other comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The RFC definitely needs an update yes; %LOCALAPPDATA%\Cargo\Cache would be my suggestion

config: AppData\Roaming\Cargo
binaries: AppData\Local\Programs\Cargo

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

%LOCALAPPDATA%\Cargo\Programs would be better - the convention in AppData is to own the subdirectory, not to create a generic space under it and then claim a subspace under that.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Incorrect.
FOLDERID_UserProgramFiles is what we want, and it defaults to %LOCALAPPDATA%\Programs which AppData\Local\Programs\Cargo falls under. Probably would be good to update the text to specify the known folder id constants specifically.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, documented https://docs.microsoft.com/en-us/windows/win32/msi/installation-context - thanks. Which also points out that we should use ShGetKnownFolderPath to work with this.

```

## Unixy systems (macOS, Linux, BSDs)

Here, we're following the [XDG specification](https://specifications.freedesktop.org/basedir-spec/basedir-spec-0.7.html).
By default, if no further variables are set, this means that we'll be using the
following subdirectories of the home directory.

```
cache: .cache/cargo
config .config/cargo
binaries: .local/bin
```

**There is currently an on-going discussion about standardizing the location of
`.local/bin` together with a new XDG variable `XDG_BIN_HOME`. The
implementation of this RFC should be delayed until that discussion has finished
and use the result. This RFC will be amended with that result.**
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Has there been any news about this?

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd just go forward with it, because there are no negative consequences from not doing it.

The non-existence of $XDG_BIN_HOME, just like the possible non-existence of other XDG env vars, is not a problem.

If the XDG spec update ships unchanged, the cargo code will "just" work as people start using the env var, and if there are any changes (like a different name of the env var) the code can just be updated.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm still waiting for poettering to reply to my emails.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd suggest sending a follow-up, and in that follow-up I'd suggest asking if it'd be possible to move forward given the absence of objections.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The problem is mostly just that I was depending on pottering to actually push it, since no one else who could showed up. But I'm planning on poking the list about it again, yeah.



## Rustup

Rustup will replicate Cargo's priorisation algorithm. If the results differ
from what the executed version of Cargo will do, Rustup will add environment
variables `CARGO_BIN_DIR`, `CARGO_CACHE_DIR`, `CARGO_CONFIG_DIR` for the new
versions of Cargo, and add symlinks for the old versions of Cargo.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Uh I'd like some details on these symlinks please. That or a patch.



## New subcommand

Cargo (and Rustup) are going to gain a new subcommand, `cargo dirs`. It will
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could this be added to the metadata command instead of adding a whole new command?

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Or maybe to 'rustup show' since thats usably small :).

display the directories currently in use, in a human-readable format. In order
to support other programs and scripts, this subcommand will also have switches
to print the data in machine-readable form, at least `--json` for JSON output
and maybe `--shell` for env-compatible output.

Example JSON (specifics left to the implementation):
```
{
"bin_dir": "C:\\Users\\User\\AppData\\Local\\Programs\\Cargo",
"cache_dir": "C:\\Users\\User\\AppData\\Local\\Temp\\Cargo",
"config_dir": "C:\\Users\\User\\AppData\\Roaming\\Cargo"
}
```

Example (env-compatible):
```
CARGO_BIN_DIR=/home/user/.local/bin
CARGO_CACHE_DIR=/home/user/.cache/cargo
CARGO_CONFIG_DIR=/home/user/.config/cargo
```


# Drawbacks

* This increases the complexity of where to find the files Cargo uses. This can
be alleviated by providing library functions for Cargo related tools and a
command line to tell which paths are in use.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's not just this though; it's also things like documentation, explaining how cargo works to the user, etc

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point, added to the section.


* It can also complicate tutorials that tell users how Cargo works. The
tutorials should probably reference the Cargo subcommand that displays the
used locations. However it's still more complicated than static directory
name (it's in a weird location for Windows users though).


# Alternatives

* macOS could also use the `Library` folder for storing its data. This is mostly
done by UI applications.

* One could only change the Windows paths, as the Windows integration is
currently the worst.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The most notable thing about the Windows integration is probably the non-hiddenness of the folders, I should have that fixed in a release or two at most now it has been pointed out.



# Unresolved questions

* None so far.