std::env::{user_config_path, user_cache_path}: per-user application paths

[ChrisDenton]

Proposal

Problem statement

It would be useful for cli applications to have a simple solution for finding a place to store its own per-user files.

Motivating examples or use cases

The following are generally regarded as the wrong way to get a directory to put the application files, especially in cross-platform scenarios.

// Most platforms discourage creating directories in the root of the user's home directory
let app_dir = PathBuf::from(std::env::home_dir()?).join("app_name");

// This has the same problem as above but also uses `HOME` which may not be correct on all platforms
let app_dir = PathBuf::from(env::var("HOME")?).join("app_name");

Solution sketch

Expose a cross-platform subset of the XDG Base Directory specification (hereinafter simply referred to as "XDG" for brevity).

// in std::env

/// For storing the application's user configuration.
pub fn user_config_path() -> Result<PathBuf, Error>
/// For non-essential files. Data stored here should be re-creatable when necessary.
pub fn user_cache_path() -> Result<PathBuf, Error>

These will be simply implemented using environment variables and nothing else. More complex solutions can be left to third party crates. The following table shows the XDG environment variables used and the fallback for each platform

fn	XDG	Unix	MacOs	Windows
user_config_path()	$XDG_CONFIG_HOME	$HOME/.config	$HOME/.config	$APPDATA
user_cache_path()	$XDG_CACHE_HOME	$HOME/.cache	$HOME/Library/Caches	$LOCALAPPDATA

This could be extended in the future by implementing XDG_DATA_HOME and XDG_STATE_HOME. However, the downside of providing these in std is we're committing to them forever. So I think it prudent to act cautiously

Alternatives

• Leave this to crates
• Do something more complex, perhaps using specific OS APIs. if available

Links and related work

• Go has UserConfigDir and UserCacheDir
• dirs and related crates
• etcetera
• Config file management rust-cli/team#7

What happens now?

This issue contains an API change proposal (or ACP) and is part of the libs-api team feature lifecycle. Once this issue is filed, the libs-api team will review open proposals as capability becomes available. Current response times do not have a clear estimate, but may be up to several months.

Possible responses

The libs team may respond in various different ways. First, the team will consider the problem (this doesn't require any concrete solution or alternatives to have been proposed):

• We think this problem seems worth solving, and the standard library might be the right place to solve it.
• We think that this probably doesn't belong in the standard library.

Second, if there's a concrete solution:

• We think this specific solution looks roughly right, approved, you or someone else should implement this. (Further review will still happen on the subsequent implementation PR.)
• We're not sure this is the right solution, and the alternatives or other materials don't give us enough information to be sure about that. Here are some questions we have that aren't answered, or rough ideas about alternatives we'd want to see discussed.

[BlackHoleFox]

What about the inconsistency between macOS's config and Window's config paths? If being pedantically correct, macOS should point inside $HOME/Library/Application Support if Windows is pointing to %APPDATA%. I have macOS apps that write to both places and Windows apps that write to both too, so declaring one "the right way" for std may be bumpy.

Related(ish): std would need to fix the behavior of and un-deprecate std::env::home to be consistent with providing user-based directories. Otherwise saying yes you can get the cache path, but just don't reach for the home directory would raise eyebrows.

[ChrisDenton]

What about the inconsistency between macOS's config and Window's config paths? If being pedantically correct, macOS should point inside $HOME/Library/Application Support if Windows is pointing to %APPDATA%. I have macOS apps that write to both places and Windows apps that write to both too, so declaring one "the right way" for std may be bumpy.

The mac users I've talked to (admittedly a limited number) have been clear that for cli apps they strongly preferred not using "Application Support". In a poll conducted by nushell, mac users were also very vocal about wanting it to be more Unix-ish. The cache path is however a bit special because $HOME/Library/Caches is treated specially (e.g. ignored by time machine). It also matters less because unlike with config, it's not intended to be a directory that users interact with much (unless something has gone wrong). There's also a precedent set by golang so we wouldn't be out on a limb.

I don't think we can reasonably have entirely consistent cross-platform behaviour. However, the one thing they do appear to have in common is to say you shouldn't create application directories in the root of the home directory so nudging programmers in a better direction is good, I feel. Third party crates will still exist for more involved application directory handling, this merely intends to raise the baseline.

Related(ish): std would need to fix the behavior of and un-deprecate std::env::home to be consistent with providing user-based directories. Otherwise saying yes you can get the cache path, but just don't reach for the home directory would raise eyebrows.

Sure. I wanted this to be a separate discussion though because, as you say, if something like this is accepted (how ever it's implemented) then it strengthens the argument for a home_dir in some form. At least on *nix platforms.

[VorpalBlade]

This API would be very unfortunate since it doesn't expose the difference between configuration and state, that XDG makes a difference between.

This cause all sorts of issues on Linux already with non-XDG compliant applications. For example, some of us like to version control out config files in git (this is very useful when you use many computers). However many programs store temporary state such as a list of recently opened files or most recent window positions in config files. According to XDG that should be stored in XDG_STATE_HOME ($HOME/.local/state normally).

A dumbed down API like proposed here will simply result in many rust programs doing this incorrectly. Instead it is better the expose the full nuance, but on platforms that don't make a difference map multiple of these to the same directory.

[ChrisDenton]

For sure state can be added. I don't know if that's essential though as many apps won't be storing state and the uses you mention seem more appropriate to GUI apps then CLI.

[VorpalBlade]

For sure state can be added. I don't know if that's essential though as many apps won't be storing state and the uses you mention seem more appropriate to GUI apps then CLI.

That is true, it is less common (but not unheard of) to store state in CLI programs. One example that immediately comes to mind is the .gitcredentials file that git uses if you don't have a platform keyring. I would consider this long lived state rather than configuration.

But why would GUI programs also not use APIs from the standard library? After all, it is the first and most obvious place people will look for an API at, and many won't read the documentation in detail.

With recent increased focus on supply chain people might be less willing to take on additional dependency in order to do the right thing as well, especially if they primarily use Windows/Mac or don't care about this issue personally (which I gather is also a driving factor for wanting this at all in the first place).

And you could argue that GUI programs handle everything through their GUI frameworks anyway, which may be true sometimes (GTK, Qt), but there are lighter weight GUI libraries too. Neither iced nor slint has any API for storing config files from what I can find from a quick search (I'm not involved at all with Rust GUI programming, so I don't know what other crates to check).

And what about TUI programs, that straddle the border between GUI and CLI? Last I looked at ratatui it did not concern itself with how to store state, or anything not directly related to formatting things to the terminal.

[ChrisDenton]

The trouble is, especially with macos, there can be a difference in conventions between cli and gui. E.g. zsh, homebrew, etc all seem to use unix conventions on macos whereas GUI apps use the Apple specific directories.

So if I'm forced to make a choice I think I'd choose to focus on the CLI use case. Most GUI applications will be using a framework any way and if not then there are a number of platform differences to consider (e.g. how recent/frequent files are handled) that we can't abstract over.

[VorpalBlade]

Interesting, I haven't used Mac since they release Mac OS X (so sometime in the 90s would have been the last time), so I can't speak to the complexities there. And that means the std API will still come with footguns for that platform (hopefully at least documented). I would not be surprised if the API ends up deprecated again (like home_dir) down the road if we don't do this properly.

But even with that said: I think my other points are fairly compelling still, TUI programs often still have state and many CLI programs do. For example the history file of shells and other programs with interactive prompts.

However, doing a quick inventory on my system there seems to be a general confusion between XDG_STATE_HOME and XDG_DATA_HOME. Gnuplot puts it's history file in XDG_STATE_HOME, but some GUI programs put their history of recent files in XDG_DATA_HOME. And of course, many programs (bash, zsh etc) still put this directly in the home directory.

Personally for me that doesn't really matter. What matters to me is that there are separate files for configuration and state. The issues really starts when everything is mixed in one file. Making the difference between configuration and state clear is important to nudge people into doing the right thing and not just dumping it all in one file.

I have written an entire tool to deal with this sort of mixup in INI files (which KDE uses), but I would rather see programs just do the right thing so such a tool isn't needed any more.

[lolbinarycat]

For sure state can be added. I don't know if that's essential though as many apps won't be storing state and the uses you mention seem more appropriate to GUI apps then CLI.

also worth noting that the xdg basedir specification draws a distinction between "data" and "state", the main difference being that state is less important (eg. a browser may want to store bookmarks in the data dir, but store history and cookies in the state dir)

[ChrisDenton]

This was discussed in today's libs-api meeting. The feeling was we should implement a full ProjectDirs like API (that includes passing the project name and encompassing a larger number of directories).

This would be a fairly large project and require feedback from a range of interested parties (including authors of existing crates) at the design stage. It's not something that could be done iteratively by starting with a more limited API.

I'm closing this as I don't think I'm the right person to spearhead such a project and honestly I do have misgivings about having such an API in std. But anyone else should feel free to pick this up if they wish to.

[VorpalBlade]

Perhaps a way forward would be to have a rust project maintained crate for this, or if there is a clear best option in the ecosystem to incorporate it, either just the API or the code as well.

There is precedence for both of these options, I believe regex is under the rust-lang organisation for example. The standard library also uses and reexports parts of hash brown as the standard map type. Then there is once_cell that got incorporated into the standard library.

Perhaps (if the cargo team alone didn't have to maintain it) the home crate could serve as a starting point for this. The advantage of doing it in a "blessed" crate is that it can be made available on stable while still allowing breaking changes while we figure out the best API. Then at some future point it can be incorporated into std if people feel there is a strong argument for it.

[ChrisDenton]

I'm sceptical that there can ever be one true high level API because there's just too many possible ways to slice the onion. Having a blessed set of base paths seems like a more tractable problem to me (although not without issues). Another direction would perhaps be to only offer target-specific paths (e.g. in std::os::unix, std::os::macos and std::os::windows) but I'm not sure how helpful that is on its own.