v2: Honor OS-specific file system storage conventions and migrate all assets to new locations #2955
Description
I typically prefer simplicity and consistency when trying to make an app cross-platform, and it is difficult.
I think it may be best to adhere to OS conventions when choosing where to store data.
(This issue arises as I finish implementing a feature that auto-saves configs so that the last-saved config can be resumed after the server is restarted; since a good choice must be made as to where to store the config file.)
Currently (Caddy 2.0 up to and including beta 11), data is stored in $XDG_DATA_HOME or:
$HOME/.local/share/caddy/
And if $HOME is empty, the current working directory is used in place of $HOME.
But this is a little awkward on Mac, Windows, Plan 9, Android, and others. In fact, this only really makes sense on Linux and BSD (which is, granted, probably 99% of deployments).
It can also lead to unintended consequences when data is not stored in an expected place for the OS, especially when you consider file backups, sysadmin experience, etc.
Mac typically wants things stored in ~/Library and Windows has %AppData% and %LocalAppData% folders.
Newer Go versions have os.UserCacheDir(), os.UserConfigDir(), and os.UserHomeDir() which account for these differences.
Although I will be using UserConfigDir(), it is not implemented quite how I want; and unfortunately there is no os.UserDataDir() at all. We can't use a cache directory for the data directory because Caddy data is not ephemeral -- for example, certificates must not be deleted arbitrarily. (We may use UserCacheDir in the future, but we don't need it today. Caddy 2 currently needs UserConfigDir and UserDataDir).
I will be implementing my own dataDir() function which essentially looks like this:
func dataDir() string {
if basedir := os.Getenv("XDG_DATA_HOME"); basedir != "" {
return filepath.Join(basedir, "caddy")
}
switch runtime.GOOS {
case "windows":
appData := os.Getenv("AppData")
if appData != "" {
return filepath.Join(appData, "Caddy")
}
case "darwin":
home := homeDir()
if home != "" {
return filepath.Join(home, "Library", "Application Support", "Caddy")
}
case "plan9":
home := homeDir()
if home != "" {
return filepath.Join(home, "lib", "caddy")
}
case "android":
home := homeDir()
if home != "" {
return filepath.Join(home, "caddy")
}
default:
home := homeDir()
if home != "" {
return filepath.Join(home, ".local", "share", "caddy")
}
}
return "./caddy"
}Notice that the XDG spec is honored regardless of the OS. I think this is reasonable and useful.
For a config directory, I will use os.UserConfigDir(), but only if $XDG_CONFIG_HOME is not set. The standard lib function only accepts that variable on Unix systems, but I want it to be able to work on any OS, for consistency's sake.
This will take effect starting in Caddy 2 beta 12.
From now until the release candidates, Caddy 2 will attempt to migrate the data folder to the new location for you until the release candidates. This will just be a simple "rename" syscall, so if the old and new locations are on different devices, the move will fail. Also, if the new destination already exists, Caddy will not perform any migration out of an abundance of caution. That'd be unusual, but I'll post instructions here in a reply for moving the files over yourself. You'll see an error message in the logs that links you to the instructions. Errors with migration will not prevent the server from running.
I'll also be adding the locations to our new documentation, and relevant logs will also indicate file/folder paths where that is useful.
I've also updated the --environ flag and environ subcommand to include information about the Go runtime and the directories that Caddy uses, as well as an error in the logs at startup if necessary env variables are missing.
tl;dr
Where Caddy stores stuff is changing for non-Linux/BSD systems to conform to OS specifications. They will be moved for you, but if there's an error, you can move them manually using the instructions below. If it succeeds, this is a transparent change.