Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

v2: Honor OS-specific file system storage conventions and migrate all assets to new locations #2955

Closed
mholt opened this issue Dec 31, 2019 · 2 comments
Assignees
Milestone

Comments

@mholt
Copy link
Member

@mholt mholt commented Dec 31, 2019

I typically prefer simplicity and consistency when trying to make an app cross-platform, and it is difficult.

I think it may be best to adhere to OS conventions when choosing where to store data.

(This issue arises as I finish implementing a feature that auto-saves configs so that the last-saved config can be resumed after the server is restarted; since a good choice must be made as to where to store the config file.)

Currently (Caddy 2.0 up to and including beta 11), data is stored in $XDG_DATA_HOME or:

$HOME/.local/share/caddy/

And if $HOME is empty, the current working directory is used in place of $HOME.

But this is a little awkward on Mac, Windows, Plan 9, Android, and others. In fact, this only really makes sense on Linux and BSD (which is, granted, probably 99% of deployments).

It can also lead to unintended consequences when data is not stored in an expected place for the OS, especially when you consider file backups, sysadmin experience, etc.

Mac typically wants things stored in ~/Library and Windows has %AppData% and %LocalAppData% folders.

Newer Go versions have os.UserCacheDir(), os.UserConfigDir(), and os.UserHomeDir() which account for these differences.

Although I will be using UserConfigDir(), it is not implemented quite how I want; and unfortunately there is no os.UserDataDir() at all. We can't use a cache directory for the data directory because Caddy data is not ephemeral -- for example, certificates must not be deleted arbitrarily. (We may use UserCacheDir in the future, but we don't need it today. Caddy 2 currently needs UserConfigDir and UserDataDir).

I will be implementing my own dataDir() function which essentially looks like this:

func dataDir() string {
	if basedir := os.Getenv("XDG_DATA_HOME"); basedir != "" {
		return filepath.Join(basedir, "caddy")
	}
	switch runtime.GOOS {
	case "windows":
		appData := os.Getenv("AppData")
		if appData != "" {
			return filepath.Join(appData, "Caddy")
		}
	case "darwin":
		home := homeDir()
		if home != "" {
			return filepath.Join(home, "Library", "Application Support", "Caddy")
		}
	case "plan9":
		home := homeDir()
		if home != "" {
			return filepath.Join(home, "lib", "caddy")
		}
	case "android":
		home := homeDir()
		if home != "" {
			return filepath.Join(home, "caddy")
		}
	default:
		home := homeDir()
		if home != "" {
			return filepath.Join(home, ".local", "share", "caddy")
		}
	}
	return "./caddy"
}

Notice that the XDG spec is honored regardless of the OS. I think this is reasonable and useful.

For a config directory, I will use os.UserConfigDir(), but only if $XDG_CONFIG_HOME is not set. The standard lib function only accepts that variable on Unix systems, but I want it to be able to work on any OS, for consistency's sake.

This will take effect starting in Caddy 2 beta 12.

From now until the release candidates, Caddy 2 will attempt to migrate the data folder to the new location for you until the release candidates. This will just be a simple "rename" syscall, so if the old and new locations are on different devices, the move will fail. Also, if the new destination already exists, Caddy will not perform any migration out of an abundance of caution. That'd be unusual, but I'll post instructions here in a reply for moving the files over yourself. You'll see an error message in the logs that links you to the instructions. Errors with migration will not prevent the server from running.

I'll also be adding the locations to our new documentation, and relevant logs will also indicate file/folder paths where that is useful.

I've also updated the --environ flag and environ subcommand to include information about the Go runtime and the directories that Caddy uses, as well as an error in the logs at startup if necessary env variables are missing.

tl;dr

Where Caddy stores stuff is changing for non-Linux/BSD systems to conform to OS specifications. They will be moved for you, but if there's an error, you can move them manually using the instructions below. If it succeeds, this is a transparent change.

@mholt mholt added in progress v2 labels Dec 31, 2019
@mholt mholt added this to the v2.0.0-beta12 milestone Dec 31, 2019
@mholt mholt self-assigned this Dec 31, 2019
@mholt
Copy link
Member Author

@mholt mholt commented Dec 31, 2019

If Caddy fails to migrate its assets automatically, you may have to perform a file copy from the old folder to the new one. Once you are sure it is working, you can delete the old folder.

Linux and BSD users, or anyone using XDG_DATA_HOME, should not need to migrate anything.

You only need to follow these instructions if an error in the log told you to do so.

Manual migration instructions

Make the new path. This depends on your OS:

  • Windows: %AppData%\Caddy
  • Mac: $HOME/Library/Application Support/Caddy
  • Plan 9: $HOME/lib/caddy
  • Android: $HOME/caddy ($HOME may be /sdcard)

Make sure only the user that Caddy runs as can read, write, or execute the new folder. On Linux, this would be like chmod 0700.

Find the old path: $HOME/.local/share/caddy

Copy the contents of the old folder (not the folder itself) into the new folder you just created.


That's it! Restart Caddy 2, and if there are no errors in the logs and if your server is working like it was, you can delete the old path.

@mholt mholt removed the in progress label Jan 1, 2020
@mholt
Copy link
Member Author

@mholt mholt commented Jan 1, 2020

This is now pushed, and will go out with the next beta.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant