Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Don't store state data in $XDG_CONFIG_HOME #1285

Closed
rpdelaney opened this issue Dec 20, 2018 · 23 comments
Closed

Don't store state data in $XDG_CONFIG_HOME #1285

rpdelaney opened this issue Dec 20, 2018 · 23 comments
Assignees
Labels
feature New feature request
Milestone

Comments

@rpdelaney
Copy link

rpdelaney commented Dec 20, 2018

Feature description

In general weechat does not support the XDG directory specification. #391 added limited support for moving the weechat configuration directory without the use of symlinks. Critically, weechat still provides no options that I'm aware of to divide configuration data from state data.

  1. Configurations should be read from "$XDG_CONFIG_HOME/weechat/", then "$XDG_CONFIG_DIRS", then "$HOME/.config/weechat/", before falling back to "$HOME/.weechat/" (to maintain backward compatibility).
  2. State data such as logs and authentication credentials should be read from "$XDG_DATA_HOME/weechat/", then "$HOME/.local/share/weechat/", before resorting to fallbacks.
    • Authentication data (i.e. passwords) should not be stored alongside configuration files since this dangerous deviation from the protocol may cause users to back them up by insecure means to insecure locations.
    • Logs of private channels and queries may also present a security hazard for users.
  3. Any temporary files such as session keys should be stored in "$XDG_CACHE_HOME" (although I wasn't able to find any of these in my "~/.weechat" directory).

For emphasis, users who do not expect state data to be stored here may accidentally back it up and synchronize it across systems improperly. Configuration data are fine, but prefer storing state data somewhere in $XDG_DATA_HOME to conform to the freedesktop.org / XDG standard.

Reference: https://standards.freedesktop.org/basedir-spec/basedir-spec-latest.html

$XDG_CONFIG_HOME defines the base directory relative to which user specific 
configuration files should be stored. If $XDG_CONFIG_HOME is either not set or 
empty, a default equal to $HOME/.config should be used.
$XDG_CONFIG_DIRS defines the preference-ordered set of base directories to 
search for configuration files in addition to the $XDG_CONFIG_HOME base 
directory. The directories in $XDG_CONFIG_DIRS should be seperated with a 
colon ':'. 
$XDG_DATA_HOME defines the base directory relative to which user specific data 
files should be stored. If $XDG_DATA_HOME is either not set or empty, a default 
equal to $HOME/.local/share should be used.
@weechatter weechatter added the feature New feature request label Dec 20, 2018
@teto
Copy link

teto commented Sep 12, 2019

looking forward to that. I tend to grep in my dotfiles a lot, and having logs there messes up the results. (and XDG is a good thing by itself)

Seirdy added a commit to Seirdy/dotfiles that referenced this issue Feb 27, 2020
Rationale: WEECHAT_HOME contains both data that should belong in
XDG_DATA_HOME and configurations that should belong in XDG_CONFIG_HOME;
therefore, WEECHAT_HOME belongs in neither until
weechat/weechat#1285 is fixed.
@ifohancroft
Copy link

ifohancroft commented Mar 17, 2021

I am sorry to be commenting on such an old ticket but this is still an issue, so I am curious: @flashcode what do you think about it?
I have read in GNU Savannah a comment from 2013 saying there is no plan to implement this because WeeChat doesn't use X-Window lib and not to break compatibility and confuse existing users, but I have a couple of ideas on how to achieve that while still supporting the XDG Basedir Specification:

  1. The XDG specification is not just for GUI applications. Generally, the idea is to separate cache, from data, from configuration and to not pollute the home directory. I think we can all benefit from WeeChat splitting those three and going under the appropriate subdirs of the home folder.
  2. Support the XDG spec according to what it says (I believe the XDG envs goes first, then the directory paths if the envs aren't set, etc) but as a fallback, keep looking directly in the homedir for everything. This way, there is backwards compatibility and nothing changes for existing users unless they wipe everything and make a new install without transfering their config files.
  3. Keep the WEECHAT_HOME environment variable and leave it working just like it currently is (if it's set, it sets precedence over everything and it can still be used to store there everything, if you want everything in a single place) but keep it empty by default (like it currently is)

Some might argue that my proposed fallbacks aren't ideal and I would agree with that. Feel free to implement them in a different way. Honestly, the important thing for me, from what I suggested is the full support of the XDG Basedir Specification. I have way too many things in my home directory and my .profile is full with too many workarounds (such as WeeChat's WEECHAT_HOME environment variable) in-order to move as many things as possible to subfolders and hopefully split data from cache from configuration.

@flashcode
Copy link
Member

@ifohancroft: thank you for your comment, an old issue can still of course be commented!

Unlike my comment in 2013, I'm in favor of such change.

This require some work though, and specifications about what to do exactly should be debated, for example the initial request asks to extract credentials from config to put them in another directory, this is not easy at all.

I think as a first step, I could just split config, data, etc. and fallback on the current home if the directory exists, like you proposed.
There's still some work needed, just for this, in WeeChat itself but also all plugins/scripts that can store some data in the WeeChat home directory.

So it has a lot of impacts, that's why I didn't implement this yet, we need to think about the proper way to do it before developing it.

@rpdelaney
Copy link
Author

the initial request asks to extract credentials from config to put them in another directory, this is not easy at all.

As strong as my opinions are about this, I admit there is some room for interpretation in the XDG spec about whether this is actually wrong or not. While not ideal from my POV, for my purposes it would be enough if passwords and so on were stored in a secrets.conf file or such so that it could be easily ignored by git, rsync etc.

@ifohancroft
Copy link

ifohancroft commented Mar 17, 2021

@ifohancroft: thank you for your comment, an old issue can still of course be commented!

Unlike my comment in 2013, I'm in favor of such change.

I wasn't sure what was the policy on that when it comes to tickets and PRs, since it's frowned upon in forums for example.

This require some work though, and specifications about what to do exactly should be debated, for example the initial request asks to extract credentials from config to put them in another directory, this is not easy at all.

I've always had this problem of specific situations that seem not fit either case and not knowing how to handle it because of that.

I think as a first step, I could just split config, data, etc. and fallback on the current home if the directory exists, like you proposed.

That would be awesome!

There's still some work needed, just for this, in WeeChat itself but also all plugins/scripts that can store some data in the WeeChat home directory.

Yeah, it's never as easy as just changing a variable to point to another directory unfortunately :X
How is it currently handled? Do scripts and plugins ask WeeChat for the path?

So it has a lot of impacts, that's why I didn't implement this yet, we need to think about the proper way to do it before developing it.

Yeah. Like you mentioned above, the requirements need to be better specified, so I'd suggest instead of trying to come up with a list of things we want from this change, it should perhaps just follow the specification. Like we can quote the part we are discussing or if you want to highlight something that's in the spec but can't be done or at least not at this point (like the separation of credentials from config).

@ifohancroft
Copy link

I couldn't write a good summary of the spec, that's better written or more clean than the spec, so I am just linking the spec itself. If needed by someone, I can clarify things as I've read it more than I care to admit:

https://specifications.freedesktop.org/basedir-spec/basedir-spec-latest.html

Basically, the gist of is:

  • Store cache in the dir specified by env $XDG_CACHE_HOME or under $HOME/.cache/ if the env is empty or not set.
  • Store configuration in the dir specified by env $XDG_CONFIG_HOME or under $HOME/.config/ if the env is empty or not set.
  • Store data in the dir specified by env $XDG_DATA_HOME or under $HOME/.local/share/ if the env is empty or not set.
  • Store runtime data in the dir specified by env $XDG_RUNTIME_DIR or under a directory with the same properties (as explained in the spec for env $XDG_RUNTIME_DIR if the env is empty or not set).

However, I would still recommend reading the spec as I might have missed or may have been wrong about it.

@ifohancroft
Copy link

Here's my take on things:

  • The config files should go under $XDG_CONFIG_HOME/weechat/
  • The log files should go under $XDG_DATA_HOME/weechat/

Regarding cache and runtime data - I don't know if WeeChat uses any.
Regarding the credentials: This counts as configuration and I believe it should stay in the config folder. However, @rpdelaney's suggestion about moving the credentials to their own config file if not hard to do, would definitely provide some benefits, but it also raises a lot of questions:

  • Should it be a file per server or one file containing the credentials for all configured servers?
  • What about if you set a password to lock WeeChat itself (unless I remember WeeChat having this option)? Should it share the file or?

@flashcode
Copy link
Member

How is it currently handled? Do scripts and plugins ask WeeChat for the path?

Yes, in script you do something like:

path = weechat.info_get("weechat_dir", "")

Probably all these calls will have to be changed to store files in the "data" directory.

Regarding cache and runtime data - I don't know if WeeChat uses any.

There are, for example (this list is not exhaustive):

  • scripts: directories python, perl, etc.
  • list of scripts: ~/.weechat/script/plugins.xml.gz (it's a cache)
  • FIFO pipe: ~/.weechat/weechat_fifo (present only while WeeChat is running, destroyed upon exit)
  • any data stored by scripts (configuration files, text files, SQLite databases, etc.)

@ifohancroft
Copy link

Yes, in script you do something like:

path = weechat.info_get("weechat_dir", "")

Perhaps keeping the same method but exposing a path for each of the 4 different types would be the least painful way for scripts to switch to the new method?

Btw some ramblings about the spec:

$XDG_CONFIG_DIRS (I didn't mention this earlier, but like I said I was just giving a summary, so the full spec should still be read as I can't give a better explanation of things than the spec itself) - This is a var, that contains more dirs where configs can be stored. $XDG_CONFIG_DIRS can contain multiple paths, separated with colon ':' where the second path takes precedence over the third, the first takes precedence over the first and $XDG_CONFIG_HOME takes precedence over the first path in $XDG_CONFIG_DIRS.

The spec doesn't mention if this should be implemented and how. I think it's fine if a spec doesn't require a particular implementation, but I wish it at least marked which parts must be implemented and which are optional.

I mean for example, let's say WeeChat decides to implement $XDG_CONFIG_DIRS. It can be used as a system wide config directory (most software that uses $XDG_CONFIG_DIRS uses it that way). Whatever is not configured currently, in WeeChat has a default value but it's not present in a config file, in the case of $XDG_CONFIG_DIRS, it can instead be present in a config file there (although what should happen if someone deletes the system wide config file and the local one?) and the local one can be used like now to override settings. However, should this be implemented? Does it even matter?

I feel like maybe we should all write down our questions and things on the topic then join the xdg mailing list and perhaps contribute to the spec at least with discussion.

@flashcode (Very cool nickname btw) WeeChat doesn't currently use a system wide config - where/how are the settings currently stored?

What do you think about the idea of using a system wide config for storing them? In such a case (system wide config file) what should be the behavior if the file is deleted? Perhaps all settings can show without value but should WeeChat be expected to work in such a case? Should maybe WeeChat have defaults like now but to be used only if there is no config (if both a user-wide and system wide are missing)?

To clarify: I am not pushing for a system-wide config, I mean I feel like it's fine either way (how it currently is or with a system wide config instead) I am just considering the different scenarios out loud.

@flashcode
Copy link
Member

@flashcode (Very cool nickname btw) WeeChat doesn't currently use a system wide config - where/how are the settings currently stored?

What do you think about the idea of using a system wide config for storing them? In such a case (system wide config file) what should be the behavior if the file is deleted? Perhaps all settings can show without value but should WeeChat be expected to work in such a case? Should maybe WeeChat have defaults like now but to be used only if there is no config (if both a user-wide and system wide are missing)?

To clarify: I am not pushing for a system-wide config, I mean I feel like it's fine either way (how it currently is or with a system wide config instead) I am just considering the different scenarios out loud.

For now, by design, the WeeChat home directory is used for both read/write of configuration and must be writeable by the user.

Using system wide config was another feature request (#547), but I'm not sure we should implement everything at same time, it requires more work than just following XDG settings. Anyway it's nice to keep this in mind when implementing this issue as it's somewhat related as well.

@ifohancroft
Copy link

For now, by design, the WeeChat home directory is used for both read/write of configuration and must be writeable by the user.

Using system wide config was another feature request (#547), but I'm not sure we should implement everything at same time, it requires more work than just following XDG settings. Anyway it's nice to keep this in mind when implementing this issue as it's somewhat related as well.

Yeah, implementing it all at once would be too much.

I was actually wondering if a system-wide config should be implemented at all, not if it should be implemented at the same time with everything else. Although, I guess it would be nice to also have a system wide config.

P.S. If I want to bounce a couple of ideas off of you, that I'd like to hear your opinion on (not related to the XDG spec) what would it be the best way to do so? Should I ping you on IRC or should I open an RFC ticket here?

@flashcode
Copy link
Member

flashcode commented Mar 18, 2021

P.S. If I want to bounce a couple of ideas off of you, that I'd like to hear your opinion on (not related to the XDG spec) what would it be the best way to do so? Should I ping you on IRC or should I open an RFC ticket here?

IRC is OK if you need a fast answer.
And actually it's nice in my opinion to first discuss features on IRC first, to be sure your feature request has some chances to be implemented. Example: you want to integrate in WeeChat a chat protocol which is closed/not free, this has zero chances to be integrated as an official plugin in WeeChat itself.

But an issue is better for long-term discussions before implementing it, and it's public, unlike IRC. Anyway before opening a new issue, you could check if there's not an old one waiting for your comments 😉

@ifohancroft
Copy link

ifohancroft commented Mar 18, 2021

Thank you! That's exactly why I want to bounce ideas off of you first, before creating a feature request as you might be against some things being implemented.

@flashcode
Copy link
Member

As strong as my opinions are about this, I admit there is some room for interpretation in the XDG spec about whether this is actually wrong or not. While not ideal from my POV, for my purposes it would be enough if passwords and so on were stored in a secrets.conf file or such so that it could be easily ignored by git, rsync etc.

@rpdelaney: that's already the case, you can define and use "secured data" in options. This is stored in file sec.conf and can be protected by a passphrase (highly recommended!). And since WeeChat 3.1, you can use an external program (eg a password manager) to get the passphrase on startup.

For more info:

@rpdelaney
Copy link
Author

that's already the case, you can define and use "secured data" in options.

That's good to know. Curious that this is not default behavior.

For my part I am using a certificate to authenticate, since I can store the cert elsewhere.

@flashcode
Copy link
Member

flashcode commented Mar 28, 2021

@ifohancroft, @rpdelaney: I initiated a new repository with WeeChat specifications, that will be used for new features that have a lot of impacts, to discuss about them before they are implemented (easier to track than in a GitHub issue).

So the first specification is this one, you can find it here: https://specs.weechat.org/specs/2021-001-follow-xdg-base-dir-spec.html (or GitHub repository: https://github.com/weechat/specs.weechat.org, which uses GitHub pages to serve the site).

Note: it's still a "draft" and in progress, there are some question marks in the document (like scripts that have to be modified by this feature).

@ifohancroft
Copy link

ifohancroft commented Mar 29, 2021

@ifohancroft, @rpdelaney: I initiated a new repository with WeeChat specifications, that will be used for new features that have a lot of impacts, to discuss about them before they are implemented (easier to track than in a GitHub issue).

So the first specification is this one, you can find it here: https://specs.weechat.org/specs/2021-001-follow-xdg-base-dir-spec.html (or GitHub repository: https://github.com/weechat/specs.weechat.org, which uses GitHub pages to serve the site).

Note: it's still a "draft" and in progress, there are some question marks in the document (like scripts that have to be modified by this feature).

That's an awesome idea!

What is the process for commenting on existing WeeChat specifications? Should we open a ticket in the Spec repo to list our comments for the spec or perhaps while a spec is still a draft / not implemented, to have a ticket per Spec where people should post comments?

I was going to ask what is the process for recommending new specs but I believe this is just opening a ticket here to submit a feature request?

@flashcode flashcode self-assigned this Apr 17, 2021
@flashcode flashcode added this to the 3.2 milestone Apr 17, 2021
@flashcode
Copy link
Member

flashcode commented Apr 17, 2021

For your information, I updated the spec (main changes are in functions string_eval_path_home and mkdir_home): https://specs.weechat.org/specs/2021-001-follow-xdg-base-dir-spec.html

I'm starting the development now, in a separate branch xdg-directories that will be rebased frequently on master (with push force).

The target version is 3.2 (tentative, it may be postponed to 3.3 depending on how long the development will take, and there are also a lot of scripts to update).

flashcode added a commit that referenced this issue May 11, 2021
flashcode added a commit that referenced this issue May 11, 2021
flashcode added a commit that referenced this issue May 11, 2021
flashcode added a commit that referenced this issue May 11, 2021
flashcode added a commit that referenced this issue May 11, 2021
flashcode added a commit that referenced this issue May 11, 2021
flashcode added a commit that referenced this issue May 11, 2021
…r IRC server option "sasl_key" (user's guide) (issue #1285)
flashcode added a commit that referenced this issue May 11, 2021
…r IRC server option "ssl_cert" (user's guide) (issue #1285)
flashcode added a commit that referenced this issue May 11, 2021
flashcode added a commit that referenced this issue May 11, 2021
flashcode added a commit that referenced this issue May 11, 2021
flashcode added a commit that referenced this issue May 11, 2021
flashcode added a commit that referenced this issue May 11, 2021
flashcode added a commit that referenced this issue May 11, 2021
This is to prevent two WeeChat using the same runtime directory to use the same
FIFO pipe.
flashcode added a commit that referenced this issue May 11, 2021
@flashcode flashcode removed the in progress Someone is working on this issue label May 11, 2021
@flashcode
Copy link
Member

flashcode commented May 11, 2021

Implemented! ✌🏼
And the impacted scripts have been updated as well.

Feedback is welcome before the final release, to be sure everything works fine and as expected.

@teto
Copy link

teto commented Jun 3, 2021

thank you very much for implementing this. It worked for me (no fancy testing), Dont forget to unset $WEECHAT_HOME like I did though because initially I was like why isn't xdg working :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature New feature request
Projects
None yet
Development

No branches or pull requests

5 participants