Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Quod Libet doesn't match FreeDesktop directories specs #138

Closed
lazka opened this issue Mar 14, 2015 · 31 comments
Closed

Quod Libet doesn't match FreeDesktop directories specs #138

lazka opened this issue Mar 14, 2015 · 31 comments
Labels

Comments

@lazka
Copy link
Member

@lazka lazka commented Mar 14, 2015

Original issue 138 created by thibaut.bethune on 2009-02-24T23:52:29.000Z:

I've tried Quod Libet 2.0 on Ubuntu 9.04 alpha 4
and it seems that Quod Libet places its configuration files in
/home/.quodlibet directory which doesn't match FreeDesktop directories specs :

The default for $XDG_CONFIG_HOME is $HOME/.config, the default for
$XDG_DATA_HOME is $HOME/.local/share. So all applications should look for
those environment variables and use those default values if the variables
are not set.

This is quite important since it's not possible to easily backup config
& data files if these files are not stored at the right place

See http://www.freedesktop.org/wiki/Specifications/basedir-spec
See also
http://ploum.frimouvy.org/?184-cleaning-user-preferences-keeping-user-data
(main post and comment#8)

@lazka
Copy link
Member Author

@lazka lazka commented Mar 14, 2015

Comment #1 originally posted by fverschelde on 2009-03-07T18:44:03.000Z:

Hello,

First I would like to add support for this request.

Implementing the Freedesktop specs for this means that two problems must be solved:

  1. Discerning what is "user-specific data files" and "user-specific configuration".
  2. Implementing the standard while remaining backwards compatible (i.e.: what happens
    if a user upgrades from 2.0 to say 2.1, where 2.1 follows the Freedesktop XDG Base
    Directory Specification).

FIRST ISSUE: WHAT GOES WHERE?

Quod Libet would need to put "user-specific data files" in $XDG_DATA_HOME/quodlibet/
or, if $XDG_DATA_HOME is not defined, in $HOME/.local/share/quodlibet/.
It would need to put "user-specific configuration" in $XDG_CONFIG_HOME/quodlibet/ or,
if $XDG_CONFIG_HOME is not defined, in $HOME/.config/quodlibet/.

But what is "user-specific data files" and what is "user-specific configuration"?
Configuration is anything that is defined by the user but for which there is a
predefined sensible default in the application. If it is lost, the user might to
configure a few things to get their previous setup back, but nothing of value will be
lost.
Data is anything that is produced by the activity of the user. For example: song
ratings, number of plays (if Quod Libet tracks those), playlists.

SECOND ISSUE: BACKWARD COMPATIBILITY

My take on this is that most applications that want to switch (from storing files in
$HOME/.appname/ to storing them in
$XDG_CONFIG_HOME/.appname/+$XDG_DATA_HOME/.appname) should do it like this: when
looking for data, it should first check if there is a $HOME/.appname/ directory (and
relevant files maybe). If it's there, it could either copy it to the new location, or
continue using it. If it's not there, it would then use the Freedesktop-compliant
locations.

It basically means that users upgrading won't lose configuration or data, and for new
users Quod Libet will follow the XDG Base Directory Specification.

(I have no opinion on whether the application, if the $HOME/.appname/ directory
exists, should keep using it, or copy its content to new locations and use those, or
move its content to new locations and use them. First solution seems simpler to
implement, the third one is maybe cleaner.)

@lazka
Copy link
Member Author

@lazka lazka commented Mar 14, 2015

Comment #2 originally posted by fverschelde on 2009-03-07T21:24:51.000Z:

Two more things:

  1. There is a move towards supporting this Freedesktop specification. (Which is
    logical if application developers agree that Freedesktop specs are "a good thing" and
    strive to adopt them. I'm not sure if there's a very strong consensus towards
    Freedesktop specs, though.) Right now most applications still use a $HOME/.appname
    directory, or worse (VMWare uses a $HOME/vmware directory, annoying as hell). But
    things are on the move. Applications that support this spec since at least
    summer/fall 2008 include VLC, Totem, Sound Juicer, Transmission and others.
  2. I played with the Quod Libet source (from a SVN checkout). From my experiments,
    very basic support for the Freedesktop spec implies:
  3. removing the USERDIR constant defined in quodlibet/const.py;
  4. replacing it with two constants, for instance USERCONFDIR and USERDATADIR;
  5. replacing every occurrence of USERDIR in quodlibet source with either USERCONFDIR
    and USERDATADIR.

I did just that (around 25 occurrences of USERDIR to replace), using those constants:

USERCONFDIR = os.path.join(HOME, ".config", "quodlibet")
USERDATADIR = os.path.join(HOME, ".local", "share", "quodlibet")

Which means I'm not checking whether there is a $HOME/.quodlibet directory already,
or doing fancier stuff (I'm not a developer, I'm just learning a bit of Python but
mostly for web development with Django). Well, I tried this and it worked fine.

With my limited knowledge of both Python and Quod Libet, I should be able to submit a
patch for implementing the first solution I suggested (checking if $HOME/.quodlibet
exists and use it if it does, and use the standard directories otherwise). I could
also ask for some information from people who understand the Freedesktop spec better
than I do, so that I may know where different data should go.

Tell me what you think. :)

@lazka
Copy link
Member Author

@lazka lazka commented Mar 14, 2015

Comment #3 originally posted by reiter.christoph on 2009-03-09T10:11:44.000Z:

Imho, before any coding someone should decide which files
go in which directory.

GNOME wiki quote:
"if you remove the .cache folder, the user will not notice it except maybe
for performance (might be huge performance). If you remove the .config folder,
the user will see all the preferences reset to the default but without any
data loss. In fact, it would not be a problem if the user didn't customize
his desktop."

The simplest, less error prone migration would be: only use the new schema
if there is no %HOME/.quodlibet or if there is a $XDG_CONFIG_HOME/.quodlibet
directory.

Here is a list of all used files in QL:

  1. All plugin files in XXX/plugins/(playorder|songsmenu|editing|events)
  2. Dump/MiniDump files
  3. accels file
  4. queue file
  5. songinfo file
  6. control file
  7. config file
  8. current file
  9. songs file
  10. logs folder (*.log files in XXX/logs/)
  11. list folder:
    tagpatterns / tagpatters.saved
    renamepatterns / renamepatterns.saved
    queries /queries.saved
  12. stations file
  13. playlists file
  14. browsers folder (containes py/pyc files)
  15. feeds file
  16. devices file
  17. cache file

The directories could be different if he environmen variable is set...
I guess this is right:

DATADIR = os.path.join(os.getenv("XDG_DATA_HOME", HOME, ".local", "share"), 'quolibet')
CONFIDIR = os.path.join(os.getenv("XDG_CONFIG_HOME", HOME, ".config")), 'quolibet')
CACHEDIR = os.path.join(os.getenv("XDG_CACHE_HOME", HOME, ".cache")), 'quolibet')

@lazka
Copy link
Member Author

@lazka lazka commented Mar 14, 2015

Comment #4 originally posted by reiter.christoph on 2009-03-09T14:55:58.000Z:

Found some more.. here is a first attempt.
Feedback please.

  1. All plugin files in XXX/plugins/(playorder|songsmenu|editing|events) -> data
  2. Dump/MiniDump files -> data
  3. accels file -> config
  4. queue file -> data
  5. songinfo file -> data
  6. control file -> data
  7. config file -> config
  8. current file -> data
  9. songs file -> data
  10. logs folder (*.log files in XXX/logs/) -> data/logs
  11. list folder:
    tagpatterns / tagpatters.saved
    renamepatterns / renamepatterns.saved
    queries /queries.saved -> config/list
  12. stations file -> data
  13. playlists file -> data
  14. browsers folder (containes py/pyc files) -> data/browsers
  15. feeds file -> data
  16. devices file -> data
  17. cache file -> cache
  18. cover.png -> data (notify plugin)
  19. album_pattern -> config (album browser)
  20. scrobbler_cache -> cache (qlscrobbler plugin)
@lazka
Copy link
Member Author

@lazka lazka commented Mar 14, 2015

Comment #5 originally posted by fverschelde on 2009-04-05T16:54:18.000Z:

Christoph, thanks for that work. I had missed the “cache” part.

Ploum, cited in Thibault's opening message, has a follow-up post which presents the
distinction between user data, user configuration and cache in plain English:
http://ploum.frimouvy.org/?207-modify-your-application-to-use-xdg-folders

Here is some feedback for your list of Quod Libet data. I must say i'm not a
developer per se, and don't know a lot about Quod Libet, though. ;)

Note 1: when my choice differs from yours, i indicate yours in parenthesis.
Note 2: sometimes i'm not sure what a file is about, so i only indicate your choice.
I'm not on a Linux computer right now so i can't check those files.
Note 3: i don't write subdirectories in my feedback. Of course, once a category (user
data/user config/cache) is chosen, the files can go in subdirectories like "list" or
whatever's needed.

  1. All plugin files in XXX/plugins/(playorder|songsmenu|editing|events) -> CONFIG?
    (data)
  2. Dump/MiniDump files -> not sure what it is (data)
  3. accels file -> not sure what it is (config)
  4. queue file -> CONFIG (data), whereas default setting is "empty queue"
  5. songinfo file -> not sure what it is (data)
  6. control file -> not sure what it is (data)
  7. config file -> CONFIG, obviously
  8. current file -> CACHE (data)
  9. songs file -> DATA, could have been CACHE if it did not contain metadata that
    cannot be retrieved by refreshing the library (like song ratings, number of plays, etc.)
  10. logs folder (*.log files in XXX/logs/) -> CACHE? (data/logs) since it's not user
    data per se
  11. list folder:
    tagpatterns / tagpatters.saved -> CONFIG
    renamepatterns / renamepatterns.saved -> CONFIG
    queries /queries.saved -> CONFIG
  12. stations file -> DATA, actually could be config as well but if you only want to
    backup your user data you'll probably want to backup this
  13. playlists file -> DATA, actually could be config as well but if you only want to
    backup your user data you'll probably want to backup this
  14. browsers folder (containes py/pyc files) -> not sure (data/browsers)
  15. feeds file -> not sure (data)
  16. devices file -> CONFIG? (data)
  17. cache file -> CACHE
  18. cover.png (notify plugin) -> CACHE (data), like current file that's volatile data
  19. album_pattern (album browser) -> CONFIG
  20. scrobbler_cache (qlscrobbler plugin) -> CACHE

I'll have to revise/complete this.

ON A SIDE NOTE: i guess a quick 'n dirty solution would be to move HOME/.quodlibet to
HOME/.config/quodlibet, and not to sort between user data, user preference and cache.
I can see this could be acceptable (though non conforming) for the following reasons:

  • Quod Libet doesn't generate lots of user data. The only data that's clearly uer
    data is playlists, and part of the songs database IF the user stores information in
    the songs database that can't be retrieved from the files themselves. The main user
    data quodlibet uses and even writes (through Ex Falso) is music files, and those are
    managed by the user in a non-hidden directory or other places.
  • Quod Libet and its plugins use little cache (not sure how the cache file itself is,
    but i think it's quite small), so going to the extent of using a specific cache
    folder might be overkill.
@lazka
Copy link
Member Author

@lazka lazka commented Mar 14, 2015

Comment #6 originally posted by steven.strobe.cc on 2009-06-18T08:15:20.000Z:

Since QL won't be switching names, this change should not be rushed; I'd prefer to
wait on this one until after the rather large list of changes in 2.1 have been tested
and released.

@lazka
Copy link
Member Author

@lazka lazka commented Mar 14, 2015

Comment #7 originally posted by fverschelde on 2009-06-18T11:41:18.000Z:

Long term sounds good if that means work on this issue is scheduled and it gets done
eventually. :)

@lazka
Copy link
Member Author

@lazka lazka commented Mar 14, 2015

Comment #8 originally posted by jason52lh on 2010-11-23T13:44:42.000Z:

I'd really like to see this change. this issue exist for almost 2 years but not implement...

I don't think the config files are that important that application can not switch the restore place.

This can be done in a major version change. just a warning window would be good enough. if there is any problem, user can easily solve them by google it.

Acturally i think change like this should implement as soon as possible, cuz this application is more and more widely used.

And apps like AWN, cairo-dock, rhythmbox didn't wait that long.

@lazka
Copy link
Member Author

@lazka lazka commented Mar 14, 2015

Comment #9 originally posted by jason52lh on 2010-11-23T14:15:48.000Z:

Patch: Simply make ~/.quodlibet to $XDG_CONFIG_HOME. use glib way to do this job will be much easier.

@lazka
Copy link
Member Author

@lazka lazka commented Mar 14, 2015

Comment #10 originally posted by joe.wreschnig on 2010-11-23T17:02:29.000Z:

That patch makes it impossible for me to have ~/.quodlibet, which is what I want.

@lazka
Copy link
Member Author

@lazka lazka commented Mar 14, 2015

Comment #11 originally posted by jason52lh on 2010-11-23T17:25:49.000Z:

Is there a big different to access ~/.quodlibet than ~/.config/quodlibet? if you want you can make soft link.

Do you like to see a huge number of hiden files when enable show hiden files in filemanager or use tab to complete command? It's in your $HOME dir so you have to see them a lot no matter you want to or not.

@lazka
Copy link
Member Author

@lazka lazka commented Mar 14, 2015

Comment #12 originally posted by joe.wreschnig on 2010-11-23T19:35:42.000Z:

I don't see the value in having half my files in ~/.config and half my files in ~/. I don't find value in change for the sake of change. I don't find any arguments for ~/.config persuasive. (Is the only argument "We can crappy up this folder instead of that folder?")

I've been using Unix for 18 years now and I've never been bothered by having rc files in my home directory, at least not in any way that moving them all to another directory would help, and certainly not in a sense that moving some random subset to another directory would help.

@lazka
Copy link
Member Author

@lazka lazka commented Mar 14, 2015

Comment #13 originally posted by thibaut.bethune on 2010-11-23T22:19:33.000Z:

One of the advantage of move is to allow users to easily backup their system choosing to restore only their data or their config or both easily.
It can't be done easily at the moment.
It will be very easy to create backup tools that will provide that kind of choice then.
Actually, it is a move to allow users to better handle its files to my mind.
it is definitely worth it.

@lazka
Copy link
Member Author

@lazka lazka commented Mar 14, 2015

Comment #14 originally posted by jason52lh on 2010-11-24T04:56:21.000Z:

example: you can easily clean up cache files by delete ~/.cache dir, if every apps match XDG standards, or it would be really hard for a user to do taht.

besides, there is no advantage to put all files into $HOME, either. so why not just match XDG standards?

@thibaut.bethune, actually it woundn't be done easily at any moments, and if more and more user using this, it would be harder and harder to handle all the situation.

@lazka
Copy link
Member Author

@lazka lazka commented Mar 14, 2015

Comment #15 originally posted by joe.wreschnig on 2010-11-24T12:43:38.000Z:

Why is there this arbitrary distinction between "config" and "data"? When am I going to be in a situation where I don't want to back up both? It seems like a very un-Unix split also - what's my ~/.emacs? Surely it's a configuration file, but it contains more code than many programs!

Why is there no way to configure all XDG-aware apps to behave as applications have for decades? (There's trivial ways to do this, like using XDG_CONFIG_ROOT as a string prefix rather than a parent directory, but the standard forbids them.)

Why did XDG not standardize on existing practice, as was the common way for e.g. the LSB and other open and closed standards bodies to work?

Changes like this contribute to nothing but bitrot of otherwise stable software and unnecessary code churn for active ones. This whole thing reminds me of how GConf was going to be the be-all-end-all solution for configuration storage some years ago, and it was vitally important for application authors to support that.

@lazka
Copy link
Member Author

@lazka lazka commented Mar 14, 2015

Comment #16 originally posted by thibaut.bethune on 2010-11-24T13:33:29.000Z:

Traditionally programs don't make a clear distinction between config and data (and cache) files therefore the move can't be automated.
It would have been possible if there still were /.config and /.data subfolders in any ~/.program folder but it is not the case.
The spec asks developpers to make that distinction obvious.
That lloks like a progress to me since a better classification will allow new developpements that weren't possible before.

@lazka
Copy link
Member Author

@lazka lazka commented Mar 14, 2015

Comment #17 originally posted by joe.wreschnig on 2010-11-24T16:28:38.000Z:

No, traditionally programs don't make a clear distinction between config and data because they're not different, and to pretend they are different is to pretend programmable, extensible, really configurable applications don't exist.

Traditionally, programs don't write cache files, because cache files only matter for the small set of programs that download large resources over slow links. (I think the only thing QL caches are thumbnails - which there's a separate fd.o spec for, and it doesn't use ~/.cache either. Great thing about standards, etc.) (The scrobbler cache file is identified as a cache above, but it's not, it's data.)

And there's no place in the XDG spec for QL's control, songinfo, cover.png, and other interface files. Their lifetime would be appropriate for a cache folder, but they're not cache files.

This is change, not progress.

@lazka
Copy link
Member Author

@lazka lazka commented Mar 14, 2015

Comment #18 originally posted by thibaut.bethune on 2010-11-24T22:12:57.000Z:

If my memory is correct, Rhythmbox treats covers as cache since the application automatically downloads them from the Internet.
Therefore if you delete them, they won't miss you since the application will bring them back. It will only take some more time : like caching. No data will be lost from that point of view.

@lazka
Copy link
Member Author

@lazka lazka commented Mar 14, 2015

Comment #19 originally posted by joe.wreschnig on 2010-11-24T22:58:05.000Z:

Quod Libet writes cover files alongside the audio files, where they should be. (Or in metadata tags.) The cover cache is to avoid loading sometimes extremely large files off slow media and rescaling. That's identical in purpose to the thumbnail cache, a standard for which already exists, and is not in ~/.cache. I don't understand why you brought that up though.

@lazka
Copy link
Member Author

@lazka lazka commented Mar 14, 2015

Comment #20 originally posted by thibaut.bethune on 2010-11-24T23:11:14.000Z:

There is indeed a conflit between the 2 specs : Thumbnail Managing Standard and FreeDesktop directories.

I only wanted to help by answering "there's no place in the XDG spec for QL's control, songinfo, cover.png, and other interface files. Their lifetime would be appropriate for a cache folder, but they're not cache files."

If you don't think that this spec can give the user the ability to control its configuration then i guess you should mark this bug as wontfix.
Since you are the developer, you decide : no offense :-)

@jrobeson
Copy link

@jrobeson jrobeson commented Sep 26, 2015

seems like it should be like so:

covers: $XDG_CACHE_DIR/quodlibet
control: $XDG_RUNTIME_DIR/quodlibet
playlist: $XDG_DATA_DIR/quodlibet
lists: $XDG_DATA_DIR/quodlibet
config: $XDG_CONFIG_DIR/quodlibet

anything that has trouble finding a home that isn't configuration should probably go in $XDG_DATA_DIR/quodlibet

@lazka lazka added the enhancement label Sep 19, 2016
@lazka lazka removed the bug label Sep 19, 2016
@jcrben
Copy link

@jcrben jcrben commented Nov 13, 2016

Any movement around this? Would you be happy to take a pull request?

@lazka
Copy link
Member Author

@lazka lazka commented Nov 13, 2016

No movement.

I'm for simply moving "~/.quodlibet" to "~/.config/quodlibet" and adding a symlink.
And if possible do that as atomically as possible. We'd need similar functionality for osx, see #1950.

And for cache stuff we can just switch to "~/.cache/quodlibet" without migrating data; we already use that for some cover caching atm.

@jcrben
Copy link

@jcrben jcrben commented Nov 22, 2016

I guess setting QUODLIBET_USERDIR functions as a workaround for now, right?

@lazka
Copy link
Member Author

@lazka lazka commented Nov 22, 2016

I guess setting QUODLIBET_USERDIR functions as a workaround for now, right?

yeah

@ptitjes
Copy link
Collaborator

@ptitjes ptitjes commented Mar 10, 2017

@jcrben Are you working on this ?

@jcrben
Copy link

@jcrben jcrben commented Mar 10, 2017

No, not using quod libet right now

@Socialdarwinist
Copy link

@Socialdarwinist Socialdarwinist commented Jul 14, 2017

Don’t have fixes in the Python standard library or elsewhere to be considered? For elucidation, os.environ does not support XDG directories, and as for additional libraries, there is just the abandonware pyxdg for Python 2. Right now, the access practice to the XDG variables is ugly, like reading the specifying files directly or starting subprocesses of programs that ought to be installed.

@lazka
Copy link
Member Author

@lazka lazka commented Sep 17, 2018

This was fixed by #2466

@lazka lazka closed this Sep 17, 2018
@mclark-newvistas
Copy link

@mclark-newvistas mclark-newvistas commented May 7, 2019

This is not fixed by #2466. While I appreciate taking .quodlibet out of ~, I keep .config on a different filesystem which doesn't expect very many writes. Keeping state files in .config is not okay. Above you comment that some files - ie, thumbnails - are maybe more appropriate for .cache and maybe for .local/var. Either way they aren't appropriate for .config. If you must pick a single directory, I would rather lose my config file for quodlibet in a disaster than lose the storage for all of my config files due to the medium being used as a cache directory. Data classification matters.

@CreamyCookie
Copy link
Collaborator

@CreamyCookie CreamyCookie commented Jul 24, 2019

This is now tracked in #2202.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
7 participants
You can’t perform that action at this time.