XDG Base Directory Specification not followed

[ghost]

Describe the bug
Ghidra does not follow the XDG Base Directories Specification.

To Reproduce
Steps to reproduce the behavior:

1. Install ghidra
2. Run ghidra

Expected behavior
Following the XDG Base Directory Specification.

Environment (please complete the following information):

• OS: Arch Linux
• Java Version:

openjdk version "1.8.0_222"
OpenJDK Runtime Environment (build 1.8.0_222-b05)
OpenJDK 64-Bit Server VM (build 25.222-b05, mixed mode)

• Ghidra Version: 8369ef1

[ghost]

The homepage claims support of Linux and macOS. Perhaps it would be useful to use tags to differentiate between various priority levels of bugs, rather than treating minor bugs as "enhancements". Not sure how the application's config directory is handled on Windows, but I hope it is with APPDIR, and not the same dotfolder which aren't even hidden on Windows.

[nsajko]

@billop To be fair, the specification has nothing to do with Linux (the kernel). So this is not a defect/bug.

But this is coming from someone who sometimes uses some Environment variables defined by the specification.

[ghost]

Good point, I meant a mainstream linux distro, speaking of "Linux" as an OS. I assumed that Ghidra touting support for Linux meant that as well.

[hadess]

This is blocking my attempt at packaging Ghidra for Flatpak, as the application expects to be installed in a directory which is writable by the user, which is not going to be the case for Flatpak or system-installed software.

For reference: flathub/flathub#1068

[ryanmkurtz]

How does XDG handle remote home directories? If you are storing a lot of large cached files in ~/.cache/, is there anything following the standard that will stop those files from being written to the remote location?

[Avamander]

@ryanmkurtz

How does XDG handle remote home directories? If you are storing a lot of large cached files in ~/.cache/, is there anything following the standard that will stop those files from being written to the remote location?

Safe to assume it's handled better in those setups than a random dotfolder in $HOME.

[ryanmkurtz]

Ghidra's cache directory is not in $HOME...

[ryanmkurtz]

Do be more clear, I'm asking if XDG offers something similar to %LOCALAPPDATA% in Windows. We need to support "roaming profiles" and can't store large cached files in the user's home directory unless it can somehow be exempt from remote storage while in there like it can on Windows. That's why Ghidra's cache directory is currently in the far-from-ideal temp directory on linux.

[nsajko]

@ryanmkurtz I don't know anything about Windows, but it sounds like you want XDG_CACHE_HOME. It's described on the Arch wiki page which Billop referenced in the original post above, and this seems to be the official reference: https://specifications.freedesktop.org/basedir-spec/basedir-spec-latest.html

Basically, if I defined XDG_CACHE_HOME to be the name of a directory, Ghidra should put all cache files recursively within that directory. (To be clear, the cache files would most appropriately go within that subdirectory, e.g. into $XDG_CACHE_HOME/ghidra.)

[ryanmkurtz]

@nsajko, thanks, that is my understanding too. And by default, that will be at $HOME/.cache/Ghidra for Linux users. Now, if Ghidra gets deployed onto a network where people use network-mounted home directories, it will be a problem if $HOME/.cache/Ghidra gets copied to the network, as it can get quite large. It would also be an issue to ask every user to set $XDG_CACHE_HOME to something outside of their home directory. My main question is how does XDG address this use case?

[nsajko]

Now, if Ghidra gets deployed onto a network where people use network-mounted home directories, it will be a problem if $HOME/.cache/Ghidra gets copied to the network, as it can get quite large. It would also be an issue to ask every user to set $XDG_CACHE_HOME to something outside of their home directory.

Presumably whoever deployed Ghidra would set the XDG_CACHE_HOME environment variable. Basic Unix usage as far as I see.

I'm not sure if you already know this, but in Unix (and Linux) environment variables commonly get inherited from the parent process, so it is enough to set the environment variable once in some config file. For example: /etc/profile, or $HOME/.profile, or it could be done with Systemd. Or you could set it specifically for Ghidra, with a wrapper script, for example.

In any case, I don't quite see what's the problem here, but I hope I'm helping.

[LCRERGO]

Actually the distro maintainers do not assume that the XDG variables are set, what happens generally when you develop an application that follows the XDG specification is that they try to use the XDG variables if they are set, if not they use what should be "the default" location. e.g:
XDG_CACHE_HOME = $HOME/.cache

It should also be noted that there 5 directories/variables that are included in the XDG specification, but normally on most applications only XDG_CACHE_HOME, XDG_CONFIG_HOME and XDG_DATA_HOME are used. For more details on that I suggest XDG Base Directory.

[sirus20x6]

Please follow xdg to the letter. There is no reason to clutter the home directory with this stuff.

[Jappie3]

Bumping this issue... it shouldn't be difficult to implement (check env var, otherwise default to the correct location - which is NOT ~/.ghidra) and the XDG spec should really be followed. Apps cluttering the home directory is extremely frustrating for end users.

[sirus20x6]

You know what. If I have time maybe I'll fix it and make a pull request.

[liff]

If it’s of any help, I’ve a small experiment here. Only ever tested briefly on Linux.

[ryanmkurtz]

Marking this Internal-only because if we decide to implement it, we will want a consistent multi-platform solution and there are a lot of subtleties through the codebase that will have to be addressed.

[Avamander]

@ryanmkurtz XDG is only relevant for one platform though, just like for example %appdata% on Windows.

If anything, switching to XDG-BD is fixing a bug coming from the mistaken assumption the current approach is multiplatform.

[ryanmkurtz]

I know that it's only for Linux, but if we make a change, we will want to use the standard locations on all platforms, not just do something special for Linux (i.e., not create a ~./ghidra/ directory on macOS or Windows either).

We do not consider Ghidra creating a ~/.ghidra directory on Linux a bug. It is intended behavior.

[sirus20x6]

We do not consider Ghidra creating a ~/.ghidra directory on Linux a bug. It is intended behavior.

It maybe intended behavior, but maybe intend something better

[Adamcake]

Four years since this issue was opened. XDG isn't a very big spec folks, you can read it in about 5 minutes. All I see in this thread is you making excuses, none of which are anywhere near acceptable, and the combined time it took you to type all of them would probably be less than it would have taken to fix this.

[hadess]

Four years since this issue was opened. XDG isn't a very big spec folks, you can read it in about 5 minutes. All I see in this thread is you making excuses, none of which are anywhere near acceptable, and the combined time it took you to type all of them would probably be less than it would have taken to fix this.

I'm sure this will make it faster for the code to be written and merged. No, wait, the opposite.

[dragonmacher]

I imagine the big issue with this request is that it would require a solution tailored to each major platform we support. Currently Ghidra uses 1 model that is the same for all OSes. This makes working on the system simpler for the maintainers. While implementing XDG may be simple enough, reworking this part of the system to have a cohesive model for all OSes is less so.

[ryanmkurtz]

I think changing the directories for each platform would be pretty quick and easy, as would be incorporating the XDG environment variables for Linux and possibly macOS. There are non-obvious changes that would have to be made in addition to these easy things though too:

• Continue to support the legacy locations so an upgrade could find the old files
• Provide something in the GUI that would show the current location of all of these directories, maybe in the About window
• Provide a script that would wipe out all or some of these directories. It's very convenient for the devs to tell people to just wipe out their ~/.ghidra directory to fix certain issue or rule out others. Since these locations would now be unpredictable...we'd need a script.
• Ensure we are no longer writing anything to the ghidra installation directory. We have been slowly addressing this over the years, but certain things still do it, like the recompilation of the .sla files. I am not sure if there are more things like that, but we'd have to fix each one individually.

So, not the hardest thing in the world to accomplish, but also not a quick change to some hardcoded paths either. I'll bring this up again with the team though.

[Adamcake]

Currently Ghidra uses 1 model that is the same for all OSes.

If Ghidra was coded under the mistaken assumption that there's a cross-platform data storage location, then that's the root of the issue. Start there and work upwards. It's APPDATA on Windows, XDG basedir on Linux, and I don't know what it is on MacOS but probably XDG too.

Continue to support the legacy locations so an upgrade could find the old files

Ideally, software should just not have any code that writes to the home directory at all. But the way lots of people are solving the "legacy location" issue is by having an environment variable that tells it the location of its data folder, and default to ~/.ghidra if that env var isn't set. This was an acceptable solution for Audacity, Python, and Git, just for a few examples, so I'd say it'd be an acceptable solution here too. Not ideal, of course... but acceptable.

[ryanmkurtz]

I'm working on this now and have a question for people who actually use XDG. For Ghidra's "user temp" and "user cache" directories, we have been putting them in /tmp/<user>-Ghidra on Linux. In my branch this can now be overridden with XDG_RUNTIME_DIR and XDG_CONFIG_HOME, and will default to java.io.tmpdir and $HOME/.config Typically, java.io.tmpdir will just be /tmp, so it seems necessary to incorporate the user's name into the subdirectory within /tmp. Other times though, the directory may land within the user's home directory, making it redundant to include the user's name. Ideally I wouldn't ever have to incorporate the user's name...I don't recall seeing this from other program's too often. Does anyone know how directory isolation is achieved in practice when the specified locations are in common directories like /tmp?

[Adamcake]

XDG_RUNTIME_DIR and XDG_CONFIG_HOME are always user-specific, and the default value of $HOME/.config is obviously user-specific, so no need to worry about those. When it comes to java.io.tmpdir, anything you store in there should be handled as if it were specific not just to the current user, but to the current process. That is, anything you create in /tmp should not be expected to persist between runs and should be deleted on program exit when possible. If two or more Ghidra processes run in parallel, they should not have access to each other's tmpdirs. If that doesn't sound like your use-case then it probably isn't the right place to store those things. (in which case may I suggest XDG_CACHE_HOME instead?)

If you do want to use tmpdir, then, once you've decided on either XDG_RUNTIME_DIR or java.io.tmpdir, I think you would use Files.createTempDirectory to create a new and unique directory there. I've never used it myself but I've just done a bit of searching around and it looks like this method wraps libc's mkdtemp() which is the correct way to do it. Below code is untested.

String prefix = "Ghidra";
Path temp = Files.createTempDirectory(java.io.tmpdir, prefix);
temp.toFile().deleteOnExit();

So no need to incorporate the user's name as far as I can see.

[ryanmkurtz]

Thank you. So is it an error to define XDG_RUNTIME_DIR to be in a subdirectory of /tmp that is owned by the user? If so, what other directory meets the criteria of being automatically deleted when the system reboots?

Or maybe we don't need to control Ghidra's temp usage via XDG, and just have it always use java.io.tmpdir with per-process calls to Flies.createTempDirectory? Is that objectionable?

Also, generally speaking, if XDG_CONFIG_HOME gets set to something not user specific, is that considered a user-error (i.e., a case I don't need to worry about)?

[Adamcake]

My advice would be to get the value of XDG_RUNTIME_DIR, or if none is set then use the value of java.io.tmpdir instead; store the value and use it for any and all calls to File.createTempDirectory. That would be entirely fine. Temp directories should be specific to one Ghidra session, which makes them implicitly user-specific regardless of where they actually are.

Just out of interest, the value of XDG_RUNTIME_DIR on my system is /run/user/1000.

generally speaking, if XDG_CONFIG_HOME gets set to something not user specific, is that considered a user-error (i.e., a case I don't need to worry about)?

Correct. The basedir spec defines XDG_CONFIG_HOME as: "the base directory relative to which user-specific configuration files should be stored". In fact, that's what the _HOME part means, it implies it will be inside the user's home directory. In general you can just assume these paths are correct and not do any validation on them (well not beyond reporting and exiting on i/o errors anyway.)

[ryanmkurtz]

Ok, thanks again!