Allow to set phive home directory via environment variable

[dnaber-de]

I know I can already change the default home directory of phive via command line option:

https://github.com/phar-io/phive/blob/master/src/shared/config/Config.php#L34-L39

But this is a bit cumbersome and error prone like I would loose all cached phars if I forget it in one command or just add a typo.

It would be convenient if I could change this via an environment variable like PHIVE_HOME. The implementation could look like this:

    public function getHomeDirectory(): Directory {
        if ($this->cliOptions->hasOption('home')) {
            $dir = new Directory($this->cliOptions->getOption('home'));
            $dir->ensureExists();

            return $dir;
        }
        
        if(false !== getenv('PHIVE_HOME')) {
            $dir = new Directory(getenv('PHIVE_HOME'));
            $dir->ensureExists();
            
            return $dir;
        }

        $dir = $this->environment->getHomeDirectory()->child('.phive');
        $dir->ensureExists();

        return $dir;
    }

Background is that I am working with a CI environment that does not cache files in ${HOME} but only in ${HOME}/cache so I could speed up the build process when I could set PHIVE_HOME=${HOME}/cache/.phive. I do the same with GNUPGHOME and several other tools. If you agree on that I'd be happy to provide a PR including unit tests of course.

[theseer]

I like the idea. So yes, please provide a PR 👍

[ktomk]

Would also be great if $HOME/.config/phive is preferred over $HOME/.phive (preferred = only if directory exists) and $XDG_CONFIG_HOME/phive also preferred over $HOME/.config/phive if $XDG_CONFIG_HOME is set.

PHIVE_HOME should override still any of these, only topped by --home.

Compare with COMPOSER_HOME, this is useful for usability and to not clutter the users home-dir so much with dot-files.

Let me know what you think, if it has some initial support, looks like a good follow-up to me on this new PHIVE_HOME enviroment parameter.

[theseer]

Hi @ktomk, thanks for adding some thoughts..

The idea of following XDG was already shortly discussed in #233 and I'm still not any further in terms of deciding.

I do see the point of not cluttering ~ with additional folders, yet I don't really like the separation into completely independent directories.

While I guess we could implement a lookup hierarchy as you mentioned, it makes things rather complex: There is no $basedir + /foo anymore but more of a $basehome + (if A then /foo, if B then /foo/bar, if C /foo/bar/baz) plus deciding what to actually create in case it wasn't there yet...

Not sure what is best.

[ktomk]

Yes, these are all valid considerations and it must not be something complicated with such changes. Skimming #233 it appears to me the XDG directories are looked for from the wrong side ("das Pferd wird von hinten aufgezäumt" as it could be said in German).

To support XDG dirs in my view is not to introduce new directories where phive does not have them but instead make use of them where applicable. Otherwise it would indeed make it complicated as it would put something over the current design of phive and I would defer design changes to when its due (albeit there may be such future reasons I thoughtfully ignore these now), as me reading you disliking it as well: Introducing new directories for what reason? Just because we can?

What I would suggest instead is that phive has its current runtime parameter - let's call it the same as the new environment variable - PHIVE_HOME (in italics following if it's the runtime parameter) whichs current default value is $HOME/.phive (correct me if I'm wrong).

Right now (before the environment variable PHIVE_HOME was landing), it is hard-encoded for the /.phive part apart from $HOME for its value. the utility name prepended with the dot (".") character.

So by default PHIVE_HOME is $HOME/.phive. If the directory does not exists, I assume that phive creates it when needed, e.g. that is expected to happen somewhere after command-line options, arguments and the environment have been parsed.

And PHIVE_HOME can currently be controlled via the --home option with its directory option-argument. In pseudo bash-like code:

# A-1: default
PHIVE_HOME="$HOME/.phive"

# A-2: --home option and option-argument
if has_option_argument "--home"; then
  buffer="$(get_option_argument "--home")"
  if [[ ! -d $buffer ]]; then # test $buffer is not a directory
      invalid_argument "--home" "$(printf "not a directory: '%s'" "$buffer")"
      exit(1)
  fi
  PHIVE_HOME=buffer
fi

# B: file-system init/re-use
mkdir -p -- "$PHIVE_HOME"

Just a quick simplification as I understand it of the current PHIVE_HOME parameter with the default (A-1), the command-line option (A-2) and it's initialization for the phive utility (B).

In this issue (#286), it is extended by the PHIVE_HOME environment variable:

# A-1: default
PHIVE_HOME="${PHIVE_HOME-$HOME/.phive}"

# ...

That is: If the environment variable PHIVE_HOME is set, it will be used instead of the default.

The name of the internal phive runtime parameter PHIVE_HOME does not change, nor does it change of how it is handled, see (B).

Let's consider this already done, the PHIVE_HOME environment variable is considered working.

What would then be needed is to support $HOME/.config/phive as PHIVE_HOME default given the directory exists:

# A-1: default
buffer="$HOME/.phive"
if [[ -z ${PHIVE_HOME+x} ]]; then # test PHIVE_HOME environment variable is set
    buffer="$PHIVE_HOME"
elif [[ -d "$HOME/.config/phive" ]]; then # test new default is directory
   # A-1-NEW-1: new default phive home directory (changed compared to old phive releases)
   buffer="$HOME/.config/phive"
elif [[ ! -d "$buffer" ]]; then  # test old default is _not_ a directory
  # A-1-NEW-2: new default (new phive release is on a fresh system)
  buffer="$HOME/.config/phive"
fi
PHIVE_HOME="$buffer"

# ...

This example enlarges the single-liner A-1 default from previous by keeping the PHIVE_HOME environment variable override and introduces the new default $HOME/.config/phive either if the user moved the configuration already there (A-1-NEW-1) or phive is running for the first time (A-1-NEW-2).

It shows again, that the handling of the PHIVE_HOME runtime parameter is not affected, only the way how the default PHIVE_HOME value is obtained.

It adds a little bit of file-system I/O though, there are one or two more directory tests due to the changes/transition of the current default $HOME/.phive to $HOME/.config/phive.

So how would the XDG directory standard align with this? As phive only has a configuration dir (data and cache directories are subdirectories of it in phives' design), it is just a check for the XDG_CONFIG_HOME environment variable in the string expression:

• from: "$HOME/.config/phive"
• to: "${XDG_CONFIG_HOME:-$HOME/.config}/phive" (use XDG_CONFIG_HOME or if unset $HOME/.config, then append /phive)

(I spare to add another code-example as this should be straight-forward to see in the previous one at the A-1-NEW-* locations).

So different to the outline in #233, here is and remains the single phive directory that currently is. The XDG dir spec is made use of for the part that applies to phive. In case XDG does not apply on the system, the user already benefits from the less cluttered home dir as it is expected that other tools also use the $HOME/.config directory, with a sub-directory of their utility names.

There is relatively little to test (one or two directories) and by properly identifying the PHIVE_HOME runtime parameter (for the rest of all code in phive) there should be no changes on how to treat the directory it represents within the existing procedures.

The change is limited to how the PHIVE_HOME runtime parameter is passed to phive via the environment, utility arguments (and then as well state of the file-system), I'm not totally into the code structure of phive, but I can imagine this is kind of limited to the module parsing CLI input and setting up phive runtime parameters (if it not even already exists).

Last but not least I'd like to note that with this outline it's possible that existing users can migrate back to a previous older version of phive. When they want to benefit from the new default directory, they can just move it from the old path to the new. New users don't have to learn about the past unless they want to play with an old version. This all works without touching users files (no magic directory migrations behind the scenes necessary).

So while this might appear all shining, what practical downsides did I ran over? If I compare it with composer, their docs got stale. Originally defaulting to $HOME/.composer with my (very, very old install) when I wrote some documentation for pipelines where caching is involved, I was reading about it in the composer docs where it stated $HOME/.config/composer which I could not use to warm-up pipeline (container) caches from my local system as the directory was not in use. But that was minor and appears to me as a documentation issue, not prolonging about previous formats. Displaying the config setting on the command-line revealed the correct location immediately. A pretty minimal, but a not so good experience at least.

Composer changed away from the ~/.composer directory quite some time ago and from what I know there was a long-time migration window (multiple years, interesting to find out) in which a mixed-mode was supported. I don't know the details, from my outline above I'm not so creative right now to bring it to fall, maybe you have some ideas to attack it, maybe there is something to gather from the composer project as well, e.g. checking a bit of the history and issues. What they have and I left out is Windows support, from what I know the same principles should apply and composer has some battle-proven lines of code there under a compatible license that can be helpful when it comes to implementation.

[theseer]

Wow. Thanks for the long response.

While it may seem like it will do the effort you put into writing this up an injustice, let me quote one small fraction to summarize the most imporant aspect of your post for me:

The change is limited to how the PHIVE_HOME runtime parameter is passed to phive via the environment, utility arguments (and then as well state of the file-system), I'm not totally into the code structure of phive, but I can imagine this is kind of limited to the module parsing CLI input and setting up phive runtime parameters (if it not even already exists).

That is - unfortunately - not correct because it's not the full picture.
We do have various files and things to take care off, only a fraction being configuration:

• an actual config
• auth data for private repositories like passwords or keys
• our "private" gpg keyring
• our repositories.xml meta file whith alias definitions (which phive will download/update as needed)
• caching data from github and others
• The phars downloaded and installed
• The usage database file

All these files currently reside in a "custom" (you may probably call it arbitrary or proprietary) directory structure within $HOME/.phive. Following XDG's recommendation would require us to split this into various subfolders, @MacFJA outlined in the ticket I referenced. Let me quote that fraction again, to make it easier to follow:

For directories in $HOME$, I prefer the XDG/Freedesktop way to avoid dozen of .$PROGRAM$/ in my home directory:

* `$HOME$/.config/phive/` (`auth.xml`, `repositories.xml`, `local.xml`)

* `$HOME$/.cache/phive/` (`http-cache/`)

* `$HOME$/.local/share/phive/` (`phars/`, `gpg/`, `phars.xml`)

* `/tmp/$RAND$-phive/` (`_tmp_wrk/`)

If we were to support all what has been proposed and what you outlined, you won't have a single "$PHIVE_HOME", because conceptually that doesn't exist. And that would, imho, render the very effort of this PR useless ;)

Don't get me wrowng: I'm not per se against implementing this, I just don't (yet?) see how this would make the world better rather than just more complex without any actual benefit or in best case just different.

[ktomk]

@Theser, all fine and thanks for your reply. I think the crucial point is this:

Following XDG's recommendation would require us to ...

Such a requirement appears imagined to me. I even think it turns my suggestion into borderline malicious if I would consider it only at far so. And your reply then very, very clam calm and kind.

I proposed to use one single environment variable XDG_CONFIG_HOME:

$XDG_CONFIG_HOME defines the base directory relative to which user specific configuration files should be stored. If $XDG_CONFIG_HOME is either not set or empty, a default equal to $HOME/.config should be used.

(Environment variables - XDG Base Directory Specification)

Maybe you have a problem because you think these aren't all "user specific configuration files", then maybe it helps if you try to see it with my eyes: If I as user use phive, I'm pretty sure all these files created are because of my user-specific configuration.

As far as I know the XDG standard does not define and would therefore not require utility authors to re-define what "user specific configuration files" construe. Neither is there a test that could be applied nor is there a body that certifies an implementation.

Even if my suggestion would violate the standard in real or thought, isn't the suggested use fail-safe, portable, in the interest of the user and even assuming a wrong or incomplete implementation of the XDG base-dir specs in this case not still much more towards it?

Maybe you can elaborate a bit more about the motivation to change the single, working base-directory-structure phive has? My first tendency is to defer such an invasive action as much as possible as I see no factual requirement to do so, nor do I see any practical benefit of doing it (now). Please help me to learn more. What do I not see?

[theseer]

I admit I didn't spent much time looking into the XDG specs so far. But looking at how even composer is cluttering round, my previous statement still holds.

Composer stores files in:

• .config/composer
• .cache/composer
• .local/share/composer

It's the .cache/composer directory, where composer stores all downloaded "packages", It's .local/share/composer where composer keeps backups of itself (why ever that location was choosen..).

To compare, JetBrains does the same:

• .config/JetBrains
• .cache/JetBrains
• .local/share/JetBrains

In .config all the actual global config settings and license information is stored, .cache is used for actual cache stuff, and in .local they keep the installed plugins and related files.

So again, my point of criticism (if you want to call it that way) is that instead of having one directory to store all our things in (following our own directory layout), we'd fan out into at least 3 if not more directories that do not have a common base anymore - unless you consider ~ a common base directory ;)

So while the specs may say you can use the $XDG_CONFIG_HOME for a pointer to find the .config, directory, that's probably correct and nice, it's not helping us to locate the other 2 (or more) directories.

Yes, in theory we could simply move all our stuff in .config/phive, but i'm pretty certain that was not the idea behind XDG's .config approach. Again, I didn't yet have time to really read the specs.

Maybe I'm still missing something or took a wrong turn anywhere, but I fail to see how this is better ;)

[ktomk]

@theseer Me also does not want to make the impression that I have the best understanding of it. I like this sentence to nail it:

Yes, in theory we could simply move all our stuff in .config/phive, but i'm pretty certain that was not the idea behind XDG's .config approach. Again, I didn't yet have time to really read the specs.

I'm feeling lucky, because I don't have to decide on it. I would opt for the simple theory and then see if the computer verifies it after written down but not without saying that it's totally fine to do it the way you like. The main point for me is that I find it risky and error-prone to destroy the common base this way.

Btw, I have the tendency to put XDG a bit aside, I don't know how you feel about. I'm thinking more towards temporary files within that structure and if it wouldn't be a good test to start with those first.

That is perhaps also a benefit for the reported issue here, that is, temporary files should not end up in build caches (also not in $HOME I guess) where they become a burden to maintain. No idea how composer does it thought, in pipelines I have a couple of routines that take care of temporary files but I don't care about windows much so no idea how portable the code is.

[theseer]

I'll close this as implemented.

We can open a new discussion issue for XDG directories, but I'm still not convinced we should support it.

[ktomk]

@theseer ack, convinced or not convinced if I may ask?

[theseer]

not ;) sorry, updated the comment.

[ktomk]

all fine.