Replace ~ only works at beginning of a word

[danielmatz]

It seems like the ~ only gets replaced with the home directory when it is at the beginning of a word. Is that intentional?

I tried this in fish:

$ env DIR=~ bash -c 'echo $DIR'
~

And the same thing in bash:

$ env DIR=~ bash -c 'echo $DIR'
/Users/dmatz

But then in fish this works as expected:

$ echo ~
/Users/dmatz

[ridiculousfish]

It looks like bash will expand ~ after an = sign, but not after a letter. Anyone care to track down what the actual rules are?

[KamilaBorowska]

The exact rules are undocumented, but I assume the rules look like (regex) ^([A-Za-z_]+=)?~. Well, technically bash documentation says that it's done for part after = for variable assignment, but this is not a variable assignment, is it? Perhaps it's some quirk of bash grammar, which barely sees any difference between commands and arguments.

[cben]

testing bash

https://www.gnu.org/savannah-checkouts/gnu/bash/manual/bash.html#Tilde-Expansion

Each variable assignment is checked for unquoted tilde-prefixes immediately following a : or the first =. In these cases, tilde expansion is also performed.

Yikes, the : only works after an equal sign:

bash
$ echo foo=~/bar
foo=/home/bpaskinc/bar
$ echo foo:~/bar
foo:~/bar
$ echo ~/bar:~/baz
/home/bpaskinc/bar:~/baz
$ echo PATH=~/bar:~/baz
PATH=/home/bpaskinc/bar:/home/bpaskinc/baz

More precisely, it really seems to check the left side looks like a variable name:

$ echo =~/bar:~/baz
=~/bar:~/baz
$ echo X=~/bar:~/baz
X=/home/bpaskinc/bar:/home/bpaskinc/baz
$ echo --X=~/bar:~/baz
--X=~/bar:~/baz
$ echo --X ~/bar:~/baz
--X /home/bpaskinc/bar:~/baz
$ echo 1=~/bar:~/baz
1=~/bar:~/baz

So, this really looks like a heuristic to match variable assignments, but doesn't actually check the position in the command line, so also works for env FOO=~/bar, make FOO=~/bar etc.
I could have sworn bash also expands --option=~/bar, reinforcing which is well established equivalence with --option ~/bar but I can't reproduce.

Bash in posix mode

didn't test as fully, seems to behave similarly, except this really weird corner:

$ set -o posix  # POSIX mode
$ echo PATH=~/bar:~/baz:~/quux  # WAT? all EXCEPT first expanded??
PATH=~/bar:/home/bpaskinc/baz:/home/bpaskinc/quux
$ echo ~/bar:~/baz:~/quux  # same as normal mode
/home/bpaskinc/bar:~/baz:~/quux

$ set +o posix  # regular GNU mode
$ echo PATH=~/bar:~/baz:~/quux  # all expanded
PATH=/home/bpaskinc/bar:/home/bpaskinc/baz:/home/bpaskinc/quux
$ echo ~/bar:~/baz:~/quux
/home/bpaskinc/bar:~/baz:~/quux

[tested on GNUa «bash», versio 5.0.7(1)-release (x86_64-redhat-linux-gnu)]

Does POSIX have an opinion?

https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_06_01

A "tilde-prefix" consists of an unquoted <tilde> character at the beginning of a word, followed by all of the characters preceding the first unquoted <slash> in the word, or all the characters in the word if there is no <slash>. In an assignment (see XBD Variable Assignment), multiple tilde-prefixes can be used: at the beginning of the word (that is, following the <equals-sign> of the assignment), following any unquoted <colon>, or both. A tilde-prefix in an assignment is terminated by the first unquoted <colon> or <slash>. If none of the characters in the tilde-prefix are quoted, the characters in the tilde-prefix following the <tilde> are treated as a possible login name from the user database.

Sounds pretty much like bash. The weird corner in bash posix mode actually sounds like a bug?

~user expansion

There is also the rarer ~username form looking up other users home dir.

bash
$ echo HOMEDIRS=~root:~nosuchuser:~bpaskinc
HOMEDIRS=/root:~nosuchuser:/home/bpaskinc

fish supports it too, only at start of word:

fish
> echo ~root ~nosuchuser ~bpaskinc
/root ~nosuchuser /home/bpaskinc

Saner proposal for fish

(Ignoring backward compatibility concerns, optimizing for new fish users.)

1. IMHO there is no reason to match posix/bash 100%. Those rules are heuristic and violate law of orthogonality.

2. Expanding after = is good, and should be always done. Use cases:

   • env FOO=~/bar (which fish encourages lacking builtin VAR=val cmd syntax)
   • make FOO=~/bar
   • prog --long-option-dir=~/bar ⬅️ really big "least surprise" win imho!
     • In fish, = option form works especially well with arrays / command expansion:
       compiler --include=~/libs/$LIBS -> compiler --include=/home/me/libs/lib1 --include=/home/me/libs/lib2

   "Always expanded after =" is much simpler to remember than any conditions on left side.

3. Would it be convenient if fish just expanded ~ everywhere? No:

   • Backup files (Emacs convention): rm file~, diff file{~,}
   • URLs: curl https://example.edu/~professor/paper.tex — it's important NOT to expand ~ after /!
   • gitrevisions: git show HEAD~2

4. Not sure about colons. fish somewhat abstracts away colons by preferring arrays, but env PATH=~/bin:~/.local/bin and similar are useful. I lean towards yes, let's do it.
   If we do, this should be "always expanded after :" with no conditions on left side. Most importantly ~/a:~/b must match behavior of FOO=~/a:~/b.

5. Would it be convenient and simple to always expand after non-letters?
   I don't know, URLs use case suggests at least "except slash".
   I guess it's better to start from = and : and expand later if needed than from "any punctuation" and have to add exceptions later...

[cben]

zsh

https://unix.stackexchange.com/questions/373519/expansion-of-tilde-in-zsh
By default only expands per POSIX — start of word and in actual assignment (I mean with empty zshrc, don't know what zsh-newuser-install tends to set up in practice...)
But it offers magic_equal_subst option which is slightly wider than the bash:

All unquoted arguments of the form anything=expression appearing after the command name have filename expansion (that is, where expression has a leading ~ or =) performed on expression as if it were a parameter assignment.

The practical difference is:

% echo --long-opt=~/foo:~/bar
--long-opt=/home/bpaskinc/foo:/home/bpaskinc/bar

However, you still need the =. Well, it's also expanded at start of word, but then following : are not expanded, leaving some weird rough corners:

% echo --long-opt=~/fly:~/fools=~/bar:~/baz=~/great:~/quux
--long-opt=/home/bpaskinc/fly:/home/bpaskinc/fools=~/bar:/home/bpaskinc/baz=~/great:/home/bpaskinc/quux
% echo colon:~/foo                                                           
colon:~/foo
% echo ~/bar:~/foo                                               
/home/bpaskinc/bar:~/foo
% echo ~/bar:~/foo=~/causality/violation
/home/bpaskinc/bar:/home/bpaskinc/foo=/home/bpaskinc/causality/violation

---

So I stand by my recommendation for fish to always expand after = or : as simplest possible rule covering common use cases.

[zanchey]

Great research - thank you!

I certainly think starting with expanding after = is a reasonable thing to do. Colons, in view of the new path variable mechanics, may not be worth doing.

[kuon]

I also encountered the make path=~/... issue. Took me a while to understand why it was failing.

I realize there is no fix at present, but is there a workaround?

[ridiculousfish]

I still would like to fix this, but as a workaround you can use $HOME instead of ~.