Mega abbr

[ridiculousfish]

Enhanced Abbreviations

This implements enhanced abbreviations as discussed in #8854. I think I have addressed all of the concerns raised in that discussion, with the exception of first-class support for subcommands ("expand this abbreviation if it's an argument to this command") - but users can cobble together something like that with functions, as we'll see later.

This makes abbreviations a powerful, highly customizable feature, similar to key bindings:

• Abbreviations may optionally expand as any token, not just as commands.
• Abbreviations may match a PCRE2 regex instead of a literal string. We're still limited to a single token for now.
• Abbreviations may be marked "quiet" which prevents printing the new command line. The history receives the job before it is expanded, which means the pre-expanded command may be autosuggested.
• Abbreviations can specify where they want the cursor to go after expansion. This is useful for abbreviations which want to add a suffix like "| less".
• The expansion of an abbreviation may be given by a fish function, which can inspect the matched token (or even the entire command line) and generate output.
• Abbreviations may trigger only after space (or other closing characters), or only after enter (or other exec bindings).

The new signature follows, the features are discussed later:

abbr --add NAME [--quiet] [--position command | anywhere] [--regex PATTERN]
                [--set-cursor SENTINEL] [--trigger-on entry | exec]...
                [-f | --function] EXPANSION

Breaking Changes

• Abbreviations can no longer be made universal - they have too much metadata to cram into a variable. For compatibility we will continue to import any existing universal abbreviations, but for new ones, users should instead add them to a function. We look for and call fish_user_abbreviations if it exists, analogous to fish_user_key_bindings.

Open Questions

• Previous versions of fish would install an abbr.fish function. I'm not sure if packagers will correctly erase this function when upgrading fish. If it's a problem we might mark abbr as a protected builtin so any abbr function is ignored.

• Is fish_user_abbreviations a useful addition? This is the function that is called next to fish_user_key_bindings.

• There are four different "trigger modes":

  • Expands on space or enter (current and remains the default)
  • Expands on space only, via --trigger-on entry
  • Expands on enter only, via --trigger-on exec
  • Expands after enter and after being sent to history, via --quiet

  I feel this is a bit confusing and I welcome better naming / organization ideas here.

• I'm not sure if we should aim for fish 3.6 or later - it depends how much confidence we have in the interface I guess.

Detailed Discussion

(This is just copy and pasted from the new docs.)

With --quiet, the expansion occurs without printing the new command line. The original, unexpanded command is saved in history. Without --quiet the new command line is shown with the abbreviation expanded. If a --quiet abbreviation results in an incomplete command or syntax error, the command line is printed as if it were not quiet.

With --position command, the abbreviation will only expand when it is positioned as a command, not as an argument to another command. With --position anywhere the abbreviation may expand anywhere in the command line. The default is command.

With --regex, the abbreviation matches using the regular expression given by PATTERN, instead of the literal NAME. The pattern is interpreted using PCRE2 syntax and must match the entire token. If multiple abbreviations match the same token, the last abbreviation added is used.

With --set-cursor, the cursor is moved to the first occurrence of SENTINEL in the expansion. That SENTINEL value is erased.

With --trigger-on entry, the abbreviation will expand after its word or pattern is ended, for example by typing a space. With --trigger-on exec, the abbreviation will expand when the enter key is pressed. These options may be combined, but are incompatible with --quiet. The default is both entry and exec.

With -f or --function, EXPANSION is treated as the name of a fish function instead of a literal replacement. When the abbreviation matches, the function will be called with the matching token as an argument. If the function's exit status is 0 (success), the token will be replaced by the function's output; otherwise the token will be left unchanged.

Examples

Here is the much-longed-for history expansion (#288). I think we should ship this, but it would be a separate PR:

function last_history_item
    echo $history[1]
end
abbr -a !! --position anywhere --function last_history_item --quiet

We can use --set-cursor to append a | less and leave the cursor BEFORE the bar:

abbr -a L --position anywhere --set-cursor ! "! | less"

---

Here we can use a regex to implement zsh suffix aliases: you can "execute" a text file to edit it in vim.

function vim_edit
    echo vim $argv[1]
end
abbr -a vim_edit_texts --position command --regex ".+\.txt" --function vim_edit

---

Here we make an abbreviation c expanding to checkout, but ONLY if the current job starts with git. Note using commandline to inspect text outside of the matched token:

function git_c
    string match --quiet 'git c ' -- (commandline -j); or return 1
    echo checkout
end
abbr -a git_c --position anywhere --regex c --function git_c

---

This abbreviation creates a "template" for processing directories:

abbr 4DIRS --trigger-on entry --set-cursor ! "$(string join \n -- 'for dir in */' 'cd $dir' '!' 'cd ..' 'end')"

after triggering it looks like this:

for dir in */
    cd $dir
    _
    cd ..
end

with the cursor positioned at the _.

[floam]

This looks fantastic. I

Here is the much-longed-for history expansion (#288). I think we should ship this, but it would be a separate PR:

That'll be a popular feature to get out and it will be nice to have a working abbreviation example out of the box.

[faho]

This does look very nice.

I'll have more thoughts later, for now:

Is fish_user_abbreviations a useful addition? This is the function that is called next to fish_user_key_bindings.

I don't think it is, tbh. The reason for fish_user_key_bindings was that it was necessary because you couldn't add bindings in config.fish. Now that we have binding presets, which allow you to just put bindings in config.fish, I don't see a lot of use of it anymore.

For this, the function isn't necessary and you can just put them in config.fish, or if you want nicer organization put them in a snippet - conf.d/abbrs.fish.

I'd nix it.

Previous versions of fish would install an abbr.fish function. I'm not sure if packagers will correctly erase this function when upgrading fish

Package managers should remove the file. Packages typically are built in a clean environment (not as incremental builds from older releases, possibly even a clean chroot/container or build server).

But people who installed using make install would be affected.

Abbreviations may be marked "quiet" which prevents printing the new command line. The history receives the job before it is expanded, which means the pre-expanded command may be autosuggested.

This would make abbreviations more of "an actual thing", and I'm not sold on it.

Currently, one of the cool things about abbreviations over aliases is that they don't exist. You type the keys, the expansion happens, the commandline contains what is actually executed. So there can never be any confusion about what would actually run - you can see it, you can inspect it.

Of course with this, abbreviations are more complicated so having a quiet abbr makes you keep the abbr that you used in mind, so there's a trade-off, but I'm not sure --quiet is right.

This now looks like actual syntax features, so people might be tempted to use it in scripts or functions.

--

Also some preliminary thoughts before I try this myself: What happens if I add a costly function and run it on the regex .*? Will people use this to implement #8371?

[faho]

Alright, some observations after I tried it out, purposely without looking at the code. In no particular order:

1. This is a nice, free, speedup for people with abbrs in their config.fish - from 500micros per-call to 20micros. The time for sourcing (the abbrs from) https://github.com/lewisacidic/fish-git-abbr/blob/master/git-abbr.fish goes from 26ms to 0.644ms.
2. --global and --universal are ignored, so this should keep people's config working
3. -f as a short form for --function doesn't exist?
4. Things still happen on a per-word basis. This should be okay, and should prevent a lot of complaints about how slow it is because people add slow functions.
5. The first matching abbr is used, in definition order:

abbr --add literal --regex 'foob' literal
abbr --add regex --regex 'foo.' regex

Enter "foob ", it is replaced with "literal". Turn the definitions around and it's replaced with "regex". That's probably okay.

6. The expand-abbr bind function doesn't seem to be what triggers the expansion - I can't seem to find a way to not expand an abbr where it would match. That seems like it's necessary (sorry, I appear to have tested this wrong)
7. Even --regex abbrs still need a name - this is probably for --erase?
8. Could we add similar regex-matching to bind? Might be a way to get a better vi-mode?
9. --function takes a function name, not a script. You can't do

abbr a --function 'string replace a b -- $argv[1]'

This will spew an "unknown command 'string replace...'" error. I'm not sure what to do about it?

[faho]

One small option incompatibility that I noticed in my personal config:

This no longer ignores options after the first non-option, so my abbrs:

abbr --add e emacs -nw
abbr --add usc systemctl --user

failed. Adding a -- separator works. This seems like it might be reasonably common in practice, but I'm not sure if fixing it is worth it? With the new options it would effectively force you to use the name as a very late argument.

[krobelus]

this commit breaks this test:

diff --git a/tests/checks/tmux-prompt.fish b/tests/checks/tmux-prompt.fish
index e1f129e56..e8d82aad6 100644
--- a/tests/checks/tmux-prompt.fish
+++ b/tests/checks/tmux-prompt.fish
@@ -18,3 +18,15 @@ set -U prompt_var changed
 tmux-sleep
 isolated-tmux capture-pane -p
 # CHECK: prompt 0> <changed>
+
+set -U prompt_var
+# Execute a multi-line command with the cursor on the first line, to make sure we move the
+# cursor down before execution.
+isolated-tmux send-keys 'echo multi-line' M-Enter 'echo command-line' C-p Enter
+tmux-sleep
+isolated-tmux capture-pane -p
+# CHECK: prompt 0> <> echo multi-line
+# CHECK: echo command-line
+# CHECK: multi-line
+# CHECK: command-line
+# CHECK: prompt 1> <>

[floam]

The on-exec behavior should be a nicer approximation of real aliases than our alias create-a-wrapper function (for interactive session, natch).

[floam]

The abbr completions need updated.

abbr --
-a  --add                                  (Add abbreviation)
-e  --erase                              (Erase abbreviation)
-g  --global        (Store abbreviation in a global variable)
-h  --help                                             (Help)
-l  --list                     (Print all abbreviation names)
-q  --query                 (Check if an abbreviation exists)
-r  --rename                         (Rename an abbreviation)
-s  --show                          (Print all abbreviations)
-U  --universal  (Store abbreviation in a universal variable)

[floam]

This is bike shedding but I think it'd be easier to use if this followed function --on-event, et cetera and just required --on-exec rather than an argument to --trigger-on to change the behavior. --on-entry need not even be created, since the immediate switcheroo is the default behavior.

[floam]

Except, it'd be nice if we got wrapped completions e.g. abbr --add foo --trigger-on=exec clang showed clang completions with foo <TAB> just like alias foo clang.

[floam]

We don't require --add for complete or bind, why should abbr? Especially if it's not setting a variable or managing some backing store in the fishd file.

[faho]

Here's another idea: Expand variables used as commands:

function replace_command_var
    set -l varname (string replace -r '^\$' '' -- $argv[1])
    set -q $varname
    or return 1

    echo -- $$varname
end

abbr -a commandvariables --regex '\$.*' --function replace_command_var --position command

This will work if something is defined globally, so you can do $EDITOR foo.txt, and it will turn it into e.g. emacs -nw foo.txt.

If the variable is only local, the function won't have access to it.

---

Another idea: Remove implicit cd and replace it with this:

function autocd
    test -d "$argv[1]"
    or return 1

    type -q "$argv[1]"
    and return 1

    echo -- cd $argv[1]
end

abbr -a autocd --regex '.*' --function autocd --position command

I do actually find this worth thinking about, I'm not just kidding here.

---

So far, I have not found any use case for --trigger-on.

[faho]

We don't require --add for complete or bind, why should abbr? Especially if it's not setting a variable or managing some backing store in the fishd file.

@floam:

This, like the old function, defaults to --add mode. --add is not required.

[floam]

Strange, I can't remember what error I got when trying to use abbr that led me to think it needed --add.

[faho]

Okay, once more about --quiet:

If we shipped a default global abbreviation for !! (which isn't out of the question), making it quiet seems like it has potential for confusion.

Because that means you type sudo !!, and you see sudo !! in your history, but that's not actually what ran, and !! isn't a part of the language, but just some reader trickery. I find it too hard to find out what really ran, actually.

Plus, bash has the "histverify" option that allows you to edit the command before execution, because one of the big downsides of its history expansion is how immediate it is. This is reasonably popular.

To be honest I'd probably drop --quiet entirely.

[floam]

Minor thing, this should say the option expects an argument, not that it is an unknown option.

> abbr --trigger-on
abbr: --trigger-on: unknown option

[krobelus]

This looks like amazing work.

So far, I have not found any use case for --trigger-on.

Because that means you type sudo !!, and you see sudo !! in your history, but that's not actually what ran, and !! isn't a part of the language, but just some reader trickery. I find it too hard to find out what really ran, actually.

I agree.

I guess the motivation for both --trigger-on and --quiet is to be able to emulate Bash's !!, !$ and so on.
I wonder what the motivation for Bash's behavior is, perhaps they don't want to rewrite what the user typed, but we already do that with normal abbreviations.
Haven't done research but I don't see a reason to emulate Bash's complex UI here.

Of course we didn't think of this when discussing the design :(

But people who installed using make install would be affected.

Ok, I guess enough reason to add a wrapper function for a few releases.

[faho]

I wonder what the motivation for Bash's behavior is, perhaps they don't want to rewrite what the user typed, but we already do that with normal abbreviations.

My guess is that they only thought of histverify (which does rewrite without executing, but only after you press enter) after the rest was implemented and didn't want to change the defaults, or it worked like this in other shells.

Ok, I guess enough reason to add a wrapper function for a few releases.

An empty file would be enough. If that doesn't work, just some comments in there.

[faho]

I'm not sure if we should aim for fish 3.6 or later - it depends how much confidence we have in the interface I guess.

What I would do is remove the other trigger modes and --quiet. It's much easier to add them later than to remove them.

That leaves as the signature:

abbr --add NAME [--position command | anywhere] [--regex PATTERN]
                [--set-cursor SENTINEL]
                [-f | --function] EXPANSION

I think we could change --position to one option --anywhere and then add more options if we wanted to add more positions, but the current way is also fine.

We could also add more short options (-r for symmetry with string match), but then of course they clash with the already existing abbr --rename and such. A break here would, imho, be acceptable (you don't rename abbrs in a script).

Anyway, those are all niggles. I think this interface - the one without the additional trigger modes - would be acceptable to me for 3.6. We'd have to figure out some of the bugs (-f doesn't exist) and differences (options are now parsed after nonoptions, so some existing calls break). Afer that we'd be missing all the auxilliary stuff - completions, documentation (the !! FAQ entry should probably be reworded?).

Then, once that's done, we can think about where and how we would add default abbreviations (can't really go in __fish_config_interactive because that's done too late), and how much we want to do with them. It ranges from !! to implicit cd to actually just completely ripping out alias and replacing it with a wrapper around abbr.

[faho]

One more idea for the interface: Have --set-cursor use a default sentinel, say %.

This means:

abbr foo --set-cursor 'bar % baz'

expands "foo" to "bar baz" and leaves the cursor in-between the two, where the % was. To give a different sentinel, because you use a "%" there, you would have to use --set-cursor='{}' as an optional argument (are multi-char sentinels accepted?).

The idea is that "%" is reasonably rarely used (or should be, maybe there's a better char?), and so most uses won't have to care about a sentinel.

Also I'd rename "sentinel" to "marker" in the docs.

[mqudsi]

This is incredibly powerful and extensible, as in both at the surface level and in a "the more that you think about it, the more you can (ab)use it to do interesting things." Which is (mostly!) a really good thing.

Things that immediately struck me reading the (excellent) writeup, before reading the code or trying this myself:

• While the programmer in me loves the idea of having a bitmask of triggers, I'm not sold on that being ideal from a user-facing or support-related aspect. Can we fold the triggers so that they're mutually exclusive and one is a superset of the other? Are there cases worth supporting for each bit on its own and each bit in conjunction with each of the others?
• Setting the sentinel via a placeholder rather than position is clever because it lets you avoid "how many chars is it after expansion" but I don't like using anything that's already got a meaning in the default docs (such as !). I also don't like that people would have a tendency to space-delimit the sentinel (to make their commands make sense and work on their own) but when you elide it, there will be double-space (but not a big deal).

More technical matters:

• Can we document (and enforce) from the start that abbreviations are not loaded in non-interactive mode and make abbr a complete no-op in that case? I would like to prevent anyone using them to make their scripts more opaque, and I would like to someday make it possible (or at least remotely possible) to generally load user scripts without user config.
• Mostly OT, but I've been bitten by make install not dropping existing functions before (in my case, writing a completion that depended on a removed function). I cannot remember if I committed it or not, but pre-cmake I clearly remember bringing up checking for and using rsync instead of install if were present (originally I was of the opinion we should nuke the existing folders before install but someone, David I think, pointed out a reason not to). I think doing that again w/ our cmake-based installer is still a good idea. In all cases, I don't think the previous presence of an existing function should be something that factors too heavily in our considerations, as it is quite the crutch.
• Depending on when @zanchey wants to do a 3.6 (and going by just some of the more pressing recent bugfixes, I think it's going to be soon) I wouldn't want this to land before it can be really tested and refined.

[faho]

Can we document (and enforce) from the start that abbreviations are not loaded in non-interactive mode and make abbr a complete no-op in that case?

They always have been a no-op. The reader does the actual abbreviation handling, so registering them does nothing. Even read --shell mode doesn't have access (see

fish-shell/src/builtins/read.cpp

Line 212 in 022f42c

conf.expand_abbrev_ok = false;

).

This is another case for abbr --quiet being weird - because abbreviations don't "exist", type my-cool-abbr won't say anything, so you now have something you can run that doesn't exist. Of course it would be possible for type to be adjusted, but then you have a discrepancy between interactive and non-interactive (or just "inside a function").

[faho]

Another neat possiblity:

abbr --add dotdot --regex '^\.\.+$' --function multicd
function multicd
    echo cd (string repeat -n (math (string length -- $argv[1]) - 1) ../)
end

Turns ..<space> into cd ../ and ...<space> into cd ../../

[faho]

Another behavior change I'm not sure is on purpose:

> abbr --add foo echo bar

> if foo<press return>

will no longer expand "foo". With fish 3.5.1 it does.

[burneyy]

Great stuff! I have been waiting for the --set-cursor option for some time and am actively using the compiled fish shell from your branch.

Some feedback regarding the --trigger-on option:
Why not just rename the options to space and enter? Of course people might have remapped expand-abbr to a different key but I feel like it would still be more intuitive than entry and exec (at least I had to double-check with the documentation to know which is which).

[faho]

Here's what I would do, summarized:

1. Remove the trigger modes and --quiet - they are confusing and could lead to a lot of awkward support requests
2. Remove fish_user_abbreviations - it's not helpful
3. Add an empty abbr.fish function file for easier upgrading
4. Add "-f" short form for "--function" or remove it from the docs
5. Possibly make it ignore options after the first non-option for compatibility - this is awkward but not doing it might cause a lot of old abbrs to fail
6. Make --set-cursor use a default sentinel of "%", making the sentinel argument optional?
7. Figure out why if someabbr<return> fails to expand the abbr
8. Fix freeze that happens with echo )

The most important bit here is that I have now become convinced the trigger modes and --quiet are a bad idea. They make it hard to reason about what is actually run - now potentially every single error someone reports could be down to a --quiet abbr breaking the actual input. That's the worst part of bash's history expansion (without histverify) and zsh's global aliases (what does ls L even mean - considering that L could be a global alias for | less - and yes, people do this). Also it circumvents the "expand-abbr" function, so you can't skip these by pressing ctrl-space instead of space.

[floam]

I agree with @faho's laundry list except for the discomfort with '--quiet'. I'd vote to retain that ability.

[ridiculousfish]

Thank you all for the thoughtful feedback. I have removed --quiet and --trigger-on, and fixed the other issues identified.

abbr --add e emacs -nw is a good catch. For compatibility, any unrecognized option after the first two positionals will be considered part of the abbreviation. This is a little unusual but I think it will be fine.

• faho: Remove fish_user_abbreviations function
• faho: Make abbr.fish empty so it overwrites existing function
• faho: Remove quiet abbreviations
• faho: Add -f as short form for function
• faho: The first matching abbr is used - it should be the reverse, the most-recently added abbr is used first. LMK if you still see this.
• faho: Even --regex abbrs still need a name - this is probably for --erase? Right
• faho: This no longer ignores options after the first non-option.
• krobelus: broken test
• floam: completions need updating
• floam: --on-entry instead of --trigger-on
• floam: wrapped completions for trigger-on exec. No longer relevant since abbreviations expand eagerly.
• floam: don't require add. This works and continues to work.
• faho: cool ideas about generalized commands
• faho: remove trigger-on
• faho: Switch -r from --rename to --regex
• faho: default sentinel
• faho: rename to marker
• mqudsi: fold triggers (these just got removed)
• mqudsi: document and enforce that abbreviations are interactive only. I documented this more forcefully but don't want to make abbr a no-op
• mqudsi: keep the abbr.fish function file, but now it only has a comment.
• faho: abbreviations don't expand for incomplete command lines
• faho: What to do about function, not script? This is a function so that it may be passed arguments. I made this error on functions which contain spaces, hopefully that catches some of these.

[krobelus]

the other day I was confused by this weird universal variable

$ set -U | grep abbr
_fish_abbr__21_21_ '--position anywhere --trigger-on exec --function bangbang'

turns out that was because I temporarily downgraded to a version without this PR while having this in my config.fish:

abbr !! --position anywhere --function bangbang

if I remove that from my config, then fish import that universal variable, so !! would actually expand to

--position anywhere --trigger-on exec --function bangbang 

Anyway this is a user error, likely not to affect many users and easy to fix.

[glyh]

I stumbled on this from #5003 , could anyone provide an example on how to use this to make sudo expand abbreviations correctly?

like sudo y -> sudo yay where y is a defined abbreviation.

I feel --position anywhere is not a correct solution...

[faho]

@glyh Please in future open a new issue with your question, a large PR like this is terrible for answering related things after the fact.

That being said, see #9411 - currently, making an abbr just for a specific command is harder than it should be, but that one contains some examples.