Improvements to switch-buffer: tagged buffers; and better interaction with buffers when the prompt is open: keybinding to switch to/from prompt to buffer

[Kabouik]

switch-buffer with the power of fuzzy search is one of the great features Nyxt offers. One thing I love about it is it can live-preview buffers (provided they have been loaded, else it will load them first) and then cycling through the preview of each buffer is very fast.

However, unless the buffer selection is confirmed with Return and the prompt is closed (which implies losing any keyword filtering, and also implies extra keystrokes as well as some UI delays), one cannot interact much in the previewed tabs: scrolling works, but text input and clicking on hyperlinks are not possible. Yet, there are multiple cases where one would be interested in interacting with buffers without closing the prompt, particularly when switching buffers frequently to just peek at them or copy/paste stuff from one buffer to a text field of another one, or when filtering the buffer list with a search in the prompt input.

The video below doesn't show much, every Nyxt user already knows about the power of fuzzy search in the switch-buffer view. I recorded it just to illustrate a simple case with buffers opened on git-related stuff and on work-related stuff. Keeping the prompt open could be convenient for a user ready to sacrifice some vertical space, if full interaction was possible when the prompt is open. For instance I could cycle through git buffers only, or work buffers only, and keep a clean view of my dynamic buffer list without compromising the browsing experience.

Possible improvements could be:

• allow interaction with web buffers and internal buffers when the prompt is open; I suppose one major limitation is the prompt would have to be updated when following a link in a buffer
• support switching focus between prompt and buffers with a keybinding toggle (it already works with the mouse), perhaps with an indicator like the vi I and N indicators, or a mode
• support selecting buffers with the mouse in the switch-buffer list (I tried at 0'18" but eventually used the keyboard, sometimes the mouse is convenient too for lazy browsing)
• ability to add tags to currently opened buffers: in the video below I was lucky that most my work-related buffers had "org" in their top-level domain and all the others had "git" in their URL, but that won't always be the case. There was no way to group the IPCC buffer with the "org" buffers in my search for instance. Adding tags to buffers could allow filtering them by tag later on, regardless of their domain, URL, or title; and optionally those tags could be saved per domain with some rules in auto-config.lisp without creating bookmarks
• maybe it's already possible: exclude matches from the buffer list with some regex sorcery?
• probably already doable: combination with macros for pre-configured filters

screenrecorder-2021-07-23_10.59.01.mp4

I think there would be a wide range of uses for this. And also more specific ones: with tags, I could even filter my buffers with "work" if I have to discuss something with my boss in front of my computer and don't them to see that I also have 10 tabs open on puppies and cats of the Internet. Of course I could also temporarily reduce unwanted buffers, but the use case is not exactly the same and I couldn't hide those tags in just a keybinding, while I could define commands or macros to switch-buffers on a certain keyword with just a keystroke.

[Ambrevar]

Lots of great ideas here! :)

Technically nothing seems to be very difficult, but first we must
discuss how we want this to happen in terms of UX.

• allow interaction with web buffers and internal buffers when the prompt is open

• support switching focus between prompt and buffers with a keybinding
  toggle (it already works with the mouse), perhaps with an indicator
  like the vi I and N indicator, or a mode.

That the mouse enables focus is actually an not well defined behaviour
:p

The first two points are actually the same thing: we need to be able to
move focus between the main buffer, panels and the prompt buffer.

We could also anticipate window manager and support multiple main
buffers.

Suggestion: We need a current-focused-buffer that returns which GTK
view is focused.
Then in on-signal-key-press-event and friends instead of

(or (current-prompt-buffer)
    (active-buffer sender))

we would just cal (current-focused-buffer).

• support selecting buffers with the mouse in the switch-buffer list

Already listed in atlas-engineer/prompter#32.

• ability to add tags to currently opened buffers: in the video below
  I was lucky that all my work-related buffers had "org" in their
  domain, but that won't always be the case. Adding tags to buffers
  could allow filtering them by tag later on, regardless of their
  domain, URL, or title; and optionally those tags could be saved per
  domain in auto-config.lisp without being bookmarks.

We've discussed buffer groups in the past, tags are probably a better
approach.

See also #1573
and #565.

maybe it's already possible: exclude matches from the buffer list with some regex sorcery?

It's possible by defining a custom prompt and setting prompter:filter,
but there is currently no example of this.

We would need a more systematic approach, for instance a default filter
which support regexp if some flag is on.

But I'm also wondering if regexps are the right approach. There are
annoying to write, most people won't ever use them beside trivial
expressions.

Could we do better with a more graphical and interactive approach? Like
some sort of match rule selector?

probably already doable: combination with macros for pre-configured filters

Can you expand?

I think there would be a wide range of uses for this. And also more
specific ones: with tags, I could even filter my buffers with "work"
if I have to discuss something with my boss in front of my computer
and don't them to see that I also have 10 tabs open on puppies and
cats of the Internet. Of course I could also temporarily reduce
unwanted buffers, but the use case is not exactly the same and I
couldn't hide those tags in just a keybinding, while I could define
commands or macros to switch-buffers on a certain keyword with just a
keystroke.

What about using another Nyxt window for this kind of things?

In #178 we discuss how to
map a window to a buffer, but we could also map a window to a subset of buffers.

[Kabouik]

As someone who doesn't understand anything in regex, I couldn't agree more. I was just thinking it might be good if regex was supported so that advanced users can define complex rules in their config, but was hoping some shortcuts would be implemented as well so that the filter function can actually be used on the fly too without going into complex syntaxes. In the end, perhaps only a very small fraction of users would use regex anyway, so simple rules might be enough.

Maybe something similar to what search engines usually offer would work, like +git -lab to include all buffers containing git but not lab in their domain? Of course that filter would not make much sense in real world, it's just a silly example. Maybe we could still prepend special characters with \ when we want to filter on them instead of using them as operators, like \+ if we want to filter buffers that contains + in their description or title. I guess \ prefixes are still a regex thing. I know kakoune's search for instance uses that to search special characters in buffers.

Can you expand?

What I had in mind was a way to automatically append a string to the input field of switch-buffer. Forgive me as this is not correct CL syntax, but something along those lines (switch-buffer STRING) where STRING could be a search term, and therefore the corresponding filter could be set to a keybining, like M-g to show all git buffers, and M-r to show all Reddit buffers, and so on. After quick discussion on IRC, macros in their current form are probably not the best way to do that, but perhaps it's already doable with more complex commands. If the tag system is implemented and if domains can be given some tags automatically from config file, this could get very powerful and allow working around domains that do not contain the given filter (such as notabug or codeberg for "git", or a given set of URLs for "perso", etc.).

What about using another Nyxt window for this king of things?

This would work for some users, but other users tend to have a strict view on whether they should keep concurrent windows for their web browser. I'm not saying I can explain why with relevant arguments, but I certainly can say I belong to that group. I suppose this would use more resources, may pose some challenges to restore multiple sessions, and most importantly, having several windows of the same application can be annoying in tiling window managers. For instance if you have a rule that moves your web browser to a given workspace, having two or three windows for the browser will tile them on that workspace. You can of course use a tabbed or stacked view of the windows with some WMs, and users of tiling WM would probably know how to do it, but this will eat some space in most cases by adding a tab bar to the windows.

[Ambrevar]

I believe special syntaxes are used interactively mostly for the following operations:

• "starts with"
• "does not contain"

Anything else?

Regexps are actually not very useful for "does not contain".

So either we go with our own special syntax, like - as you suggested, or we
try to mold an interactive interface for this.

For instance, we could have multiple prompt buffer inputs:

• line 1 would be what to match;
• line 2 what to exclude.

What I had in mind was a way to automatically append a string to the input field
of switch-buffer.

This you can easily do by writing a custom command which takes the suffix as
parameter and appends it to the prompt buffer input.

Let me know if you want a concrete example.

Of course it would solve your problem only in a clunky way.

I suppose this would use more resources,

Not really, it's the same Nyxt process and the same WebKit processes.

may pose some challenges to restore multiple sessions,

should not be a problem either, session persistence is independent of windows.
Should we add a window information to buffers, we would have nothing to update,
it would automatically be persisted and the "1-to-1" map mode would be in charge
of remapping the restored buffers.

[Kabouik]

I believe special syntaxes are used interactively mostly for the following operations:

• "starts with"
• "does not contain"

Anything else?

Regexps are actually not very useful for "does not contain".

So either we go with our own special syntax, like - as you suggested, or we
try to mold an interactive interface for this.

For instance, we could have multiple prompt buffer inputs:

• line 1 would be what to match;
• line 2 what to exclude.

I can see how an interactive interface with consecutive prompt inputs would help learning, but I assume most Nyxt user would rather just use operators to directly type what they want in one go. This is what most web applications offer too: one text field where you can use things like + - | & or date: from: before: and whatnot for chats/emails. I think we would need some operator for "OR` and "AND" in addition to "contains" and "does not contain". Why using "starts with" instead of "contains"?

What I had in mind was a way to automatically append a string to the input field
of switch-buffer.

This you can easily do by writing a custom command which takes the suffix as
parameter and appends it to the prompt buffer input.

Let me know if you want a concrete example.

I was confident this would be doable with :input in the command but an example would be great!

I suppose this would use more resources,

Not really, it's the same Nyxt process and the same WebKit processes.

Good to know!

may pose some challenges to restore multiple sessions,

should not be a problem either, session persistence is independent of windows.
Should we add a window information to buffers, we would have nothing to update,
it would automatically be persisted and the "1-to-1" map mode would be in charge
of remapping the restored buffers.

That's good news! However my personal use case would very probably still be to use only one window as much as possible, and probably reconsider what I am doing if it requires multiple windows and window manipulation. But filter-buffer and some filter-buffer-preset1 in my configuration would already help me a lot. And so would tags and custom rules to assign tags to certain domains too; although I realize this might be some work and may not happen soon or even ever.

[Ambrevar]

I can see how an interactive interface with consecutive prompt inputs would help learning, but I assume most Nyxt user would rather just use operators to directly type what they want in one go. This is what most web applications offer too: one text field where you can use things like + - | & or date: from: before: and whatnot for chats/emails. I think we would need some operator for "OR` and "AND" in addition to "contains" and "does not contain". Why using "starts with" instead of "contains"?

What I dislike with this approach is that computer users got to learn a new syntax for
every new app and new web service out there.

Ideally I'd like to offer an unambiguous, trivially accessible and
all-powerful search.

Another option would be to have buttons or actions in a menu to insert
the syntax for you. A bit like a completion framework.

I'd also love to stick to a Lispy syntax:

• it'd be more consistent,
• it has ideal expressiveness,
• it will save us from the legion of pitfalls the inevitably come
  together with home brewed syntaxes.

In the past (Nyxt 1.5) we had Lispy bookmark filtering, like so

(or (and foo bar) (and qux baz))

I'd love to re-introduce it in a generalized way.

With some smart input, the about can be really fast and we can enable
completion upon ( (and auto-insert )).

That's good news! However my personal use case would very probably
still be to use only one window as much as possible, and probably
reconsider what I am doing if it requires multiple windows and window
manipulation. But filter-buffer and some filter-buffer-preset1 in
my configuration would already help me a lot. And so would tags and
custom rules to assign tags to certain domains too; although I realize
this might be some work and may not happen soon or even ever!

Sure, I can totally see how buffer tags have their use.
Would you be interesting in contributing this feature?

[Ambrevar]

I was confident this would be doable with :input in the command but an example would be great!

Something along these lines (untested):

(define-command switch-buffer-suffix (&key id (current-is-last-p nil) suffix)
  "Switch the active buffer in the current window.
Buffers are ordered by last access.
With CURRENT-IS-LAST-P, the current buffer is listed last so as to list the
second latest buffer first."
  (if id
      (set-current-buffer (buffers-get id))
      (prompt
       :prompt "Switch to buffer"
       :input suffix
       :sources (list (make-instance 'user-buffer-source
                                     :constructor (buffer-initial-suggestions
                                                   :current-is-last-p current-is-last-p))))))

Then you can bind a key to

(make-command switch-buffer-FOO ()
   (switch-buffer :suffix "foo"))

and the resulting command will have "foo" pre-inserted in the prompt. Is this what you want?

[Kabouik]

I was confident this would be doable with :input in the command but an example would be great!

Something along these lines (untested):

(define-command switch-buffer-suffix (&key id (current-is-last-p nil) suffix)
  "Switch the active buffer in the current window.
Buffers are ordered by last access.
With CURRENT-IS-LAST-P, the current buffer is listed last so as to list the
second latest buffer first."
  (if id
      (set-current-buffer (buffers-get id))
      (prompt
       :prompt "Switch to buffer"
       :input suffix
       :sources (list (make-instance 'user-buffer-source
                                     :constructor (buffer-initial-suggestions
                                                   :current-is-last-p current-is-last-p))))))

Then you can bind a key to

(make-command switch-buffer-FOO ()
   (switch-buffer :suffix "foo"))

and the resulting command will have "foo" pre-inserted in the prompt. Is this what you want?

Awesome. That's exactly what I was looking for, yes! Then I can see that becoming even more useful when web-buffers can be interacted with while the prompto is expanded (text input, following links with the mouse, etc.) in the future or when/if tags can be automatically assigned to domains. I had to add (in-package :nyxt) to make the first block work, but I am still getting a warning with the second where I set the suffix, resulting in that second command not being listed in the prompt:

; file: /home/mathieu/.config/nyxt/init.lisp
; in: MAKE-COMMAND SWITCH-BUFFER-GIT
;     (NYXT:SWITCH-BUFFER :SUFFIX "git")
; 
; caught WARNING:
;   :SUFFIX is not a known argument keyword.
; 
; compilation unit finished
;   caught 1 WARNING condition

[Kabouik]

I can see how an interactive interface with consecutive prompt inputs would help learning, but I assume most Nyxt user would rather just use operators to directly type what they want in one go. This is what most web applications offer too: one text field where you can use things like + - | & or date: from: before: and whatnot for chats/emails. I think we would need some operator for "OR` and "AND" in addition to "contains" and "does not contain". Why using "starts with" instead of "contains"?

What I dislike with this approach is that computer users got to learn a new syntax for
every new app and new web service out there.

Ideally I'd like to offer an unambiguous, trivially accessible and
all-powerful search.

Another option would be to have buttons or actions in a menu to insert
the syntax for you. A bit like a completion framework.

I'd also love to stick to a Lispy syntax:

• it'd be more consistent,
• it has ideal expressiveness,
• it will save us from the legion of pitfalls the inevitably come
  together with home brewed syntaxes.

In the past (Nyxt 1.5) we had Lispy bookmark filtering, like so

(or (and foo bar) (and qux baz))

I'd love to re-introduce it in a generalized way.

With some smart input, the about can be really fast and we can enable
completion upon ( (and auto-insert )).

That makes perfect sense. I am not familiar with Lisp at all (as you noticed, I'm sure) which is probably why I didn't even think about it for filters, and I was thinking about keeping the number of keystrokes as low as possible, pretty much like what some search engines use (combination of + - operators and quotes). However, it is true that it is not necessarily standard or optimal.

Just keeping the number of keystrokes low is important in my opinion, so that people can realistically use the filter from their prompt and not only from predefined commands in their configuration. I am not a big fan of multiple steps in the prompt if it can be done in one line, but you are right that it would help discovering/learning/mastering the feature. Autocompletion and buttons as you suggested might be a great solution to all this, but I suspect it would be a lot of work for something maybe just a fraction of users would use on a regular basis? I don't want to distract you guys from more important things.

That's good news! However my personal use case would very probably
still be to use only one window as much as possible, and probably
reconsider what I am doing if it requires multiple windows and window
manipulation. But filter-buffer and some filter-buffer-preset1 in
my configuration would already help me a lot. And so would tags and
custom rules to assign tags to certain domains too; although I realize
this might be some work and may not happen soon or even ever!

Sure, I can totally see how buffer tags have their use.
Would you be interesting in contributing this feature?

Of course! Keep in mind that my skills are very limited though. You are lucky you are not on IRC to see all the silly questions I ask to aartaka and jmercouris every so often. I am happy to test anything though, including broken things, and hopefully by the time this is on the short-term to-do list, I will be able to write a couple simple commands in common lisp and really help.

[Ambrevar]

Sorry, I mistyped the example, should obviously be switch-buffer-suffix, since switch-buffer does not have a :suffix keyword argument aindeed.

[Kabouik]

That fixed the warning but switch-buffer-GIT still isn't listed in the prompt, and if I try assigning a binding to it in my override-map, I get:

<WARN> [11:33:42] Warning: Error on GTK thread: The value
                       SWITCH-BUFFER-GIT
                     is not of type
                       NYXT::NYXT-KEYMAP-VALUE

[Ambrevar]

We could keep the number of keystrokes minimal by auto-inserting Lispy snippets.
Example, where | is the caret position after insertion.

• ( would insert (| ) and open a new sub prompt-buffer with all available commands.
• ! would insert (not |)
• TAB would exit the local parentheses and understand nesting. For instance, pressing TAB in
  (and (not foo|) ) would move the caret as in (and (not foo) |).

[Ambrevar]

About your warning: can you show me the whole code?

[Kabouik]

I pushed the whole config here (the command is defined in init.lisp to get as much info in terminal as possible, the keybinding is set in base/keybindings.lisp). And below is the full terminal output:

<INFO> [11:33:41] Listening to socket "/run/user/1000/nyxt/nyxt.socket".
Nyxt version 2.1.1-411-g06db9aad
<INFO> [11:33:41] Loading Lisp file "/home/mathieu/.config/nyxt/auto-config.lisp".
<INFO> [11:33:41] Loading Lisp file "/home/mathieu/.config/nyxt/init.lisp".
<INFO> [11:33:41] Loading Lisp file "~/.config/nyxt/themes/kabouik-standard-dark.lisp".
<INFO> [11:33:41] Loading Lisp file "~/.config/nyxt/base/keybindings.lisp".
<INFO> [11:33:41] Loading Lisp file "~/.config/nyxt/base/urlprompt.lisp".
<INFO> [11:33:42] Loading Lisp file "~/.config/nyxt/base/commands.lisp".
<INFO> [11:33:42] Loading Lisp file "~/.config/nyxt/base/glyphs.lisp".
<INFO> [11:33:42] Loading Lisp file "~/.config/nyxt/base/domainrules.lisp".
<INFO> [11:33:42] Loading Lisp file "~/.config/nyxt/ex/specificurl.lisp".
<INFO> [11:33:42] Loaded config.
<WARN> [11:33:42] Warning: Error on GTK thread: The value
                       SWITCH-BUFFER-GIT
                     is not of type
                       NYXT::NYXT-KEYMAP-VALUE
^C^C<INFO> [11:33:47] Deleting socket "/run/user/1000/nyxt/nyxt.socket".

Nyxt doesn't start until I remove the keybinding that broke GTK.

[Ambrevar]

You misunderstood, you must bind a key to the make-command.
make-command has no effect as a top-level expression.

More specifically,

(define-key *my-keymap* "C-x C-b" (make-command ...))

[Ambrevar]

You should use (in-package :nyxt-user), not :nyxt.

[Kabouik]

Then I might have other issues because I get warnings with (define-command switch-buffer-suffix …) unless I add (in-package :nyxt) in init.lisp. I initially didn't have it in my configuration.

I'm also struggling with adding the key to my other custom bindings even if I directly embed (make-command …) there, but I'll keep trying and maybe ask on the forum to keep this issue clear.

[Inc0n]

I would like to bring attention to column ordering, where a good example that I wish the order to be changed is switch buffer, where I think title before url, instead of url at first column to allow better visibility, personally at the very least.

Is there a way to configure this?

[jmercouris]

We don't have a way to change ordering, only to toggle columns. I wonder how we might extend the prompt buffer to support ordering.

[Ambrevar]

It would be nice if the user could drag-and-drop the columns, thus providing a familiar user experience. Another key aspect is that current the first column is special in the sense that it is cannot be hidden, plus it is used for things like copying and pasting. If we are to reorder columns, we would need to visually and programmatically mark the "key column".