New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

History synchronization #825

Closed
dag opened this Issue May 24, 2013 · 38 comments

Comments

Projects
None yet
@dag
Contributor

dag commented May 24, 2013

The question of whether fish can synchronize history between running shells came up twice on IRC today, and has been asked before as well.

With fishd I imagine we have some of the infrastructure for this in place already, so perhaps implementing it wouldn't be too hard. However, I also think this is something not everyone wants to have all the time, and we also want to avoid adding configuration options. So the question then is, can we accommodate most needs without configuration?

One idea is to have something like say, history --synchronize, and then provide events like "executed a commandline" or perhaps it's enough to do a function -v history, and then people who want this can script it.

I don't think that idea is optimal: it's not discoverable (you have to script it) and you can only have either behavior at any given time.

I instead propose making the history wrap around to a synchronized history; that is, if you step or search forward in the history when you are at the bottom (such as is the normal state when on the commandline and you haven't used history-search-backward yet), you're acting on a reversed history of commands entered in any shell. This way, the current behavior of history-search-backward is retained and can be used just like we've always use it, but we can also initiate a history search forwards to navigate the universally synchronized history.

Remaining questions:

  • Should the synchronized history also include items from the local history? I think it should, because some people will prefer to always use the synchronized history, and because I think it'll be easier to remember to use the synchronized history when you need it than the opposite.
  • What should happen if you do something like C-p C-n C-n? I think this should work just like only doing C-n, which is to say "get the latest command used in any shell", and which is in line with the view of these two histories "wrapping around" above and below the commandline.

(BTW, this idea comes from zsh, which I hear can be made to do something like it, although I haven't actually studied exactly how it behaves.)

@haarts

This comment has been minimized.

Show comment
Hide comment
@haarts

haarts May 24, 2013

Contributor

I was there during the discussion and I find you've put it quite elegantly!
I support the proposed solution, including the two raised questions. Perhaps we should abandon the naming of 'backward' and 'forward' and instead call it 'universal' and 'local'.

Contributor

haarts commented May 24, 2013

I was there during the discussion and I find you've put it quite elegantly!
I support the proposed solution, including the two raised questions. Perhaps we should abandon the naming of 'backward' and 'forward' and instead call it 'universal' and 'local'.

@dag

This comment has been minimized.

Show comment
Hide comment
@dag

dag May 24, 2013

Contributor

@haarts Well, those readline functions also cycle between [matching] items in the history you're browsing. They should continue to do this for local history, and they should do the same (but in reverse) for universal history.

Contributor

dag commented May 24, 2013

@haarts Well, those readline functions also cycle between [matching] items in the history you're browsing. They should continue to do this for local history, and they should do the same (but in reverse) for universal history.

@ridiculousfish

This comment has been minimized.

Show comment
Hide comment
@ridiculousfish

ridiculousfish May 28, 2013

Member

What would be the order of the forward history commands? Would down arrow produce the latest command in any shell, or the earliest command from after the current session started?

Member

ridiculousfish commented May 28, 2013

What would be the order of the forward history commands? Would down arrow produce the latest command in any shell, or the earliest command from after the current session started?

@dag

This comment has been minimized.

Show comment
Hide comment
@dag

dag May 29, 2013

Contributor

@ridiculousfish That's what I mean by "in reverse"; forward history would get you the latest command in any shell.

Contributor

dag commented May 29, 2013

@ridiculousfish That's what I mean by "in reverse"; forward history would get you the latest command in any shell.

@bartdag

This comment has been minimized.

Show comment
Hide comment
@bartdag

bartdag Jan 7, 2014

For what it's worth, I achieve this history synchronization in bash with these commands in my .bash_profile:

export HISTCONTROL=ignoredups:erasedups  # no duplicate entries
shopt -s histappend                      # append to history, don't overwrite it

# Save and reload the history after each command finishes
export PROMPT_COMMAND="history -a; history -c; history -r; $PROMPT_COMMAND"

I really miss history across concurrent sessions in fish because I use tmux and can keep my shell sessions open for days. A useful command ran in one session (e.g., a complex git log command) cannot be easily reused in other sessions.

bartdag commented Jan 7, 2014

For what it's worth, I achieve this history synchronization in bash with these commands in my .bash_profile:

export HISTCONTROL=ignoredups:erasedups  # no duplicate entries
shopt -s histappend                      # append to history, don't overwrite it

# Save and reload the history after each command finishes
export PROMPT_COMMAND="history -a; history -c; history -r; $PROMPT_COMMAND"

I really miss history across concurrent sessions in fish because I use tmux and can keep my shell sessions open for days. A useful command ran in one session (e.g., a complex git log command) cannot be easily reused in other sessions.

@dag

This comment has been minimized.

Show comment
Hide comment
@dag

dag Jan 8, 2014

Contributor

The history is written after each commandline execution, so an easy hack is to simply start a new shell on other sessions by running fish. Alas, it's a hack not a solution, but it works. Another option is to recall the command line in the first session and then cut it to the kill ring with Ctrl-U and then you can paste it in the other session with Ctrl-Y at least if you're running an X server. Just to give you some ideas in the mean time until we have proper history synchronization.

Contributor

dag commented Jan 8, 2014

The history is written after each commandline execution, so an easy hack is to simply start a new shell on other sessions by running fish. Alas, it's a hack not a solution, but it works. Another option is to recall the command line in the first session and then cut it to the kill ring with Ctrl-U and then you can paste it in the other session with Ctrl-Y at least if you're running an X server. Just to give you some ideas in the mean time until we have proper history synchronization.

@Ironlenny

This comment has been minimized.

Show comment
Hide comment
@Ironlenny

Ironlenny Feb 5, 2014

Is there a way to periodically flush the current history to disk and reload it?

Ironlenny commented Feb 5, 2014

Is there a way to periodically flush the current history to disk and reload it?

@ridiculousfish

This comment has been minimized.

Show comment
Hide comment
@ridiculousfish

ridiculousfish Feb 5, 2014

Member

I'd be open to a parameter you can pass to builtin history that loads history from other instances.

Member

ridiculousfish commented Feb 5, 2014

I'd be open to a parameter you can pass to builtin history that loads history from other instances.

@dontdieych

This comment has been minimized.

Show comment
Hide comment
@dontdieych

dontdieych Feb 5, 2014

Is it some kind of 'merge' from other fish instance history? If that's
right, +1.

On Wed, Feb 5, 2014 at 2:11 , ridiculousfish notifications@github.com
wrote:

I'd be open to a parameter you can pass to builtin history that loads
history from other instances.


Reply to this email directly or view it on GitHub.

dontdieych commented Feb 5, 2014

Is it some kind of 'merge' from other fish instance history? If that's
right, +1.

On Wed, Feb 5, 2014 at 2:11 , ridiculousfish notifications@github.com
wrote:

I'd be open to a parameter you can pass to builtin history that loads
history from other instances.


Reply to this email directly or view it on GitHub.

@alphapapa

This comment has been minimized.

Show comment
Hide comment
@alphapapa

alphapapa May 6, 2014

I think this is an important issue. I prefer history being per-shell-instance, because each shell is typically for performing a different task, so having the up-arrow load the previous command from a different, unrelated shell is usually unhelpful, and occasionally dangerous (e.g. retrieving an rm command run in a different directory could have unfortunate results).

However, sometimes I need to search history across all shells, because I may not remember which shell I ran a command in. For example, a few minutes ago I needed to search my shell histories to see if I had run a command on a certain URL. I searched the history in two fish instances, and there were no results, so I thought I hadn't run it. Later I realized that I had run it, but it was in a different fish instance, so the history searches in the other two instances found nothing.

In Bash, I have an hr alias that runs history -r, which reloads history from disk. I use shopt -s histappend, which appends history from all instances to the history file, so whenever I need to search all my shell history, I just run hr and then h, which is aliased to history | grep -i.

Not being able to do this in fish is a major drawback. It means that I have to search history manually in every fish instance. This is enough to keep me from using fish. :/

alphapapa commented May 6, 2014

I think this is an important issue. I prefer history being per-shell-instance, because each shell is typically for performing a different task, so having the up-arrow load the previous command from a different, unrelated shell is usually unhelpful, and occasionally dangerous (e.g. retrieving an rm command run in a different directory could have unfortunate results).

However, sometimes I need to search history across all shells, because I may not remember which shell I ran a command in. For example, a few minutes ago I needed to search my shell histories to see if I had run a command on a certain URL. I searched the history in two fish instances, and there were no results, so I thought I hadn't run it. Later I realized that I had run it, but it was in a different fish instance, so the history searches in the other two instances found nothing.

In Bash, I have an hr alias that runs history -r, which reloads history from disk. I use shopt -s histappend, which appends history from all instances to the history file, so whenever I need to search all my shell history, I just run hr and then h, which is aliased to history | grep -i.

Not being able to do this in fish is a major drawback. It means that I have to search history manually in every fish instance. This is enough to keep me from using fish. :/

@ridiculousfish

This comment has been minimized.

Show comment
Hide comment
@ridiculousfish

ridiculousfish May 6, 2014

Member

It's not quite as bad as "search history manually in every fish instance." You can spawn a new instance of fish, which will inherit everyone's history, and search there.

Member

ridiculousfish commented May 6, 2014

It's not quite as bad as "search history manually in every fish instance." You can spawn a new instance of fish, which will inherit everyone's history, and search there.

@alphapapa

This comment has been minimized.

Show comment
Hide comment
@alphapapa

alphapapa May 12, 2014

Thanks, that will help. It would still be very helpful to have a command to reload history. :)

alphapapa commented May 12, 2014

Thanks, that will help. It would still be very helpful to have a command to reload history. :)

@Ironlenny

This comment has been minimized.

Show comment
Hide comment
@Ironlenny

Ironlenny May 15, 2014

I would like to implement this feature, but I'm not quite sure where to get started. Are there any suggestions?

Ironlenny commented May 15, 2014

I would like to implement this feature, but I'm not quite sure where to get started. Are there any suggestions?

@siteshwar

This comment has been minimized.

Show comment
Hide comment
@siteshwar

siteshwar May 15, 2014

Member

If I remember correctly history synchronization was disabled in commit fd4df6f, you may want to have a look at changes made in this commit.

Member

siteshwar commented May 15, 2014

If I remember correctly history synchronization was disabled in commit fd4df6f, you may want to have a look at changes made in this commit.

@ridiculousfish

This comment has been minimized.

Show comment
Hide comment
@ridiculousfish

ridiculousfish May 15, 2014

Member

Yes, the key is the birth_timestamp variable in history.cpp. See how it is used in offset_of_next_item. The fix is likely to be as simple as advancing birth_timestamp to now.

Member

ridiculousfish commented May 15, 2014

Yes, the key is the birth_timestamp variable in history.cpp. See how it is used in offset_of_next_item. The fix is likely to be as simple as advancing birth_timestamp to now.

@ridiculousfish

This comment has been minimized.

Show comment
Hide comment
@ridiculousfish

ridiculousfish Jul 25, 2014

Member

I've been thinking about what to call this. I dislike "sync" or "synchronize" because it smacks of one of those things you run "just to be safe." Like this! I also dislike "reset" for the same reason.

How about "merge?" "history --merge incorporates history from other sessions into this session." history --unify sounds OK too.

Member

ridiculousfish commented Jul 25, 2014

I've been thinking about what to call this. I dislike "sync" or "synchronize" because it smacks of one of those things you run "just to be safe." Like this! I also dislike "reset" for the same reason.

How about "merge?" "history --merge incorporates history from other sessions into this session." history --unify sounds OK too.

@bartdag

This comment has been minimized.

Show comment
Hide comment
@bartdag

bartdag Jul 25, 2014

+1 for merge.

bartdag commented Jul 25, 2014

+1 for merge.

@ridiculousfish

This comment has been minimized.

Show comment
Hide comment
@ridiculousfish

ridiculousfish Jul 25, 2014

Member

e9f870e adds support for history --merge. This immediately incorporates changes from other sessions, including insertions and deletions.

Member

ridiculousfish commented Jul 25, 2014

e9f870e adds support for history --merge. This immediately incorporates changes from other sessions, including insertions and deletions.

@milentrifonov

This comment has been minimized.

Show comment
Hide comment
@milentrifonov

milentrifonov Mar 29, 2016

@dag The thing is that history is saved after the command completes. What I am doing mostly is to ssh-ing to some of my hosts from one terminal session, and then I want to ssh to the last host from another new terminal session, but the last ssh command is not available because I am still logged from the first window and the command is not finished and write to history is not happened yet. This bothers me a lot.
Anyone knows a workaround for this ?

milentrifonov commented Mar 29, 2016

@dag The thing is that history is saved after the command completes. What I am doing mostly is to ssh-ing to some of my hosts from one terminal session, and then I want to ssh to the last host from another new terminal session, but the last ssh command is not available because I am still logged from the first window and the command is not finished and write to history is not happened yet. This bothers me a lot.
Anyone knows a workaround for this ?

@faho

This comment has been minimized.

Show comment
Hide comment
@faho

faho Mar 29, 2016

Member

@milentrifonov:

Try

function save_history --on-event fish_preexec
    history --save
end

(define in your config.fish or a sourced file, not an autoloaded function)

Member

faho commented Mar 29, 2016

@milentrifonov:

Try

function save_history --on-event fish_preexec
    history --save
end

(define in your config.fish or a sourced file, not an autoloaded function)

@milentrifonov

This comment has been minimized.

Show comment
Hide comment
@milentrifonov

milentrifonov Apr 10, 2016

@faho Thanks, this actually did the trick.

milentrifonov commented Apr 10, 2016

@faho Thanks, this actually did the trick.

@yefim

This comment has been minimized.

Show comment
Hide comment
@yefim

yefim Jun 16, 2016

Is there a way to have all my fish sessions share history automatically? I usually have 5 (or more) iTerm tabs open each with a different fish session. I don't want to have to run history --merge every time I want to re-run a command from one tab in another. I just want one history across my iTerm tabs. Will the following do what I want?

function sync_history --on-event fish_preexec
    history --save
    history --merge
end

yefim commented Jun 16, 2016

Is there a way to have all my fish sessions share history automatically? I usually have 5 (or more) iTerm tabs open each with a different fish session. I don't want to have to run history --merge every time I want to re-run a command from one tab in another. I just want one history across my iTerm tabs. Will the following do what I want?

function sync_history --on-event fish_preexec
    history --save
    history --merge
end
@faho

This comment has been minimized.

Show comment
Hide comment
@faho

faho Jun 17, 2016

Member

@yefim: Yeah, that should work. It might cause a bit of slowdown, though.

Member

faho commented Jun 17, 2016

@yefim: Yeah, that should work. It might cause a bit of slowdown, though.

@faho faho added the enhancement label Jun 17, 2016

@krader1961

This comment has been minimized.

Show comment
Hide comment
@krader1961

krader1961 Jun 17, 2016

Contributor

I don't think that will do quite what you want. Consider the case where you have two fish sessions A and B. Both have issued the prompt and are waiting for you to enter a command. In A you enter a command. In B you press [up-arrow]. Oops! You haven't yet merged the new history so you won't see the most recently entered command in session A.

Contributor

krader1961 commented Jun 17, 2016

I don't think that will do quite what you want. Consider the case where you have two fish sessions A and B. Both have issued the prompt and are waiting for you to enter a command. In A you enter a command. In B you press [up-arrow]. Oops! You haven't yet merged the new history so you won't see the most recently entered command in session A.

@krader1961

This comment has been minimized.

Show comment
Hide comment
@krader1961

krader1961 Jun 17, 2016

Contributor

Also, as @faho pointed out, that's going to be hideously expensive. Possibly enough to be noticeable.

Contributor

krader1961 commented Jun 17, 2016

Also, as @faho pointed out, that's going to be hideously expensive. Possibly enough to be noticeable.

@yefim

This comment has been minimized.

Show comment
Hide comment
@yefim

yefim Jun 17, 2016

Thank you for the quick response @faho and @krader1961 (and for finding that edge case). So is there a way for me to just have one history across all my sessions without it being so expensive?

yefim commented Jun 17, 2016

Thank you for the quick response @faho and @krader1961 (and for finding that edge case). So is there a way for me to just have one history across all my sessions without it being so expensive?

@krader1961

This comment has been minimized.

Show comment
Hide comment
@krader1961

krader1961 Jun 17, 2016

Contributor

So is there a way for me to just have one history across all my sessions without it being so expensive?

Not at this time. Even if we were to implement this in the core fish code (as opposed to a fish script) it would be very expensive and thus noticeable to the user. Which is why we're unlikely to ever implement this feature.

What I do is define an alias:

alias hr 'history --merge'  # read and merge history from disk

When I want to have access to the commands I've typed in a different session I just type hr. Most of the time I don't want my local command history to be polluted by history from other sessions.

Contributor

krader1961 commented Jun 17, 2016

So is there a way for me to just have one history across all my sessions without it being so expensive?

Not at this time. Even if we were to implement this in the core fish code (as opposed to a fish script) it would be very expensive and thus noticeable to the user. Which is why we're unlikely to ever implement this feature.

What I do is define an alias:

alias hr 'history --merge'  # read and merge history from disk

When I want to have access to the commands I've typed in a different session I just type hr. Most of the time I don't want my local command history to be polluted by history from other sessions.

@ridiculousfish

This comment has been minimized.

Show comment
Hide comment
@ridiculousfish

ridiculousfish Jun 19, 2016

Member

An alternative is to define the up arrow key binding to perform the merge. Then you only merge right before you read history.

Member

ridiculousfish commented Jun 19, 2016

An alternative is to define the up arrow key binding to perform the merge. Then you only merge right before you read history.

@wprater

This comment has been minimized.

Show comment
Hide comment
@wprater

wprater Jun 24, 2016

An alternative is to define the up arrow key binding to perform the merge. Then you only merge right before you read history.

@ridiculousfish can you show an example? would this interfere with a up arrow binding to a history search?

wprater commented Jun 24, 2016

An alternative is to define the up arrow key binding to perform the merge. Then you only merge right before you read history.

@ridiculousfish can you show an example? would this interfere with a up arrow binding to a history search?

@ridiculousfish

This comment has been minimized.

Show comment
Hide comment
@ridiculousfish

ridiculousfish Jun 24, 2016

Member

Well, it would be something like:

bind -k up 'history --merge ; up-or-search'

though now that I try it, I see that history --merge doesn't properly interleave items - items new to this session are still the most recent in history. That's a separate issue we'd have to fix.

Member

ridiculousfish commented Jun 24, 2016

Well, it would be something like:

bind -k up 'history --merge ; up-or-search'

though now that I try it, I see that history --merge doesn't properly interleave items - items new to this session are still the most recent in history. That's a separate issue we'd have to fix.

@ridiculousfish

This comment has been minimized.

Show comment
Hide comment
@ridiculousfish

ridiculousfish Jun 24, 2016

Member

That's me rediscovering #2312

Member

ridiculousfish commented Jun 24, 2016

That's me rediscovering #2312

@bittrance

This comment has been minimized.

Show comment
Hide comment
@bittrance

bittrance Aug 22, 2016

FWIW @ridiculousfish solution does not work for me on 2.3.1 on MacOS X 10.11.5. Instead mapping

bind \e\[A 'history --merge ; up-or-search'

works fine. I suppose this depends on which binding "wins" when both gets defined (one from default, one from fish_user_key_bindings).

bittrance commented Aug 22, 2016

FWIW @ridiculousfish solution does not work for me on 2.3.1 on MacOS X 10.11.5. Instead mapping

bind \e\[A 'history --merge ; up-or-search'

works fine. I suppose this depends on which binding "wins" when both gets defined (one from default, one from fish_user_key_bindings).

@edouard-lopez

This comment has been minimized.

Show comment
Hide comment
@edouard-lopez

edouard-lopez Nov 23, 2016

I'm on the same situation than @bartdag use multiple fish instance in tmux with different pane/window.

I asked how to How do I execute a command on pane/window switch? tmux/tmux/issues/666

edouard-lopez commented Nov 23, 2016

I'm on the same situation than @bartdag use multiple fish instance in tmux with different pane/window.

I asked how to How do I execute a command on pane/window switch? tmux/tmux/issues/666

@mindreader

This comment has been minimized.

Show comment
Hide comment
@mindreader

mindreader Jan 9, 2017

Could you use inotify on each shell instance to "watch" for changes to the history and merge them after second or two delay if they are idle?

mindreader commented Jan 9, 2017

Could you use inotify on each shell instance to "watch" for changes to the history and merge them after second or two delay if they are idle?

@introom

This comment has been minimized.

Show comment
Hide comment
@introom

introom Jul 2, 2017

so I am using this binding,

the key seq is \c-; on my terminal.

bind \e"[27;5;59~" "history merge; commandline -f force-repaint"

The shell has the history "echo 3 5", and when I type "echo",
I got the correct hint:
image

but when I run "echo 3 8" in another fish shell, and I switch back to the original shell and run the above key binding, the hint doesn't get updated to the new "echo 3 8".
Only after I back-delete a char or use the up/down arrow key or some changes to the command line will that be updated.

introom commented Jul 2, 2017

so I am using this binding,

the key seq is \c-; on my terminal.

bind \e"[27;5;59~" "history merge; commandline -f force-repaint"

The shell has the history "echo 3 5", and when I type "echo",
I got the correct hint:
image

but when I run "echo 3 8" in another fish shell, and I switch back to the original shell and run the above key binding, the hint doesn't get updated to the new "echo 3 8".
Only after I back-delete a char or use the up/down arrow key or some changes to the command line will that be updated.

@krader1961

This comment has been minimized.

Show comment
Hide comment
@krader1961

krader1961 Jul 2, 2017

Contributor

@introom, That is the expected behavior. The command reader, highlighter, and suggestion mechanisms "remember" the position in the local history and thus the command that is suggested. Simply doing a history merge won't affect that. The force-repaint isn't intended to change that state.

Contributor

krader1961 commented Jul 2, 2017

@introom, That is the expected behavior. The command reader, highlighter, and suggestion mechanisms "remember" the position in the local history and thus the command that is suggested. Simply doing a history merge won't affect that. The force-repaint isn't intended to change that state.

@introom

This comment has been minimized.

Show comment
Hide comment
@introom

introom Jul 2, 2017

@krader1961 any graceful solution in your mind that can update the suggestion?

introom commented Jul 2, 2017

@krader1961 any graceful solution in your mind that can update the suggestion?

@krader1961

This comment has been minimized.

Show comment
Hide comment
@krader1961

krader1961 Jul 2, 2017

Contributor

It's not obvious that behavior is desirable, @introom. But you can achieve it by clearing the command line then reinstating the original typed text. For example:

bind \cx "history merge; set -l x (commandline -b); commandline -r ''; commandline -r \$x"

Note that you don't even need to force a repaint (which makes it slower). No guarantees that trick will work in the future.

Contributor

krader1961 commented Jul 2, 2017

It's not obvious that behavior is desirable, @introom. But you can achieve it by clearing the command line then reinstating the original typed text. For example:

bind \cx "history merge; set -l x (commandline -b); commandline -r ''; commandline -r \$x"

Note that you don't even need to force a repaint (which makes it slower). No guarantees that trick will work in the future.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment