History can't handle empty lines (was: Keep losing history items)

[jdelStrother]

I keep randomly (?) losing history items. Commands that I know I typed in the past - long before my current shell was opened - are suddenly missing from my shell instance (they're not visible in history or history search --contains 'my-missing-command', but are present in ~/.local/share/fish/fish_history).

Opening a new tab or running history --merge brings back the missing command.

(Anecdotally, it seems like the commands that go missing always have a paths: entry in the history file, but I've not found much on how the paths entry affects the history command.)

What can I try to narrow down the problem?

---

fish, version 3.0.2
OS X 10.14.5
iTerm2 3.3.20190715-nightly with TERM=xterm-256color

Please tell us if you tried fish without third-party customizations by executing this command and whether it affected the behavior you are reporting:

I didn't, I haven't found a good way of simply reproducing the problem.

[faho]

What can I try to narrow down the problem?

The obvious thing is checking any configuration, in particular that touches $fish_history or does a history merge or save or... If you have any third-party stuff like oh-my-fish, that's often of interest because you don't exactly know if it touches this, so disable it.

(Anecdotally, it seems like the commands that go missing always have a paths: entry in the history file, but I've not found much on how the paths entry affects the history command.)

Could we have the offending entries? Ideally without any changes, so if you have a sample that doesn't include private information that'd be nice (we don't need or want to know your bank details), but if not a censored one might also work.

[jdelStrother]

I do have these plugins installed via fisher:

jethrokuan/z
barnybug/docker-fish-completion
laughedelic/brew-completions
arbelt/fish-plugin-scmpuff
oh-my-fish/plugin-foreign-env

although grepping through them for history doesn't show anything likely-sounding. I'm also using fzf (https://github.com/junegunn/fzf/blob/master/shell/key-bindings.fish).

Could we have the offending entries?

My latest one is:

- cmd: docker-compose -f docker-compose.services.yml up
  when: 1565184205
  paths:
    - docker-compose.services.yml

I'll carry on trying to narrow it down, although I think I only run into every couple of days so might take a while.

[faho]

My latest one is:

Okay, so the purpose of the paths: attribute is to not show this entry in suggestions (the greyed-out things in the commandline) if the given paths don't exist.

So you'd expect docker-co to suggest mpose -f docker-compose.services.yml up if the current directory has a docker-compose.services.yml, and something else if it doesn't.

Also, if a history entry is selected to be used as a suggestion, it'll be skipped when navigating history (the idea being that you could have picked it before). Are you sure you're not just seeing that behavior?

And since this is a docker example: Are all the others also docker examples? Can you try without the docker-fish-completion plugin?

[jdelStrother]

So you'd expect docker-co to suggest mpose -f docker-compose.services.yml up if the current directory has a docker-compose.services.yml

There's definitely a docker-compose.services.yml file in my current directory.

And since this is a docker example: Are all the others also docker examples? Can you try without the docker-fish-completion plugin?

It's not just docker, but I'll try removing the plugin and start keeping a note of the other commands that seem to frequently go missing.

---

Something I missed from my original report that seems weird/interesting: I pretty much also use history via fzf (hitting Ctrl-R to invoke it). For a while I assumed it was a fzf bug, but then noticed that the same command is missing from history. However, the missing command does show up as an auto-suggestion if I start to type docker-com...

[faho]

However, the missing command does show up as an auto-suggestion if I start to type docker-com...

Okay, that is indeed weird.

@ridiculousfish Any ideas? Is it possible we're skipping these too hard?

[ridiculousfish]

I'm at a loss too. It's very hard to think of a reason why it would not be in history but would be offered as a completion.

One remote possibility is that we're rejecting these based on timestamp - we think they're commands from a new session. That could happen if the clock were being set back.

Another possibility is that there's a bug in the history decoding code causing us to fail to decode certain items.

This is the sort of place where we would want a --debug flag to trace what's happening.

[zanchey]

There was some suggestion in #5136 (comment) that the hardware clock being reset may have caused a similar behaviour.

[mqudsi]

@jdelStrother if you are able to reproduce this with a clean (free of sensitive data!) history file (just temporarily point fish's history to a new location) and can induce this behavior by using that file in a future fish session, perhaps you could share that via email or similar with one of us to attempt to debug the issue.

[jaelpark]

The problem occurs when sometimes fish writes an empty command entry to the history. See the following history file as an example, and particularly line 23.

- cmd: sudo pacman -Syu --ignore rofi --ignore root
  when: 1567551190
- cmd: cd build/vim-x11/
  when: 1567551211
  paths:
    - build/vim-x11/
- cmd: vim PKGBUILD
  when: 1567551217
  paths:
    - PKGBUILD
- cmd: cat PKGBUILD
  when: 1567553811
  paths:
    - PKGBUILD
- cmd: makepkg -Acsr
  when: 1567553817
- cmd: sudo pacman -U vim-x11-8.1.1968-1-x86_64.pkg.tar.xz
  when: 1567553908
  paths:
    - vim-x11-8.1.1968-1-x86_64.pkg.tar.xz
- cmd: r
  when: 1567584641
- cmd: 
  when: 1567584666
- cmd: r
  when: 1567584667
- cmd: r
  when: 1567882393

If I add anything to the empty entry, the full history becomes available again.

[ridiculousfish]

Wild and nice detective work! Any guess on how the empty command got in there? Do you think you entered and executed an empty command or is something else happening?

[jaelpark]

It looks like the offending entry gets written if I manage to put a line return to the prompt without fish registering it. One way to reliably reproduce this is to put fish into vi mode, enter normal mode, and immediately after quickly hit enter to insert the line return. Press enter again to evaluate the empty command. I usually run the previous command by entering normal mode, pressing 'k' and then enter. But sometimes it happens that I "miss" the 'k' key, and only enter is pushed when I quickly attempt this 3-key combo. I think this is why I sometimes randomly lose the command history.

[jdelStrother]

I'm still not able to reproduce this at-will (and not using vi mode, FWIW). I'll look for empty entries next time this crops up, though.

[jaelpark]

Another way actually is to enter a single quote character, add two new lines, and then navigate back up and remove the quote. One line break should remain, and evaluating this will result in an empty command.

Edit: So the history might get erased for example when user starts typing a command, enters a quote and adds a new line, but then decides to erase the prompt while ignoring the new line that was also inserted. The new line will remain and gets evaluated when user hits enter for a new prompt line.

[ridiculousfish]

That reproduces for me. Amazing!

[zanchey]

I can reproduce with:

Alt-Enter; Enter

and fix with

history --delete '
'

[ridiculousfish]

Let's do a targeted fix for 3.1 and fix it properly with a sane history format in 3.2.