Remove duplicates when refilling readline history #1000

Closed
takluyver opened this Issue Nov 14, 2011 · 7 comments

Projects

None yet

5 participants

@takluyver
IPython member

Readline by default only keeps one copy of consecutive duplicates in history, but when we refill it from the history database, we push those duplicates multiple times. It should be an easy fix in refill_readline_hist.

@antoine-levitt

Are you saying that the way to fix this is to put duplicate lines in the history, but not refill readline's? Isn't it better to simply not add them in the first place?

@takluyver
IPython member

Our history is supposed to be a record of what you've executed - for instance, running a += 1 three times is different from running it once. Also, it's numbered to correspond with the In [1]: prompt numbers you see in a session, so one history entry is stored each time that number is incremented.

@minrk
IPython member

Right - we do not want to de-dupe the actual record of your actions, because then it is no longer accurate, and breaks reproducibility. We only want to de-dupe readline, so that we skip repeating entries when hitting the up arrow, etc.

@fperez
IPython member

Agreed. In fact, it used to work without the dupes a while back, I'm not exactly sure when it changed. Was it when we introduced the sqlite backend or later, do you guys know?

@takluyver
IPython member

I don't think I accounted for it when I did the sqlite stuff initially, so my guess would be then. I can reproduce it in 0.11.

PR coming up in a few minutes.

@takluyver takluyver was assigned Nov 15, 2011
@fperez
IPython member

Closed by #1003, I forgot to put that in the commit comment.

@fperez fperez closed this Nov 20, 2011
@mdboom mdboom pushed a commit to mdboom/ipython that referenced this issue Dec 2, 2011
@takluyver takluyver Do not push consecutive duplicates when we refill readline history.
Closes gh-1000, earns t-shirt
a421824
@mattvonrocketstein mattvonrocketstein pushed a commit to mattvonrocketstein/ipython that referenced this issue Nov 3, 2014
@takluyver takluyver Do not push consecutive duplicates when we refill readline history.
Closes gh-1000, earns t-shirt
a5c79d7
@takluyver takluyver referenced this issue Nov 20, 2014
Closed

issue 7000 #7000

@dashesy

I see duplicates when going back in history in 3.0.0 (QtConsole), it is not something that can be reproduced with confidence but anyone using IPython for long enough time should have noticed this. Maybe QtConsole is not used as much and thus not get tested like the Notebook?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment