Skip to content
This repository

HISTIGNORE for IPython #943

Closed
antoine-levitt opened this Issue October 29, 2011 · 16 comments

2 participants

Antoine Levitt Thomas Kluyver
Antoine Levitt

There's a HISTIGNORE env variable in bash that makes the history ignore certain commands (a common value is to ignore ls / cd commands). Could something like this be added to IPython?

I also have another reason to use such as setting: I'm working on some emacs integration code, and would like to send commands to ipython (typically: run file.py) without it cluttering the history.

Thomas Kluyver
Collaborator

One way of doing that might be to use a separate profile when connecting from emacs - history is stored per profile, so commands in one profile won't show up in the other.

If that's not a good solution:

Which kind of history are you concerned about? There's the readline history, which is what you get when you hit the up arrow in the terminal, and the history database, which is what's queried when you use magic commands like %hist or %save.

Also, how flexible would it need to be for your purposes? I'm imagining checking the first word of a command against a set of ignored commands - would that do what you need.

Antoine Levitt

The idea is to have ipython run from emacs, and bindings in python files would send the file to the ipython process. I still want a history, but don't want code run by emacs (like "run file.py") to show up in the history. It's a marginal problem anyway, but still.

I took for granted that the readline and database history were synced, is that not the case? I'm more concerned with readline.

I don't need a lot of flexibility for my purpose, but there's no point in destroying flexibility: ipython could very well expose a variable the user could set to a function that takes the command line as parameter, and decide whether to ignore it or not. Users could set it as they wish. I've got an emacs mindset for these things: if it can be customized, expose a setting so that users can plug their own code in it. (fortunately, in python as in emacs lisp, "expose" simply means "create a variable for it").

Thomas Kluyver
Collaborator
Antoine Levitt
Antoine Levitt

Ok, another issue is that ipython records duplicate and blank entries. Typically, for matlab-style operation with pylab, you'd often run a script many times, maybe changing a parameter inside it, and then plot the result, running the script again many times, and plotting again. This requires sifting through many "run script.py" before finding that "plot(x,y,options)" line. See also htty/htty@2173fd7

A nice generic solution would be to have a function someplace that gets called to determine whether the current input should be inserted in the history. This function could answer False if the line is the same as the previous line, or a blank line, as well as call a user function (settable from ipython_config.py). This is complicated further by multiline inputs, the various frontends and the fact that readline and ipython history have to be synchronised. I was able to do it by tweaking the "interact" function of TerminalInteractiveShell, but only for ipython history, not readline. Could someone take a look at this ?

Thomas Kluyver
Collaborator

Readline stores its own history directly from stdin, and we haven't got any direct control over that. We can modify it later, but it's somewhat awkward, and I think it's probably not worth doing without a pretty good reason.

Just testing in a terminal here, readline only stores consecutive duplicate entries once, so if you %run script.py 20 times, you only see it once as you press up-arrow to go through previous commands.

If you're just using a terminal emulator, there's not really any alternative to sending %run xyz to stdin. The new ZMQ framework can do what you're after - run the code without it appearing as an input - but it would clearly require a very different architecture. If there's a Qt version of Emacs, it might be possible to get most of the architecture for free by embedding the Qt console.

Antoine Levitt
Thomas Kluyver
Collaborator

I've just checked with trunk, and it seems we've broken it since 0.11. I'll file a new issue for that - I've got an idea about what might have caused it. For reference, I've got readline 6.2-2.

There are some things where it's worth dealing with readline history. When we removed the multiline history feature for 0.11, we had a number of users vociferously calling for its return. I certainly want to sort out the regression in duplicates and blanks, but I my feeling is that there isn't enough demand for configurable ignoring of history lines to merit adding that complexity.

Antoine Levitt
Thomas Kluyver
Collaborator

The blanks problem should be fixed by the open pull request #929, and I've commented there asking Julian to fix the problem with duplicates as well.

Thanks for what you're doing - we've had quite a few people looking for better Emacs bindings for IPython, so it should be well received. Our existing ipython.el file is very out of date, because none of the core developers know emacs lisp.

@fperez: just to keep you updated, I felt that there wouldn't be much demand for this feature, at least once we fix some regressions in readline history, so it doesn't warrant adding complexity. Feel free to reopen if you disagree.

Thomas Kluyver takluyver closed this October 30, 2011
Antoine Levitt

For the record, here's a very simple code you can put in your init scripts to make ipython (at least, the readline part) forget all about lines satisfying a certain predicate. It's so hackish it's almost like emacs lisp. I'm starting to love python. :-)

Antoine Levitt
# our history_ignore predicate
history_ignore = lambda str: str.startswith("run")

# hijack the readline.add_history method
get_ipython().readline.old_add_history = get_ipython().readline.add_history
def my_add_history(str):
    if not history_ignore(str):
        get_ipython().readline.old_add_history(str)

get_ipython().readline.add_history = my_add_history

#force a readline history reload
get_ipython().readline.clear_history()
get_ipython().refill_readline_hist()
Thomas Kluyver
Collaborator

Feel free to add it to the cookbook: http://wiki.ipython.org/Cookbook

Note that your code depends on us always removing and re-adding items in readline history. So it won't work if the user has multiline_history set to False, or if we decide to skip doing that dance for single-line cells.

Antoine Levitt

Not really, it only depends on refill_readline_hist working correctly, no matter if it's multiline or single-line. refill_readline_hist exists because you need a persistent history across sessions. I'll add it to the cookbook with a big warning sign.

Thomas Kluyver
Collaborator

But to keep working, you'd need to call refill_readline_hist after every input, which is wildly inefficient (it reads 1000 entries from a database, loops over them, encodes them, and feeds them to readline). I'm sure it should be possible to use something like get_ipython().register_post_execute, and inspect readline history. It might be tricky to do reliably (multiline cells, raw_input()s inside the cell...), but a rough implementation should be simple enough.

Antoine Levitt
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.