input_prefilter hook not called

[vharitonsky]

There is an input_prefilter hook in IPython documentation, but judging from the source it is not called anywhere.

[Carreau]

For the record, I think this came from http://stackoverflow.com/questions/20633985/ipython-input-prefilter/20634792#20634792 as the delta of time between this issues and the SO question is 7 minutes.

[vharitonsky]

Sorry, I am in despair.

[Carreau]

No problem, I'm not an expert on this part of the code and most of the other core dev are asleep at this time, they should be available in a few hours. You can also pre-describe exactly what you want to do it might help !

[vharitonsky]

I am writing a profiling extension for ipython which will measure resource usage of each executed line.
I imagine this kind of workflow:

In [1]: %profile
Profiling is enabled

In [2]:a = [a**a for a in range(100)]
CPU: 100ms MEM: +200 Mb

In [3]:%stats.cpu
Out[3]:100ms

I am using pre_prompt_hook and pre_run_code_hook in an object to achieve this. But the trick is NOT measuring anything when you call methods and attributes in %stats object which holds the profiling data. So I need some kind of a hook which runs BEFORE the pre_run_code_hook and gives me the ability to temporary disable profiling when i call %stats magic.

It's not very clear how to add custom input transformer, as the page only describes how to declare one, maybe I am missing something ?

[takluyver]

I guess you're talking about this page? http://ipython.org/ipython-doc/stable/config/inputtransforms.html

Looking back at it, I agree it's not very clear how to add an input transformer. The closest it gets to describing that is this sentence: "An InteractiveShell instance actually has two IPythonInputSplitter instances, as the attributes input_splitter, to tell when a block of input is complete, and input_transformer_manager, to transform complete cells. If you add a transformer, you should make sure that it gets added to both."

From an extension, that would look something like this:

def load_ipython_extension(ip):
    ip.input_splitter.logical_line_transforms.append(my_transformer())
    ip.input_transformer_manager.logical_line_transforms.append(my_transformer())

Note that you have to decide which of physical_line_transforms, logical_line_transforms and python_line_transforms your transformations act on:

• physical transforms get a line each time the user presses enter (e.g. stripping prompts)
• logical transforms get lines with backslash continuations stitched together (e.g. transforming line magics)
• Python transforms get multi-line strings, lists, etc. assembled into blocks

You can also define AST transforms, which can inspect and modify the parsed Python code before it is executed.

[vharitonsky]

That's exactly what I need. Thanks alot.

That also gives me the idea that maybe input_prefilter hook can be also implemented as a transformer. I mean inside IPython it will look as just another physical line transformer. This could provide a backward compatibility in a very clean way.

[takluyver]

If no-one has complained until now, there's probably no-one using the hook, so we might just get rid of it to keep things simple.

[vharitonsky]

It's always exciting to be the first :) So, should I close the issue then ?

[takluyver]

I'll improve the docs a bit and close it from the commit.