I try to do my data munching and statistics in IPython notebooks. I start the notebook server with "--script" and my notebooks (up to now...) are mixing imports, functions and data munching (loading, apply the functions, save result). I tried to reuse parts of the notebook in other notebooks but run into two problem / annoyances:
So, it would be nice if the notebook could add metadata to a cell, which would basicly instruct the exporter to "export this cell" or "do not export this cell". This could then also show an error when a magi command is encountered in a "marked for export"-cell. Another idea would be to simple comment out any encountered magic command.
Additionally it would be nice if the notebook server could be instructed to a) export the notebook if such metadata is found, even if not run with "--script" and b) use a specified name for the python file.
--script is a terrible hack (that is also totally broken, since it saves untransformed IPython input in a .py file). A more generic post-save hook should allow people to do much more sensible and sophisticated things that the current status quo.
So how should a generic post-save hock look like?
Also relevant for #3291
--script should at the very least be killed in favor of something using nbconvert once it is merged. If we do replace it with a proper hook, it should probably be a callable that takes some/all of: the notebook ID, notebook manager, and destination filename. It may also require hooks to be called on rename and deletion to match the cleanup we currently do.
Ok, so something along this lines?
def add_save_hook(self, hook):
def remove_save_hook(self, hook):
[... in FileNotebookManager.write_notebook_object()...]
for hook in hooks:
hook.postsave(notebook_id, new_name, path, self)
How would a plugin then register itself?
I don't think the notebook manager is where these things will live. We are quickly moving to a model that supports multiple directories. In that context, each directory might want to have a different configuration of these types of hooks. However the notebook manager would be the same object for each directory. IOW, we won't instantiate a new notebook manager for each directory. This represents an entirely different type of configuration abstraction than we current have. We will need to think carefully about how to represent that well. But, these things are very much in flux and I think we just need to wait for some of this other work to land first.
Ok, nothing to be done about this issue until that refactoring is done? If yes: when will this refactoring land in IPython?
@JanSchulz Regarding the injection part you can use the functions in IPython.core.display.
I'm currently thinking about using the IPython.core.hooks mechanism:
get_ipython().hooks.notebookmanager_post_save(nb, notebook_id, old_name, new_name, path)
An extension could then simple add a function to the hook via ip.set_hooks(...) (or get_ipython().set_hooks()) and be done.
The --script case would then become a simple extension which would copy the IPython.nbformat.v3.nbpy.PyWriter.writes() function and the copy/rename/delete logic from the hack in FileNotebookManager.write_notebook_object()
So, before I start coding: would such a thing be considered for merging at all and before the refactoring mentioned above?
So, the idea is implemented in https://github.com/JanSchulz/ipython/commits/define_hooks (last 4 commits). It still needs a --skript parameter to the notebook startup and the js part needs to be loaded via
%load_ext saveasscript (and before that you need to copy the file to the extension directory)
Unfortunately the %load_ext directly for both parts (hook and js) does not work: the ip object in the extension is a different one than the get_ipython() one in the filenbmanager :-( I'm not sure how to get an extension into the notebook application...
And a cleaner implmentation, which uses the notebook metadata to define a postsavehook:
To check which cells will be written to the .py file choose 'Cell Toolbar: auto export hin' and change the setting for each cell.
My current thinking is to define a cell level magic %%save_and_run -f "file" -i "identifier", which would save the transformed cell into the file and surround it with lines which include the identifier and afterwards would run the code. The identifier can then be used to change only that portion of the file.
%%save_and_run -f "file" -i "identifier"
My usecase would be fixed by minrk/ipython_extensions#11
This could also be included in ipython itself by adding this methods to IPython/core/magic/code.py (there is a %%save <linenumbers> and a magic to save to a gist...