Clear-all on save option #1280

Closed
lucasw opened this Issue Jan 17, 2012 · 24 comments

Comments

Projects
None yet

lucasw commented Jan 17, 2012

I sometimes forget to clear all before saving and checking the notebook into revision control, and it produces harder to read diffs and file sizes larger than I want to be commiting. Being able to set an option within a notebook to only save in cleared format would be very useful. Even nicer would be if it didn't clear the open notebook, just the saved version.

Owner

takluyver commented Jan 17, 2012

If you think about this the other way round (save output cells, True/False), it sounds pretty simple.

Owner

fperez commented Jan 17, 2012

While we continue thinking about the best long-term solutions for VC integration, this could be an easy way to offer a good workflow right away that doesn't require any format changes or any decision on the more complex idea of splitting the nb into input/output files that we discussed a while ago.

Owner

ellisonbg commented Jan 17, 2012

We are planning on implementing a configuration panel soon and we can put this there.

Owner

fperez commented Jan 17, 2012

Yes, that will be perfect. We can discuss on-list a little bit what we want in it, what pieces of information visible in the panel will be available as worksheet metadata (so they persist across runs of the same notebook), etc.

mforbes commented Nov 30, 2013

I am not sure I understand how this would relate to the soon-to-appear configuration panel (or where that is being discussed), but I have hacked together a potential solution to this problem:

https://github.com/mforbes/ipython

If this is an appropriate direction, I can clean this up and submit a proper PR. I have been trying this out for a while, and am discussing this here:

http://stackoverflow.com/questions/18734739/using-ipython-notebooks-under-version-control

Owner

Carreau commented Dec 1, 2013

I think this is slighlty too much restrictive. There should be another issue open about that, bu we would like a post-save hook that run any kind of conversion(s) with nbconvert afterward, so that you could, each time you save, have a clean version but also html, pdf, ... etc potentially with custom templates. (nbconvert support ipynb->ipynb also)

Finally for the clean part, most of us woudl agree that this should be done with git clean and smudge filter, not by the application itself.

mforbes commented Dec 1, 2013

I totally agree with this being too restrictive, but could not find a way of adding a hook. There was PR #3312 about hooks – closed as not "the direction we want to go" – and minimal discussion in issue #977. I cannot find any further discussion about adding hooks: Is progress towards hooks being discussed somewhere?

The problem I have with relying on clean and smudge filters is that this means a separate solution is required for each version control system. In particular, I could not find a good way of doing this in mercurial. To minimize maintenance, it would be best if the cleaning process could be kept VC agnostic.

The second issue (outstanding in all methods as far as I can see) is: how does one incorporate pulled "clean" changes into one's working notebook? Probably something along the lines of first merging the two "clean" versions, then inserting the new content into the working notebook (removing any outdated output). Any suggestions? (Some way of updating one notebook from another seems to be required for this. I am not sure if that is the jurisdiction of nbconvert or something else.)

Has there been any progress on this issue?

I've noticed that @minrk has posted scripts (https://gist.github.com/minrk/6176788 and https://gist.github.com/minrk/3719849; thank you for posting!). These are great in the short term, and their only disadvantage is kludging together a configuration that works, and like the original poster noted, remembering to clear all output before saving is something I'm bound to screw up if I don't script it. Is there any plan to bundle these scripts (or similar functionality in a command line script) as part of IPython?

I also noticed that there was some discussion of adding configuration options so that users could disable saving output. Last time I accessed http://ipython.org/ipython-doc/dev/config/options/notebook.html, I didn't see any options that would save only input cells. Is there still a plan to implement this feature?

Owner

minrk commented Jul 24, 2014

There is no immediate plan to implement these as part of IPython itself.

mpschr commented Aug 8, 2014

I agree that the autosave / save as script (--script) would be most useful if they could be set at document level.

For example if I start up a ipynb server with the --script flag it'll save all files as .py. It is a very useful feature but in many cases I may not be interested in exporting all ipynb files to .py, but just those that contain common functions/workflows

This would be extremely useful for us, please consider adding just an optional flag!

aldanor commented Aug 26, 2014

Would be really nice if IPython supported this natively since the notebooks would become much more version control friendly.

mforbes commented Sep 21, 2014

Please implement a post_save hook soon, especially since --script support has been removed. Not all version control systems have smudge filters that can work as a universal replacement yet.

@minrk minrk modified the milestones: 3.0, wishlist Nov 14, 2014

having a notebook option to clear output (and probably input cell number) on save would be great, as it would make ipynb much more VCS-friendly by keeping the diff noise lower.

aldanor commented Dec 12, 2014

Agreed, you pretty much have to do it if you check in the notebooks into version control.

@minrk minrk closed this in #6896 Dec 12, 2014

Owner

minrk commented Dec 12, 2014

Fortunately, #6896 just landed, adding a generic pre-save hook, with which you can do any manipulations on the notebook structure prior to saving, including stripping out any information you don't want to be saved.

that's great, thanks! any chance of getting a usage example for the pre-save hook into the docs? E.g., right now I'm not entirely clear how I would convince Ipython to clean my notebooks before saving...

Owner

minrk commented Dec 12, 2014

The PR description includes such an example.

hm, I saw this previously but didn't grok it correctly, apparently. Now I do. Still, the PR description will probably soon be lost in the mists of time, right?

Owner

Carreau commented Dec 12, 2014

hm, I saw this previously but didn't grok it correctly, apparently. Now I do. Still, the PR description will probably soon be lost in the mists of time, right?

Do I hear a PR coming from your repo to our doc ?

heh, you're of course correct to call me out on this. :D
I'll have to figure out and set up everything at home first - nowadays I (sadly) mostly use Ipython (at work) on Windows via an Anaconda installation. Let's see what I can do.

Owner

Carreau commented Dec 13, 2014

Advantages of docs is that you don't need to install to test :-P. You should even be able to edit the docs directly from github.

Finally getting around to this (soon I hope).
Would the appropriate place to add those usage examples be right after here, as an additional section "Example IPython notebook file"?

irhs commented Jul 1, 2015

@Carreau PR to doc with usage examples for pre/post-save hooks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment