-
Notifications
You must be signed in to change notification settings - Fork 97
Notebook normalization for version control #174
Comments
nbdime can hide metadata changes, which may be some help. We also try to leave fields like those out of the metadata when they would have their default value, to minimise clutter in version control. |
Yes, we use nbdime in our workflow, but I didn't realize that it can be used as a git filter; can it? Would it be reasonable to remove output-related metadata when the corresponding output is removed? |
Possibly not as a filter, looks like it integrates with the diff and merge operations themselves: Yes, I think metadata related to outputs should probably be cleared when the outputs are. |
Without having a filter there's extra VCS noise generated.
Where shall I open the issue? Definitely nbconvert, but also the notebook UI doesn't clean the metadata on clearing output. |
I'd go for it on the notebook repo. |
In the meantime, is there a way to clear output cells with just images? A major issue in committing notebooks in repositories is, of course, the size |
I don't think there's anything ready-made to do that, but it should be pretty easy (~10 lines in Python) to load a notebook, filter out image outputs and write the result out. |
@takluyver I suppose the filter could be a sed one-liner, but |
I wouldn't try to |
closing as answered |
nbstripout provides git filters for notebooks for doing this kind of thing. If it doesn't cover these metadata keys, they can be added pretty easily. It may be appropriate to bring nbstripout's functionality into nbdime eventually if we want to have a single package providing "notebooks in version control" toolkit, but for now nbdime is only a renderer and merge tool. |
@minrk Thanks for your great suggestion. Does nbstripout Re: stripping only images: kynan/nbstripout#58 |
For a while we are using nbconvert with
ClearOutputPreprocessor
to make notebooks git-friendly. However I have observed that there are several cell and notebook metadata fields that are left intact by it. Looking at the latestnbformat
, it appears that the following cell metadata doesn't belong to VC:collapsed
autoscroll
isolated
I cannot quite find the standard kernelspec json entries, but I remember that at the very least the minor language version was causing troubles sometimes.
Now to my question: what is the best way to strip the above metadata and is there anything available short of writing an own preprocessor?
The text was updated successfully, but these errors were encountered: