Join GitHub today
GitHub is home to over 31 million developers working together to host and review code, manage projects, and build software together.
Sign upadd step to validate preprocessors that relies on nbformat 4.4 #645
Conversation
mpacer
force-pushed the
mpacer:validate_preprocessors
branch
from
8ac540f
to
d7841b3
Aug 10, 2017
takluyver
reviewed
Aug 10, 2017
nbconvert/exporters/exporter.py
Outdated
@@ -308,4 +308,5 @@ def _preprocess(self, nb, resources): | |||
#to each preprocessor | |||
for preprocessor in self._preprocessors: | |||
nbc, resc = preprocessor(nbc, resc) | |||
nbformat.validate(nbc, relax_add_props=True) |
This comment has been minimized.
This comment has been minimized.
takluyver
Aug 10, 2017
Member
I wonder if it's worth catching the validation error and re-raising with a message saying which preprocessor made it invalid?
This comment has been minimized.
This comment has been minimized.
mpacer
Aug 15, 2017
Author
Member
That is surprisingly non-trivial to do given the way that NotebookValidationError
is written.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
takluyver
Aug 15, 2017
Member
I've posted the same idea on that nbformat issue, but for easy reference, this is what I'd suggest:
try:
nbformat.validate(nbc, relax_add_props=True)
except nbformat.ValidationError:
self.log.error('Notebook is invalid after preprocessor %r', preprocessor)
raise # With no args, raise re-raises the same exception
It would be neater to have it in the exception message itself, but I think having it in logging is good enough.
Aug 15, 2017
This was referenced
Merged
Open
This comment has been minimized.
This comment has been minimized.
Ok should be good to go now once nbformat is released |
mpacer
modified the milestone:
5.3
Aug 18, 2017
This comment has been minimized.
This comment has been minimized.
Reran CI now that nbformat 4.4 is released and it passes. This should be good to merge. |
minrk
merged commit 2fedaf9
into
jupyter:master
Aug 21, 2017
1 check passed
continuous-integration/travis-ci/pr
The Travis CI build passed
Details
minrk
changed the title
add step to validate preprocessors that relies on jupyter/nbformat#103
add step to validate preprocessors that relies on nbformat 4.4
Aug 21, 2017
mpacer
added
unlogged
and removed
unlogged
labels
Aug 31, 2017
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
mpacer commentedAug 10, 2017
This PR will break not pass on travis until jupyter/nbformat#103 is changed.
We had thought (in our discussion in #643) that preprocessors currently would not be able to violate the NotebookNode format… turns out, that isn't the case!
Lo, behold the
PizzaPreprocessor
:which is part of our
TestExporter
tests and is almost 4 years old.This is definitely not a valid attribute, so it seems we've not been strict about enforcing the "nb→nb" aspect of Preprocessors for a while.
This change would enforce the constraint that @minrk suggested in #643, that we allow only additional attributes to be added to the intermediate representation of the notebook.
That would mean that the PizzaPreprocessor would now be ok (at least as far as the intermediate representation goes), as well as the
cell.transient
field used in #643.It also would enforce this constraint on 3rd-party extensions relying on the native preprocessor support in the core Exporter (which is probably going to break some extensions but we probably want to know what extensions actually cause non-valid notebooks to be written).