Plugin doesn't work with nbconvert >= 6.x

[andreagrandi]

Hello,

I think latest Pelican version may have introduced some breaking changes for plugin (see this post here https://blog.getpelican.com/pelican-4.5-released.html ). If I upgrade from pelican==4.2.0 to pelican==4.5.0 I get this error when I build my website containing an .ipydb file:

ERROR: Could not process 2018/1-PimaIndiansDiabetes.ipynb
  | basic

I haven't been able to get more information. It just works fine with 4.2.0 but it doesn't work with 4.5.0

Thanks

[andreagrandi]

I did a few more tests and there seems to be an incompatibility with nbconvert >= 6.

I had accidentally upgraded that package and the plugin couldn't work anymore.

As soon as I installed back the latest 5.x available, the plugin worked (with Pelican 4.5.0).

I think we should pin the version in requirements to be >=5,<6 (or whatever the syntax is, I don't remember it by memory).

This as a short term fix of course. For the long term it would be interesting to understand what changed in 6.x

Cheers

[danielfrg]

Thanks for the report @andreagrandi a lot of packages are broken right now because of nbconvert 6.0.0, so yeah we should totally pin this.

I am personally not using this plugin anymore so I wont be working on this in the near future. If you want to take look at it please do so.

[andreagrandi]

Thanks for your quick reply Daniel!
I've done a tiny PR to pin the version #127
I'm not sure if you are still available to merge and release it by the way.
In case you think you are not able to maintain this anymore, would it be possible to become maintainer for the PYPI package here https://pypi.org/project/pelican-jupyter/ ? This way I could keep developing and fixing the package on my own repository but this would not disrupt future releases.
Cheers

[andreagrandi]

Actually I'm struggling to make my PR work, since there are a few isort problems which are failing on CI but I can't reproduce on my local machine, so don't bother yet.

[danielfrg]

@andreagrandi thanks for the offer, i am happy to add you as maintainer here and on PyPI! I will do it on GitHub now. Let me know your PyPI username and I will also add you there.

isort recently broke the CI linting on some of my projects, I personally just removed that from the CI as its not critical but up to you to just disable it.

[andreagrandi]

I'm andreagrandi on PyPI too.

About isort: where did you remove it from exactly? It seems that it's still running when I push the code, that's why the build is failing. I will try to use exactly the same version (or pin its version to the same I use locally), because I haven't been able to reproduce the same problem locally on my machine.

[danielfrg]

Added you to pypi.

I removed it from the github actions until i see why are they failing.

[dclong]

@danielfrg Are you using another better alternative or do you simply give up using Jupyter for Pelican?

[danielfrg]

I am no longer using pelican for any websites. I use mkdocs for docs so I wrote: https://github.com/danielfrg/mkdocs-jupyter
And for my website I just use hugo and a similar script.

[flothesof]

I believe I have the same problem: rendering of notebooks have suddenly failed when my automated build updated to the latest version.

Using the built-in option for specifying a template, I managed to find a workaround.

LIQUID_CONFIGS = (("IPYNB_EXPORT_TEMPLATE", "base", ""),)

If my reasoning is correct, the template called "basic" has been renamed to "base" by nbconvert, which causes the trouble of the notebooks not rendering anymore.

I therefore conclude that changing this line might solve this problem:

pelican-jupyter/pelican_jupyter/core.py

Line 89 in ce29872

template_file = "basic"

Judging by bug reports over at nbconvert, it seems that quite a lot of template-related things have been broken by release 6.0:

jupyter/nbconvert#1451 (comment)

[shanewilkins]

Confirm that the upgrade to nbconvert 6.0 is the issue. I downgraded back to 5.6 and the error went away. I haven't been able to test whether @flothesof 's fix will work.

[johnmathews]

As well as downgrading nbconvert to 5.6.0, I also needed to downgrade notebook to 5.6.0. The suggestion from @flothesof didn't make any difference for me.

[flothesof]

@johnmathews prompted by your finding I tried again and was able to replicate my workaround from (almost) scratch.
Version info:

• ipython 7.11.1
• notebook 6.1.5
• pelican 4.5.3
• jupyter 1.0.0
• jupyter-client 6.1.7
• jupyter-console 6.2.0
• jupyter-core 4.6.1
• jupyterlab-pygments 0.1.2
• pelican-jupyter 0.10.0
• nbconvert 6.0.7

Here's what I had to put into my pelicanconf.py (only relevant lines included):

from pelican_jupyter import liquid as nb_liquid # import pelican-jupyter
...
PLUGINS = ['liquid_tags.img', nb_liquid, 'representative_image']
...
# configuring notebook integration
LIQUID_CONFIGS = (("IPYNB_FIX_CSS", "False", ""), 
                  ("IPYNB_SKIP_CSS", "False", ""), 
                  ("IPYNB_EXPORT_TEMPLATE", "base", ""),)

Maybe this works better for you?

[shanewilkins]

@flothesof This seems to have done the trick for me. All of the plotly plots in my notebooks have disappeared, but that's a separate issue.

[AndreasThinks]

Glad I'm not the only one! It looks like it's an issue pulling out the Jinja2 template or similar...May have to explore alternative solutions to good Python/Jupyter static pages! Have been pondering swapping to Hugo for awhile, may be the time to make the jump.

[alex000kim]

downgrading to python 3.6, nbconvert=5.6.0, notebook=5.6.0 helps, but then plotly plots stop rendering...

[davidwagner]

Changing basic.tpl to classic/base.html.j2 in plugins/ipynb/core.py seems to also fix this, without needing to downgrade nbconvert. I have a somewhat older version of pelican-jupyter though so this might need to be translated to the latest version.

I wonder if something like

template_file = 'classic/base.html' if int(nbconvert.__version__.partition(".")[0]) >= 6 else 'basic'

might work.

[canyon289]

Im having this same issue as well. Tagging myself onto the discussion here!

[magsol]

Can confirm this is still an issue. Here is my relevant environment:

• ipython 7.29.0
• notebook 6.4.6
• pelican 4.6.0
• jupyter 1.0.0
• jupyter_client 7.1.0
• jupyter_console 6.4.0
• jupyter_core 4.9.1
• pelican-jupyter 0.10.0
• pelican-liquid-tags 1.0.1
• nbconvert 6.3.0

Can also confirm that the fix posted by @flothesof still works, at least in terms of the LIQUID_CONFIGS.

I did have a slight modification I had to make, as it appeared that having both pelican-jupyter and pelican-liquid-tags register the same notebook tag was causing problems: I could either get Jupyter notebooks to render, OR any of the other liquid tags (images, videos, code, etc). So I moved the pelican-jupyter liquid.py file over to the pelican-liquid-tags folder, renamed it notebook.py, and slightly modified the imports.

After this, my pelicanconf.py looked like this:

from pelican.plugins import liquid_tags
PLUGINS = [
        liquid_tags,                # regular ol' liquid tags
]
LIQUID_TAGS = ["img", "include_code", "gram", "video", "youtube", "notebook"]
LIQUID_CONFIGS = (("IPYNB_FIX_CSS", "False", ""), 
                  ("IPYNB_SKIP_CSS", "False", ""), 
                  ("IPYNB_EXPORT_TEMPLATE", "base", ""),)

Seems to work. It's not pretty but I couldn't find a way to have the two plugins co-exist.

[ozak]

Same error. After following the advice above I am getting the following error

could not find information about 'title'

But I have both the nbdata and ipynb-meta files for each notebook in the same folder as the notebook, and it used to work with previous versions. Any ideas would be appreciated.

[flothesof]

I do get this error for some files that are in my repository, but this is never a problem in my case. Did it not generate any pages with your setup? Le mer. 19 janv. 2022 à 03:39, Ömer Özak ***@***.***> a écrit :

…

Same error. After following the advice above I am getting the following error could not find information about 'title' But I have both the nbdata and ipynb-meta files for each notebook in the same folder as the notebook, and it used to work with previous versions. Any ideas would be appreciated. — Reply to this email directly, view it on GitHub <#126 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AASAKXLN3VNJ3ZKBG2CAKSTUWYP5RANCNFSM4SCXTLLA> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>. You are receiving this because you were mentioned.Message ID: ***@***.***>

[ozak]

No, it fails completely and does not generate anything. I also get an additional error

CRITICAL UndefinedError: 'gettext' is undefined

which I have no idea what is causing it.

[flothesof]

@danielfrg Do you have any ideas how we could get out of this situation that the current version of the plugin is not working as expected anymore? Would you be willing to transfer maintainership to someone else to get the plugin to work with current versions of nbconvert since you don’t seem to have time to work on this?
Regards,
Florian

[andreagrandi]

@danielfrg Do you have any ideas how we could get out of this situation that the current version of the plugin is not working as expected anymore? Would you be willing to transfer maintainership to someone else to get the plugin to work with current versions of nbconvert since you don’t seem to have time to work on this? Regards, Florian

I also "maintain" (I did a release but stuff in real life happened and couldn't do anymore) the package on PyPI https://pypi.org/project/pelican-jupyter/ but I've no idea how to fix this. Plus I haven't used the library in ages (I haven't since then written anything which requires this package to be working).

[flothesof]

@andreagrandi thank you for coming back on this topic
Actually, support for notebooks also works with the liquid-tags plugin and seems to be functional. This could help some people stuck here: https://github.com/pelican-plugins/liquid-tags#ipython-notebooks