Fix render of PDF chart images

[trevorcampbell]

@joelostblom I've got the PDF build working now (just run build_pdf.sh), but all the charts don't render properly. They all look like:

Perhaps you know the fix for this? I tried playing with stuff here but no luck.

[joelostblom]

This seems to be a hard nut to crack... I have spent most of today trying to figure this out but can't find something that works well:

1. Using another renderer (like in the link you posted) does not work in itself because Altair still requires a javascript frontend to create the chart that is then converted to svg/png (I think at least). When jupyter book runs in latexpdf mode it seems to execute cells in a different way than when it runs in html or htmlpdf mode, so there is not js frontend available for altair (at least this is what I think happens). htmlpdf works, but is terrible in splitting the document into pages and sentences are cut off
2. I also tried pre-executing notebooks so that the charts are created ahead of time and then just stored in the notebook as output but I can't get this to work either. When I set the execution to 'off' jupyterbook just stops showing outupt for all cells in PDF format.

I'm not sure what to try next (maybe saving to disk or using vlconvert) but I need a break to focus on a few other things for the rest of the day. I'll open an issue in the jupyter book repo as well.

[joelostblom]

As an update I am able to display them now, but have no way to control size and resolution, and I'm not sure my current workaround will allow for that.

[joelostblom]

After some additional hours on this, it seems like the main issue is that jupyterbook's latex output does not play nicely with SVGs. If it did, I think all we need would be to enable the svg renderer and be done with it. I posted more details and asked for help in executablebooks/MyST-NB#370 (comment)

[joelostblom]

I've heard nothing from the Juptyer Book community regarding displaying SVGs in the pdflatex build, so it seems like this is not a priority for them. I have worked with one of the other altair developers to have a solution directly via altair instead that allows us to see high dpi pngs in the book, but we are depending on a new version of an external package being released before bringing the feature into altair. It seems like this could happen within a month, so I just wanted to check in what is the timeline on having this fixed (we could get low resolution images already if we just want to play around with things).

[trevorcampbell]

Thanks for all your efforts @joelostblom -- I think holding off for a month on the PDF build is just fine. We can wait for the altair fix while working on the actual content of the book. I imagine getting the PDF build looking nice will take a week or so in total once content is settled, so not a big deal if I can only start on that in a month.

[trevorcampbell]

@joelostblom just following up on this since you're mostly done your sweep through all the chapters / worksheets / tutorials. Any update?

[joelostblom]

Yup! We just merged this functionality into Altair in vega/altair#3163. The new release (5.1) should be cut this week.

We need to silently set something like this at the beginning of each notebook only for the PDF build (adjusting scale_factor and ppi as needed):

alt.renderers.enable("png", scale_factor=0.8, ppi=200)

I am thinking we can either do it by putting that line in the ipython startup file in the docker container, or if this for some reason does not work with jupyter book execution, we could create a hidden cell at the top of each notebook and drop a unique string pattern as a comment there and then use sed to replace it with the altair command during the pdf build only. Actually I might try the latter and attempt to deal with #143 in the same PR.

[trevorcampbell]

@joelostblom there should be a way to check for the PDF vs HTML build idiomatically in python during build... I would at least check for that before going the sed or ipython startup file routes.

[trevorcampbell]

Also, if that isn't the case, here's a better option than seding source files: in build_pdf.sh, replace

docker run --rm -v $(pwd):/home/jovyan ubcdsci/py-intro-to-ds:202308182146260106a8 /bin/bash -c "jupyter-book build source --builder pdflatex"

with

docker run --rm -v $(pwd):/home/jovyan ubcdsci/py-intro-to-ds:202308182146260106a8 /bin/bash -c "export BOOK_BUILD_TYPE='PDF'; jupyter-book build source --builder pdflatex"

do similar with the HTML version

and then in the source files just check the BOOK_BUILD_TYPE environment variable

[joelostblom]

Clever idea, thank you! Too bad I spent time on the plane with my sed approach... being offline with sed trying to understand odd escaping behavior is not the most productive time I've had... I switched to your approach and put up a PR.