Allow/default to disabling all CDN operations

[bollwyvl]

Thanks again for getting this going.

As I've hinted at over on #41, a huge benefit of static sites is that they can be, well static. By this I mean: once you get a build working and put someplace, it will work, provided some future browser hasn't disabled a critical API, even if that someplace is a USB stick or DVD in a computer that can't access the internet.

I don't think it's reasonable in the long term to trust any of the major CDNs to necessarily be around in five years, much less bespoke ones like mathjax and pyodide. Further, CDNs are a source of lapses in user privacy and can be abused/disabled to affect harm/censorship by any number of agents.

I would be tempted to give a pass to content-addressable systems (e.g. git, fossil, bittorent, hypercore, ipfs) that can be verifiably cached, locally, and differentiably updated, but for now I think part of being able to fully realize the potential of this system is its ability to generate archival-grade content that can be used without access to The Internet, even if that isn't the default mode of operation.

The Challenges

Right now, this looks like:

• mathjax:
  • understandable
    • there is (at least) jupyter_server_mathjax
  • I feel like lab core or server should probably just suck it up and distribute it
    • but failing that, another, another copy of mathjax in a distributed package a la Integrate into existing Jupyter-based workflows #41 isn't going to kill anybody's internet
• pyodide:
  • sticky indeed!
    • import pandas, matplotlib actually only pulls 32mb of data, which honestly is within the range of "sane," i'd say, to distribute as an sdist... and even a whl, if it doesn't get duplicated.
      • over on ipydrawio, i've done some of the dirty work to get path-perfect things in place with !!file-loader!, so it's definitely doable to make something that quacks like their CDN.

Once it is working, "offline," keeping it that way is always going to be a challenge... I've had some luck with gross /etc/hosts stuff in VM testing, but perhaps there are better, browser-based ways to do it now.

[jtpio]

Thanks!

Right this should be able to run offline. Especially when publishing assets via GitHub releases (https://github.com/jtpio/jupyterlite/issues/16). Folks should just be able to download the tarball, run python -m http.server and have things work out of the box.

https://github.com/jtpio/jupyterlite/pull/26 tried to incrementally improve this for Pyodide, by making it possible to configure the URL via a page config option.

[jtpio]

We are also now installing traitlets and ipython in the worker initialization script:

jupyterlite/packages/pyolite-kernel/src/worker.ts

Lines 23 to 36 in 90d6c04

	await pyodide.runPythonAsync(`
	import micropip
	await micropip.install([
	'traitlets',
	'${_widgetsnbextensionWheelUrl}',
	'${_nbformatWheelUrl}',
	'${_ipykernelWheelUrl}'
	])
	await micropip.install([
	'${_pyoliteWheelUrl}'
	]);
	await micropip.install('ipython');
	import pyolite
	`);

Ideally this should be made configurable so a JupyterLite website can fully work offline with extra requests to PyPI. For now the workaround for an offline mode is to preload the packages so they are cached in the browser.

Also related to jupyterlite/pyodide-kernel#60 and #225

[bollwyvl]

Whew! With #419 in, we're really just down to calling out to CDN for pyodide. As discussed on #310, the full pyodide distribution is 178mb: we don't even really want this in our tests, etc. Instead, I'm thinking, this would go well on the demo site... or perhaps a separate one altogther, e.g. reproducibility-demo.

[bollwyvl]

Well, we don't have a packaged solution for a fully-self-hosted solution, but with #425 and #427 in, it is now possible to configure a jupyter lite that doesn't create any third-party requests 🎉 !

[jtpio]

Thanks @bollwyvl for working on this!

As part of #393 we could have an how-to guide showing how to use these, and how to generate a site that works offline.

[jtpio]

Well, we don't have a packaged solution for a fully-self-hosted solution

Maybe the available options and flags are enough for now?

As part of #393 we could have an how-to guide showing how to use these, and how to generate a site that works offline.

There is now some documentation to configure this in https://jupyterlite.readthedocs.io/en/latest/howto/configure/advanced/offline.html.

[jtpio]

Closing as fixed by https://jupyterlite.readthedocs.io/en/latest/howto/configure/advanced/offline.html via #393.

Thanks!

[bollwyvl]

Yeah, there could probably still be a top -level [extra] and flag that configured everything, but this is mostly solid. But never forget!