Remove ipykernel and IPython as dependencies

[maartenbreddels]

Looking at how ipykernel and IPython are used, I wonder if we can remove them as dependencies.
I think in xeus-python, and in pyodide environments we do not use the real packages as they are on pypi (right @martinRenou?)

Also, I'm working on a package that uses ipywidgets outside of a Jupyter context, and I don't need or use ipykernel and IPython.

Could we remove them as dependencies are say they should be provided by the 'host environment'/'kernel'?

Related #1892

[jasongrout]

Does our code still import ipython? If it is generic, then +1 on removing the dependency.

[jasongrout]

It looks like we import things directly from IPython and ipykernel. Is that part just not working for you?

[maartenbreddels]

Yes, but i think those dependencies will always come from the "environment" you use ipywidgets in. * Creating a jupyter notebook/lab environment * Xeus-python (not sure about that) * The environment ( jupyterlite i think has a shim package, and we plan to do something like that as well) . So i wonder if we can remove the declared dependency without getting into trouble, i.e. we will in practice always be in an environment where those packages are available. (from mobile phone) Op vr 8 jul. 2022 21:58 schreef Jason Grout ***@***.***>:

…

It looks like we import things directly from IPython and ipykernel. — Reply to this email directly, view it on GitHub <#3514 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AANPEPKUY2FJ622PIEB2SADVTCB5RANCNFSM53BYIL6Q> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[martinRenou]

👍🏽 👍🏽 👍🏽 on removing the ipykernel dependency, which will make it easier for the xeus-python and jupyterlite cases. For IPython I'm less opinionated because we use it in xeus-python and jupyterlite anyway.

IIRC we only need ipykernel in ipywidgets for the comms implementation. So xeus-python and jupyterlite have to monkey patch ipykernel to provide their own comm implementations. See https://github.com/jupyter-xeus/xeus-python/blob/master/src/xinterpreter.cpp#L77 and https://github.com/jupyterlite/jupyterlite/blob/main/packages/pyolite-kernel/py/ipykernel/ipykernel/comm.py.

I've been wondering about having a generic way for kernels to expose their Comm implementation for libraries to use. Maybe through entry points or similar: jupyter-xeus/xeus-python#342.

[maartenbreddels]

Currently, if you install notebook en jupyter lab, you get ipykernel as a dependency. Which I think is also a bit odd, but maybe can be considered user-friendly. But that means that in the most common environments we already have ipykernel.

But, given that there are other options to use ipywidgets (xeus-python, *-lite, others planned!), I don't think we should have ipykernel as a dependency. We could maybe live a while with saying: 'The ipykernel function that we use in ipywidgets is the interface that we require others to implement'.

Maybe start by dropping ipykernel, and keep the IPython dependency for now?

[jasongrout]

Previous discussion also at #3209

[jasongrout]

I commented on @Carreau's approach he proposed at jupyter-xeus/xeus-python#342 (comment).

[jasongrout]

The only place I see that we have a hard dependency on ipykernel is where we actually import the Comm package:

ipywidgets/python/ipywidgets/ipywidgets/widgets/widget.py

Line 12 in ad14ad1

from ipykernel.comm import Comm

We also have a softer dependency at

ipywidgets/python/ipywidgets/ipywidgets/widgets/widget.py

Line 424 in ad14ad1

comm = Instance('ipykernel.comm.Comm', allow_none=True)

where we have the default comm class referencing ipykernel.

If we took one of the approaches discussed on jupyter-xeus/xeus-python#342, and ipykernel were changed to instead import this new comm package and register itself as a comm provider, we could import from the comm package instead of ipykernel and use whatever implementation was provided, rather than depending on ipykernel (and we could also set that traitlet default to the comm package Comm class). This would require upstream work in ipykernel first, though. Anyone up for it?

We also have an implicit dependency on at least the interface provided by ipykernel at

ipywidgets/python/ipywidgets/ipywidgets/widgets/widget_output.py

Lines 117 to 127 in ad14ad1

	if kernel:
	parent = None
	if hasattr(kernel, "get_parent"):
	parent = kernel.get_parent()
	elif hasattr(kernel, "_parent_header"):
	# ipykernel < 6: kernel._parent_header is the parent *request*
	parent = kernel._parent_header
	if parent and parent.get("header"):
	self.msg_id = parent["header"]["msg_id"]
	self.__counter += 1

, but I presume other kernels are providing that interface?

[martinRenou]

This would require upstream work in ipykernel first, though. Anyone up for it?

I'd be happy to do it!

but I presume other kernels are providing that interface?

Yes, we had to implement those interfaces in xeus-python and jupyterlite's pyolite kernel. We could probably also improve these parts.

[maartenbreddels]

@martinRenou my guess is that https://github.com/jupyterlite/jupyterlite/blob/main/packages/pyolite-kernel/py/ipykernel/ipykernel/comm.py can be the basis for that comm package right?

I'd like to have a similar comm implementation, one that does not use traitlets (the overhead is large when using a lot of widgets).

[martinRenou]

Just FYI I'm currently working on this new comm package.

[Carreau]

@martinRenou and @jasongrout you should both have received PyPI invite to be owner of comm (currently a dummy package), on PyPI.

[martinRenou]

Thank you!

[Carreau]

No problem, thanks to you for working on this.

[martinRenou]

I created https://github.com/martinRenou/comm (should probably be moved to the IPython org). @jasongrout @maartenbreddels it'd be great if you could tell me what you think of the approach.

I pushed some PRs here and there to update ipykernel and xeus-python:
jupyter-xeus/xeus-python#548
ipython/ipykernel#973

And one in ipywidgets: #3533

[maartenbreddels]

Awesome work, I would prefer the default/base implementation to not use trailets. I liked your clean version you had I lite. What do you think?

On Mon, 1 Aug 2022 at 15:46, martinRenou ***@***.***> wrote: I created https://github.com/martinRenou/comm (should probably be moved to the IPython org). @jasongrout <https://github.com/jasongrout> @maartenbreddels <https://github.com/maartenbreddels> it'd be great if you could tell me what you think of the approach. I pushed some PRs here and there to update ipykernel and xeus-python: jupyter-xeus/xeus-python#548 <jupyter-xeus/xeus-python#548> ipython/ipykernel#973 <ipython/ipykernel#973> And one in ipywidgets: #3533 <#3533> — Reply to this email directly, view it on GitHub <#3514 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AANPEPNBRTCEQBAGGSHL77DVW7ILDANCNFSM53BYIL6Q> . You are receiving this because you were mentioned.Message ID: ***@***.***>

-- Maarten Breddels Software engineer / consultant / data scientist Python / C++ / Javascript / Jupyter www.maartenbreddels.com / vaex.io ***@***.*** +31 6 2464 0838 <+31+6+24640838> [image: Twitter] <https://twitter.com/maartenbreddels>[image: Github] <https://github.com/maartenbreddels>[image: LinkedIn] <https://linkedin.com/in/maartenbreddels>[image: Skype]

[martinRenou]

Thank you! I'm totally fine removing traitlets in the base implementation. Although, will this break some configuration options in ipykernel?

[maartenbreddels]

Maybe put this version in ipykernel? And a clean version in comm? Maybe the version in ipykernel can still inherit these clean classes, or use composition

On Mon, 1 Aug 2022 at 16:35, martinRenou ***@***.***> wrote: I'm totally fine removing traitlets in the base implementation. Although, will this break some configuration options in ipykernel? — Reply to this email directly, view it on GitHub <#3514 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AANPEPPDSPVMEJYT4N73VLLVW7OE3ANCNFSM53BYIL6Q> . You are receiving this because you were mentioned.Message ID: ***@***.***>

-- Maarten Breddels Software engineer / consultant / data scientist Python / C++ / Javascript / Jupyter www.maartenbreddels.com / vaex.io ***@***.*** +31 6 2464 0838 <+31+6+24640838> [image: Twitter] <https://twitter.com/maartenbreddels>[image: Github] <https://github.com/maartenbreddels>[image: LinkedIn] <https://linkedin.com/in/maartenbreddels>[image: Skype]

[martinRenou]

Indeed, I'll do this! Thank you for the review :)

[martinRenou]

Done! ipython/comm@86c8447

[martinRenou]

Comm is now a package under the IPython org https://github.com/ipython/comm (huge thanks to @ivanov for moving the repo!).

[manzt]

is ipython still a required dep with comm? / would it be possible to dynamically import and call to get_ipython(), only when necessary?