Please make the dependency on debugpy optional

[mgorny]

DebugPy is a packaging nightmare, and doesn't seem to support non-x86 platforms. Could you please make the dependency optional?

[JohanMabille]

That would break the debugging feature of ipykernel. I'm not sure we want to require users to do extra manipulations to have the debugger in ipykernel because of technical difficulty to package debugpy. Besides, I'm not sure to understand what you mean by "DebugPy doesn't seem to support non-x86 platforms": DebugPy has been packaged for ppc and aarch64 on conda-forge and seems to work.

[mgorny]

Well, right now forcing debugpy means we can't package new ipykernel at all due to bug #715. Though I suppose that's a problem of its own, and needs to be fixed independently of debugpy.

[frenzymadness]

It seems that the latest stable debugpy is not compatible with Python 3.10, although some commits exist in the main branch.

[frenzymadness]

I've tried to update ipykernel in Fedora to the latest stable release but I've failed because of debugpy. Debugpy itself is not yet compatible with Python 3.10 even when you use the content of the main branch. According to upstream, the reason is that the bundled pydevd does not support 3.10 fully yet: microsoft/debugpy#714
But this is not the biggest issue I have. The problem is that in Fedora, we do not provide JupyterLab as an RPM package - we only have Jupyter Notebook. And Jupyter Notebook does not support debuggers yet so trying to package and maintain debugpy just to be able to update ipykernel when our users have no benefit from the debugger implementation does not seem to be the best way how to spend my time.
Right now, I can either keep the latest version where debugpy is not required or try to disable the debugger implementation in ipykernel.

Should I try to prepare a PR making the dependency on debugpy optional?

[martinRenou]

It might be simpler and safer to use pip, or even better miniconda, to install your Python packages, instead of dnf.

That way you don't mess with your system. You could even start using JupyterLab :)

[hroncok]

Some of our Fedora packages (integrated into the distro) depend on ipykernel. Using pip or miniconda is not an option. Lumír has provided some feedback about why is this new dependency on DebugPy undesired and problematic for us and saying "just use pip/miniconda" does not bring any value to this discussion.

[martinRenou]

Is the issue with Python 3.10 the only blocker? It might be worth just waiting for debugpy to be released with Python 3.10 support?

[martinRenou]

saying "just use pip/miniconda" does not bring any value to this discussion.

I was just trying to help by suggesting another approach to install it. I didn't understand you were actually making a Fedora package depending on ipykernel. But now I get why conda/pip will not be helpful to you.

[hroncok]

I was just trying to help ...

Apologies if my response was a bit too unfriendly. I guess I assumed it was obvious that we actually talk about the distro package of ipykernel, but I see now that it was not. Glad that we are now on the same page, thank you!

The problem is that in Fedora, we do not provide JupyterLab as an RPM package - we only have Jupyter Notebook. And Jupyter Notebook does not support debuggers yet so trying to package and maintain debugpy just to be able to update ipykernel when our users have no benefit from the debugger implementation does not seem to be the best way how to spend my time.

I think this is the main issue here. Not all use cases of ipykernel require debugging and always bringing the new dependency is kinda undesired in most cases. What if Jupyter Lab requires ipykernel[debugger] and we make the dependency optional -- using the [debugger] extra. That way, all use cases should be satisfactory.

[JohanMabille]

I think having an option to disable the dependency on debugpy (and the debugging features) is fine. Indeed, now that the Debug Adapter Protocol is part of the Jupyter Kernel Protocol, the default should be to depend on debugpy. Not sure what the best implementation would be, though (an argument for setup.py? Patching setup.py?). If I remember correctly, the dependency on the debugger lies in ipkernel.py only.

Clients detect the support for debugger in the kernelspec. Therefore when the debugger is not supported, the debugger field of the kernelspec should be set to False.

I would be happy to review and test your PR.

[minrk]

from a setuptools/conda/most packaging environment stand point, ipykernel should depend on debugpy in its metadata. However, for cases like fedora/debian packaging, which have different priorities about dependencies than most, they can change the default dependencies to offer more stripped-down functionality. pip doesn't have a 'default extras', so our only reasonable choice is to put it in install_requires.

I think the appropriate level of support from our side is to try to handle debugpy being unavailable at runtime (catch ImportErrors). Since external-packaging tools ignore dependencies in the package metadata, I don't think we need to touch that, they can just exclude debugpy from the dependencies on their side, and as long as the code handles debugpy being unavailable, it should be okay. We've had similar workarounds for e.g. stdlib sqlite3 being unavailable - that shouldn't ever happen, but it does in weird packaging environments.

[hroncok]

Since external-packaging tools ignore dependencies in the package metadata...

Ours don't. We would need to patch it out. But otherwise, your assessment is correct, the code handling debugpy being unavailable would be helpful, as we would only need to patch out the metadata.

[frenzymadness]

I think the appropriate level of support from our side is to try to handle debugpy being unavailable at runtime (catch ImportErrors).

My plan is to prepare a patch for this next week. Ideally including a test run with debugpy unavailable which makes us sure that the support for that installation won't break in the future.