broken repr in ipython 8 w/ ipywidgets installed

[dchudz]

In [1]: from dask.distributed import LocalCluster

In [2]: cluster = LocalCluster()

In [3]: cluster
Out[3]: Tab(children=(HTML(value='<div class="jp-RenderedHTMLCommon jp-RenderedHTML jp-mod-trusted jp-OutputArea-outpu…

Some relevant versions:

ipython                        8.1.1
distributed                    2022.2.0
ipywidgets                     7.6.5

Either of the following make the cluster repr go back to what it should be:

• uninstalling ipywidgets
• downgrading to ipython 7

This is the relevant change going from ipython 7 to 8: ipython/ipython#12315. It looks like ipython is attempting to render the widget repr, which doesn't make sense. I don't know whether this is something dask should fix, or if the "real" fix is further upstream in ipython.

[ian-r-rose]

Thanks @dchudz. Some additional context for people who don't want to click through a bunch of links/PRs:

The IPython terminal used to disable rich reprs and just display text-ish stuff. So the ipywidgets repr on the distributed.deploy.Cluster object was ignored, and a simple text repr was shown instead. In IPython 8 rich reprs were enabled in the terminal (the idea being that frontends with fancy terminals could try to render other mimetypes if they wanted to). So now instead of getting useful text representations of a Cluster in the terminal, users see a text representation of an ipywidgets box.

Now, we could try to sniff out whether a user is running in a notebook or a terminal, but that is explicitly discouraged and difficult to do in IPython (the idea is that the backend should be agnostic to who is rendering the content, it could be a terminal, a notebook, colab, etc). I think the best way forward is to make sure that the Cluster repr always includes a text/plain option (we currently only send a widget if ipywidgets is available). That way when the ipywidget mimetype fails to render we get a better text fallback. Unfortunately, this is a bit tricky to do with ipywidgets 7, but it's relatively straigtforward to do with the soon-to-be-released ipywidgets 8. See here for a fix.

So, I think I'd recommend revisiting this when ipywidgets 8 is released and making the above fix.