Upgrade to support jupyter_client 8 (#1778)

When upgrading the dependency spec to include v8, some python tests
broke (see #1728). The tests
were failing with a timeout due to indefinitely blocked IO.

This originates in an infinitely-looping notebook and is caused by a
misconfiguration of the client connection. Currently using
`nbclient.NotebookClient` with a `jupyter_client.manager.KernelManager`
results in a deadlock. nbclient doesn't seem aware of this issue because
there is no test covering it. `NotebookClient` uses `AsyncKernelManager`
by default.

So why isn't nbgrader using it? nbgrader uses the NotebookClient via
nbconvert which just realized the same issue. Up until v7.3.1, nbconvert
has been hard-coding the default kernel manager to the sync version.
(see jupyter/nbconvert#1964)

Rather than require nbconvert>=7.3.1, this commit sets our own default
value to the async kernel manager.

nbgrader/preprocessors/execute.py

@@ -5,6 +5,7 @@
 from . import NbGraderPreprocessor
 from nbconvert.exporters.exporter import ResourcesDict
 from nbformat.notebooknode import NotebookNode
+from jupyter_client.manager import AsyncKernelManager
 from typing import Any, Optional, Tuple
 
 
@@ -67,6 +68,11 @@ class Execute(NbGraderPreprocessor, ExecutePreprocessor):
         """)
     ).tag(config=True)
 
+    def __init__(self, *args, **kwargs):
+        # nbconvert < 7.3.1 used the sync version of this, which doesn't work for us.
+        kwargs.setdefault('kernel_manager_class', AsyncKernelManager)
+        super().__init__(*args, **kwargs)
+
     def on_cell_executed(self, **kwargs):
         cell = kwargs['cell']
         reply = kwargs['execute_reply']

pyproject.toml

@@ -36,7 +36,7 @@ dependencies = [
     "ipywidgets>=7.6",
     "Jinja2>=3",
     "jsonschema>=3",
-    "jupyter_client<8",
+    "jupyter_client<9",
     "jupyter_server>=1.12,<2",
     "jupyterlab<4",
     "jupyterlab_server",