nbdev_build_docs fails with NotImplementedError raised in asyncio

[jnareb]

After upgrading all Python modules with conda update --all, the nbdev_build_docs now fails with the following error:

$ nbdev_build_docs
converting: C:\Users\jnare\GitHub\git-commit-graph-ext\index.ipynb

Error in atexit._run_exitfuncs:
Traceback (most recent call last):
  File "c:\programdata\anaconda3\lib\site-packages\nbclient\client.py", line 365, in _async_cleanup_kernel
    await ensure_async(self.kc.stop_channels())
  File "c:\programdata\anaconda3\lib\site-packages\jupyter_client\client.py", line 125, in stop_channels
    if self.shell_channel.is_alive():
  File "c:\programdata\anaconda3\lib\site-packages\jupyter_client\client.py", line 151, in shell_channel
    socket = self.connect_shell(identity=self.session.bsession)
  File "c:\programdata\anaconda3\lib\site-packages\jupyter_client\connect.py", line 561, in connect_shell
    return self._create_connected_socket('shell', identity=identity)
  File "c:\programdata\anaconda3\lib\site-packages\jupyter_client\connect.py", line 545, in _create_connected_socket
    sock = self.context.socket(socket_type)
  File "c:\programdata\anaconda3\lib\site-packages\zmq\sugar\context.py", line 226, in socket
    s = self._socket_class(self, socket_type, **kwargs)
  File "c:\programdata\anaconda3\lib\site-packages\zmq\_future.py", line 144, in __init__
    self._init_io_state()
  File "c:\programdata\anaconda3\lib\site-packages\zmq\asyncio\__init__.py", line 53, in _init_io_state
    self.io_loop.add_reader(self._fd, lambda : self._handle_events(0, 0))
  File "c:\programdata\anaconda3\lib\asyncio\events.py", line 501, in add_reader
    raise NotImplementedError
NotImplementedError 
Exception ignored in: <function Socket.__del__ at 0x000002036B509160>
Traceback (most recent call last):
  File "c:\programdata\anaconda3\lib\site-packages\zmq\sugar\socket.py", line 67, in __del__
    self.close()
  File "c:\programdata\anaconda3\lib\site-packages\zmq\_future.py", line 160, in close
    self._clear_io_state()
  File "c:\programdata\anaconda3\lib\site-packages\zmq\asyncio\__init__.py", line 60, in _clear_io_state
    self.io_loop.remove_reader(self._fd)
  File "c:\programdata\anaconda3\lib\asyncio\events.py", line 504, in remove_reader
    raise NotImplementedError
NotImplementedError:
Exception ignored in: <function Socket.__del__ at 0x000002036B509160>
Traceback (most recent call last):
  File "c:\programdata\anaconda3\lib\site-packages\zmq\sugar\socket.py", line 67, in __del__
    self.close()
  File "c:\programdata\anaconda3\lib\site-packages\zmq\_future.py", line 160, in close
    self._clear_io_state()
  File "c:\programdata\anaconda3\lib\site-packages\zmq\asyncio\__init__.py", line 60, in _clear_io_state
    self.io_loop.remove_reader(self._fd)
  File "c:\programdata\anaconda3\lib\asyncio\events.py", line 504, in remove_reader
    raise NotImplementedError
NotImplementedError:
Conversion failed on the following:
index.ipynb

I use Python 3.8.5 (from Anaconda3), Jupyter Notebook 6.1.4, JupyterLab 2.2.6

The following version of libraries are currently installed:

• nbclient 0.5.1
• jupyter_client 6.1.7
• pyzmq 19.0.2
• nbdev 1.1.5

My operating system is Windows 10 Home 64-bit, version 1909

[jnareb]

It turns out that the same problem exists in nbdev_test_nb and was caused, from what I understand, by the change in Python 3.8, where the default event loop on Windows changed to proactor (and not "selector" event loop).

See for example issue mkdocs/mkdocs#1885 fixed in mkdocs/mkdocs#1922

It looks like for Windows one needs to add the following line

asyncio.set_event_loop_policy(asyncio.WindowsSelectorEventLoopPolicy())

[miklevin]

This may be the same issue. I'm on Windows 10 with WSL2. On Linux I'm running Ubuntu 20.04.4 LTS. I use Jupyter Desktop Version 3.3.2-2. I'm on nbdev 1.2.5 on both the Linux and Windows sides. So I do Notebooks under Windows Desktop JupyterLab and the nbdev_[commands] under WSL2/Linux/focal 20.04.4. The problem is extremely consistent and reproducible.

The process is:

Make new folder.
Create Notebook.
Turn it into git repo.
Add git endpoint.
Run nbdev_new.
Put in hidden showdoc block.
Add an async function to an export cell.
Attempt running nbdev_build_docs. Get this error:

(py310) ubuntu@LunderVand:/mnt/c/Users/mikle/github/issues$ git branch -M main
(py310) ubuntu@LunderVand:/mnt/c/Users/mikle/github/issues$ git remote add origin git@github.com:miklevin/issues.git
(py310) ubuntu@LunderVand:/mnt/c/Users/mikle/github/issues$ nbdev_new
Executing: git config --local include.path ../.gitconfig
Success: hooks are installed and repo's .gitconfig is now trusted
(py310) ubuntu@LunderVand:/mnt/c/Users/mikle/github/issues$ nbdev_build_docs
converting: /mnt/c/Users/mikle/github/issues/index.ipynb
converting: /mnt/c/Users/mikle/github/issues/Untitled.ipynb
An error occurred while executing the following cell:
------------------
show_doc(foo, default_cls_level=2, show_all_docments=False)
------------------

---------------------------------------------------------------------------
NameError                                 Traceback (most recent call last)
/tmp/ipykernel_3836/4040911153.py in <module>
----> 1 show_doc(foo, default_cls_level=2, show_all_docments=False)

NameError: name 'foo' is not defined
NameError: name 'foo' is not defined

Conversion failed on the following:
Untitled.ipynb
Traceback (most recent call last):
  File "/home/ubuntu/py310/bin/nbdev_build_docs", line 8, in <module>
    sys.exit(nbdev_build_docs())
  File "/home/ubuntu/py310/lib/python3.10/site-packages/fastcore/script.py", line 113, in _f
    tfunc(**merge(args, args_from_prog(func, xtra)))
  File "/home/ubuntu/py310/lib/python3.10/site-packages/nbdev/export2html.py", line 649, in nbdev_build_docs
    if fname is None: make_sidebar()
  File "/home/ubuntu/py310/lib/python3.10/site-packages/nbdev/export2html.py", line 717, in make_sidebar
    create_default_sidebar()
  File "/home/ubuntu/py310/lib/python3.10/site-packages/nbdev/export2html.py", line 709, in create_default_sidebar
    dic = {get_config().lib_name: _create_default_sidebar()}
  File "/home/ubuntu/py310/lib/python3.10/site-packages/nbdev/export2html.py", line 701, in _create_default_sidebar
    for t,f in groupby(names, _get_title).items():
  File "/home/ubuntu/py310/lib/python3.10/site-packages/fastcore/basics.py", line 536, in groupby
    for o in x: res.setdefault(key(o), []).append(val(o))
  File "/home/ubuntu/py310/lib/python3.10/site-packages/nbdev/export2html.py", line 690, in _get_title
    with open(fname, 'r') as f: code = f.read()
FileNotFoundError: [Errno 2] No such file or directory: '/mnt/c/Users/mikle/github/issues/docs/Untitled.html'
(py310) ubuntu@LunderVand:/mnt/c/Users/mikle/github/issues$

[miklevin]

Code can be examined here https://github.com/miklevin/nbdev_async