Extension Manager return error 500

[franckjulliard]

Hello,
I am running latest version of JupyterLab (0.35.4) in docker based on the jupyter/datascience-notebook latest image.
The docker is running inside company network with limited access to the internet by all the necessary route are open to install packages and have jupyter and python up and running.

When enabling the Extension Manager I can see the following error :

Error communicating with server extension. Consult the documentation for how to ensure that it is enabled.

Reason given:

Error: 500 ()

The log is showing a lot of things but I cannot get what's going on

[E 02:22:23.694 NotebookApp] 500 GET /jultest/lab/api/extensions?1555467298192 (172.18.0.2) 445098.84ms referer=https://suvm-rid02/jultest/lab
[E 02:22:23.695 NotebookApp] Uncaught exception GET /jultest/lab/api/extensions?1555467298206 (172.18.0.2)
HTTPServerRequest(protocol='http', host='suvm-rid02', method='GET', uri='/jultest/lab/api/extensions?1555467298206', version='HTTP/1.1', remote_ip='172.18.0.2')
Traceback (most recent call last):
File "/opt/conda/lib/python3.7/site-packages/tornado/web.py", line 1699, in _execute
result = await result
File "/opt/conda/lib/python3.7/site-packages/tornado/web.py", line 1699, in _execute
result = await result
File "/opt/conda/lib/python3.7/site-packages/tornado/gen.py", line 736, in run
yielded = self.gen.throw(*exc_info) # type: ignore
File "/opt/conda/lib/python3.7/site-packages/jupyterlab/extension_manager_handler.py", line 216, in get
extensions = yield self.manager.list_extensions()
File "/opt/conda/lib/python3.7/site-packages/tornado/gen.py", line 729, in run
value = future.result()
File "/opt/conda/lib/python3.7/site-packages/tornado/gen.py", line 736, in run
yielded = self.gen.throw(*exc_info) # type: ignore
File "/opt/conda/lib/python3.7/site-packages/jupyterlab/extension_manager_handler.py", line 216, in get
extensions = yield self.manager.list_extensions()
File "/opt/conda/lib/python3.7/site-packages/tornado/gen.py", line 729, in run
value = future.result()
File "/opt/conda/lib/python3.7/site-packages/tornado/gen.py", line 736, in run
yielded = self.gen.throw(*exc_info) # type: ignore
File "/opt/conda/lib/python3.7/site-packages/jupyterlab/extension_manager_handler.py", line 86, in list_extensions
pkg_info = yield self._get_pkg_info(name, data)
File "/opt/conda/lib/python3.7/site-packages/tornado/gen.py", line 729, in run
value = future.result()
File "/opt/conda/lib/python3.7/site-packages/tornado/gen.py", line 736, in run
yielded = self.gen.throw(*exc_info) # type: ignore
File "/opt/conda/lib/python3.7/site-packages/jupyterlab/extension_manager_handler.py", line 86, in list_extensions
pkg_info = yield self._get_pkg_info(name, data)
File "/opt/conda/lib/python3.7/site-packages/tornado/gen.py", line 729, in run
value = future.result()
File "/opt/conda/lib/python3.7/site-packages/tornado/gen.py", line 736, in run
yielded = self.gen.throw(*exc_info) # type: ignore
File "/opt/conda/lib/python3.7/site-packages/jupyterlab/extension_manager_handler.py", line 154, in _get_pkg_info
outdated = yield self._get_outdated()
File "/opt/conda/lib/python3.7/site-packages/tornado/gen.py", line 729, in run
value = future.result()
File "/opt/conda/lib/python3.7/site-packages/tornado/gen.py", line 736, in run
yielded = self.gen.throw(*exc_info) # type: ignore
File "/opt/conda/lib/python3.7/site-packages/jupyterlab/extension_manager_handler.py", line 154, in _get_pkg_info
outdated = yield self._get_outdated()
File "/opt/conda/lib/python3.7/site-packages/tornado/gen.py", line 729, in run
value = future.result()
File "/opt/conda/lib/python3.7/site-packages/tornado/gen.py", line 736, in run
yielded = self.gen.throw(*exc_info) # type: ignore
File "/opt/conda/lib/python3.7/site-packages/jupyterlab/extension_manager_handler.py", line 189, in _load_outdated
logger=self.log,
File "/opt/conda/lib/python3.7/site-packages/tornado/gen.py", line 729, in run
value = future.result()
File "/opt/conda/lib/python3.7/concurrent/futures/_base.py", line 425, in result
return self.__get_result()
File "/opt/conda/lib/python3.7/concurrent/futures/_base.py", line 384, in __get_result
raise self._exception
File "/opt/conda/lib/python3.7/concurrent/futures/thread.py", line 57, in run
result = self.fn(*self.args, **self.kwargs)
File "/opt/conda/lib/python3.7/site-packages/jupyterlab/commands.py", line 357, in get_latest_compatible_package_versions
return handler.latest_compatible_package_versions(names)
File "/opt/conda/lib/python3.7/site-packages/jupyterlab/commands.py", line 1408, in latest_compatible_package_versions
raise ValueError(msg % keys)
ValueError: "['@jupyter-widgets/jupyterlab-manager@0.38.1', '@jupyterlab/hub-extension@0.12.0', 'jupyterlab_bokeh@0.6.3', '@jupyterlab/toc@0.6.0']" is not a valid npm package

[W 02:22:23.696 NotebookApp] Unhandled error
[E 02:22:23.697 NotebookApp] {
"Host": "suvm-rid02.ad-its.credit-agricole.fr",
"User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/73.0.3683.86 Safari/537.36",
"Accept": "/",
"Accept-Encoding": "gzip, deflate, br",
"Accept-Language": "en-US,en;q=0.9",
"Content-Type": "application/json",
"Cookie": "username-suvm-rid02="2|1:0|10:1555465146|45:username-suvm-rid02|44:NTQ3OTY4ZWRiMGQzNDBjOTgxNTQxYTgzZDQzYmJkMGM=|7bcc8663503e738faab18293eb334b8e8ee46eaa0b7a729e2eafce753078c863"; _xsrf=2|9f52a378|0084787aa4b2e0da6a7c0e05eb00da4b|1553471738; username-suvm-rid02-40015="2|1:0|10:1553493907|51:username-suvm-rid02-40015|44:NTgyNTAyYmQ1OWUzNDU1ODg5OTNlNTU0YWE4NzU0M2I=|697aa5e6931a0e96b58804be26d6fbea18534560b7f255011a9779d8ec11970c"; username-suvm-rid02--40023="2|1:0|10:1553501669|51:username-suvm-rid02-40023|44:MDIwNWNjNTM3MGFiNDhmNDgwMWJjMzdkOWZiYzFlMGQ=|47625d6579a0384895e10ccccd3ef01f1c620f5071cae059c5e36d8cec4a1824"",
"Referer": "https://suvm-rid02/jultest/lab",
"X-Forwarded-For": "10.198.240.59",
"X-Forwarded-Host": "suvm-rid02",
"X-Forwarded-Port": "443",
"X-Forwarded-Proto": "https",
"X-Forwarded-Server": "67f7427b34e4",
"X-Real-Ip": "10.198.240.59",
"X-Xsrftoken": "2|9f52a378|0084787aa4b2e0da6a7c0e05eb00da4b|1553471738"
}
[E 02:22:23.697 NotebookApp] 500 GET /jultest/lab/api/extensions?1555467298206 (172.18.0.2) 445100.97ms referer=https://suvm-rid02/jultest/lab

The version of node.js is v11.11.0
version of npm is 6.7.0

The extension I install manually during the build of the image are working fine.

Any help is welcome,
Franck

[ian-r-rose]

Thanks for the report @franckjulliard, it looks like there may be a bug in the logic for installing extensions via the extension manager. I'm particularly looking at this line:

ValueError: "['@jupyter-widgets/jupyterlab-manager@0.38.1', '@jupyterlab/hub-extension@0.12.0', 'jupyterlab_bokeh@0.6.3', '@jupyterlab/toc@0.6.0']" is not a valid npm package

It would be helpful if you could narrow down your setup to a minimal example. Is there a small set of commands/actions that you can see that trigger this error?

[franckjulliard]

My Dockerfile is very simple actually, just a few lines to access some local, proxy ...

```docker
# Distributed under the terms of the Modified BSD License.
FROM jupyter/datascience-notebook

MAINTAINER Jupyter Project <jupyter@googlegroups.com>

USER root

RUN chown -R 1000:100 /home/$NB_USER/.jupyter

COPY ipython_config.py /home/$NB_USER/.ipython/profile_default/
COPY jupyter_notebook_config.py /home/$NB_USER/.jupyter/
COPY jupyter_notebook_config.json /home/$NB_USER/.jupyter/

RUN chown -R 1000:100 /home/$NB_USER/.ipython
RUN chown -R 1000:100 /home/$NB_USER/.jupyter


ENV http_proxy=http://proxyjpn:8080
ENV https_proxy=http://proxyjpn:8080
ENV ftp_proxy=http://proxyjpn.8080
ENV no_proxy="localhost, 127.0.0.1"

# DB Connection 

RUN apt-get update && \
    apt-get install -y --no-install-recommends \
    freetds-bin unixodbc unixodbc-dev odbcinst tdsodbc python-pyodbc libmysqlclient-dev vim

ADD odbc.ini odbcinst.ini /etc/
ADD freetds.conf /etc/freetds/

# Install additional Packages
RUN pip install --upgrade pip

RUN pip install pyodbc openpyxl lxml xlwt altair suds-jurko retrying quandl yapf \
	jupyter_contrib_nbextensions pandas-datareader missingno tqdm cvxopt fredapi \
	mysqlclient mibian bokeh zeep pytrends adjustText xlsxwriter img2pdf 

RUN pip install --upgrade --user pandas numpy matplotlib

RUN jupyter labextension install @jupyterlab/toc

# Patch of Suds 
COPY client.py /opt/conda/lib/python3.6/site-packages/suds/

# Final setup
RUN chown -R 1000:100 /home/$NB_USER/.local
RUN chown -R 1000:100 /opt/conda/share/jupyter/lab
USER $NB_USER

I could remove the installation of @jupyterlab/toc eventually
This is the only thing that have some relation with jupyterlab extension.

[ian-r-rose]

Do you find the same problem outside of the proxied environment?

[franckjulliard]

I have no place to run it with direct access to internet unfortunately.
Do you think it might be related to that?
Some proxy setting of node.js to set-up?

[franckjulliard]

I made it work!
it was related to npm.
I had to add a .npmrc file with the proxy configuration and also change some access right to some folder to be able to rebuild jupyterlab
/opt/conda/share/jupyter/lab
and download npm date
home/.npm

anyway thank you for showing me the right direction to investigate.