Jupyter Lab Has (Almost) Blank Login Screen

[TheCatster]

What docker image you are using?

Example: jupyter/datascience-notebook

What complete docker command do you run to launch the container (omitting sensitive values)?

version: "3"
services:
    jupyterlab:
    image: jupyter/datascience-notebook
    restart: always
    ports:
        - 8888:8888
    volumes:
        - ./jupyter-data:/home/jovyan
        - ./notebooks:/home/jovyan/notebooks
    environment:
        - NB_UID=1000
        - NB_GUID=1000
        - JUPYTER_ENABLE_LAB=yes
        - RESTARTABLE=yes

What steps do you take once the container is running to reproduce the issue?

Example:

1. Visit https://jupyter.domain.com

What do you expect to happen?

Expect to see a login screen, as usual.

What actually happens?

This rarely works, for no apparent reason, when I remove the JUPYTER_ENABLE_LAB variable, but usually results in a blank screen with a message and textbox to "Enter password or token:". My normal password does not work. I cannot do anything else. Once I remove the variable, it goes back to being a classic notebook, but it does work. This also does not work through an SSH tunnel, so I do not believe it has anything to do with my reverse proxy setup.

[romainx]

Hello,

I've answered to an issue today #1247 that reported the same behavior.

For the blank notebook I don't know, maybe you can check if you have a config file containing a not compliant configuration that has persisted in your home or something in your browser cache (try to clear the cache)?

Apparently it was solved by the user that made the following feedback.

Yes, there was an incompatible file in the .jupyter/lab directory. When I wipe that directory the Lab loads fine. Much appreciated.

Could you check on your side if the problem is the same?

Thanks.
Best.

[TheCatster]

Clearing the .jupyter/lab directory did not fix it. It's not browser cache, since it's multiple devices experiencing it, and I don't believe it has anything to do with my notebooks, as they haven't been touched in about a month and worked for months before that (I got busy with work and didn't have the time to continue working on my projects). The only thing that could've changed in that time was the image itself, since I use watchtower on that server. Was something deprecated in the past few months that I'm unaware of? I know of the deprecation of classic notebook itself, but that shouldn't break the Lab.

[romainx]

@TheCatster the last major change was switching to JupyterLab v3.0.x, you can check with a prior tagged version jupyter/datascience-notebook:lab-2.2.9.
You can also try to not mount the volumes you are using because they may be related to this issue.
I do not reproduce it with a fresh setup.
Best.

[TheCatster]

I'll try now without mounting, that's a good idea.

[TheCatster]

Nothing mounted, no special env vars except JUPYTER_ENABLE_LAB. Not behind a reverse proxy, and this still happens. I'll try the 2.2.9 image now then.

[romainx]

It looks like a know issue, could you check #1035 and #1057?

[romainx]

Hello @TheCatster, any update on your side.

[TheCatster]

Sorry about that, haven't had a moment. I run this on a remote machine without any of the scenarios described, except for in that month or so I didn't use the Jupyter Lab I added a code-server container. I'm wondering if the Python plug-in there is interfering. I will setup a separate VM to test out just Jupyter Lab.

[romainx]

We have received several issues of users reporting token problems and most of them were related to VS Code or (code-server) running a Jupyter on the same port. So it may be the same cause here.

[TheCatster]

I'm quite confused. I did as I said, rid the code-server containers, and moved Jupyter Lab to a lone VM. Still, I have the same screen.

[TheCatster]

To add, which is quite interesting, if I include the token in the URL there is no issue, and the proper Jupyter Lab loads. It seems to be only the login page.

[Bidek56]

I use jupyter/pyspark-notebook on a AKS cluster and https://jupyter.domain.com works all the time.
The issue maybe in your /home/{{ .Values.username }}/.jupyter/jupyter_lab_config.py
I use the following values if it helps:

# ------------------------------------------------------------------------------
# NotebookApp configuration
# ------------------------------------------------------------------------------
c.IPKernelApp.pylab = 'inline'
c.ServerApp.ip = '*'
c.ServerApp.port = 8888
c.ServerApp.open_browser = False
c.ServerApp.base_url = '/'
c.ServerApp.trust_xheaders = True
c.ServerApp.tornado_settings = {'static_url_prefix': '/static/'}
c.ServerApp.root_dir = '/home/notebook'
c.ServerApp.allow_origin = '*'
c.ServerApp.allow_remote_access = True
c.ServerApp.token = ''

[TheCatster]

I had no such file when I first looked, and after adding one and what you had (editing the root dir), it seems that the file is not being read. Why would that be?

Edit: This is bizarre. The styling is still completely gone, but when I wiped the lab directory, added your config, and restarted the container then the style is still gone, but my password works and it loads correctly.

[Bidek56]

You styling is gone because of the c.ServerApp.tornado_settings = {'static_url_prefix': '/static/'} setting, please check your browser console for any errors.

[TheCatster]

There are no console errors. Is there a way to see where the static files are actually kept?

[Bidek56]

There seems to be a bug related to c.ServerApp.base_url which maybe causing your issue, can you try removing it from the config?

[TheCatster]

No change from that.

[IrtazaHussain1]

I have customized start-notebook.sh file and created the new image from jupyter/datascience-notebook:67b8fb91f950
bellow is updated code

set -e

export JUPYTER_PASSWORD
export JUPYTER_BASE_URL


wrapper=""
if [[ "${RESTARTABLE}" == "yes" ]]; then
    wrapper="run-one-constantly"
fi

# set jupyter password, if provided
if [ ! -z "$JUPYTER_PASSWORD" ]
then
  # hash password string
  password_cmd="from notebook.auth import passwd;print(passwd('${JUPYTER_PASSWORD}'))"
  password=$( python -c "${password_cmd}" )
  notebook_password="--NotebookApp.password='${password}'"
else
  notebook_password=""
fi

if [[ ! -z "${JUPYTERHUB_API_TOKEN}" ]]; then
    # launched by JupyterHub, use single-user entrypoint
    exec /usr/local/bin/start-singleuser.sh "$@" "$notebook_password"
elif [[ ! -z "${JUPYTER_ENABLE_LAB}" ]]; then
    . /usr/local/bin/start.sh $wrapper jupyter lab "$@" "$notebook_password" --NotebookApp.base_url=$JUPYTER_BASE_URL
else
    echo "WARN: Jupyter Notebook deprecation notice https://github.com/jupyter/docker-stacks#jupyter-notebook-deprecation-notice."
    . /usr/local/bin/start.sh $wrapper jupyter notebook "$@" "$notebook_password" --NotebookApp.base_url=$JUPYTER_BASE_URL
fi

Issue i am facing is when I enable lab using environment variable JUPYTER_ENABLE_LAB in start-notebook.sh file suddenly all the styling of login page disappear while if I make JUPYTER_ENABLE_LAB empty string it start working perfectly. means if jupyter run command with lab it's styling is not working for login page only but if it run using notebook command it's start working perfectly.

[mathbunnyru]

@TheCatster I tried to run your docker-compose file on a local docker and everything works just fine.

Could you please, try to run it on a local machine as well?
And without your mounts - there might be a problem with them.

After that try to add your mounts - if everything works well, it might be a network setup issue.
If not, it means that something is wrong in the folder(s) you mount:

It might me .conda/, .local, .jupyter/ and many other things, I suppose.
But you will know that something is odd with your files.

[TheCatster]

I've tried without mounts a few times earlier, so I'll try on a local machine. I wouldn't be surprised if it's something on the network side at this point. I'm waiting for a new router to come in, so my homelab isn't currently running.

[TheCatster]

Now I'm even more confused. Same blank screen on my main workstation, nothing mounted. Only the docker compose that I pasted in the original issue.

[mathbunnyru]

Ok, at least this gives much more understanding.
Could you confirm then that it is working on another machine you have?

Also, what OS are you using / browser?
I was building everything under macOS and Safari.

[mathbunnyru]

And I would like to ask you to try visiting jupyter in a freshly installed browser - you could use Vivaldi for example.

[TheCatster]

Could you confirm then that it is working on another machine you have?

I can't at this moment, but I'll try it on another few laptops. So far it does not work on my main workstation, running Arch Linux, my Hackintosh, running Big Sur, my main laptop running Arch, and another laptop running Void.

Also, what OS are you using / browser?

Arch, macOS Big Sur, Void, and I've tried accessing it from my iPhone as well on Safari. Using Safari on Big Sur/iOS, and Firefox on all others. I've tried using my default profile (which is an arkenfox based privacy setup), or my blank profile. The homelab where I normally run Jupyter Lab is running Proxmox, and the Jupyter VM is just a simple Ubuntu Server 20.10.

And I would like to ask you to try visiting jupyter in a freshly installed browser - you could use Vivaldi for example.

I can try this just to rule out the browser as best as possible.

[TheCatster]

Brand new install of Vivaldi, I've never used it before. No difference

[mathbunnyru]

That's so unfortunate. I have no idea how the same setup doesn't work on all your machines but works on the one I have.

Please, could you explain step-by-step, how do you launch your docker-compose, which website you visit (http://127.0.0.1:8888/?token=TOKEN_HERE) and versions of docker-compose and docker (please, try updating them).

[mathbunnyru]

Sorry for such stupid requests, but I don't see another way of debugging this issue.

[TheCatster]

No no, not stupid at all, I completely understand. I always run the latest versions, being on a rolling release distro, so my docker version is 20.10.5, build 55c4c88966 and my docker-compose is 1.28.5.

1. I make a directory called jupyterlab. On my workstation, where I'm currently testing, it's located at ~/Programming/Docker/jupyterlab.
2. Within jupyterlab, I make a file called docker-compose.yml and copy and paste in the default configuration (doesn't matter, if I use mine above it also doesn't work)
3. I run docker-compose up to test. Otherwise, I run it with docker-compose up -d.
4. I get a bunch of the normal output, and a message saying go to http://127.0.0.1:8888/?token=TOKEN_HERE. On my homelab, I use nginx as a reverse proxy, so I go to jupyter.domain.com, but on my workstation I am going to the given address.
5. Once there, Jupyter Lab works perfectly, no complaints.
6. If I log out, I get the black and white screen above. Trying to make a password doesn't work, entering the token doesn't work. Only if I pass the token as a parameter in the URL.

[mathbunnyru]

First of all, I have the exact same versions as you do (on my mac).

6. If I log out, I get the black and white screen above. Trying to make a password doesn't work, entering the token doesn't work. Only if I pass the token as a parameter in the URL.

Wow, this is something new to me. I didn't log out, so the issue you're running into is that you can't log in after logout.

Still, I tried your instruction and get no errors.
I did run with docker-compose up to see the logs and here is what I see (302 GET is a successful login).

Do you see something odd in your logs? Could you please share the same logs as me?

[TheCatster]

Following are my logs, with nothing particularly strange to me except for that error with tornado.

[mathbunnyru]

Oh, the image I'm using seems to be a bit old. I will make docker pull jupyter/datascience-notebook and retry your steps.

[mathbunnyru]

I updated my image and now my logs look like yours, but without an error. I think the tornado error is causing the issue you see.

Please, run docker pull jupyter/datascience-notebook, this might easily resolve the problem

[mathbunnyru]

This is what I see now - note, no error.

[mathbunnyru]

And, I hope, you use the jupyter/datascience-notebook from dockerhub, not your (custom) build of jupyter/datascience-notebook, because the images might be different.

[TheCatster]

Edit: Yup, says up to date. I also do use watchtower on my homelab, so it should always be up to date.

Yeah, I use it from docker hub, and I just pulled mine when you asked me to try on local. But, I'll try again.

[mathbunnyru]

I took a careful look at your logs.
Your log misses one line Writing notebook server cookie secret.
You might be still attaching your volumes, because with volumes attached I don't see this line as well.
Please, take a more careful at your docker-compose.yaml.

And let's try to use docker-compose up with --force-recreate option.

[romainx]

@mathbunnyru @Bidek56 thank you for all investigations made.
@TheCatster Based on these discussions, we are most definitely on an issue specific to your setup. I'm not sure we should spend more time on specific cases like this that only affect a specific setup. So if you don't mind, we will close this issue.

Thank you for your understanding.

[mathbunnyru]

Let me first explain the main idea on how to debug these things.
General idea is that you need to make the setup as simple as possible.
It will most likely be repeatable by other people.
And the more simple thing you have, the more simple it will be to debug.

Things we've excluded so far:

• network (tried on a local machine)
• volumes attached (but as I said, I'm not sure that this is true)
• other stuff related to os / browser / versions

I would like to suggest you try to use the raw docker command because this will remove docker-compose as a possible problem.
Also, please, try to think about how your setup differs from the most simple one.

I think you will be able to resolve this issue by yourself after these steps.

[jeffyjefflabs]

I found my way here after also noticing the lack of styling on the login page. I can't reproduce @TheCatster's token problem either, but just for reference, the lack of styling is easily reproducible with docker run -e JUPYTER_ENABLE_LAB=yes -p8888:8888 jupyter/base-notebook:latest and then going to http://localhost:8888 without including the token in the URL. I believe that's due to jupyter-server/jupyter_server#271 (since Lab 3.x runs on the jupyter_server back-end now) and not anything specific to the docker-stacks image.

[TheCatster]

I finished setting up my new network, and I no longer have the token issue. I do still have no styling, but if what @jeffyjefflabs just linked is the answer, then I can live with no styling. I think that solves it for me! I honestly did nothing different, at all. The network is set up almost identically to the previous one (just a new router, using FreshTomato firmware, and a new managed switch). It is the same homelab, same everything else.

[mathbunnyru]

Great, I'll close this issue as it seems to be resolved.