Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

jupyterhub 4.0.1 doesn't load jupyterlab automatically if Spawner.default_url is "/lab" #4510

Closed
Frank-Steiner opened this issue Jul 14, 2023 · 7 comments
Labels

Comments

@Frank-Steiner
Copy link

Bug description

After upgrading from jupyerhub 3 to 4.0.1 using the same config as before, any user logging in or starting a server from hub home ends up with 404 : Not Found. You are requesting a page that does not exist!

Expected behaviour

jupyterlab should open.

Actual behaviour

I get the 404 error page.

How to reproduce

No idea... I also uninstalled jhub compeletely, removed the sqlite db, and installed 4.0.1 from fresh, and received the same problem. Reverting back to 3.1.1 works find, the jupyterlab page is found again, going to 4.0.1 again shows the 404 page. But I guess that's not happening for everyone else :-)

Your personal set up

  • OS: SuSE Linux Enterprise 15 SP4

  • Version(s): jupyterhub 4.0.1, python 3.10

Full environment
aiohttp==3.8.4
aiosignal==1.3.1
alembic==1.11.1
ansi2html==1.8.0
anyio==3.7.1
argon2-cffi==21.3.0
argon2-cffi-bindings==21.2.0
arrow==1.2.3
asttokens==2.2.1
async-generator==1.10
async-lru==2.0.3
async-timeout==4.0.2
attrs==23.1.0
Babel==2.12.1
backcall==0.2.0
beautifulsoup4==4.12.2
bleach==6.0.0
certifi==2023.5.7
certipy==0.1.3
cffi==1.15.1
charset-normalizer==3.2.0
click==8.1.5
comm==0.1.3
cryptography==41.0.2
dash==2.11.1
dash-core-components==2.0.0
dash-html-components==2.0.0
dash-table==5.0.0
debugpy==1.6.7
decorator==5.1.1
defusedxml==0.7.1
exceptiongroup==1.1.2
executing==1.2.0
fastjsonschema==2.17.1
Flask==2.2.5
fqdn==1.5.1
frozenlist==1.4.0
greenlet==2.0.2
idna==3.4
ipykernel==6.24.0
ipython==8.14.0
ipython-genutils==0.2.0
ipywidgets==8.0.7
isoduration==20.11.0
itsdangerous==2.1.2
jedi==0.18.2
Jinja2==3.1.2
json5==0.9.14
jsonpointer==2.4
jsonschema==4.18.3
jsonschema-specifications==2023.6.1
jupyter-dash==0.4.2
jupyter-events==0.6.3
jupyter-lsp==2.2.0
jupyter-telemetry==0.1.0
jupyter_client==8.3.0
jupyter_core==5.3.1
jupyter_server==2.7.0
jupyter_server_proxy==4.0.0
jupyter_server_terminals==0.4.4
jupyterhub==4.0.1
jupyterlab==4.0.3
jupyterlab-dash==0.1.0a3
jupyterlab-pygments==0.2.2
jupyterlab-widgets==3.0.8
jupyterlab_server==2.23.0
jupytext==1.14.7
Mako==1.2.4
markdown-it-py==3.0.0
MarkupSafe==2.1.3
matplotlib-inline==0.1.6
mdit-py-plugins==0.4.0
mdurl==0.1.2
mistune==3.0.1
multidict==6.0.4
mysql-connector==2.2.9
nbclassic==1.0.0
nbclient==0.8.0
nbconvert==7.6.0
nbformat==5.9.1
nest-asyncio==1.5.6
notebook==6.5.4
notebook_shim==0.2.3
numpy==1.24.2
oauthlib==3.2.2
overrides==7.3.1
packaging==23.1
pamela==1.1.0
pandocfilters==1.5.0
parso==0.8.3
pexpect==4.8.0
pickleshare==0.7.5
platformdirs==3.8.1
plotly==5.15.0
prometheus-client==0.17.1
prompt-toolkit==3.0.39
psutil==5.9.5
ptyprocess==0.7.0
pure-eval==0.2.2
pycparser==2.21
Pygments==2.15.1
pyOpenSSL==23.2.0
python-dateutil==2.8.2
python-json-logger==2.0.7
PyYAML==6.0
pyzmq==25.1.0
referencing==0.29.1
requests==2.31.0
retrying==1.3.4
rfc3339-validator==0.1.4
rfc3986-validator==0.1.1
rpds-py==0.8.10
ruamel.yaml==0.17.32
ruamel.yaml.clib==0.2.7
Send2Trash==1.8.2
simpervisor==1.0.0
six==1.16.0
sniffio==1.3.0
soupsieve==2.4.1
SQLAlchemy==2.0.18
stack-data==0.6.2
sudospawner==0.5.2
tenacity==8.2.2
terminado==0.17.1
tinycss2==1.2.1
toml==0.10.2
tomli==2.0.1
tornado==6.3.2
traitlets==5.9.0
typing_extensions==4.7.1
uri-template==1.3.0
urllib3==2.0.3
wcwidth==0.2.6
webcolors==1.13
webencodings==0.5.1
websocket-client==1.6.1
Werkzeug==2.2.3
widgetsnbextension==4.0.8
yarl==1.9.2

Configuration
# jupyterhub_config.py
c = get_config()  #noqa
c.Spawner.default_url = '/lab'
c.JupyterHub.hub_port=8792
c.JupyterHub.cleanup_servers = False
c.ConfigurableHTTPProxy.should_start = False
c.JupyterHub.cookie_secret_file = '/etc/jhub/jupyterhub_cookie_secret'
c.ConfigurableHTTPProxy.api_url = 'http://localhost:8791'
c.Authenticator.delete_invalid_users = True
c.JupyterHub.pid_file = '/run/jhub/jhub.pid'
c.Spawner.cmd = '/opt/jhub/bin/jupyterhub-singleuser'
c.ServerApp.shutdown_no_activity_timeout = 168 * 60 * 60
c.MappingKernelManager.cull_idle_timeout = 168 * 60 * 60
c.MappingKernelManager.cull_interval = 60 * 60
c.JupyterHub.allow_named_servers = True
c.JupyterHub.named_server_limit_per_user = 3
c.JupyterHub.log_level = 'DEBUG'
c.ServerApp.tornado_settings = {
    "max_buffer_size": 1024*1024*1024,
}
c.Authenticator.admin_users= {'fst'}
c.JupyterHub.db_url = '/var/lib/jhub/jupyterhub.sqlite'
Logs That's the full set of journal log messages from starting jupyterhub, andlogging in until receiving the 404.
Jul 14 16:16:00 abel configurable-http-proxy[20284]: 16:16:00.661 [ConfigProxy] info: Adding route / -> http://localhost:8792
Jul 14 16:16:00 abel configurable-http-proxy[20284]: 16:16:00.667 [ConfigProxy] info: Proxying http://*:8790 to http://localhost:8792
Jul 14 16:16:00 abel configurable-http-proxy[20284]: 16:16:00.667 [ConfigProxy] info: Proxy API at http://127.0.0.1:8791/api/routes
Jul 14 16:16:00 abel configurable-http-proxy[20284]: 16:16:00.667 [ConfigProxy] info: Writing pid 20284 to /run/chp/chp.pid
Jul 14 16:16:00 abel configurable-http-proxy[20284]: 16:16:00.669 [ConfigProxy] info: Route added / -> http://localhost:8792
Jul 14 16:16:01 abel start_jhub[20285]: [I 2023-07-14 16:16:01.213 JupyterHub app:2859] Running JupyterHub version 4.0.1
Jul 14 16:16:01 abel start_jhub[20285]: [D 2023-07-14 16:16:01.214 JupyterHub app:2834] Writing PID 20285 to /run/jhub/jhub.pid
Jul 14 16:16:01 abel start_jhub[20285]: [I 2023-07-14 16:16:01.214 JupyterHub app:2889] Using Authenticator: jupyterhub.auth.PAMAuthenticator-4.0.1
Jul 14 16:16:01 abel start_jhub[20285]: [I 2023-07-14 16:16:01.214 JupyterHub app:2889] Using Spawner: jupyterhub.spawner.LocalProcessSpawner-4.0.1
Jul 14 16:16:01 abel start_jhub[20285]: [I 2023-07-14 16:16:01.214 JupyterHub app:2889] Using Proxy: jupyterhub.proxy.ConfigurableHTTPProxy-4.0.1
Jul 14 16:16:01 abel start_jhub[20285]: [D 2023-07-14 16:16:01.219 JupyterHub app:2816] Could not load pycurl: No module named 'pycurl'
Jul 14 16:16:01 abel start_jhub[20285]:     pycurl is recommended if you have a large number of users.
Jul 14 16:16:01 abel start_jhub[20285]: [I 2023-07-14 16:16:01.219 JupyterHub app:1664] Loading cookie_secret from /etc/jhub/jupyterhub_cookie_secret
Jul 14 16:16:01 abel start_jhub[20285]: [D 2023-07-14 16:16:01.219 JupyterHub app:1833] Connecting to db: sqlite:////var/lib/jhub/jupyterhub.sqlite
Jul 14 16:16:01 abel start_jhub[20285]: [D 2023-07-14 16:16:01.267 JupyterHub orm:1016] database schema version found: 0eee8c825d24
Jul 14 16:16:01 abel start_jhub[20285]: [D 2023-07-14 16:16:01.309 JupyterHub app:2104] Loading roles into database
Jul 14 16:16:01 abel start_jhub[20285]: [I 2023-07-14 16:16:01.320 JupyterHub app:1984] Not using allowed_users. Any authenticated user will be allowed.
Jul 14 16:16:01 abel start_jhub[20285]: [D 2023-07-14 16:16:01.324 JupyterHub app:2343] Purging expired APITokens
Jul 14 16:16:01 abel start_jhub[20285]: [D 2023-07-14 16:16:01.327 JupyterHub app:2343] Purging expired OAuthCodes
Jul 14 16:16:01 abel start_jhub[20285]: [D 2023-07-14 16:16:01.329 JupyterHub app:2179] Loading role assignments from config
Jul 14 16:16:01 abel start_jhub[20285]: [D 2023-07-14 16:16:01.340 JupyterHub app:2502] Initializing spawners
Jul 14 16:16:01 abel start_jhub[20285]: [D 2023-07-14 16:16:01.342 JupyterHub app:2633] Loaded users:
Jul 14 16:16:01 abel start_jhub[20285]:     
Jul 14 16:16:01 abel start_jhub[20285]: [I 2023-07-14 16:16:01.342 JupyterHub app:2928] Initialized 0 spawners in 0.002 seconds
Jul 14 16:16:01 abel start_jhub[20285]: [I 2023-07-14 16:16:01.347 JupyterHub metrics:278] Found 0 active users in the last ActiveUserPeriods.twenty_four_hours
Jul 14 16:16:01 abel start_jhub[20285]: [I 2023-07-14 16:16:01.348 JupyterHub metrics:278] Found 0 active users in the last ActiveUserPeriods.seven_days
Jul 14 16:16:01 abel start_jhub[20285]: [I 2023-07-14 16:16:01.348 JupyterHub metrics:278] Found 0 active users in the last ActiveUserPeriods.thirty_days
Jul 14 16:16:01 abel start_jhub[20285]: [I 2023-07-14 16:16:01.349 JupyterHub app:3142] Not starting proxy
Jul 14 16:16:01 abel start_jhub[20285]: [D 2023-07-14 16:16:01.350 JupyterHub proxy:880] Proxy: Fetching GET http://localhost:8791/api/routes
Jul 14 16:16:01 abel start_jhub[20285]: [D 2023-07-14 16:16:01.358 JupyterHub proxy:953] Omitting non-jupyterhub route '/'
Jul 14 16:16:01 abel start_jhub[20285]: [I 2023-07-14 16:16:01.359 JupyterHub app:3178] Hub API listening on http://127.0.0.1:8792/hub/
Jul 14 16:16:01 abel configurable-http-proxy[20284]: 16:16:01.359 [ConfigProxy] info: 200 GET /api/routes
Jul 14 16:16:01 abel start_jhub[20285]: [D 2023-07-14 16:16:01.359 JupyterHub proxy:389] Fetching routes to check
Jul 14 16:16:01 abel start_jhub[20285]: [D 2023-07-14 16:16:01.359 JupyterHub proxy:880] Proxy: Fetching GET http://localhost:8791/api/routes
Jul 14 16:16:01 abel configurable-http-proxy[20284]: 16:16:01.361 [ConfigProxy] info: 200 GET /api/routes
Jul 14 16:16:01 abel start_jhub[20285]: [D 2023-07-14 16:16:01.361 JupyterHub proxy:953] Omitting non-jupyterhub route '/'
Jul 14 16:16:01 abel start_jhub[20285]: [D 2023-07-14 16:16:01.361 JupyterHub proxy:392] Checking routes
Jul 14 16:16:01 abel start_jhub[20285]: [I 2023-07-14 16:16:01.361 JupyterHub proxy:477] Adding route for Hub: / => http://127.0.0.1:8792
Jul 14 16:16:01 abel start_jhub[20285]: [D 2023-07-14 16:16:01.361 JupyterHub proxy:880] Proxy: Fetching POST http://localhost:8791/api/routes/
Jul 14 16:16:01 abel configurable-http-proxy[20284]: 16:16:01.363 [ConfigProxy] info: Adding route / -> http://127.0.0.1:8792
Jul 14 16:16:01 abel configurable-http-proxy[20284]: 16:16:01.363 [ConfigProxy] info: Route added / -> http://127.0.0.1:8792
Jul 14 16:16:01 abel configurable-http-proxy[20284]: 16:16:01.364 [ConfigProxy] info: 201 POST /api/routes/
Jul 14 16:16:01 abel start_jhub[20285]: [I 2023-07-14 16:16:01.364 JupyterHub app:3247] JupyterHub is now running, internal Hub API at http://127.0.0.1:8792/hub/
Jul 14 16:16:01 abel start_jhub[20285]: [D 2023-07-14 16:16:01.364 JupyterHub app:2852] It took 0.156 seconds for the Hub to start
Jul 14 16:16:05 abel configurable-http-proxy[20284]: 16:16:05.865 [ConfigProxy] debug: PROXY WEB /hub/static/components/font-awesome/fonts/fontawesome-webfont.woff2 to http://127.0.0.1:8792
Jul 14 16:16:05 abel configurable-http-proxy[20284]: 16:16:05.871 [ConfigProxy] debug: PROXY WEB /hub/login to http://127.0.0.1:8792
Jul 14 16:16:05 abel start_jhub[20285]: [D 2023-07-14 16:16:05.881 JupyterHub log:191] 200 GET /hub/static/components/font-awesome/fonts/fontawesome-webfont.woff2?v=4.7.0 (@<ip>) 6.84ms
Jul 14 16:16:05 abel start_jhub[20285]: [D 2023-07-14 16:16:05.899 JupyterHub roles:281] Assigning default role to User fst
Jul 14 16:16:05 abel start_jhub[20285]: [D 2023-07-14 16:16:05.902 JupyterHub base:589] Setting cookie for fst: jupyterhub-hub-login
Jul 14 16:16:05 abel start_jhub[20285]: [D 2023-07-14 16:16:05.902 JupyterHub base:585] Setting cookie jupyterhub-hub-login: {'httponly': True, 'path': '/hub/'}
Jul 14 16:16:05 abel start_jhub[20285]: [I 2023-07-14 16:16:05.902 JupyterHub base:835] User logged in: fst
Jul 14 16:16:05 abel start_jhub[20285]: [D 2023-07-14 16:16:05.902 JupyterHub user:431] Creating <class 'jupyterhub.spawner.LocalProcessSpawner'> for fst:
Jul 14 16:16:05 abel start_jhub[20285]: [I 2023-07-14 16:16:05.903 JupyterHub log:191] 302 POST /hub/login?next= -> /hub/spawn (fst@@<ip>) 21.87ms
Jul 14 16:16:05 abel configurable-http-proxy[20284]: 16:16:05.903 [ConfigProxy] debug: Not recording activity for status 302 on /
Jul 14 16:16:05 abel configurable-http-proxy[20284]: 16:16:05.951 [ConfigProxy] debug: PROXY WEB /hub/spawn to http://127.0.0.1:8792
Jul 14 16:16:05 abel start_jhub[20285]: [D 2023-07-14 16:16:05.953 JupyterHub base:297] Recording first activity for <User(fst 0/1 running)>
Jul 14 16:16:05 abel start_jhub[20285]: [D 2023-07-14 16:16:05.960 JupyterHub scopes:863] Checking access via scope servers
Jul 14 16:16:05 abel start_jhub[20285]: [D 2023-07-14 16:16:05.960 JupyterHub scopes:677] Unrestricted access to /hub/spawn via servers
Jul 14 16:16:05 abel start_jhub[20285]: [D 2023-07-14 16:16:05.960 JupyterHub pages:217] Triggering spawn with default options for fst
Jul 14 16:16:05 abel start_jhub[20285]: [D 2023-07-14 16:16:05.960 JupyterHub base:959] Initiating spawn for fst
Jul 14 16:16:05 abel start_jhub[20285]: [D 2023-07-14 16:16:05.960 JupyterHub base:963] 0/100 concurrent spawns
Jul 14 16:16:05 abel start_jhub[20285]: [D 2023-07-14 16:16:05.960 JupyterHub base:968] 0 active servers
Jul 14 16:16:05 abel start_jhub[20285]: [I 2023-07-14 16:16:05.975 JupyterHub provider:659] Creating oauth client jupyterhub-user-fst
Jul 14 16:16:05 abel start_jhub[20285]: [D 2023-07-14 16:16:05.985 JupyterHub user:794] Calling Spawner.start for fst
Jul 14 16:16:05 abel start_jhub[20285]: [I 2023-07-14 16:16:05.986 JupyterHub spawner:1689] Spawning /opt/jhub/bin/jupyterhub-singleuser
Jul 14 16:16:05 abel start_jhub[20285]: [D 2023-07-14 16:16:05.996 JupyterHub spawner:1384] Polling subprocess every 30s
Jul 14 16:16:06 abel start_jhub[20298]: [I 2023-07-14 16:16:06.786 ServerApp] Package jupyterhub took 0.0000s to import
Jul 14 16:16:06 abel start_jhub[20298]: [I 2023-07-14 16:16:06.846 ServerApp] Package jupytext took 0.0596s to import
Jul 14 16:16:06 abel start_jhub[20298]: [I 2023-07-14 16:16:06.847 JupyterHubSingleUser] Starting jupyterhub single-user server extension version 4.0.1
Jul 14 16:16:06 abel start_jhub[20298]: [I 2023-07-14 16:16:06.847 JupyterHubSingleUser] Using default url from environment $JUPYTERHUB_DEFAULT_URL: /lab
Jul 14 16:16:06 abel start_jhub[20298]: [I 2023-07-14 16:16:06.851 ServerApp] jupyterhub | extension was successfully linked.
Jul 14 16:16:06 abel start_jhub[20298]: [I 2023-07-14 16:16:06.851 ServerApp] jupytext | extension was successfully linked.
Jul 14 16:16:06 abel start_jhub[20285]: [I 2023-07-14 16:16:06.917 JupyterHub log:191] 200 GET /hub/api (@127.0.0.1) 1.01ms
Jul 14 16:16:06 abel start_jhub[20298]: [I 2023-07-14 16:16:06.917 JupyterHubSingleUser] Updating Hub with activity every 300 seconds
Jul 14 16:16:06 abel start_jhub[20298]: [I 2023-07-14 16:16:06.917 ServerApp] jupyterhub | extension was successfully loaded.
Jul 14 16:16:06 abel start_jhub[20298]: [W 2023-07-14 16:16:06.917 ServerApp] [Jupytext Server Extension] Async contents managers like AsyncLargeFileManager are not supported at the moment (https://github.com/mwouts/jupytext/issues/1020). We will derive a contents manager from LargeFileManager instead.
Jul 14 16:16:06 abel start_jhub[20298]: [I 2023-07-14 16:16:06.917 ServerApp] [Jupytext Server Extension] Deriving a JupytextContentsManager from LargeFileManager
Jul 14 16:16:06 abel start_jhub[20298]: [I 2023-07-14 16:16:06.919 ServerApp] jupytext | extension was successfully loaded.
Jul 14 16:16:06 abel start_jhub[20298]: [I 2023-07-14 16:16:06.925 ServerApp] Serving notebooks from local directory: /home/users/fst
Jul 14 16:16:06 abel start_jhub[20298]: [I 2023-07-14 16:16:06.925 ServerApp] Jupyter Server 2.7.0 is running at:
Jul 14 16:16:06 abel start_jhub[20298]: [I 2023-07-14 16:16:06.925 ServerApp] http://127.0.0.1:45197/user/fst/lab?token=...
Jul 14 16:16:06 abel start_jhub[20298]: [I 2023-07-14 16:16:06.925 ServerApp]     http://127.0.0.1:45197/user/fst/lab?token=...
Jul 14 16:16:06 abel start_jhub[20298]: [I 2023-07-14 16:16:06.925 ServerApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).
Jul 14 16:16:06 abel start_jhub[20285]: [D 2023-07-14 16:16:06.934 JupyterHub base:297] Recording first activity for <APIToken('789e...', user='fst', client_id='jupyterhub')>
Jul 14 16:16:06 abel start_jhub[20285]: [D 2023-07-14 16:16:06.939 JupyterHub scopes:863] Checking access via scope users:activity
Jul 14 16:16:06 abel start_jhub[20285]: [D 2023-07-14 16:16:06.939 JupyterHub scopes:690] Argument-based access to /hub/api/users/fst/activity via users:activity
Jul 14 16:16:06 abel start_jhub[20285]: [D 2023-07-14 16:16:06.940 JupyterHub users:874] Activity for user fst: 2023-07-14T14:16:06.903366Z
Jul 14 16:16:06 abel start_jhub[20285]: [D 2023-07-14 16:16:06.940 JupyterHub users:892] Activity on server fst/: 2023-07-14T14:16:06.903366Z
Jul 14 16:16:06 abel start_jhub[20285]: [I 2023-07-14 16:16:06.943 JupyterHub log:191] 200 POST /hub/api/users/fst/activity (fst@127.0.0.1) 12.14ms
Jul 14 16:16:06 abel start_jhub[20285]: [I 2023-07-14 16:16:06.961 JupyterHub log:191] 302 GET /hub/spawn -> /hub/spawn-pending/fst (fst@<ip>) 1009.31ms
Jul 14 16:16:06 abel configurable-http-proxy[20284]: 16:16:06.961 [ConfigProxy] debug: Not recording activity for status 302 on /
Jul 14 16:16:07 abel configurable-http-proxy[20284]: 16:16:07.017 [ConfigProxy] debug: PROXY WEB /hub/spawn-pending/fst to http://127.0.0.1:8792
Jul 14 16:16:07 abel start_jhub[20285]: [D 2023-07-14 16:16:07.020 JupyterHub scopes:863] Checking access via scope servers
Jul 14 16:16:07 abel start_jhub[20285]: [D 2023-07-14 16:16:07.020 JupyterHub scopes:677] Unrestricted access to /hub/spawn-pending/fst via servers
Jul 14 16:16:07 abel start_jhub[20285]: [I 2023-07-14 16:16:07.020 JupyterHub pages:398] fst is pending spawn
Jul 14 16:16:07 abel start_jhub[20285]: [I 2023-07-14 16:16:07.044 JupyterHub log:191] 200 GET /hub/spawn-pending/fst (fst@<ip>) 25.26ms
Jul 14 16:16:07 abel configurable-http-proxy[20284]: 16:16:07.151 [ConfigProxy] debug: PROXY WEB /hub/static/components/requirejs/require.js to http://127.0.0.1:8792
Jul 14 16:16:07 abel configurable-http-proxy[20284]: 16:16:07.153 [ConfigProxy] debug: PROXY WEB /hub/static/components/jquery/dist/jquery.min.js to http://127.0.0.1:8792
Jul 14 16:16:07 abel configurable-http-proxy[20284]: 16:16:07.153 [ConfigProxy] debug: PROXY WEB /hub/static/components/bootstrap/dist/js/bootstrap.min.js to http://127.0.0.1:8792
Jul 14 16:16:07 abel start_jhub[20285]: [D 2023-07-14 16:16:07.155 JupyterHub log:191] 200 GET /hub/static/components/requirejs/require.js?v=bd1aa102bdb0b27fbf712b32cfcd29b016c272acf3d864ee8469376eaddd032cadcf827ff17c05a8c8e20061418fe58cf79947049f5c0dff3b4f73fcc8cad8ec (@<ip>) 0.72ms
Jul 14 16:16:07 abel start_jhub[20285]: [D 2023-07-14 16:16:07.156 JupyterHub log:191] 200 GET /hub/static/components/jquery/dist/jquery.min.js?v=de027062931edd07b01842eff24fc15fdbdcaa1af245dcd133155faba9e0c965f0a34dc6144ce3b149bc43b4597073c792cb6dabbfc6168c63095523923bcf77 (@<ip>) 0.62ms
Jul 14 16:16:07 abel start_jhub[20285]: [D 2023-07-14 16:16:07.157 JupyterHub log:191] 200 GET /hub/static/components/bootstrap/dist/js/bootstrap.min.js?v=a014e9acc78d10a0a7a9fbaa29deac6ef17398542d9574b77b40bf446155d210fa43384757e3837da41b025998ebfab4b9b6f094033f9c226392b800df068bce (@<ip>) 0.60ms
Jul 14 16:16:07 abel configurable-http-proxy[20284]: 16:16:07.192 [ConfigProxy] debug: PROXY WEB /hub/static/favicon.ico to http://127.0.0.1:8792
Jul 14 16:16:07 abel start_jhub[20285]: [D 2023-07-14 16:16:07.194 JupyterHub log:191] 200 GET /hub/static/favicon.ico?v=fde5757cd3892b979919d3b1faa88a410f28829feb5ba22b6cf069f2c6c98675fceef90f932e49b510e74d65c681d5846b943e7f7cc1b41867422f0481085c1f (@<ip>) 0.55ms
Jul 14 16:16:07 abel configurable-http-proxy[20284]: 16:16:07.198 [ConfigProxy] debug: PROXY WEB /hub/api/users/fst/server/progress to http://127.0.0.1:8792
Jul 14 16:16:07 abel start_jhub[20285]: [D 2023-07-14 16:16:07.200 JupyterHub scopes:863] Checking access via scope read:servers
Jul 14 16:16:07 abel start_jhub[20285]: [D 2023-07-14 16:16:07.200 JupyterHub scopes:677] Unrestricted access to /hub/api/users/fst/server/progress via read:servers
Jul 14 16:16:09 abel start_jhub[20298]: [I 2023-07-14 16:16:09.765 ServerApp] 302 GET /user/fst/ -> /user/fst/lab? (@127.0.0.1) 0.49ms
Jul 14 16:16:09 abel start_jhub[20285]: [D 2023-07-14 16:16:09.765 JupyterHub utils:278] Server at http://127.0.0.1:45197/user/fst/ responded with 302
Jul 14 16:16:09 abel start_jhub[20285]: [W 2023-07-14 16:16:09.765 JupyterHub _version:37] Single-user server has no version header, which means it is likely < 0.8. Expected 4.0.1
Jul 14 16:16:09 abel start_jhub[20285]: [I 2023-07-14 16:16:09.765 JupyterHub base:988] User fst took 3.805 seconds to start
Jul 14 16:16:09 abel start_jhub[20285]: [I 2023-07-14 16:16:09.765 JupyterHub proxy:330] Adding user fst to proxy /user/fst/ => http://127.0.0.1:45197
Jul 14 16:16:09 abel start_jhub[20285]: [D 2023-07-14 16:16:09.766 JupyterHub proxy:880] Proxy: Fetching POST http://localhost:8791/api/routes/user/fst
Jul 14 16:16:09 abel configurable-http-proxy[20284]: 16:16:09.767 [ConfigProxy] info: Adding route /user/fst -> http://127.0.0.1:45197
Jul 14 16:16:09 abel configurable-http-proxy[20284]: 16:16:09.768 [ConfigProxy] info: Route added /user/fst -> http://127.0.0.1:45197
Jul 14 16:16:09 abel configurable-http-proxy[20284]: 16:16:09.768 [ConfigProxy] info: 201 POST /api/routes/user/fst
Jul 14 16:16:09 abel start_jhub[20285]: [I 2023-07-14 16:16:09.768 JupyterHub users:768] Server fst is ready
Jul 14 16:16:09 abel start_jhub[20285]: [I 2023-07-14 16:16:09.769 JupyterHub log:191] 200 GET /hub/api/users/fst/server/progress?_xsrf=[secret] (fst@<ip>) 2569.56ms
Jul 14 16:16:09 abel configurable-http-proxy[20284]: 16:16:09.786 [ConfigProxy] debug: PROXY WEB /hub/spawn-pending/fst to http://127.0.0.1:8792
Jul 14 16:16:09 abel start_jhub[20285]: [D 2023-07-14 16:16:09.788 JupyterHub scopes:863] Checking access via scope servers
Jul 14 16:16:09 abel start_jhub[20285]: [D 2023-07-14 16:16:09.788 JupyterHub scopes:677] Unrestricted access to /hub/spawn-pending/fst via servers
Jul 14 16:16:09 abel start_jhub[20285]: [I 2023-07-14 16:16:09.789 JupyterHub log:191] 302 GET /hub/spawn-pending/fst -> /user/fst/ (fst@<ip>) 1.45ms
Jul 14 16:16:09 abel configurable-http-proxy[20284]: 16:16:09.789 [ConfigProxy] debug: Not recording activity for status 302 on /
Jul 14 16:16:09 abel configurable-http-proxy[20284]: 16:16:09.795 [ConfigProxy] debug: PROXY WEB /user/fst/ to http://127.0.0.1:45197
Jul 14 16:16:09 abel start_jhub[20298]: [I 2023-07-14 16:16:09.796 ServerApp] 302 GET /user/fst/ -> /user/fst/lab? (@<ip>) 0.45ms
Jul 14 16:16:09 abel configurable-http-proxy[20284]: 16:16:09.797 [ConfigProxy] debug: Not recording activity for status 302 on /user/fst
Jul 14 16:16:09 abel configurable-http-proxy[20284]: 16:16:09.801 [ConfigProxy] debug: PROXY WEB /user/fst/lab to http://127.0.0.1:45197
Jul 14 16:16:09 abel start_jhub[20298]: [W 2023-07-14 16:16:09.819 ServerApp] 404 GET /user/fst/lab? (@<ip>) 16.62ms
Jul 14 16:16:09 abel configurable-http-proxy[20284]: 16:16:09.819 [ConfigProxy] debug: Not recording activity for status 404 on /user/fst
Jul 14 16:16:09 abel configurable-http-proxy[20284]: 16:16:09.864 [ConfigProxy] debug: PROXY WEB /user/fst/static/logo/logo.png to http://127.0.0.1:45197
Jul 14 16:16:09 abel configurable-http-proxy[20284]: 16:16:09.884 [ConfigProxy] debug: PROXY WEB /user/fst/static/favicon.ico to http://127.0.0.1:45197

@Frank-Steiner Frank-Steiner changed the title jupyterhub 4.0.1 x gives 404 for /user/xx/lab jupyterhub 4.0.1 doesn't load jupyterlab automatically if Spawner.default_url is "/lab" Jul 17, 2023
@Frank-Steiner
Copy link
Author

I was able to find the reason as we had different users with some being able to access the GUIs, others getting the 404. The difference was that the working users had a file ~/.local/etc/jupyter/jupyter_server_config.d/jupyterlab.json in their home with content

{
  "ServerApp": {
    "jpserver_extensions": {
      "jupyterlab": true
    }
  }
}

Thus, jupyterhub 4.x is not enabling the jupyterlab extension automatically if

c.Spawner.default_url = '/lab`

is set in the config. 3.x does. So I'm not sure if this is a bug or a design change, but I guess it makes sense to activate the jupyterlab extension automatically if that's set to be the default interface...

@minrk
Copy link
Member

minrk commented Aug 1, 2023

Indeed, jupyterhub 4 loads itself as a jupyter server extension and no longer enables any other extensions itself.

You can opt-out of the new extension implementation with JUPYTERHUB_SINGLEUSER_EXTENSION=0 to get the jupyterhub 3 behavior.

JupyterHub 3 launches jupyter-labhub by default, which had the main effect of setting the default default url to /lab and ensuring the jupyterlab extension is enabled.

But to the issue at hand, installing jupyterlab generally creates the file $PREFIX/etc/jupyter/jupyter_server_config.d/jupyterlab.json. How has jupyterlab been installed, and can you find the server config file that it creates during installation?

You might double check the permissions on /opt/jhub/etc/jupyter/jupyter_server_config.d/ and its parents to make sure all users can read that config.

It's unclear to me why it's not being enabled by default (possibly a peculiar environment, path, or permissions issue), but the solution is to ensure that the jupyterlab extension is enabled, e.g.

jupyter server extension enable jupyterlab --sys-prefix

Exactly how/when/where that should be done will depend on how you manage the environment your users use, but from the looks of it, it should work with:

/opt/jhub/bin/jupyter-server extension enable jupyterlab --sys-prefix

@vwbusguy
Copy link

vwbusguy commented Aug 1, 2023

We're seeing this today, too, when using current versions of r-notebook for singleuser image. We also have been trying the workarounds listed here (JUPYTERHUB_SINGLEUSER_EXTENSION=0 and jupyter server extension enable jupyterlab --sys-prefix in a downstream image and/or in the z2jh helm). /lab is there and is functional, but it now seems hard-coded to default to /tree no matter what. This seems like a regression from the documented behavior of /lab being the default path unless another path (/tree, /rstudio, etc.) is specified.

Interestingly, the same config works for deploying with r-notebook:r-4.2.2, but not on r-notebook:latest.

@manics
Copy link
Member

manics commented Aug 1, 2023

@vwbusguy this issue is about differences in behaviour between JupyterHub 3 and 4.
The problem you've mentioned

/lab is there and is functional, but it now seems hard-coded to default to /tree no matter what.

is a bug in the notebook repo:
jupyter/notebook#6954

@jtpio
Copy link
Contributor

jtpio commented Aug 2, 2023

Investigating this issue in jupyter/notebook#6997, which should fix it I think.

If someone from the hub team would like to review it that would be great, thanks!

@shaneknapp
Copy link
Contributor

thanks @jtpio for fixing this in jupyter/notebook#6997

fwiw, i believe that we can probably close this issue. :)

@manics
Copy link
Member

manics commented Aug 4, 2023

Thanks @shaneknapp @jtpio !

@manics manics closed this as completed Aug 4, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

6 participants