"'_xsrf' argument missing from POST" (PUT request) for jupyterlab extension behind nginx reverse proxy

[jkelling]

I am running a jupyter notebook with enabled jupyterlab extension behind an nginx reverse proxy.

The jupyter notebooks part works fine. URLs of the type
https://server/base/tree
or
https://server/base/notebooks/Untitled1.ipynb
are proxied, the _xsrf cookie is forwarded and various kernels work.

Accessing
https://server/base/lab
All requests work except for a PUT request to
https://server/base/lab/api/workspaces/base/lab?1559052959210
which comes back as 403. After the 403 log line, jupyter notebook also logs:

'_xsrf' argument missing from POST

This failure is silent on the client side, the lab GUI does completely load.

Trying to open anything from the launcher or a file yield another PUT request which returns 403 with the jupyter log again. This time the GUI reports the error and also displays a second error when something in the launcher was clicked:

Launcher Error
e is undefined

I am reporting this here, because this may a configuration error in the proxy or a bug in jupyterlab, as jupyter notebook in the same instance works without problems. However, it does not appear to use PUT requests.

Sending that same PUT request to the same proxy, only to a different path which it forwarded to a stub which only echoes the received request shows that the _xsrf cookie is not stripped.

On the other hand, when by passing the proxy (via a port forwarded through SSH) and talking directly to the jupyter notebook instance this problem does not occur.

This does seem to have something to do with the Proxy, but not actually with the _xsrf missing.

location /base {
        proxy_pass      http://jupyterNotebookHost:8888;
        proxy_http_version 1.1;

        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header Host $http_host; # keep requested host unchanged
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;

        proxy_read_timeout 86400;

        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
}

What could the error

'_xsrf' argument missing from POST

be caused by, assuming the _xsrf cookie is actually present? Could it actually be some security policy in jupyterlab which is triggered by some client or host headers being inconsistent, but which is not checked by in jupyter notebook, or something like this?

I guess as a feature request, I would like to ask for a more helpful error message.

pip3 show ...:

jupyter                           1.0.0      
jupyter-client                    5.2.3      
jupyter-console                   5.2.0      
jupyter-contrib-core              0.3.3      
jupyter-contrib-nbextensions      0.5.0      
jupyter-core                      4.4.0      
jupyter-highlight-selected-word   0.2.0      
jupyter-latex-envs                1.4.4      
jupyter-nbextensions-configurator 0.4.0      
jupyterlab                        0.35.6     
jupyterlab-launcher               0.13.1     
jupyterlab-server                 0.2.0      
jupyterlab-widgets                0.6.15
....
tornado                           5.0.2      

Update: nginx is not the problem

There is another apache responsible for both Shibboleth authentication and SSL termination in front of the nginx. When talking directly to the nginx jupyterlab works.

This for one forwards the initial https request as plain http and sets some headers as well a introducing a _shibsession_xxx=xxx cookie. However, rewriting the cookie header to only contain the _xsrf cookie does not fix the problem and jupyterlab still complains, that _xsrf is missing.

Maybe the apache is the problem. In any case the error message form jupyterlab seems to be wrong. Actually removing the _xsrf cookie leads all GET requests to fail too. Can anyone tell what could be different about that PUT request?

[blink1073]

Hi @jkelling, I believe this was fixed in #6141, which will be in the upcoming alpha 4 release.

[afshin]

@jkelling Please feel free to post this issue again if it is not resolved, but closing it for now as #6141 likely did fix the issue.

[lock]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related discussion.