VS Code keeps trying to send requests to deleted remote kernels

[abaybektursun]

Applies To

• Notebooks (.ipynb files)
• Interactive Window and/or Cell Scripts (.py files with #%% markers)

What happened?

1. Connect to a remote kernel
2. Disconnect and delete the remote kernel
3. Observe that VS Code is still trying to access the deleted kernel unless you restart VS Code

This is what I see in my remote kernel server after restarting it:

[W 2023-11-05 23:33:23.587 ServerApp] 403 GET /api/kernels?1699227203387 (@127.0.0.1) 6.16ms referer=None
[W 2023-11-05 23:33:23.588 ServerApp] wrote error: 'Forbidden'
    Traceback (most recent call last):
      File "/home/user/.local/lib/python3.10/site-packages/tornado/web.py", line 1784, in _execute
        result = method(*self.path_args, **self.path_kwargs)
      File "/home/user/.local/lib/python3.10/site-packages/tornado/web.py", line 3278, in wrapper
        url = self.get_login_url()
      File "/home/user/.local/lib/python3.10/site-packages/jupyter_server/base/handlers.py", line 753, in get_login_url
        raise web.HTTPError(403)
    tornado.web.HTTPError: HTTP 403: Forbidden

Millions of them, when I close VS Code they stop.

VS Code Version

Version: 1.83.1 Commit: f1b07bd25dfad64b0167beb15359ae573aecd2cc Date: 2023-10-10T23:57:32.750Z Electron: 25.8.4 ElectronBuildId: 24154031 Chromium: 114.0.5735.289 Node.js: 18.15.0 V8: 11.4.183.29-electron.0 OS: Darwin arm64 22.6.0

Jupyter Extension Version

v2023.10.1003070148

Jupyter logs

Don't know how to get this, let me know how

Coding Language and Runtime Version

No response

Language Extension Version (if applicable)

No response

Anaconda Version (if applicable)

No response

Running Jupyter locally or remotely?

Remote

[h4rvey-g]

Same issue here. It took me hours to realize it's a problem of VS Code instead of my docker jupyter config.

[DonJayamanne]

Thank you for filing this issue and sorry you are running into this.

Disconnect and delete the remote kernel

Please can you explain what you mean by delete the remote kernel
How do you go about deleting a kernel?
There's no such functionality in VS Code nor in jupyter, hence the question.
Are you deleting a kernle spec, or do you mean shutdown a kernel or the like.

@h4rvey-g Please could you too share the logs as well,

Are you using a Jupyter Token to connect to the remote server or are yous using some user name password combo?

@h4rvey-g Are you able to share your docker file so I can test this at my end?

[h4rvey-g]

Here is how I reproduce this on my end

1. Run a jupyter container

docker run -it -p 8888:8888 -e GRANT_SUDO=yes -e DOCKER_STACKS_JUPYTER_CMD=notebook --user root quay.io/jupyter/base-notebook start-notebook.py --ServerApp.root_dir=/home/jovyan/work

2. Connect to the jupyter server in a VS Code notebook, via the link in the container output (e.g. http://127.0.0.1:8888/tree?token=e7895033c17336f205336cc79b8a47a1f24d02a1f9c2755f)
3. Press Ctrl+C twice to shut down the jupyter server and the container
4. Run a new container from the same image, which will give a new jupyter server link. Then numerous messages of tornado.web.HTTPError: HTTP 403: Forbidden will flood out, like the log @abaybektursun posted. I tried to re-open the notebook file and connect it to the new jupyter server link but it does not help. So I think it might be some functions of the jupyter extension still tried to connect to the previous server, but it didn't use the correct token, which led to this error.
5. Using Remote SSH: Kill VS Code Server on Host and reloading VS Code will resolve this problem.

[abaybektursun]

@DonJayamanne I delete it using cross button in the GUI, in the existing servers list hover over the server and then click x that appears.

[DonJayamanne]

@abaybektursun Does this persist after you reload VS Code
I.e. delete the entry and the reload vscode, do you still see the same requests being sent?

[abaybektursun]

@DonJayamanne It's gone after I restart VSCode

[h4rvey-g]

Deleting the Jupyter server from existing servers list works for me too.

[DonJayamanne]

@ayushnoori @h4rvey-g @abaybektursun
Sorry you have been running into this issue and thanks for your patience and providing the logs.
Please could you

• Download the attached file, extract it
• Open VS Code and select the command Install from VSIX
• Select the (above) extracted VSIX and when prompted, please reload VS Code
• Try to replicate the issue, if this still persists, please do share the logs & the exact steps
• E.g. if you run a cell please mention that as well.
  ms-toolsai-jupyter-insiders.vsix (1).zip

[ayushnoori]

Hi @DonJayamanne, when I try to run Install from VSIX, I receive the following error:

Unable to install extension 'ms-toolsai.jupyter' as it is not compatible with VS Code '1.84.2'.

May you please advise how to resolve? Thanks!

[DonJayamanne]

Please try this vsix
ms-toolsai-jupyter-insiders.vsix.zip

Sorry about that, it was updated for the next version

[ayushnoori]

Hi @DonJayamanne, I was able to download the extension from the new VSIX file and unfortunately the issue still persists. The flurry of 403 errors appear after I connect to the Jupyter kernel with Select Another Kernel > Existing Jupyter Server > Enter the URL of the running Jupyter server, and then run the first cell.

After I kill the server and run it again with jupyter server --no-browser, the 403 errors appear again, even without reconnecting to the Jupyter notebook.

The only new message in the logs (copied from the output panel in VS Code) that I can discern is:

2023-12-06 14:27:55.757 [info] Extension host with pid 68641 exited with code: 0, signal: unknown.

However, I should note that the logs from previous days don't show any similar message.

[DonJayamanne]

flurry of 403 errors appear after I connect to the Jupyter kernel with

have you deleted the previous server using the ‘X’ button?

[ayushnoori]

I haven't deleted old servers using the "X" button in the GUI which appears after Select Another Kernel > Existing Jupyter Server. Do I need to do so manually each time? I tried to delete, and then when I attempted to reconnect, the entry localhost appeared again:

[DonJayamanne]

Do I need to do so manually each time? I tried to delete,

Yes you need to remove them

and then when I attempted to reconnect, the entry localhost appeared again:

Please can yoyu givem them different names & share the logs, this sounds like a totally different issue

After I kill the server and run it again with jupyter server --no-browser, the 403 errors appear again, even without reconnecting to the Jupyter notebook.

You will see this only a max of 10 times now, this unfortunately cannot be avoided
As we have no idea whether the server is down or whether its just a network issue.
Previously we'd end up with an unending list of 403 requests, now it should stop after a while.