PyLance crashes when setting node option '--no-deprecation'

[kvablack]

Environment data

• Language Server version: 2024.5.1
• OS and version: linux x64
• Python version (and distribution if applicable, e.g. Anaconda):
• python.analysis.indexing: true
• python.analysis.typeCheckingMode: off

Code Snippet

N/A

Repro Steps

Purged all VSCode cache and config files from both my local machine and the remote server I'm connected to via SSH. Created a single .py file, opened it, and installed the Python extension pack. Immediately get a crash from PyLance:

2024-05-23 14:08:48.139 [info] (Client) Pylance async client (2024.5.1) started with python extension (2024.6.0)
2024-05-23 14:08:48.230 [info] /home/black/.vscode-server/extensions/ms-python.vscode-pylance-2024.5.1/dist/server.bundle.js:1
(function(_0x4d4841,_0x1df26e){const... # I'm truncating a super long line here, looks like the entire `server.bundle.js`
2024-05-23 14:08:48.623 [info] [Error - 2:08:48 PM] Server initialization failed.
2024-05-23 14:08:48.623 [info]   Message: Pending response rejected since connection got disposed
  Code: -32097 

I tried:

• Many versions of the PyLance extension, back about 6 months
• Using a custom Node executable (suggested in a VSCode dialog)

How do I debug this further? I have absolutely no way to edit any Python files on this machine now.

[rchiodo]

Thanks for the issue.

The super long line would help. Or at least the start of the line. It should show what line is crashing. We can then use our internal source maps to figure out where that is in our code.

[kvablack]

Sure thing:
log.txt

[heejaechang]

Unfortunately the log doesn't have any meaningful info.

can you make sure the log contains things described here? https://github.com/microsoft/pylance-release/wiki/Collecting-data-for-an-investigation.#collecting-trace-logs

for your case, since it crash almost right away, set the setting in settings.json and reload vscode and we need full log in the python language server output window regardless how big it is.

[kvablack]

I have "python.analysis.logLevel": "Trace", in my settings.json (both user and remote), but it doesn't seem to change anything. Here is the full log:

log.txt

It might also be worth noting that most PyLance commands (e.g., Pylance: Start Logging) gives me "command 'pylance.logging.start' not found".

[rchiodo]

Hmm, maybe VS code is truncating the output for some reason. It should have something like this in it somewhere:

TypeError: Cannot read properties of undefined (reading 'call')
    at _0x13e5f3 (c:\Users\rchiodo\.vscode\extensions\ms-python.
2024-05-23 15:33:39.726 [info] vscode-pylance-2024.5.1\dist\server.bundle.js:1:977217)
    at 78268 (c:\Users\rchiodo\.vscode\extensions\ms-python.vscode-pylance-2024.5.1\dist\server.bundle.js:1:2537)
    at _0x13e5f3 (c:\Users\rchiodo\.vscode\extensions\ms-python.vscode-pylance-2024.5.1\dist\server.bundle.js:1:977225)
    at 80332 (c:\Users\rchiodo\.vscode\extensions\ms-python.vscode-pylance-2024.5.1\dist\server.bundle.js:1:578016)
    at _0x13e5f3 (c:\Users\rchiodo\.vscode\extensions\ms-python.vscode-pylance-2024.5.1\dist\server.bundle.js:1:977225)
    at 47100 (c:\Users\rchiodo\.vscode\extensions\ms-python.vscode-pylance-2024.5.1\dist\server.bundle.js:1:954056)
    at _0x13e5f3 (c:\Users\rchiodo\.vscode\extensions\ms-python.vscode-pylance-2024.5.1\dist\server.bundle.js:1:977225)
    at c:\Users\rchiodo\.vscode\extensions\ms-python.vscode-pylance-2024.5.1\dist\server.bundle.js:1:977459
    at _0x13e5f3.O (c:\Users\rchiodo\.vscode\extensions\ms-python.vscode-pylance-2024.5.1\dist\server.bundle.js:1:978466)
    at _0x13e5f3.x (c:\Users\rchiodo\.vscode\extensions\ms-python.vscode-pylance-2024.5.1\dist\server.bundle.js:1:977509)
    at _0x13e5f3.x (c:\Users\rchiodo\.vscode\extensions\ms-python.vscode-pylance-2024.5.1\dist\server.bundle.js:1:981005)
    at c:\Users\rchiodo\.vscode\extensions\ms-python.vscode-pylance-2024.5.1\dist\server.bundle.js:1:981053
    at Object.<anonymous> (c:\Users\rchiodo\.vscode\extensions\ms-python.vscode-pylance-2024.5.1\dist\server.bundle.js:1:981086)
    at Module._compile (node:internal/modules/cjs/loader:1356:14)
    at Module._extensions..js (node:internal/modules/cjs/loader:1414:10)
    at Module.load (node:internal/modules/cjs/loader:1197:32)
    at Module._load (node:internal/modules/cjs/loader:1013:12)
    at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:128:12)
    at node:internal/main/run_main_module:28:49

(I forced that on my machine by editing some bytes in the server.bundle.js).

Prior to that line it dumps the entire server.bundle.js for some reason.

Can you try setting this value higher?

    "terminal.integrated.scrollback": 10000,

[kvablack]

Hmm I see, I can't seems to get rid of the truncation. I set "terminal.integrated.scrollback" to a high value and it didn't change anything. When I hovered over the line, I also got a message from VSCode suggesting to modify "editor.stopRenderingLineAfter", but when I actually put that in my settings.json it says the setting does not exist. I also checked the raw log file at ~/.vscode-server/data/logs/20240523T134940/exthost23/ms-python.python/Python\ Language\ Server.log, and the line is still truncated at the same point.

[heejaechang]

have you tried to delete .vscode-server and re-install?

[kvablack]

Yep, I've tried that several times. I also tried some more aggressive purging (reinstall VSCode locally, clearing all configuration files locally + remotely, clearing ~/.cache).

[kvablack]

I just confirmed that it works on a different remote machine, so it's definitely a problem with that machine. However, I'm not sure what else to try, since I'm not familiar with what else could possibly affect PyLance. Let me more comprehensively enumerate what I tried:

• Deleting ~/.vscode-server and reloading, which also requires re-installing the Python extension pack
• Deleting ~/.cache
• Rebooting
• Uninstalling and reinstalling python3 using apt
• Using a custom Node interpreter (Node v22.0.0) instead of the one bundled with vscode-server
• Uninstalling VSCode locally, deleting all local configuration files, and reinstalling

All of these have led to the exact same outcome as far as I can tell. Any other suggestions?

[rchiodo]

What if you change server.bundle.js to just throw new Error('Server crash');? (After copying it to a new file).

server.bundle.js would be in the .vscode-server/extensions/ms-python.vscode-pylance-2024.5.1/dist folder.

Does that show a callstack?

[rchiodo]

I can only guess that some global node option on that machine is messing something up. Although if that were the case, I wouldn't expect any extensions to load either.

[kvablack]

I uninstalled everything node-related that I could find from my machine. I was using NVM so I just deleted ~/.nvm. Crash is still happening.

If I add throw new Error("server crash"); to the top of server.bundle.js, the stacktrace looks fine:

2024-05-23 16:56:36.821 [info] /home/black/.vscode-server/extensions/ms-python.vscode-pylance-2024.5.1/dist/server.bundle.js:1
throw new Error("server crash");
^

Error: server crash
    at Object.<anonymous> (/home/black/.vscode-server/extensions/ms-python.vscode-pylance-2024.5.1/dist/server.bundle.js:1:7)
    at Module._compile (node:internal/modules/cjs/loader:1256:14)
    at Module._extensions..js (node:internal/modules/cjs/loader:1310:10)
    at Module.load (node:internal/modules/cjs/loader:1119:32)
    at Module._load (node:internal/modules/cjs/loader:960:12)
    at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:86:12)
    at node:internal/main/run_main_module:23:47

Node.js v18.18.2

[rchiodo]

Does an older version work on this machine? Like 2024.4.1?

[rchiodo]

We changed our obfuscation in 2024.5.1. I wonder if that's breaking something somehow.

[kvablack]

No, I tried a bunch of different versions (all the way back to 2023.12.1) and more or less the same thing happens. I just double-checked with 2024.4.1.

[rchiodo]

Do any of the older versions give you a callstack for the crash?

[rchiodo]

Another thing to try is go on the machine and just run node server.bundle.js. It shouldn't work, but it might actually give us a callstack. Normally doing this will start the server and it will give an error message like so:

You may install and use any number of copies of the software only with Microsoft Visual Studio, Visual Studio for Mac, Visual Studio Code, Azure DevOps, Team Foundation Server, and successor Microsoft products and services (collectively, the “Visual Studio Products and Services”) to develop and test your applications. The software is licensed, not sold. This agreement only gives you some rights to use the software. Microsoft reserves all other rights. You may not: work around any technical limitations in the software that only allow you to use it in certain ways; reverse engineer, decompile or disassemble the software, or otherwise attempt to derive the source code for the software, except and to the extent required by third party licensing terms governing use of certain open source components that may be included in the software; remove, minimize, block, or modify any notices of Microsoft or its suppliers in the software; use the software in any way that is against the law or to create or propagate malware; or share, publish, distribute, or lease the software (except for any distributable code, subject to the terms above), provide the software as a stand-alone offering for others to use, or transfer the software or this agreement to any third party.

But if something is messed up in the bundle or in your node, maybe running it from the command line will give us a callstack.

[kvablack]

Ah nice, that gave me the traceback! Here it is:

TypeError: Cannot assign to read only property 'noDeprecation' of object '#<process>'
    at 47100 (/home/black/.vscode-server/extensions/ms-python.vscode-pylance-2024.5.1/dist/server.bundle.js:1:952786)
    at _0x13e5f3 (/home/black/.vscode-server/extensions/ms-python.vscode-pylance-2024.5.1/dist/server.bundle.js:1:977225)
    at /home/black/.vscode-server/extensions/ms-python.vscode-pylance-2024.5.1/dist/server.bundle.js:1:977459
    at _0x13e5f3.O (/home/black/.vscode-server/extensions/ms-python.vscode-pylance-2024.5.1/dist/server.bundle.js:1:978466)
    at _0x13e5f3.x (/home/black/.vscode-server/extensions/ms-python.vscode-pylance-2024.5.1/dist/server.bundle.js:1:977509)
    at _0x13e5f3.x (/home/black/.vscode-server/extensions/ms-python.vscode-pylance-2024.5.1/dist/server.bundle.js:1:981005)
    at /home/black/.vscode-server/extensions/ms-python.vscode-pylance-2024.5.1/dist/server.bundle.js:1:981053
    at Object.<anonymous> (/home/black/.vscode-server/extensions/ms-python.vscode-pylance-2024.5.1/dist/server.bundle.js:1:981086)
    at Module._compile (node:internal/modules/cjs/loader:1256:14)
    at Module._extensions..js (node:internal/modules/cjs/loader:1310:10)

[kvablack]

(same error in 2023.12.1, this is using vscode-server bundled node)

[rchiodo]

Not sure why we have this in our code but this is the failing line:

if (process.env.NODE_ENV === 'production') {
    // Silence Buffer deprecations from onnx.
    (process as any).noDeprecation = true;
}

It must mean you have an environment variable NODE_ENV set to production. Would explain why this is failing on just the one machine.

[kvablack]

This is very strange, since I definitely don't have the NODE_ENV variable set. In fact:

$ .vscode-server/cli/servers/Stable-dc96b837cf6bb4af9cd736aa3af08cf8279f7685/server/node -e "console.log(process.env.NODE_ENV)"
undefined

[kvablack]

Oh my god, I figured it out. It's because I had set the environment variable NODE_OPTIONS="--no-deprecation", because otherwise a completely unrelated command-line tool I was using would spit out tons of deprecation warnings.

Seems like globally setting NODE_OPTIONS is very dangerous, I definitely shouldn't have done that. I'm pretty unfamiliar with Node so I didn't think for a second that other things could be affected. I'm a bit shocked that VSCode extensions are so affected by environment variables that I set in my .bashrc, though.

[rchiodo]

Glad you figured out. Thanks for the help.

[kvablack]

Thanks for the real-time support!