Pyrefly crashes VSCode when allocating memory on glibc 2.17

[varchasgopalaswamy]

Describe the Bug

When I use the Pyrefly vscode extension, after a few minutes it causes vscode to crash. A core file is created. When I check it with gdb, I get the following:

Core was generated by `.../.vscode-server/extensions/meta.pyrefly-0.42.2-linux-x64/bin/pyrefly'.
Program terminated with signal 6, Aborted.
#0  0x00002ad2b0c30802 in __restore_sigs ()
(gdb) bt
#0  0x00002ad2b0c30802 in __restore_sigs ()
#1  0x00002ad2b0c30843 in raise ()
#2  0x00002ad2b0aa7c2f in std::io::Write::write_fmt::he598795b07ae9227 ()
Backtrace stopped: previous frame inner to this frame (corrupt stack?)

Any idea what's wrong?

IDE Information

VSCode
Version: 1.105.1
Commit: 7d842fb85a0275a4a8e4d7e040d2625abbf7f084
Date: 2025-10-14T22:33:36.618Z (1 mo ago)
Electron: 37.6.0
ElectronBuildId: 12502201
Chromium: 138.0.7204.251
Node.js: 22.19.0
V8: 13.8.258.32-electron.0

[varchasgopalaswamy]

I took at look at the language server output.

 INFO Language server processed event `LspRequest(textDocument/semanticTokens/full)` in 0.00s (0.00s waiting)
memory allocation of 15269904 bytes failed
[Info  - 12:26:05 PM] Connection to server got closed. Server will restart.
true
[Error - 12:26:05 PM] Server process exited with signal SIGABRT.

That's just 15 MB, so I'm definitely not out of memory (I have ~ 300 GB available).

[kinto0]

what's the size of the largest file you're having pyrefly analyze? it seems like we're ooming trying to send semantic tokens, which only operate on a single module. maybe we should limit the max size of that?

[varchasgopalaswamy]

I'll check the next time I have access to my code. But there are some rather large files that contain generated python code. I would strongly prefer that the code be checkable - pyright is able to do so. Is there an environment variable that controls how big the allocation size can be?

[rodrigosf672]

@austin3dickey Once we have more details and/or repro steps, this should be a good test case to verify that the Pyrefly extension does not "crash" Positron and, if it does, to understand what kind of fix would be needed (if applicable from our end).

[varchasgopalaswamy]

So the largest file in my project is 25k lines long.

(.venv)$ PYREFLY_STACK_SIZE=5242880 uvx pyrefly check -v /path/to/large/file
DEBUG Checking 1 files (listing took 0.00s)
 INFO Using stack size of 5,242,880 bytes (due to `$PYREFLY_STACK_SIZE`)
DEBUG Running with 36 threads (5 MiB stack size)
[00:00:00] ░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░       0/0      DEBUG Running epoch 1 of run 0
[00:00:07] ░░░░░░░░░░░░░░░░░░░░░░

runs and gives me a bunch of type checking errors, which is expected behavior. So I'm not sure why it crashes in vscode, when it runs just fine when checking the file itself.

Another issue I've noticed - before the crash happens, all intellisense in vscode hangs completely. Mousing over variables shows a "loading" dialog, stuff like go-to or find references just waits eternally. After about five minutes of this, I get the crash message. This time it was for a 512 byte allocate, with the same bt as above.

memory allocation of 512 bytes failed
[Info  - 2:30:56 AM] Connection to server got closed. Server will restart.
true
[Error - 2:30:56 AM] Server process exited with signal SIGABRT.
starting generic LSP server

[varchasgopalaswamy]

I tried increasing PYREFLY_STACK_SIZE to 100 MB and found no change, incidentally.

[kinto0]

is the workspace you are using public enough for me to reproduce it locally?

[varchasgopalaswamy]

is the workspace you are using public enough for me to reproduce it locally?

Unfortunately it isn't 🙁. It's easy enough for me to trigger the crash though, so let me know if there's any diagnostics I can collect for you.

[samwgoldman]

Agree with @kinto0 -- it looks like the proximal cause here is responding to the semantic token request for a very large file. The immediate error is that we can't allocate memory -- malloc returning null, Rust bailing -- but it's not yet clear what exactly we are trying to allocate and why.

It's unfortunate that the core dump appears to have a corrupt stack, so we can't see the full stack trace. A real stack trace would help a lot. @varchasgopalaswamy mind trying again to reproduce, get a fresh core dump, and hope the stack isn't corrupted?

[kinto0]

one easy thing to test might be disabling semantic tokens through the configuration. if this doesn't crash it, that's a hint that we're on the right track

(in user settings, cmd+shift+p > preferences: open user settings.json)

    "python.pyrefly.disabledLanguageServices": {
        "semanticTokens": true
    },

[varchasgopalaswamy]

"python.pyrefly.disabledLanguageServices": {
"semanticTokens": true
},

I tried this - unfortunately, I still get the same issue. Before the crash occured, though, I opened the 25k line file, and found that pyrefly very quickly provided type hinting for it - so that's nice at least! And the crash doesn't seem to occur when actually processing that file. Here's the logs from the language server

starting generic LSP server
 INFO Reading messages
 INFO Using stack size of 52,428,800 bytes (due to `$PYREFLY_STACK_SIZE`)
 INFO File ... opened, prepare to validate open files.
 INFO Unhandled notification: Notification { method: "$/setTrace", params: Object {"value": String("off")} }
 INFO Validated open files and saved non-committable transaction.
 INFO Cleared 2 dropped configs from config cache
 INFO Populating all files in the config (File("***/pyproject.toml")).
 INFO Ran task on sourcedb_queue heavy task queue. Queue time: 0.00, task time: 0.00
 INFO Language server processed event `DidOpenTextDocument` in 4.53s (0.00s waiting)
 INFO We should cancel request RequestId(I32(7))
 INFO Language server processed event `CancelRequest` in 0.00s (3.26s waiting)
 INFO We should cancel request RequestId(I32(2))
 INFO Language server processed event `CancelRequest` in 0.00s (3.26s waiting)
 INFO We should cancel request RequestId(I32(3))
 INFO Language server processed event `CancelRequest` in 0.00s (3.26s waiting)
 INFO We should cancel request RequestId(I32(11))
 INFO Language server processed event `CancelRequest` in 0.00s (2.14s waiting)
 INFO We should cancel request RequestId(I32(14))
 INFO Language server processed event `CancelRequest` in 0.00s (2.14s waiting)
 INFO We should cancel request RequestId(I32(15))
 INFO Language server processed event `CancelRequest` in 0.00s (2.14s waiting)
 INFO Request pyrefly/textDocument/typeErrorDisplayStatus (1) is canceled due to subsequent mutation
 INFO Language server processed event `LspRequest(pyrefly/textDocument/typeErrorDisplayStatus)` in 0.00s (4.53s waiting)
 INFO Request textDocument/documentSymbol (2) is canceled due to explicit cancellation
 INFO Language server processed event `LspRequest(textDocument/documentSymbol)` in 0.00s (4.19s waiting)
 INFO Request textDocument/codeAction (3) is canceled due to explicit cancellation
 INFO Language server processed event `LspRequest(textDocument/codeAction)` in 0.00s (4.19s waiting)
 INFO Request textDocument/inlayHint (4) is canceled due to subsequent mutation
 INFO Language server processed event `LspRequest(textDocument/inlayHint)` in 0.00s (4.19s waiting)
 INFO Request pyrefly/textDocument/typeErrorDisplayStatus (5) is canceled due to subsequent mutation
 INFO Language server processed event `LspRequest(pyrefly/textDocument/typeErrorDisplayStatus)` in 0.00s (4.19s waiting)
 INFO Request pyrefly/textDocument/typeErrorDisplayStatus (6) is canceled due to subsequent mutation
 INFO Language server processed event `LspRequest(pyrefly/textDocument/typeErrorDisplayStatus)` in 0.00s (4.19s waiting)
 INFO Language server processed event `LspResponse` in 0.00s (4.11s waiting)
 INFO Request textDocument/documentSymbol (7) is canceled due to explicit cancellation
 INFO Language server processed event `LspRequest(textDocument/documentSymbol)` in 0.00s (4.11s waiting)
 INFO Client configuration applied to workspace: Some(Url { scheme: "file", cannot_be_a_base: false, username: "", password: None, host: None, port: None, path: "***", query: None, fragment: None })
 INFO Client configuration applied to workspace: None
 INFO Language server processed event `LspResponse` in 0.11s (4.11s waiting)
 INFO Request textDocument/semanticTokens/full (8) is canceled due to subsequent mutation
 INFO Language server processed event `LspRequest(textDocument/semanticTokens/full)` in 0.00s (3.64s waiting)
 INFO Request textDocument/semanticTokens/range (9) is canceled due to subsequent mutation
 INFO Language server processed event `LspRequest(textDocument/semanticTokens/range)` in 0.00s (3.64s waiting)
 INFO Request textDocument/documentSymbol (10) is canceled due to subsequent mutation
 INFO Language server processed event `LspRequest(textDocument/documentSymbol)` in 0.00s (3.37s waiting)
 INFO Request textDocument/codeAction (11) is canceled due to explicit cancellation
 INFO Language server processed event `LspRequest(textDocument/codeAction)` in 0.00s (3.37s waiting)
 INFO Request textDocument/documentSymbol (12) is canceled due to subsequent mutation
 INFO Language server processed event `LspRequest(textDocument/documentSymbol)` in 0.00s (3.24s waiting)
 INFO Request textDocument/hover (13) is canceled due to subsequent mutation
 INFO Language server processed event `LspRequest(textDocument/hover)` in 0.00s (2.56s waiting)
 INFO Request textDocument/codeAction (14) is canceled due to explicit cancellation
 INFO Language server processed event `LspRequest(textDocument/codeAction)` in 0.00s (2.24s waiting)
 INFO Request textDocument/codeAction (15) is canceled due to explicit cancellation
 INFO Language server processed event `LspRequest(textDocument/codeAction)` in 0.00s (2.24s waiting)
 INFO Request textDocument/codeAction (16) is canceled due to subsequent mutation
 INFO Language server processed event `LspRequest(textDocument/codeAction)` in 0.00s (2.24s waiting)
 INFO Language server processed event `LspResponse` in 0.00s (0.10s waiting)
 INFO Language server processed event `LspResponse` in 0.00s (0.09s waiting)
 INFO Language server processed event `LspResponse` in 0.00s (0.09s waiting)
 INFO Handling non-canceled request textDocument/codeAction (17)
 INFO Language server processed event `LspRequest(textDocument/codeAction)` in 0.00s (0.00s waiting)
 INFO Handling non-canceled request textDocument/semanticTokens/full (18)
 INFO Skipping request - textDocument/semanticTokens/full service disabled
 INFO Language server processed event `LspRequest(textDocument/semanticTokens/full)` in 0.00s (0.00s waiting)
 INFO Handling non-canceled request textDocument/hover (19)
 INFO Language server processed event `LspRequest(textDocument/hover)` in 0.01s (0.00s waiting)
 INFO Handling non-canceled request textDocument/hover (20)
 INFO Language server processed event `LspRequest(textDocument/hover)` in 0.01s (0.00s waiting)
 INFO File *** opened, prepare to validate open files.
 INFO Validated open files and saved non-committable transaction.
 INFO Ran task on sourcedb_queue heavy task queue. Queue time: 0.00, task time: 0.00
 INFO Language server processed event `DidOpenTextDocument` in 2.52s (0.00s waiting)
 INFO We should cancel request RequestId(I32(21))
 INFO Language server processed event `CancelRequest` in 0.00s (2.31s waiting)
 INFO Request textDocument/codeAction (21) is canceled due to explicit cancellation
 INFO Language server processed event `LspRequest(textDocument/codeAction)` in 0.00s (2.52s waiting)
 INFO Handling non-canceled request pyrefly/textDocument/typeErrorDisplayStatus (22)
 INFO Language server processed event `LspRequest(pyrefly/textDocument/typeErrorDisplayStatus)` in 0.00s (2.52s waiting)
 INFO Handling non-canceled request textDocument/documentSymbol (23)
 INFO Language server processed event `LspRequest(textDocument/documentSymbol)` in 0.00s (2.52s waiting)
 INFO Handling non-canceled request textDocument/semanticTokens/full (24)
 INFO Skipping request - textDocument/semanticTokens/full service disabled
 INFO Language server processed event `LspRequest(textDocument/semanticTokens/full)` in 0.00s (2.52s waiting)
 INFO Handling non-canceled request textDocument/codeAction (25)
 INFO Language server processed event `LspRequest(textDocument/codeAction)` in 0.00s (2.31s waiting)
 INFO Handling non-canceled request textDocument/documentSymbol (26)
 INFO Language server processed event `LspRequest(textDocument/documentSymbol)` in 0.00s (2.27s waiting)
 INFO Handling non-canceled request textDocument/semanticTokens/range (27)
 INFO Skipping request - textDocument/semanticTokens/range service disabled
 INFO Language server processed event `LspRequest(textDocument/semanticTokens/range)` in 0.00s (1.92s waiting)
 INFO Handling non-canceled request textDocument/inlayHint (28)
 INFO Language server processed event `LspRequest(textDocument/inlayHint)` in 0.00s (1.92s waiting)
 INFO Language server processed event `LspResponse` in 0.00s (0.00s waiting)
 INFO Language server processed event `LspResponse` in 0.00s (0.00s waiting)
 INFO Language server processed event `LspResponse` in 0.00s (0.00s waiting)
 INFO Handling non-canceled request textDocument/codeAction (29)
 INFO Language server processed event `LspRequest(textDocument/codeAction)` in 0.00s (0.00s waiting)
 INFO Handling non-canceled request textDocument/inlayHint (30)
 INFO Language server processed event `LspRequest(textDocument/inlayHint)` in 0.00s (0.00s waiting)
 INFO Handling non-canceled request textDocument/hover (31)
 INFO Language server processed event `LspRequest(textDocument/hover)` in 0.00s (0.00s waiting)
 INFO Handling non-canceled request textDocument/semanticTokens/full (32)
 INFO Skipping request - textDocument/semanticTokens/full service disabled
 INFO Language server processed event `LspRequest(textDocument/semanticTokens/full)` in 0.00s (0.00s waiting)
 INFO Prepare to check 526 files.
 INFO Handling non-canceled request textDocument/inlayHint (33)
 INFO Language server processed event `LspRequest(textDocument/inlayHint)` in 0.00s (0.00s waiting)
 INFO Handling non-canceled request textDocument/hover (34)
 INFO Language server processed event `LspRequest(textDocument/hover)` in 0.01s (0.00s waiting)
 INFO Handling non-canceled request textDocument/hover (35)
 INFO Language server processed event `LspRequest(textDocument/hover)` in 0.00s (0.00s waiting)
 INFO Handling non-canceled request textDocument/hover (36)
 INFO Language server processed event `LspRequest(textDocument/hover)` in 0.00s (0.00s waiting)
 INFO Handling non-canceled request textDocument/codeAction (37)
 INFO Language server processed event `LspRequest(textDocument/codeAction)` in 0.00s (0.00s waiting)
 INFO Handling non-canceled request textDocument/hover (38)
 INFO Language server processed event `LspRequest(textDocument/hover)` in 0.00s (0.00s waiting)
 INFO Handling non-canceled request textDocument/codeAction (39)
 INFO Language server processed event `LspRequest(textDocument/codeAction)` in 0.00s (0.00s waiting)
 INFO Handling non-canceled request textDocument/hover (40)
 INFO Language server processed event `LspRequest(textDocument/hover)` in 0.00s (0.00s waiting)
 INFO Handling non-canceled request textDocument/codeAction (41)
 INFO Language server processed event `LspRequest(textDocument/codeAction)` in 0.00s (0.00s waiting)
 INFO Handling non-canceled request textDocument/hover (42)
 INFO Language server processed event `LspRequest(textDocument/hover)` in 0.00s (0.00s waiting)
 INFO Handling non-canceled request textDocument/codeAction (43)
 INFO Language server processed event `LspRequest(textDocument/codeAction)` in 0.00s (0.00s waiting)
 INFO Language server processed event `DidCloseTextDocument` in 0.00s (0.00s waiting)
 INFO File BIG FILE opened, prepare to validate open files.
 INFO Validated open files and saved non-committable transaction.
 INFO Ran task on sourcedb_queue heavy task queue. Queue time: 0.00, task time: 0.00
 INFO Language server processed event `DidOpenTextDocument` in 4.56s (0.00s waiting)
 INFO We should cancel request RequestId(I32(44))
 INFO Language server processed event `CancelRequest` in 0.00s (4.45s waiting)
 INFO We should cancel request RequestId(I32(48))
 INFO Language server processed event `CancelRequest` in 0.00s (4.06s waiting)
 INFO We should cancel request RequestId(I32(53))
 INFO Language server processed event `CancelRequest` in 0.00s (0.04s waiting)
 INFO Request textDocument/codeAction (44) is canceled due to explicit cancellation
 INFO Language server processed event `LspRequest(textDocument/codeAction)` in 0.00s (4.56s waiting)
 INFO Handling non-canceled request pyrefly/textDocument/typeErrorDisplayStatus (45)
 INFO Language server processed event `LspRequest(pyrefly/textDocument/typeErrorDisplayStatus)` in 0.00s (4.56s waiting)
 INFO Handling non-canceled request textDocument/documentSymbol (46)
 INFO Populated all files in the project path, prepare to recheck open files.
 INFO Ran task on recheck_queue heavy task queue. Queue time: 0.00, task time: 31.83
 INFO Populating up to 2000 files in the workspace ("WORKSPACE").
 INFO Language server processed event `LspRequest(textDocument/documentSymbol)` in 0.14s (4.56s waiting)
 INFO Validated open files and saved non-committable transaction.
 INFO Language server processed event `RecheckFinished` in 0.02s (0.01s waiting)
 INFO Request textDocument/semanticTokens/full (47) is canceled due to subsequent mutation
 INFO Language server processed event `LspRequest(textDocument/semanticTokens/full)` in 0.00s (4.72s waiting)
 INFO Request textDocument/hover (48) is canceled due to explicit cancellation
 INFO Language server processed event `LspRequest(textDocument/hover)` in 0.00s (4.62s waiting)
 INFO Request textDocument/codeAction (49) is canceled due to subsequent mutation
 INFO Ran task on sourcedb_queue heavy task queue. Queue time: 0.00, task time: 0.00
 INFO Language server processed event `LspRequest(textDocument/codeAction)` in 0.00s (4.61s waiting)
 INFO Request textDocument/documentSymbol (50) is canceled due to subsequent mutation
 INFO Language server processed event `LspRequest(textDocument/documentSymbol)` in 0.00s (4.56s waiting)
 INFO Request textDocument/semanticTokens/range (51) is canceled due to subsequent mutation
 INFO Language server processed event `LspRequest(textDocument/semanticTokens/range)` in 0.00s (4.36s waiting)
 INFO Request textDocument/inlayHint (52) is canceled due to subsequent mutation
 INFO Language server processed event `LspRequest(textDocument/inlayHint)` in 0.00s (4.23s waiting)
 INFO Request textDocument/hover (53) is canceled due to explicit cancellation
 INFO Language server processed event `LspRequest(textDocument/hover)` in 0.00s (3.80s waiting)
 INFO Language server processed event `LspResponse` in 0.00s (0.14s waiting)
 INFO Language server processed event `LspResponse` in 0.00s (0.13s waiting)
 INFO Language server processed event `LspResponse` in 0.00s (0.13s waiting)
 INFO Handling non-canceled request textDocument/hover (54)
 INFO Language server processed event `LspRequest(textDocument/hover)` in 0.00s (0.04s waiting)
 INFO Language server processed event `LspResponse` in 0.00s (0.00s waiting)
 INFO Handling non-canceled request textDocument/codeAction (55)
 INFO Language server processed event `LspRequest(textDocument/codeAction)` in 0.00s (0.00s waiting)
 INFO Handling non-canceled request textDocument/codeAction (56)
 INFO Language server processed event `LspRequest(textDocument/codeAction)` in 0.00s (0.00s waiting)
 INFO Handling non-canceled request textDocument/semanticTokens/full (57)
 INFO Skipping request - textDocument/semanticTokens/full service disabled
 INFO Language server processed event `LspRequest(textDocument/semanticTokens/full)` in 0.00s (0.00s waiting)
 INFO Handling non-canceled request textDocument/hover (58)
 INFO Language server processed event `LspRequest(textDocument/hover)` in 0.00s (0.00s waiting)
 INFO Handling non-canceled request textDocument/hover (59)
 INFO Language server processed event `LspRequest(textDocument/hover)` in 0.01s (0.00s waiting)
 INFO Handling non-canceled request textDocument/inlayHint (60)
 INFO Language server processed event `LspRequest(textDocument/inlayHint)` in 0.00s (0.00s waiting)
 INFO Handling non-canceled request textDocument/hover (61)
 INFO Language server processed event `LspRequest(textDocument/hover)` in 0.01s (0.00s waiting)
 INFO Handling non-canceled request textDocument/codeAction (62)
 INFO Language server processed event `LspRequest(textDocument/codeAction)` in 0.00s (0.00s waiting)
 INFO Handling non-canceled request textDocument/hover (63)
 INFO Language server processed event `LspRequest(textDocument/hover)` in 0.00s (0.00s waiting)
 INFO Prepare to check 3 files.
 INFO Populated all files in the workspace, prepare to recheck open files.
 INFO Ran task on recheck_queue heavy task queue. Queue time: 31.83, task time: 8.14

After this point, the extension hangs and eventually crashes. Unfortunately the stack was still corrupt. I find it's always corrupted :(