Pyrefly VS Code Extension uses 100GB of memory

[kootsoop]

Describe the Bug

When loading the python project I'm working on, the Pyrefly process on my Mac (info below) eventually eats up 100 GB of memory slowing down my 32 GB RAM machine a little.

The log below only as 15 GB of memory loaded. I had to kill the pyrefly process because my system was unresponsive.

Mac

Model Name: Mac mini
Model Identifier: Macmini8,1
Processor Name: 6-Core Intel Core i7
Processor Speed: 3.2 GHz
Number of Processors: 1
Total Number of Cores: 6
L2 Cache (per Core): 256 KB
L3 Cache: 12 MB
Hyper-Threading Technology: Enabled
Memory: 32 GB
System Firmware Version: 2103.100.6.0.0 (iBridge: 23.16.14242.0.0,0)
OS Loader Version: 583~2317
Serial Number (system): C07CF2XHPJJ9
Hardware UUID: F7C27459-DF0B-5A37-9935-38756EBABA2B
Provisioning UDID: F7C27459-DF0B-5A37-9935-38756EBABA2B
Activation Lock Status: Enabled

Sandbox Link

No response

(Only applicable for extension issues) IDE Information

Pyrefly:

Installation
Identifier
meta.pyrefly
Version
0.57.1
Last Updated
2 weeks ago
Size
26.56MB

VS Code

Version: 1.113.0
Commit: cfbea10c5ffb233ea9177d34726e6056e89913dc
Date: 2026-03-24T15:07:18+01:00
Electron: 39.8.3
ElectronBuildId: 13620978
Chromium: 142.0.7444.265
Node.js: 22.22.1
V8: 14.2.231.22-electron.0
OS: Darwin x64 24.6.0

Pyrefly log

starting generic LSP server
INFO Reading messages
WARN PYTHONPATH environment variable is set to /Users/kootsoop/git/suheel/ARI_search/python. Checks in other environments may not include these paths.
INFO File /Volumes/Macintosh HD 1/Users/kootsoop/git/suheel/ARI_search/python/common_crawl/collection_two/04_download_content.py opened, prepare to validate open files.
INFO Ran task on sourcedb_queue heavy task queue. Queue time: 0.00, task time: 0.00
INFO Published 0 diagnostics for file:///Volumes/Macintosh%20HD%201/Users/kootsoop/git/suheel/ARI_search/python/common_crawl/collection_two/04_download_content.py
INFO Validated open files and committed transaction.
INFO Language server processed event DidOpenTextDocument in 0.27s (0.00s waiting)
INFO Language server processed event LspResponse in 0.00s (0.26s waiting)
WARN PYTHONPATH environment variable is set to /Users/kootsoop/git/suheel/ARI_search/python. Checks in other environments may not include these paths.
INFO Client configuration applied to workspace: Some(Url { scheme: "file", cannot_be_a_base: false, username: "", password: None, host: None, port: None, path: "/Volumes/Macintosh%20HD%201/Users/kootsoop/git/suheel/ARI_search", query: None, fragment: None })
INFO Client configuration applied to workspace: None
INFO Language server processed event LspResponse in 0.05s (0.26s waiting)
INFO Language server processed event LspResponse in 0.00s (0.03s waiting)
INFO Language server processed event LspResponse in 0.00s (0.03s waiting)
INFO Invalidated config, prepare to recheck open files.
INFO Ran task on recheck_queue heavy task queue. Queue time: 0.00, task time: 0.01
INFO Published 0 diagnostics for file:///Volumes/Macintosh%20HD%201/Users/kootsoop/git/suheel/ARI_search/python/ari.pyi
INFO Ran task on recheck_queue heavy task queue. Queue time: 0.01, task time: 0.00
INFO Ran task on recheck_queue heavy task queue. Queue time: 0.01, task time: 0.00
INFO Populating up to 2000 files in the workspace ("/Volumes/Macintosh HD 1/Users/kootsoop/git/suheel/ARI_search").
INFO Published 0 diagnostics for file:///Volumes/Macintosh%20HD%201/Users/kootsoop/git/suheel/ARI_search/python/common_crawl/collection_two/04_download_content.py
INFO Validated open files and committed transaction.
INFO Language server processed event RecheckFinished in 0.00s (0.00s waiting)
INFO Language server processed event LspResponse in 0.00s (0.00s waiting)
INFO Prepare to check 159 files.

[yangdanny97]

Sounds like a memory leak of some sort - are you able to share the project/codebase you're working on?

[NathanTempest]

159 files should NOT take up 100 GB of space, the likely culprits might be circular imports or infinite recursion which might have blown the memory up, large dependency trees via which PYTHONPATH might have pulled in vast libraries, or memory leaks in LSP.

Can you help us with these questions to help narrow the culprit down -

1. Dependencies: What are the main libraries your project uses? (e.g., torch, pandas, numpy, django, etc.)

2. PYTHONPATH: Your log shows PYTHONPATH=/Users/kootsoop/git/suheel/ARI_search/python. Does this directory contain third-party packages, or just your project code?

3. Reproducibility: Does this happen every time you open the project, or intermittently?

4. Specific file: Does the memory spike happen when opening any file, or a specific one? (04_download_content.py is mentioned in the log)

5. Workaround test: Can you try creating a pyrefly.toml in your project root with:

   replace-imports-with-any = ["*"] 
   

This will skip resolving all imports. If memory stays low, it confirms the issue is in dependency resolution.

[NathanTempest]

Hey @kootsoop did our fixes help you identify the memory leaks or reduce the memory usage ? For more detailed bug handling or identifying if Pyrefly does have a memory leak, we would need more info regarding your local libraries that are in your project or details regarding your codebase.

[jeffkeller-einc]

I had this issue and I narrowed it down to imports from a local namespace package.

py
├── file1.py
└── file2.py

I have imports that look like:

import py.file1 as file1
from py.file2 import my_variable

I added an __init__.py file to the py/ directory and Pyrefly's memory usage dropped to an acceptable level.

[connernilsen]

Interesting, I bet we were finding a py directory somewhere else that had tons of imports. We can try to track it down and see why a directory like that ended up blowing up memory so much. My guess is that the py directory that was found had a really expensive module in it.

Can you run pyrefly dump-config from wherever your Pyrefly config/pyproject.toml is and paste your results here? If you don't have a config, you can run it from the import root of your project.

[jeffkeller-einc]

Sure thing. I couldn't find a flag to uncollapse the 54 additional covered files (if you think that list might contain clues).

# ~/.positron/extensions/meta.pyrefly-0.60.1-darwin-arm64/bin/pyrefly dump-config   
 INFO Checking current directory with default configuration
Default configuration
  Using interpreter: /Users/jeffkeller/git/data-science/infrastructure/aws_cdk/.venv/bin/python3
  Covered files:
    /Users/jeffkeller/git/data-science/infrastructure/aws_cdk/app.py
    /Users/jeffkeller/git/data-science/infrastructure/aws_cdk/app_bootstrap.py
    /Users/jeffkeller/git/data-science/infrastructure/aws_cdk/bootstrap/chatops.py
    /Users/jeffkeller/git/data-science/infrastructure/aws_cdk/bootstrap/tags.py
    /Users/jeffkeller/git/data-science/infrastructure/aws_cdk/py/__init__.py
    /Users/jeffkeller/git/data-science/infrastructure/aws_cdk/py/container_images.py
    /Users/jeffkeller/git/data-science/infrastructure/aws_cdk/py/env_vars.py
    /Users/jeffkeller/git/data-science/infrastructure/aws_cdk/py/iam_policies.py
    /Users/jeffkeller/git/data-science/infrastructure/aws_cdk/py/tags.py
    /Users/jeffkeller/git/data-science/infrastructure/aws_cdk/stacks/api_services/cloudwatch.py
    ...and 54 more
  Resolving imports from:
    Import root (inferred from project layout): "/Users/jeffkeller/git/data-science/infrastructure/aws_cdk"
    Site package path queried from interpreter: ["/Users/jeffkeller/git/data-science/infrastructure/aws_cdk/.venv/lib/python3.12/site-packages"]