Massive memory use on opening opencode when there is a very large file

[TimAidley]

Before trying out opencode, I had been experimenting which chunkhound, that indexes your source folder to allow searching later. It stores its database file(s) in the .chunkhound folder, and in my project that folder was around 27GB in size.

It seems like opencode was trying to load the files in that folder, because when I ran opencode the memory usage would shoot up, and opencode was using 29GB of RAM, and my computer was getting very slow.

Deleting that folder solved the problem. Unfortunately I didn't check what the sizes and filetypes of files in that folder were before I deleted it.

[github-actions]

This issue might be a duplicate of existing issues. Please check:

• Hangs if there is a multi gb .tar.gz file in current directory #683: Hangs if there is a multi gb .tar.gz file in current directory - similar issue with large files causing hangs
• Session grew to 88G! #1126: Session grew to 88G! - another case of extreme memory usage
• Running opencode causes a git diff that consumes all memory #2262: Running opencode causes a git diff that consumes all memory - similar memory consumption issue resolved by removing large files
• URGENT - "MEM LEAK", eats up gigabytes of ram until system crash #804: URGENT - "MEM LEAK", eats up gigabytes of ram until system crash - memory leak issue with large files

Feel free to ignore if none of these address your specific case.

[rekram1-node]

good to know thx for raising

[zelenkovsky]

In my case, it's 72GB of memory that got committed right after start (for other repository it's 93GB):

[rekram1-node]

@zelenkovsky can you tell me more about where you are running opencode is this a massive project or something?

[zelenkovsky]

@rekram1-node, aha, very interesting thing. If my working folder has a backup subfolder with large .tar.gz files, the opencode executable starts to grow in size (virtual memory) and gets killed by the OOM Killer.

Is there anything special in backup or tar.gz handling?

That happens on application startup. I did not even enter anything.

[zelenkovsky]

Actually, backup has nothing to do with it; it's large tar.gz files located anywhere in the project folder.

[zelenkovsky]

Analysis Summary

I've identified the root cause of the high virtual memory consumption when large .tar.gz files are present in your working directory. Here's what happens during opencode startup:

Primary Issue: File Tree Generation at Startup

1. Location: packages/opencode/src/session/system.ts:47-50
2. Function: SystemPrompt.environment() calls Ripgrep.tree() during initialization
3. Process: This scans ALL files in your project directory to build a project tree structure that gets included in the system prompt

The Problem Chain:

1. File Scanning: Ripgrep.tree() → Ripgrep.files() → ripgrep binary with --files flag
2. No Archive Filtering: The default ignore patterns in FileIgnore don't include common archive extensions like *.tar.gz, *.zip, *.tar, etc.
3. Memory Allocation: When ripgrep encounters large files (even if it's just listing them), it can cause memory allocation issues, especially with compressed archives
4. Startup Execution: This happens before any conversation starts, during the system prompt preparation

Evidence from Code:

• src/session/system.ts:47-50: Calls Ripgrep.tree() with limit of 200 files during environment setup
• src/file/ignore.ts: Missing patterns for archive files (*.tar.gz, *.zip, *.tar, *.7z, etc.)
• src/file/ripgrep.ts:206-220: The files() function uses ripgrep's --files flag which scans the entire directory

Verification Checklist: ✅ Verified: File scanning occurs at startup via SystemPrompt.environment() ✅ Verified: No archive file patterns in default ignore list ✅ Verified: Ripgrep.files() scans
entire directory structure ❓ Inferred: Large archive files cause memory allocation during scanning

Next Steps to Confirm: You could verify this by temporarily moving your .tar.gz files out of the working directory and checking if the memory usage drops during opencode startup.

The fix would involve adding archive file patterns to the default ignore list in src/file/ignore.ts.

[ofriw]

Before trying out opencode, I had been experimenting which chunkhound, that indexes your source folder to allow searching later. It stores its database file(s) in the .chunkhound folder, and in my project that folder was around 27GB in size.

It seems like opencode was trying to load the files in that folder, because when I ran opencode the memory usage would shoot up, and opencode was using 29GB of RAM, and my computer was getting very slow.

Deleting that folder solved the problem. Unfortunately I didn't check what the sizes and filetypes of files in that folder were before I deleted it.

Author of ChunkHound here. As a workaround, you can place the db file outside of your code's dir using the --db flag. See https://chunkhound.github.io/configuration/#database-options

[IlyaMaksheev]

I also experience same issue with .parquet files.