Skip to content

fix: keep the forkserver preload shared with job processes (gc.freeze) - #7118

Closed
Darshak03 wants to merge 1 commit into
livekit:mainfrom
Darshak03:fix/forkserver-gc-freeze
Closed

fix: keep the forkserver preload shared with job processes (gc.freeze)#7118
Darshak03 wants to merge 1 commit into
livekit:mainfrom
Darshak03:fix/forkserver-gc-freeze

Conversation

@Darshak03

Copy link
Copy Markdown

Problem

On Linux the worker preloads the registered plugin packages plus
livekit.agents.inference._warmup into the forkserver process so job processes
inherit the imported modules and the warmed native models copy-on-write.

The sharing does not last. Every job process runs its own cyclic GC, and a full
collection rewrites the PyGC_Head (two pointers stored immediately before each
tracked object) of every object in the oldest generation — a write, even when
nothing is collectable. Preloaded modules, classes, functions and their dicts
are long-lived, so they all live there: the first full collection in a forked
child dirties nearly every page holding a preloaded object and the kernel copies
it. Measured on a Linux worker at ~96MB moving from shared to private per job
process, and it happens with no job traffic at all — an idle prewarmed process
already crosses a full collection during prewarm.

The native weights from init_vad() / init_eot() are allocated outside the
Python heap and stay shared; it is the Python object graph that is lost.

Fix

gc.freeze() in the forkserver, once everything is preloaded. Frozen objects go
to the permanent generation, which the collector never scans, so their headers
are never rewritten and the pages stay shared for the life of the job process.
They are still freed by refcounting if they die, and module-level state lives for
the whole process anyway, so nothing is retained that would not have been.

It has to run inside the forkserver — job processes fork from there, so freezing
in the worker would not affect them. This reuses the mechanism already in the
tree for that: a side-effect module in the preload list, like
livekit.agents.inference._warmup. The new module is listed last so everything
above it is resident by the time it runs.

The module deliberately imports nothing but gc. The forkserver's preload loop
swallows ImportError, so putting the call at the end of _warmup.py would
silently skip the freeze on any host where livekit-local-inference fails to
import, even though the plugin preload — and the problem — is still there.

Also drops ~230k objects from every full collection in the job process, removing
a recurring GC pause from the audio path.

Verification

A child forked from a forkserver that preloads the new module reports
gc.get_freeze_count() == 233838, i.e. the freeze survives the fork and covers
the whole preloaded graph.

To see the memory effect on a Linux worker, sample a job process right after it
is forked and again after a full collection:

grep -E '^(Shared_Clean|Private_Dirty)' /proc/<job-pid>/smaps_rollup

Shared_Clean drops and Private_Dirty rises by the same amount without this
change; with it, both hold.

Notes

  • No behaviour change on spawn (macOS/Windows) — there is no preload there.
  • Alternatives rejected: gc.freeze() in the worker (wrong process), and
    gc.disable() / raised thresholds in the job process (leaks real cycles for
    the lifetime of a long session).

Fixes #7117

The forkserver preloads plugin packages and the warmed native models so
forked job processes inherit them copy-on-write, but the first full GC in
a job process rewrites the GC header of every preloaded object and copies
nearly every page holding one, undoing the sharing (~96MB per process).

Freeze the preloaded objects into the permanent generation from a
side-effect module listed last in the preload list, so the collector in
each job process never touches them.
@CLAassistant

Copy link
Copy Markdown

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Devin Review: No Issues Found

Devin Review analyzed this PR and found no bugs or issues to report.

Devin Review

@Darshak03

Copy link
Copy Markdown
Author

Hi @chenghao-mou,
Could you please help me understand why my changes haven’t been merged? Is there any specific format or guideline I need to follow? I’ve also signed the CLA.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

forkserver preload's COW sharing is destroyed by the GC in each job process (~96MB per idle worker)

3 participants