Skip to content

5.28.2 - Reader autorelease retention

Choose a tag to compare

@superuser404notfound superuser404notfound released this 29 Jul 07:19
7d6e6cf

One fix, reported and correctly diagnosed by bitxeno: a remote Blu-ray ISO grew memory by roughly 30 MB/s for the whole session until the process was jetsammed. Verifying the diagnosis turned up the same defect on the local and SMB readers, and retired a piece of engine folklore about URLSession.

Fixed

  • A disc image leaked every byte it read. HTTPDiscIOReader is pull-based: FFmpeg's read callback runs each range GET synchronously on the demux pump thread, and that thread spends the whole session inside one dispatch block, so nothing ever drains its autorelease pool. Every response body bridged out of the completion handler was stranded there until playback ended: up to 8 MB per request at the top of the adaptive window, several requests a second, once per reader fork (main demuxer, subtitle side demuxer, forward prefetcher). RSS looks flat while this happens, because the leaked pages are never touched again and go to the compressor, which is why vmCmp and physFP are the tell and rss is not. FileIOReader had the same defect on the local path (FileHandle.read hands back an NSData-backed Data on that same thread, one leaked byte per byte played), and SMBIOReader copies out of a dispatch_data-backed body there too. All three now drain per read.

Measured against a local range origin, 480 MB fetched in 8 MB requests:

variant malloc in use
shared session, completion handler, no pool +968 MB
session per request + invalidate, no pool +973 MB
shared session, completion handler, pool per request 0 MB
shared session, delegate-based delivery, no pool 0 MB

The second row is the folklore. The AVIOReader leak of 5.x was recorded as URLSession retaining completed completion-handler bodies inside its task pool until invalidation, and both the code comments and the fix history were written around that reading. A session per request leaks exactly as much as a shared one, so the session was never the owner: it is the never-draining autorelease pool of the thread that bridges the body out. Delegate-based delivery fixed that path because it releases the body on the delegate queue, where it never reaches the demux thread's pool at all. The comments on chunkSession and persistentSession now say so.

The local reader shows the same arithmetic without a network in the way: 624 MB read in 32 KB chunks left 625 MB in use before the fix, and 0 MB after.

Added

  • discFetchedMB in the 30 s memprobe. The disc pull path had no byte counter of any kind (avioFetchedMB covers the AVIOReader path only), so a disc-image session reports every engine-tracked pool flat while its reader forks pull tens of MB/s off the link, and this report had to be argued from arithmetic instead of a counter. It is printed only when that path is in use and is session-scoped like t=, so two probe lines give a rate.

Tests

Issue243ReaderRetentionTests reads 128 MB through each reader inside a pool the test owns and asserts that draining it frees close to nothing. Reverting either wrapper frees 134.4 MB and 135.1 MB there. Measuring the drain rather than the loop keeps the check usable in a parallel suite: concurrent allocation can only shrink the observed drop, never inflate it.

Reported by bitxeno (#243), whose report pinned the site, the mechanism and the fix, and read the compressor behaviour correctly off the memprobe lines.

No API changes.