5.23.12 - Persistent reader window bound

Patch release. The memory side of #220, reported and field-verified by rrgomes over four days of instrumented runs.

Fixed

A link only moderately faster than the content grew the reader's window without bound. (#220)

The persistent reader applies backpressure by suspending the URLSession task above a 16 MB high water. suspend() is advisory: CFNetwork keeps draining the socket and delivering to the delegate, and whether that costs anything depends entirely on the link.

Against a fast origin the socket buffer fills, TCP throttles the sender, and the suspend is never asked to hold anything. Every measurement looks correct, including this project's own regression test for the mechanism, which stalls a consumer completely against a 50 MB/s origin and therefore never leaves the regime where TCP does the work.

Against an origin delivering a moderate multiple of media rate the socket never fills and every byte is accepted while the task is flagged suspended. Measured at 911 MB post-suspend on a single reader with the window climbing linearly, and over 3 GB across both readers on a real 4K remux, which took the host machine down. Slower links are worse than faster ones, which is the opposite of what everyone involved assumed.

The resident window is now bounded at 48 MB, three times the high water. Past it the connection is ended deliberately and re-requested at the frontier once the consumer has drained it, so no delivered byte is discarded or re-fetched. Two details this depends on: the end is not charged to the unproductive-reconnect streak, which exists to detect a dead source and would otherwise kill the reader on a healthy link, and it does not take the reconnect-at-read-position fast path, which resets the window and would re-fetch the whole bound every cycle.

The bound is sized against the peak rather than the window. Crossing it is a Data realloc that holds both buffers at once, so the peak lands at roughly twice the bound: a field capture with an earlier 128 MB value in place still reached 1003 MB of live allocation, with the census naming two blocks at 153 and 129 MB.

Both readers of a subtitled source were affected, and the native path as much as the software one, since a direct-play source runs the HLS loopback and demuxes from the origin itself. Only the reader that parks ever exercises the defect: a reader that is running is its own consumer, so resident bytes are delivery rate multiplied by park duration rather than delivery rate alone. Healthy sessions sit at 16-21 MB on macOS and tvOS alike and never approach the bound.

The malloc census aborted the process it was measuring.

Its recorder runs inside malloc_zone's in-use enumerator with every zone held through force_lock, so it must not allocate. for i in 0..<Int(count) iterates a Range through IndexingIterator's protocol witness when the call is not specialized, that allocates, and allocating there takes the same os_unfair_lock recursively: libplatform aborts with "Trying to recursively lock an os_unfair_lock". Optimized builds specialize the range away, so release builds were fine while debug builds died on the first census.

Added

Per-reader window diagnostics in the periodic memory probe. pumpWinMB / prefWinMB for the resident window, AheadMB for the undrained forward extent that the suspend gates on, Susp, and PostMB for bytes the delegate accepted while the task was already flagged suspended, reported separately for the playback reader and the subtitle side reader on both paths. PostMB is the field that separates backpressure which never engaged from backpressure which engaged and was ignored; the suspend flag alone reads identically in the healthy and the failing case.

A prefetch gauge in the same line. prefetch= reports the subtitle forward prefetcher's state and, once stopped, why: eof, failed, openfail or cancelled. Alongside it prefetchLead, prefetchHarvested and prefetchTbFallback. The reader works a full lead ahead of the playhead, so it reaches EOF before playback ends, and a bare "stopped" state fired on every completed session.

Scripts/throttle-origin.py, a range-preserving throttling proxy that puts a chosen link shape in front of a real server, in single-URL or whole-server form, logging per-connection byte totals on close. A defect that only appears at a moderate multiple of media rate cannot be reached by repeating runs on a fast link, and a connection measured above media rate is itself the diagnostic: a reader whose consumer is draining it runs at media rate, because the consumer sets the pace.

Notes

This is a bound, not the correction. The reader asks for bytes=X-, the whole rest of the file, and then tries to brake with a mechanism that cannot brake. Bounded range requests would make the overshoot structurally impossible, but they need one URLSession to outlive a single connection, which is its own change to the network path.

Thanks to rrgomes for the server-side per-connection accounting that made the mechanism visible, for the census capture that named the allocation, and for retracting three of his own hypotheses on evidence along the way.