fix(io): live-correct the persistent reader - window, reconnect shape, and 509 classification - #332
Merged
superuser404notfound merged 3 commits intoAug 8, 2026
Conversation
… end-drain-reconnect The 16 MB high-water end (superuser404notfound#310) had no live branch, and live connections are open-ended by design, so ending at high water was the only thing that ever terminated a healthy live connection. Each end drained ~8 MB to low water and re-requested "at the frontier" — a byte offset that means nothing to a live origin — so everything broadcast during the drain was lost and the demuxer rejoined on a corrupt TS packet. And it never happened once: IPTV panels serve their ring buffer as a join burst at line rate on every (re)connect, so the burst refilled the window immediately and each reconnect caused the next one, forever. A field trace against an Xtream panel cycled every ~9.5 MB with a mpegts "Packet corrupt" and an h264 decode error per cycle; the loopback repro accepts 17 MB of a 24 MB burst, parks at 16.9 MB and holds no connection. - Live readers run a 64 MB high water (matching streamHighWater, the bound already accepted for the other reader that cannot bound by range request): the join burst is absorbed once and steady state plateaus at burst size — arrival rate equals media rate once the burst is over — with the connection never voluntarily ended. - The end-and-refill mechanism is unchanged and survives as the memory backstop for a "live" source that sustainedly outruns realtime (a misdeclared VOD). No suspend, no parked task: the superuser404notfound#310 contract (actively delivering flow or no flow) holds verbatim. - The high water is an init parameter for the same reason connStallTimeout is one: a process-wide test hook would leak into concurrently running suites. Test plan: swift test on macOS 26 (1564 tests green, including a new live-window suite against the loopback origin: a 24 MB serve-then-silent burst must be accepted on one open-ended connection with parked=false, and a shrunken backstop must still end, stay bounded, and refill at the frontier on drain). Both new cases verified failing on the unfixed base. Field shape from an Apple TV 4K (tvOS 26) playing 1080i H.264 live TS from an Xtream aggregator with max_connections=1 and 302->http redirects. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…adder (superuser404notfound#307 follow-up) 509 "Bandwidth Limit Exceeded" is what a connection-capped IPTV panel answers while the slot the reader is replacing has not been torn down server-side yet. It classified as a hard 5xx, so every attempt dropped the pinned post-redirect URL and re-resolved through the portal: latency per attempt, plus the second request against the very origin that has no room for it, which is the 519ae26 reasoning left incomplete. And the hammering never paced itself: ~8 MB of progress per cycle reset the unproductive streak every time, so a permanent 509 ground through 13 attempts with 12 portal re-resolves at zero backoff on the loopback repro before the unproductive cap ended it. - One classifier, isRateLimitStatus (429/503/509), used by the pin-drop exemption, both reconnect ladders, the detour fetch, and the Retry-After parse - honoured on 509 when a panel sends it. - A lingering-slot 509 now keeps the pin and retries in place; a permanent 509 pays the rate-limit streak and its bounded give-up (7 attempts, paced) instead of the hard-5xx grind. Test plan: swift test on macOS 26 (1564 tests green; new cases in the resolved-URL suite: a two-attempt 509 keeps the pin end to end, a permanent 509 gives up at the rate-limit cap without ever re-resolving through the source, and the classifier assertions cover 509 against 500/502/504). Both verified failing on the unfixed base. Field shape from an Apple TV 4K (tvOS 26) against an Xtream aggregator with max_connections=1 whose external portal answers 509 while the replaced connection's slot lingers. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…yte frontier The reconnect request carried Range: bytes=<frontier>-, but the frontier is reader bookkeeping - the window position delivered bytes are appended at - not a server-side byte address, because a live origin has none. Panels that ignore the offset and serve "from now" masked this; a panel that answers 416 to every offset it cannot satisfy turned each reconnect into an unrecoverable rejection loop. Field trace (Apple TV 4K, tvOS 26, Xtream panel): the origin cleanly completes every response after its ~14 MB ring burst, then 416'd the same frontier offset 35 generations in a row at ~1/s - the read-progress streak reset kept the ladder at streak=1 with 1 s backoff while the runway drained from 8 MB to zero, the re-resolve at streak=2 changed nothing (the offset was the problem, not the pin), and the session starved into an AVPlayer -12888. - Live persistent connections now always request `bytes=0-` - the one shape every origin serves, and the shape the join already uses. The append anchors delivered bytes at the window frontier exactly as it always has (the persistent path appends sequentially and never re-anchors on Content-Range), so the logical bookkeeping is unchanged. - VOD requests are untouched: bounded ranges and frontier offsets remain correct where byte addresses are real. Test plan: swift test on macOS 26 (1565 tests green). New case in the live-window suite: an origin that completes every response after a 4 MB burst and rejects any nonzero offset with 416 must keep delivering across reconnect cycles with every request at bytes=0-. On the unfixed code the case reproduces the field failure verbatim: exactly one burst delivered, then the rejection loop starves the read. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
tschuegy
force-pushed
the
fix/live-window-highwater-disconnect
branch
from
August 8, 2026 05:16
8269089 to
ef350fc
Compare
superuser404notfound
merged commit Aug 8, 2026
1187bd7
into
superuser404notfound:main
3 checks passed
Owner
|
Merged as 1187bd7. Thanks for this one, the writeup did most of the review work: three separate defects, each with the field trace that produced it and a loopback test that reproduces it. Verified here before merging, by disabling each guard in turn rather than trusting the green run:
Full suite green (1623 tests) on the merge result. One follow-up pushed in #336, because the live request shape needs one more distinction. #331 stays open until the retest on device. |
pull Bot
pushed a commit
to RadicalMuffinMan/AetherEngine
that referenced
this pull request
Aug 8, 2026
superuser404notfound#331 follow-up) A live reconnect asked `bytes=<frontier>-` and starved against panels that answer 416 to any offset they cannot satisfy. superuser404notfound#332 made every live request `bytes=0-`, which fixes those panels but is wrong for the other live shape: a source that IS a growing file (a Jellyfin live stream file, a misdeclared VOD) answers the frontier correctly and resumes there, and asking it for byte zero re-delivers its whole buffer on top of the window. The rejection is the signal. Keep the frontier until an origin answers 416 to a nonzero live offset, latch the join shape from that point, and name the latch in the log so a field capture separates "this origin has no byte addresses" from "the reconnect is broken". A ring-buffer panel pays exactly one rejected request per reader; a byte-addressable live source pays nothing and keeps resuming. Both directions witnessed by disabling the guard: never latching starves after one 4 MB burst (13 rejected requests at the same offset, the field trace), latching unconditionally restarts a range-honouring live origin at byte zero. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HX5zV3Fcf7Nzq4DYGdXvQE
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
A live source was stuttered on a steady ~30 s cycle by the reader's own 16 MB high-water end, starved entirely against a panel that answers 416 to frontier offsets, and paid a portal re-resolve for every 509. Three tightly-coupled fixes in
AVIOReader, one per commit.Reported in #331 (kept open until the reporter retests on device)
What changed
streamHighWaterparity), instance-captured likeconnStallTimeout. The panel's join burst is absorbed once, steady state plateaus at burst size with the connection never voluntarily ended, and the end-and-refill survives unchanged as the memory backstop for a "live" source that sustainedly outruns realtime. No suspend, no parked task - the Long-suspended data tasks holding full receive windows starve every Network.framework flow in the app process on tvOS — close-and-rerequest at high water would avoid the dormant state (field dose-response inside) #310 contract (actively delivering flow or no flow) holds verbatim.bytes=0-(the shape the join already uses and every origin serves); the append anchors delivered bytes at the window frontier exactly as it always has. VOD bounded ranges and frontier offsets are untouched.isRateLimitStatus: 429/503/509) used by the pin-drop exemption, both reconnect ladders, the detour fetch, and the Retry-After parse. A lingering-slot 509 keeps the pinned redirect target (completing 519ae26) and pays the paced rate-limit streak with its bounded give-up instead of the zero-backoff hard-5xx grind.Test plan
swift test(1565 tests green)Packet corrupt+ h264 decode error per ~9.5 MB is gone - one connection,pumpAheadMBplateaus at burst size, no periodic 509s, 10+ min clean on device. Panel B (416s nonzero offsets, cleanly completes each response after its ~14 MB ring burst): previously starved into CoreMedia -12888 within a minute; now reconnects cleanly through every burst cycle. New loopback tests pin all three behaviors (LiveWindowBackpressureTests+ 509 cases inResolvedURLInvalidationTests); each was verified failing on the unfixed base, reproducing the field traces verbatim (16.9 MB park mid-burst; one 4 MB burst then starvation; 13 attempts with 12 portal re-resolves on a permanent 509).Checklist
CHANGELOG.mdupdatedfeat(...),fix(...),chore(...))AVIOReaderis internal, the newwindowHighWaterinit parameter is internal test plumbing likeconnStallTimeout)