VOD cold-start pays a mandatory sequential head→tail→re-anchor round trip on non-fast-start MP4 sources — request for parallel fetch or moov-offset hint

[digilearn-dev]

What happened?

On every fresh VOD load against a non-fast-start MP4 source (moov atom at file end), the demuxer open sequence costs 1.2-2.8s of pure network round-trip time before frame rate / stream metadata is available, via three sequential connections:

gen=1: offset=0, len=32MB → real data connection (~1-2s)
gen=2: offset=, len=0MB → moov atom / tail probe (~130-750ms)
gen=3: offset=, len=32MB → re-anchor (~45-100ms)

This is confirmed consistent across multiple sessions and titles on our end. It's occurring before any codec/DisplayCriteria logic engages, so it's purely the cost of locating and reading the trailing moov atom.

I checked #69 and see this matches the documented non-fast-start MP4 behavior — the detour-block cache fix there collapsed what could otherwise be a reconnect storm down to this legitimate minimum sequence, but the sequential round-trip cost itself remains by design. I'm not reporting a regression from that fix; I'm asking whether there's room to reduce this legitimate minimum further, since for slower/higher-latency sources (debrid-style proxies in our case) this is now the single largest contributor to VOD time-to-first-frame — bigger than anything in the DisplayCriteria path (#274/#277).

Steps to reproduce

1. Load a VOD title from a non-fast-start remote MP4 source (moov atom at end of file) with meaningful network latency to the host (not a fast local/CDN path).
2. Observe AVIOReader logs: pump conn start gen=1 offset=0, followed by pump conn start gen=2 offset= len=0MB, followed by pump conn start gen=3 offset=<near 0>.
3. Measure elapsed time from open_started to the point frame rate/stream info becomes available — the three round trips dominate this window on higher-latency sources.

AetherEngine version or commit SHA

6.4.3

Host app

Custom / my own integration

Platform

tvOS

OS version

tvOS 26.6 (beta)

Device / chip

Apple TV 4K (A15)

Playback path

Native AVPlayer

Source media (for playback bugs)

Various VOD titles served via a remote HTTP source (debrid-style proxy) producing non-fast-start MP4 (moov atom at file end). Not codec/resolution-specific.

Error codes / log lines

[AVIOReader] pump conn start gen=1 offset=0 len=32MB
[AVIOReader] File size: 4502623452 bytes (data connection)
[AVIOReader] pump gen=1 first data after 1011ms
[AVIOReader] pump conn start gen=2 offset=4502622327 len=0MB
[AVIOReader] pump conn gen=1 ended with error: cancelled
[AVIOReader] pump gen=2 first data after 747ms
[AVIOReader] pump conn start gen=3 offset=5756 len=32MB
[AVIOReader] pump gen=3 first data after 103ms

No errors — this is expected, working behavior, just a real latency cost we're asking about.

Anything else

Checked the public LoadOptions surface (6.4.3) exhaustively before filing — confirmed there's no host-facing content-length hint, moov-offset hint, parallel head+tail fetch option, or way to influence this open sequence. probesize/maxAnalyzeDuration (#68) only bound find_stream_info after the demuxer is open; they don't affect the need to seek to a trailing moov. Also checked #70 (redundant size probe removal) and #126 (moov-at-end + unknown length) as related prior work — this report is a step beyond both, not a regression of either.

Possible directions, offered as starting points rather than a specific implementation ask:

Issue the tail probe (gen=2) concurrently with the initial data connection (gen=1) rather than sequentially, since they're logically independent reads.
A host-supplied content-length or last-known-moov-offset hint (keyed by URL) for sources the host already has size/duration metadata for via other means (e.g., from a provider API), letting the engine skip straight to the real read.
A durable moov-offset cache keyed by URL for repeat plays of the same file, if the in-process SourceContentLengthCache (#112) could be extended or paralleled for this purpose — understanding that's currently internal/enum-based and not something we're asking to be made public, just flagging it as prior art that might already be close to reusable internally.

Not a regression, not blocking — flagging as an opportunity, since this is now the largest single contributor to VOD cold-start time on our slower sources, larger than anything in the DisplayCriteria path.

[superuser404notfound]

Released as 6.4.5 (a2087a5, fix in 69b6e031). Thanks for the trace with the three generations and their first-data times, that is what made this diagnosable without access to the source.

One of the three round trips was ours, and it is gone. The report reads gen=2 and gen=3 as two costs of the layout. Only gen=2 is. seekReconnect resets the reader's window, so the seek to the trailing object discarded bytes that had been resident moments earlier, and gen=3 went back to re-fetch them. The window is now parked across a parse seek and serves that return as a copy, released once the demuxer's header pass is done (after which a far seek is a scrub, where the old window is worthless). That half costs no additional bytes anywhere.

gen=2 now overlaps gen=1 rather than queueing behind it. open() issues one speculative 64 KB suffix range. bytes=-n needs no size, which is the reason it can go out before the first response header exists, so it does not have to wait for the size resolution your gen=1 provides. It covers the small trailing objects an open reads: mfra on fragmented MP4, and a trailing moov whose sample tables fit.

Measured here against a Range-logging origin on a 51 MB moov-at-end file: three sequential requests become two concurrent ones, with the tail read and the return trip both absent.

Where it will not help, stated plainly. A moov grows with the sample count, so a feature-length file's runs into megabytes and will not fit in 64 KB. That request stays. Sizing the speculative fetch to cover it was considered and rejected: it would spend real bandwidth on a guess, competing with the first data connection on exactly the slow links this is meant to help. The parked window still removes the return trip in that case, so the worst case goes from three round trips to two rather than to one.

On your source specifically the tail sits about 1125 bytes from EOF, which fits the suffix window with room to spare, so both halves should apply. If the numbers say otherwise, the thing to capture is whether a bytes=-65536 request appears alongside the first one.

On the other two directions in the report. The durable per-URL moov-offset cache is a reasonable idea but a later one: it only pays on a repeat play of the same file, while this pays on every cold start, and the two do not conflict. The host-supplied moov-offset / content-length hint I would rather not add. The offset is not something a host generally knows (a provider API gives size and duration, not box layout), so the surface would be correct only for hosts that parse the container themselves, and a wrong hint costs a wasted fetch plus the round trip it was meant to replace.

Leaving this open until your retest.

[digilearn-dev]

Retested on 6.4.5 across several fresh cold VOD plays (MP4 sources, same file type as the original report). bytes=-65536 doesn't appear in any session — the connection pattern is unchanged from before the fix: fully sequential gen=1 → gen=2 (tail probe) → gen=3 (re-fetch of start), sometimes a 4th hop. This holds even on a fast-network session (gen=1 first data in 126ms), where the shape should be most visible if the fix were engaging — ruling out network variance as the explanation.

Three sessions, same show/source, different episodes:

Session A — ttff_ms=3577

[AVIOReader] pump conn start gen=1 offset=0 len=32MB
[AVIOReader] File size: 3399970254 bytes (data connection)
[AVIOReader] pump gen=1 first data after 1720ms
[AVIOReader] pump conn start gen=2 offset=3399969134 len=0MB
[AVIOReader] pump conn gen=1 ended with error: cancelled
[AVIOReader] pump gen=2 first data after 438ms
[AVIOReader] pump conn start gen=3 offset=265159 len=32MB
[AVIOReader] pump gen=3 first data after 66ms
[AVIOReader] pump conn start gen=4 offset=3399889966 len=0MB
[AVIOReader] pump conn gen=3 ended with error: cancelled
[AVIOReader] pump gen=4 first data after 121ms

Session B — ttff_ms=3428

[AVIOReader] pump conn start gen=1 offset=0 len=32MB
[AVIOReader] File size: 3652742915 bytes (data connection)
[AVIOReader] pump gen=1 first data after 1892ms
[AVIOReader] pump conn start gen=2 offset=3652741792 len=0MB
[AVIOReader] pump conn gen=1 ended with error: cancelled
[AVIOReader] pump gen=2 first data after 351ms
[AVIOReader] pump conn start gen=3 offset=262318 len=32MB
[AVIOReader] pump gen=3 first data after 66ms
[AVIOReader] pump conn start gen=4 offset=3652654288 len=0MB
[AVIOReader] pump conn gen=3 ended with error: cancelled
[AVIOReader] pump gen=4 first data after 91ms

Session C (fast network — shows the shape is unchanged even when latency isn't the bottleneck) — ttff_ms=614

[AVIOReader] pump conn start gen=1 offset=0 len=32MB
[AVIOReader] File size: 4502623452 bytes (data connection)
[AVIOReader] pump gen=1 first data after 126ms
[AVIOReader] pump conn start gen=2 offset=4502622327 len=0MB
[AVIOReader] pump conn gen=1 ended with error: cancelled
[AVIOReader] pump gen=2 first data after 56ms
[AVIOReader] pump conn start gen=3 offset=4502516117 len=0MB
[AVIOReader] pump gen=3 first data after 39ms
[AVIOReader] pump conn start gen=4 offset=5752 len=32MB
[AVIOReader] pump gen=4 first data after 39ms

Confirmed AetherEngine is pinned to 6.4.5 (Package.resolved), and the checkout does contain the tailPrefetchBytes/startTailPrefetch() code per our build. But at runtime, none of these sessions show the speculative suffix request firing, concurrently or otherwise. Happy to test any specific config or angle if there's something on our end worth checking first.

[superuser404notfound]

The retest is right, and the reason is worse than a partial fix: the speculative fetch could never have won its race. Released as 6.5.1 (edb0946, fix in 2530a33c).

Nothing on that end was worth checking. The three traces are exactly what 6.4.5 predicts once the timing is measured instead of assumed. The demuxer reaches the trailing object within microseconds of the data connection's first byte. The suffix fetch pays the same round trip plus a body, so on any origin whose first byte costs anything it is still on the wire at exactly the moment the read arrives, and since nothing waited on it, the reconnect happened anyway. It never once served the read it exists for. It only added a request.

Only a loopback origin, which answers before that race can be lost, made it look like it worked: six tests passed while the field trace stayed flat. The regression test now models an origin whose first byte costs something, and it fails against 6.4.5 with the same three-request shape reported here.

A read landing in the fetched range now waits for it. The bound is what a round trip against the origin was measured to cost, taken from the data connection's own time to first data, since both requests left at the same instant: past that the fetch is not about to land and a fresh connection is the better bet. Waiting up to that point can never cost more than the reconnect it replaces. An origin that declines suffix ranges reports a terminal outcome now, so the waiter wakes on the rejection rather than sitting out its budget.

The bytes=-65536 check was not a fair thing to ask for. The engine never printed that request, so its absence from a log was not evidence of anything, and this thread has now spent a round on that twice. 6.5.1 prints it:

[AVIOReader] pump tail prefetch issued bytes=-65536
[AVIOReader] pump tail prefetch installed 65536B at <offset> after <n>ms
[AVIOReader] pump tail prefetch served the read at <offset>; no reconnect for it
[AVIOReader] pump parked window served the return to <offset>; no reconnect for it

A declined fetch says how it was declined (status=416 (no suffix range support), a Content-Range that does not describe the 65536 B asked for, a short body). Slow reads gained tailWaits=.

What the traces say about these sources, since they are more informative than the original report. Two trailing objects are read per open, not one:

	first tail read	second tail read
A	EOF-1120	EOF-80288
B	EOF-1123	EOF-88627
C	EOF-1125	EOF-107335

Only the first fits a 64 KB suffix window. The second keeps its own request on every one of these files, and that is deliberate: a moov grows with the sample count, so sizing the speculative fetch to cover 107 KB here would mean spending real bandwidth on a guess, on exactly the slow links this exists to help. So the expectation for the next trace is four connections becoming two, not one: the data connection, and the deeper trailing object. gen=2 should be gone outright. The return to the head (offset 5752 in session C, 265159 in session A) is served from the parked window when the data connection had delivered that far by the time of the parse seek, which is near certain at 5752 and a coin flip at 265159 given that gen=1's first byte landed 1720 ms in.

Thanks for retesting on three separate sources and for including the fast-network session. That last one is what ruled out the obvious explanation and pointed at the shape rather than the link. Leaving this open until the next retest.

[superuser404notfound]

Follow-up, and it goes further than the fix in 6.5.1. Released as 6.5.2 (7691cef).

After 6.5.1 I took the remaining hop in these traces to aetherctl, the engine's CLI harness, against a Range-logging origin with real per-request latency, across six container layouts. It found three more defects, all the same shape as the one this issue started with: bytes already in hand were thrown away and fetched again.

• A range that finished delivering was fetched again. A completed range clears the active task exactly as a dropped one does, and the branch that reconnects when there is no connection ran at the read position regardless, dropping everything still resident. A parse pass is exactly the consumer that hurts: it reads one 256 KB AVIO buffer at a time while the transfer runs at line rate. A 764450 B trailing moov cost three connections and 1506918 delivered bytes, 1.97x its own size.
• A read at the end of the file opened a connection for it. The EOF decision sat below that reconnect, so bytes=<fileSize>- went out first and the empty 206 it came back with reset the window past the last byte.
• The parked window from 6.4.5 was cut from the wrong place. It cut from the reader's window start, on the reasoning that the demuxer returns there. It returns to the head of the FILE: landings of 48, 1161, 5752 in the fixtures and 265159 in the trace above. Those coincide only when the parse seeks away before reading anything. A fragmented MP4 reads 33 MB first, so the parked copy began megabytes past everything the return asked for and served nothing on any layout measured. The head is now retained as it arrives instead, and the park is gone.

What these sources are, as far as the matrix can say

layout	requests per open	what served the tail
faststart MP4	2	nothing
moov at end, 23337 B	2	the suffix fetch
moov at end, 764450 B	3	retained head; the moov is too big for the suffix
fragmented, mfra at end	3	suffix fetch (mfra) + retained head
MKV, Cues at end	2	nothing
MKV, Cues at front	2	nothing

The traces here match the fragmented row: a small trailing object about 1120 B from the end, which the suffix fetch covers, and a deeper read that does not.

That deeper read is a media fragment, not an index. In the fixture it sits at EOF-4739057; here at EOF-80288, EOF-88627 and EOF-107335. Its distance from the end scales with the fragment size, so there is no suffix window that covers the class, and sizing the speculative fetch toward it would spend real bandwidth on a guess. It keeps its own request, deliberately.

For the same reason the suffix fetch stays at 64 KB rather than growing, and there is now a number behind that: five runs per point at 250 KB/s with 300 ms of latency, it costs 252 ms on a layout that does not use it, exactly the link time of 64 KB, and saves 360 ms on one that does. Those are equal at about 2 Mbit/s, which cannot stream this material anyway.

What to expect on a retest: two connections rather than four, pump tail prefetch served the read at <offset> for the object near the end, and pump retained file head served the return to <offset> for the trip back. The wall-clock share is modest, since the first byte of the data connection is 1720 ms and 1892 ms of a 3577 ms and 3428 ms open in two of these three sessions, and nothing on this side can overlap that first round trip.

Leaving this open until then.