-
Notifications
You must be signed in to change notification settings - Fork 104
improved peer ranking
The goal is to rank upstream peers by how quickly they will actually
deliver what we fetch — EB bodies, closures — so fetch logic can route to
the best and demote the worst (feeding the improvements proposed in
leios-fetch-scheme.md — Improvement #1's peer classification,
adaptive chunk sizing, and the like). The ground truth is
directly-measured completion time — T_wait, the observed
request-to-full-reply timing — and where a fetch has completed we rank on
it: a paced-slow peer measures slow — its delivery is exactly what
T_wait records — so the signal can't be gamed. Its one limitation is
that it is retrospective: you cannot measure a fetch you have not made,
so to choose a peer before the fetch, and for chunk-sizing and
diagnostics, we need a forward prediction — a time-to-deliver estimate,
ramp(RTT) + estBurstS ×
size, built from network-mux's DeltaQ/RTT instrumentation
(estBurstS, seconds/byte, from its burst-SDU serialisation estimator;
RTT from its cookie-echo window; a branch feature today, not yet on
main). Reusing instrumentation the mux already provides, rather than new
plumbing, has a catch: the throughput input estBurstS is adversarially
gameable — a peer can pace its SDUs so estBurstS reads fast (high
apparent bandwidth) while it dribbles our data, earning a rank it doesn't
deserve and winning traffic it then serves slowly. (RTT is manipulable
too, but asymmetrically: a peer can't fake the cookie-echo RTT low — it
can't echo faster than the round-trip — so cheap manipulation only
inflates it, which self-penalises by raising its time-to-deliver.
Lowering its actual RTT means co-locating near the victim: real, but
expensive, where gaming estBurstS is free. A co-located peer that looks
genuinely fast while dribbling the payload is caught by the
actual-vs-predicted per-fetch check, not by RTT.)
So the prediction has to be robust: a peer must not be able to look
good in the estimate while delivering badly. This document lays out how —
a promptness-gated burst window that yields a trustworthy estBurstS,
why ranking on time-to-deliver makes a gamer self-penalise, and an
actual-vs-predicted backstop that audits the estimate against measured
T_wait and churns consistently-off peers. The hierarchy throughout is
that direct measured completion is the preferred basis: the peer ranking
— the very decision a pacer sets out to game — rests on measured
completion wherever a fetch has completed, and there pacing gains
nothing. The estBurstS hardening exists to make the prediction
trustworthy for the roles measurement can't fill: choosing a peer before
a fetch, chunk-sizing, slow-loris, and diagnostics.
All four deny us a timely fetch — an EB body or closure that doesn't arrive in time for the round's diffusion/certification deadline — cheaply for the adversary. Three of them win by looking good enough to be chosen and then not delivering; slow-loris simply drip-feeds a fetch it already holds. They differ in how they're caught.
- RTT manipulation. Make apparent RTT low to score well on the ramp term. A peer can't fake the cookie-echo RTT low — it can't echo faster than the round-trip — but it can buy a low real RTT by co-locating near us (expensive), or inflate its RTT by delaying echoes, which only self-penalises. Caught by: the ranking (inflation self-penalises); a co-located peer that then dribbles falls to the per-fetch check. → Using the RTT signals.
-
Pacing (idle-gap). Deliver in tight micro-bursts split by silent
stalls, so a gap-based
estBurstScounts only the fast bursts and reads high while delivery is slow. Gain: fake high bandwidth → win the fetch → dribble the payload. Caught by: the promptness-gated burst window (silence forfeits the extension), reinforced by theestBurstS/RTT bind. → The window-sizing tension; The adversary bind. -
Byte-padding. Serve other bytes fast (other requests / filler)
so the aggregate
estBurstSand RTT look great, while dribbling our closure — targeted withholding (the data-withholding threats T20/T21). Caught by: per-fetch completion timing and the actual-vs-predicted check, churned on persistence. → The adversary bind (byte-padding). -
Slow-loris. Accept the fetch and drip-feed it — no attempt to
hide; it ties up our fetch slot and, at scale, our concurrent-fetch
capacity, denying us throughput cheaply (we wait on it instead of a
good peer). Unlike pacing it doesn't fake fast — it looks slow.
Caught by: the ranking (it scores low), throughput-collapse detection
(
estBurstSrising), and cancelling/reissuing the stalled fetch. → The bigger picture.
Those four game a single delivery measurement, and the machinery in the rest of this document is what catches them. A distributed blockchain setting adds three that game the ranking system rather than any one measurement — across time, items, and identities — and these lean on redundancy and peer-selection layers beyond the delivery score:
- Reputation farming, then a deadline-timed defection. Behave well to earn a top rank cheaply over low-stakes rounds, then withhold the one fetch that matters — an EB near its certification deadline — and revert. Score-based churn reacts only to a sustained low score, so a single withheld body — one non-first-delivery against an hour of farmed good score — barely moves the peer's standing: the betrayal is close to free. The defence is therefore fetch redundancy for consensus-critical items — hedge across peers (Improvement #5) so no single defector is decisive — backed by weighting a consensus-critical miss far above an ordinary one in the score. Redundancy, not punishment, is what protects the fetch.
- Surgical withholding of the critical item. Byte-padding targets us across a whole connection; the per-item version serves everything fast except the single block/EB an adversary wants suppressed (censorship, fork support). A per-peer average rank barely moves, so the ranking alone won't flag it. The defence is protocol-level: multi-source diffusion fetches the item from someone else, and a missed consensus-critical fetch should weigh far above its share of the peer's history. The ranking's only job is to not rate the withholder top on the strength of the 99% it does serve.
-
Baseline poisoning via Sybils. The actual-vs-predicted threshold is
a p99 over a population baseline of honest peers; in a permissionless
network an adversary runs many peers and inflates that baseline until a
byte-padder fits under the loosened band. The impact is bounded — it
desensitises detection, it doesn't falsely accuse honest peers — and
it's mitigated by robust, per-subnet-capped aggregation — weight the
baseline by IP-prefix / ASN rather than by peer, so a Sybil cluster in
one subnet can't dominate it (identities are cheap, distinct network
locations are not) — and by capping the band with the physical
prediction rather than the empirical p99 alone: ramp +
estBurstS× size is a model of delivery time, so the plausible residual (serve time, loss-recovery, stalls) has a physical ceiling a poisoned baseline can't inflate past.
A note on the response rather than a distinct attack. In Cardano, churn is score-based: peers first to offer a header or first to serve a requested body score well, and the worst-scoring ~20% are evicted each hour and replaced with a random sample. (Transient failure is a separate, faster path — a peer that blows response timeouts under a long GC or under-provisioning is dropped immediately, not slowly churned.) This design is largely eclipse-resistant rather than a ratchet: the resample is random, so a Sybil fleet gets no better-than-proportional odds of backfilling a freed slot, and the worst-20% eviction is absolute, so it also culls the weakest of the adversary's own peers — even a brief local eclipse leaks back out as those marginal peers churn and random (mostly honest) draws replace them. What survives is only the patient top-scorer of the reputation-farming attack: an adversary can farm score to hold a hot slot, but it cannot use the churn machinery to displace honest peers or take over the set. Cheaply forcing an honest peer out isn't available — only expensive, on-path degradation (nation-state / ISP-tier, and not ranking-specific) can. Diversity-/anti-eclipse-aware selection (Improvement #8) hardens the residual, but the base mechanism already does most of the work.
Requests are small — often a single SDU — but responses arrive as a
burst of many SDUs, and the cookie on our request is echoed back on
that burst. The first response SDU's echo gives the RTT sample; the
follow-up SDUs of the same burst feed estBurstS, the serialisation
rate. That poses the question the burst window exists to answer: for how
long do we keep attributing incoming SDUs to this response's burst —
counting them toward its throughput — before a new SDU counts as a fresh
exchange instead? That horizon is rttBurstMaxAge (hard-coded 1 s
today).
It has to be two things at once: tight enough that a pacer can't stall-then-look-fast, and extensible enough to measure a genuine multi-second closure. A fixed 1 s is a poor compromise — too short for a large honest transfer, too generous to a pacer if loosened.
When a burst would otherwise time out mid-transfer there are two choices: stop counting (and leave honest data on the table if the peer was good and the stream healthy), or keep extending the deadline while the peer is still actively serving. The second is the gate — extend while the peer serves promptly, end the window when it goes quiet.
"Serving promptly" can be read off network-mux's cookie mechanism (a branch feature today, not yet on main): we piggyback cookies on our outbound SDUs, the peer echoes them back, and the still-unechoed cookies sit in a queue — the cookie PSQ — whose matched echoes yield the RTT samples. Repurposing that queue as a liveness gauge is the new part: a peer echoing promptly keeps its oldest outstanding cookie young; a peer gone quiet lets the oldest cookie age and the queue grow. So there are two readings of "serving promptly":
- Cookie echoes (general): extend while the cookie queue isn't growing / its oldest cookie isn't aging.
- Incoming SDU spacing (one-directional download, where our cookie-carrying outbound is sparse): extend while successive SDUs arrive within a tight back-to-back max-spacing.
Either way an honest large closure keeps the window open (its SDUs / echoes keep coming) while a pacer that goes silent to game the estimate forfeits the extension. The tight spacing has one honest hazard — a loss / RTO briefly stalls the in-order stream and can trip it — so the anti-gaming does not rest on the gate alone: it rests on the actual-vs-predicted backstop (below), which tolerates a one-off trip and churns only a peer whose delivery is consistently worse than its advertised rate.
The queue is a liveness gate on the burst extension (above): a
trend — is the peer serving now — read from whether the sent-cookie
queue is growing / its oldest cookie is aging. It is deliberately not a
comparison of queue duration against the self-measured RTT quantile:
that would be circular — an adversary delaying echoes uniformly
inflates the queue front-age and the measured RTT together (a cookie
echoed at t0 + RTT + D gives both a front-age peak and an RTT sample of
RTT + D), so the comparison never trips. The trend has no such
weakness: a silent peer's queue grows whatever its RTT was.
The self-measured RTT is used heavily — as an input, not against
itself. The idle-gap gamer's inflated RTT feeds the time-to-deliver
ranking (ramp(RTT) + estBurstS × size) and self-penalises: high
apparent RTT ⇒ high time-to-deliver ⇒ ranked down, no independent
reference needed (the bind and bigger-picture sections, below).
Busy-fill — keeping the burst alive with filler SDUs rather than going
silent — is caught the other way: the filler is counted, so estBurstS
reads the true, slow rate and the estimate ranks the peer down. (Filling
fast with non-closure bytes to hold estBurstS low instead is a
different attack, byte-padding — the one escape these self-measured
signals miss, caught by the per-fetch check; see the bind.) So the scheme
deprioritises a gamer on self-measured signals alone.
An independent RTT reference buys cleaner attribution, not the core
defence. A kernel tcpi_rtt or handshake baseline gives a
single-observation tell of adversarial app-layer echo-holding
(kernel RTT low, app-RTT high ⇒ the peer is sitting on cookies —
literally the existing "app-RTT vs kernel-RTT divergence" cross-check),
distinguishing it from an honestly far peer (both high). That matters
only for escalating from deprioritise — which self-measured signals
already do — to demote/blocklist.
Attribution isn't gated on it, though: the actual-vs-predicted
comparison (below) recovers a form of it portably — a persistent
actual ≫ predicted identifies targeted withholding using only
app-layer signals (direct T_wait, estBurstS, RTT), no kernel/handshake
reference. It convicts more slowly (it needs persistence to clear the
serve_time/cold-cache confound) and it catches the attack by a different
tell.
The two attacks leak differently — idle-gap through RTT, byte-padding
through the payload — as the bind section below sets out. So the
independent reference's portability caveats (Linux + tracetcpinfo,
design-phase, for tcpi_rtt; drift-prone for the handshake baseline)
cost only the fast, clean attribution signal — not attribution
outright, and certainly not the core defence.
What makes the gate sound is a genuine dilemma the adversary faces on a single measured stream:
- To make delivery slow, the stream must be idle a lot (low byte-rate).
- To game a gap-based
estBurstS, that idle must fall outside bursts (the burst timer expires during silence, so the stall isn't counted). - But idle = no SDUs = no echoes → the cookie queue ages / RTT samples inflate.
- To keep echoes prompt, it must fill the idle with SDUs — but then
the burst stays alive, the fill is folded into
estBurstS, and the estimate reads the true (slow) rate.
So it cannot simultaneously game a gap-based throughput estimate and keep the liveness/echo signal flowing: whichever activity carries the prompt echoes is measured, and going silent to hide the stall shows up as echo lag. Pick one, caught by the other.
Cardano's mini-protocols are mostly pull-based, which tightens this further: the peer sends SDUs only in response to our requests, so it cannot inject filler at will to keep echoes flowing. The fill would have to be responses to some other request we happen to have outstanding to it — an opportunity it cannot manufacture — so busy-fill is opportunistic and hard to exploit reliably. Absent such traffic, the only way to withhold our closure is to fall silent, which the echo-lag / SDU-spacing gate catches directly.
Aggregate (protocol-mixed) estBurstS is what closes the loophole.
The bind holds only if the echo-carrying SDUs are inside the
measurement — and today estBurstS is connection-level, so every
received SDU, whatever mini-protocol carries the echo, feeds the
aggregate burst. There is no unmeasured side-channel to park prompt
echoes on. The protocol-mixed aggregate we usually treat as a limitation
is, for this attack, the strength.
Caveat for a future per-protocol split. The loophole reopens if
per-protocol estBurstS (the proposed "Shape B" split — per-mini-protocol
DeltaQ buckets, not implemented today) is added without also binding the
cookie/RTT per-protocol — because then the throughput signal and the
echo signal live at different granularities. Concretely: we fetch a
12 MB closure over LeiosFetch (protocol B) and are also driving a second
mini-protocol A (keep-alives, say, or another fetch) on the same
connection.
- It delivers the closure in tight 50 kB micro-bursts, then goes silent
on B for ~500 ms, repeat. Per-protocol
estBurstS_Bcounts only the fast micro-bursts — the 500 ms silences exceed B's burst gap and fall outside its bursts — so it reads fast. Gamed. - During each B-silence the peer keeps answering our A requests promptly, each response echoing our latest cookie. The cookie PSQ is connection-level, so those A echoes keep the queue drained and RTT low — the gate stays quiet.
Result: estBurstS_B says LeiosFetch is fast and the gate says nothing
is held back → we rank the peer a great closure-deliverer, route
closures to it, and it dribbles them. A is a side-channel that
estBurstS_B doesn't see, yet it still satisfies the connection-level
gate — exactly the gap the bind depends on being closed. The loophole
is gated on A traffic frequent enough to keep draining the
connection-level queue through the B-silences, though — consistent with
the pull-based limit above, the peer can't fabricate that; it can only
ride A traffic we ourselves drive.
The aggregate estimate doesn't have this gap: those same A responses
are inside the aggregate burst, so they either keep it alive (⇒
estBurstS reads the slow closure rate) or, if the peer goes fully
silent to avoid that,
the echoes lag (⇒ caught). The fix under a per-protocol split is to keep
the two signals at the same granularity — judge B's delivery against
echoes that ride B, so a silence on B costs echo-lag on B. Rule: go
per-protocol on estBurstS ⇒ go per-protocol on the echo/RTT too.
Where the payoff hides — and how each strategy leaks. The
adversary's payoff is always the same: pace the bursts so estBurstS
reads fast, win the fetch, then dribble the payload. Stalling echoes or
inflating RTT is never the goal — it is only a leak of the pacing, and
a different signal catches each leak.
-
Busy-fill — fill the stalls with SDUs so the echoes stay prompt
and nothing looks held back. But those filler SDUs keep the burst
alive, so they fold into
estBurstS, which then reads the honest, slow rate.estBurstSitself catches it. -
Idle-gap — go silent between micro-bursts so the stalls fall
outside the burst and
estBurstSreads fast. That same silence starves the echoes: the cookie front-age grows and the RTT-quantile inflates. RTT catches it — self-penalising in the ranking, and, against an independent reference (kerneltcpi_rtt/ handshake), attributable as app-layer echo-holding.
(The queue-vs-self-RTT comparison stays circular — uniform echo delay inflates the front-age and the RTT sample together — but the bind means we never need it.) The third leak, byte-padding, escapes both of these and is treated next.
The one escape the bind does not close: byte-padding. The peer
sends other bytes fast, serving another protocol request,
so the aggregate byte-rate looks high while it dribbles
the closure you actually asked for. estBurstS reads fast,
the padding carries prompt echoes (the gate stays quiet), yet your closure
is slow. A measurement bound to that specific fetch — direct
completion timing (T_wait on the closure; next section) — catches it,
because the padding isn't the bytes you're waiting on. And comparing
that T_wait to what the peer's own signals predict — ramp(RTT) +
estBurstS × size — upgrades detection to attribution: under padding
the prediction is fast but the actual is slow, and actual ≫ predicted
means the peer has the capacity (it's spending it elsewhere) and is
withholding from us (the data-withholding threats T20/T21), not merely
slow. Lose-lose for the
padder: advertise fast and dribble ours ⇒ discrepancy; advertise slow to
match ⇒ ranked down anyway; deliver ours fast ⇒ not the attack. Caveat:
honest confounds fire the same discrepancy — server-side serve time for
our closure, HOL/contention on the shared bearer, stale estBurstS/RTT
— so deprioritise on a one-off and punish (demote/blocklist) only on
persistence with those ruled out.
The serve-time confound is the sharpest, because it produces the same
observable, not just a noisier one. estBurstS/RTT capture the
byte-flow rate and the network round-trip, but not the peer's cost to
produce the bytes — reading our closure's txs from its store,
reassembling it:
actual = RTT + serve_time + size × per-fetch-rate + loss/RTO
predicted = RTT + size × estBurstS (aggregate signals)
The prediction has no serve_time term, so actual − predicted ≈ serve_time: a large serve time alone reads as actual ≫ predicted. So
a peer whose TxCache holds others' closures (hot,
served fast) but not ours (cold, disk-read) shows a fast aggregate and a
slow closure — capacity that exists but isn't reaching our request,
exactly like byte-padding, yet benign (which closures are hot/cold is
diffusion history, not the peer choosing to starve us). Decomposing our
fetch into time-to-first-byte (which absorbs serve time) vs the
whole-fetch byte-rate separates the two if the peer batch-reads (long
TTFB, then a normal rate ≠ dribbling) — but a slow streaming read
shows a slow rate throughout and is indistinguishable from withholding
by rate alone. So the discrepancy stays a deprioritise signal (correct
either way — the peer is slow for us now); promoting it to malice
needs the slowness to be systematic across our requests to that peer
(targeting), not per-request random (cache-luck).
The cookie/promptness queue is the load-bearing structure: the gate that lets the burst window extend to measure an honest large closure while denying a pacer the same extension (above). The consistency and actual-vs-predicted checks sit on top of it as the backstop, not as a substitute for it. It reuses machinery already present — the t-digest and the cookie PSQ; what is new is gating the extension on them.
Setting the threshold for the actual-vs-predicted check. For the
byte-padding check (how much may actual exceed predicted before we
act), don't derive the band from estDeltaQVVar. It's the wrong
granularity — per-SDU micro-jitter, which ignores the correlated
per-fetch terms (ramp-round count, loss-recovery, serve_time), so it
under-estimates the real spread and the band comes out far too tight —
and the wrong shape, since the residual is heavy-tailed and a σ-band
mis-fits it. Threshold instead against an empirical high quantile (say
p99) of the residual actual − predicted, from a population baseline
across honest peers — not the suspect's own samples, which it can
inflate to widen its band and hide under.
Stratify that baseline by regime, because the residual's spread is not
one distribution. It scales with message size (more bytes ⇒ more
loss-recovery exposure and a longer transfer) and with RTT (both the
ramp-round count and the RTO magnitude grow with it — and that
dependence survives even though predicted already subtracts a
ramp(RTT) term). Loss rate is the natural third axis, but it is
non-portable — only via kernel tcpi_retrans/tcpi_lost (Linux +
tracetcpinfo, design-phase) — so a first pass stratifies on
size × RTT only and pools clean and lossy paths in the same bin. The
cost is a deliberately conservative band: the lossy fetches in a bin
inflate its p99, widening the threshold, so first-pass detection is less
sensitive (a padder can hide under the loss-inflated band) but rarely
false-positives. Add the loss stratum to tighten it once kernel
telemetry is available on the measuring node.
The pacing attack only matters to the extent we use estBurstS as a
proxy for delivery quality (peer ranking, chunk sizing). But those
decisions are moving onto direct completion timing (T_wait /
observed arrival). Against direct completion timing, pacing is
self-defeating: the adversary's paced-slow delivery is the
measured completion time, so it ranks low no matter how it games the
per-SDU gaps. So the highest-leverage defense against estBurstS-pacing
is not to trust estBurstS for the decision that pacing targets —
rank on observed completion, and pacing buys the adversary nothing.
Even short of direct timing, ranking on estimated time-to-deliver —
ramp(PeerRTT) + size × estBurstS (Improvement #1's fused metric) —
already absorbs the idle-gap gamer, which self-penalizes: faking a
low estBurstS by going silent forces its apparent RTT up (held
cookies), and the RTT term then dominates its estimate, so a genuine
low-RTT/high-throughput peer always out-ranks it (higher apparent RTT ⇒
higher time-to-deliver). This needs no independent cross-check — you
never have to distinguish the gamer from a genuinely far peer, you
just rank both down for their RTT; the tcpi_rtt/handshake anchor is
only for attributing and punishing the gamer specifically. Two
caveats: the ranking RTT-quantile must be high enough to capture the
stall inflation (occasional long stalls hide above p80), and byte-padding
evades it — it shows low RTT (see the bind). So the ordering is:
measured completion timing closes both; the fused estimate is a good
fallback that handles idle-gap for free.
That leaves estBurstS needing hardening only for its residual
uses — diagnostic decomposition, and slow-loris detection. Slow-loris is
the mirror image of pacing: the peer makes no attempt to hide its
slowness, it just accepts the fetch and trickles it, tying up our slot.
Because it looks slow, the time-to-deliver ranking deprioritises it for
free; and a peer whose estBurstS rises mid-transfer — throughput
collapsing below a floor — is the slow-loris signature (Improvement #3),
at which point we cancel the stalled fetch and reissue it elsewhere. So
estBurstS earns its keep here in the opposite direction from pacing
— flagging a peer that looks slow, not one faking fast — where
pacing-to-look-fast can't help the adversary. The estBurstS-vs-RTT
consistency check, meanwhile, is worth keeping as a sanity gate for when
you do lean on estBurstS, but it shouldn't be load-bearing.
estBurstS is low-level and gameable, so building the prediction on it
looks risky — but it's safe because of the role it plays:
-
Center-line, not tolerance. It supplies the peer's advertised
rate in ramp(RTT) +
estBurstS× size. The tolerance — how faractualmay exceed that — is a separate quantity: a high quantile (p99) of the residualactual − predictedmeasured across honest peers (the threshold discussion above). Call that honest-peer residual distribution the baseline.estBurstS's missing macro terms (serve_time, stalls, loss-recovery) don't corrupt the check: becausepredictedomits them, they surface in honest peers' residuals too, so they're already folded into the baseline — a suspect peer is flagged only by deviating above it. -
Audited, not trusted.
actual(directT_wait) is the auditor;predicted(fromestBurstS) is the claim under audit — its unreliability is fine because it's the thing being checked, not the ground truth. -
Inaccuracy cancels; gaming self-defeats. Because that baseline is
computed with the same
estBurstS, any systematic bias inestBurstS(say it reads 10% fast for everyone) shifts every peer'spredictedalike, shifts the baseline with it, and cancels in the residual-vs-baseline comparison — so the check needsestBurstSconsistent across peers, not accurate. And the only useful lie is to makeestBurstSlook low, which shrinkspredictedand makes the check more sensitive; lying it high advertises slow and ranks the peer out.
The one role estBurstS is not given — trusted ground truth for
ranking — is the one it couldn't survive; that's why ranking ultimately
prefers measured time-to-deliver. Correspondingly, the tolerance band is
the empirical residual, not the per-SDU estDeltaQVVar (the threshold
discussion above).
-
Your own GC is a confound for the gate. A GC pause makes you
timestamp echoes (and incoming SDUs) late, inflating the cookie
front-age and the SDU-spacing reading — looking like the peer stalled,
a false extension-revocation that blames the peer for your own pause.
Same class of false trip as a loss/RTO on the gate (the window-sizing
section); the backstop's persistence requirement absorbs it. Kernel
tcpi_rttis immune to your GC — a reason to prefer it for the RTT input where available. -
Lazy sampling The RTT quantile feeding
ranking/prediction is read from the t-digest every ~1 s in
processIngress, not per-SDU — preserving the finger-tree's deferred<>. (The extension gate itself is a cheap front-age / SDU-spacing check, separate from the t-digest.) - Not tuning-free. The knobs are the promptness threshold (the tight back-to-back SDU spacing / front-age), the backstop quantile (p99), and the churn-persistence count. The tight threshold's risk is false trips (loss/RTO, GC), not gaming — a steady-dribbler that stays just under it keeps the burst extending and is then measured slow, which is correct — and those false trips are what the backstop's persistence tolerates. The knobs are physically grounded (a spacing in milliseconds, an RTT, a percentile), but they still need calibration.
- Download liveness needs no cookies. In a one-directional download the incoming-SDU-spacing gate supplies liveness directly; the cookie signal (for RTT) can go stale there — cookie-carrying outbound is sparse — so inject a cookie periodically if you need a fresh RTT through a long download.
-
How the signals relate. We compute
estBurstSourselves from the peer's SDU gaps, so a peer influences it only by pacing its own SDUs; the cookie echoes are an orthogonal measurement we control. Pacing the SDUs and starving the echoes coincide only when echoes ride the measured stream — which aggregateestBurstSguarantees (the bind section); a per-protocol split would have to preserve it. - Response on detection is cheap and low-blast-radius: the ranking already deprioritises a gamer; on a persistent actual-vs-predicted discrepancy, churn/demote it. It's a peer trying to look good, so a false demotion costs little.
The window-sizing tension — tight enough to deny a pacer, extensible
enough to measure a large closure — is resolved by gating the burst
extension on peer promptness (cookie-echo front-age, or incoming SDU
spacing on a download): an honest large transfer keeps the window open,
a pacer's silence closes it. The gate is imperfect alone — a loss/RTO can
trip the tight spacing, and steady-dribble or byte-padding slip through —
so the anti-gaming rests on the actual-vs-predicted backstop: churn a
peer whose delivery is consistently worse than its advertised rate,
which also absorbs the gate's false trips via the consistency
requirement. An independent RTT reference (kernel tcpi_rtt / handshake)
is a nice-to-have for fast attribution, not a prerequisite. And all of
this hardens the prediction: the ranking itself prefers
directly-measured completion wherever a fetch has finished, where a pacer
gains nothing.