Releases: BobbyAxerol/quant-data-layer
Release list
Quant Data Layer v2.2.2
Quant Data Layer v2.2.2
Scope
Bounded execution-readiness and recovery patch on the existing Kafka-native
architecture. Kafka client retries and projector handoff are bounded; Query can
use broker-confirmed canonical hot backup without inventing price timestamps.
MARK/INDEX retains component clocks and capture lineage. SDK2.0.7 shares the
execution proof validator. Public API, entitlement, realms and quality thresholds
are unchanged. This is not a Trading System order/accounting certification.
Runtime Acceptance
Query candidate qdl-v2-python:2.2.2-d6d2637:
sha256:514506122111df8992a5dfac9dc9a3db7ccae202a398424f4dd5a033d2ddf9a9.
Only two Query roles were recreated in the final convergence;15 other active
Data Layer roles were unchanged. Both Query roles healthy, restart0/OOMfalse.
Existing Rust recovery/Stream/projector component images and evidence are retained.
Actual TS sandbox SDK2.0.7, both Query replicas:
- MARK/INDEX snapshot/reference: 44/44 usable, including OKX inverse BTC;
22 reference views preserve all six component clock/capture fields. - Total 62/64 usable. Two OKX TRADE prices older than3s were correctly refused;
no simultaneous provider witness, so quiet-market versus pipeline attribution is
not claimed. No stale execution price was admitted. - Worker20/20 sampled READY over190.224s, session22/22, execution18/22, zeroV1fallback.
This is scoped read acceptance, not another C2 or uninterrupted availability.
SDK call through validation, after metadata resolution, milliseconds:
| Venue | Path | N | Median | Maximum |
|---|---|---|---|---|
| Binance | MARK reference | 10 | 14.06 | 68.79 |
| Binance | MARK snapshot | 10 | 19.70 | 55.97 |
| OKX | MARK reference | 12 | 14.09 | 30.36 |
| OKX | MARK snapshot | 12 | 18.02 | 69.37 |
No p99 for these small groups. These are not event-to-Redis-commit measurements.
Unchanged endpoint/catalogue and prior hot-backup benchmarks remain explicitly
inherited in endpoint-report.json, not relabelled as fresh full-system tests.
Verification And Limits
Candidate64/64 packaging tests,42/42 isolated Redis tests, actual entrypoint and
six-file Git/image hash checks pass. Remote contract/SDK/native fault gates pass
at the reviewed code head; publication requires all checks at the final PR heads.
The full certificate records the exact artifacts, evidence hashes and requirement.
Same-host Kafka backup is not independent HA or zero downtime. Historical refusal
and failed rollout/probe windows remain retained. Binance3d and DNSE/VN V2 remain
excluded. No alpha activation, mainnet authority or funded sizing claim.
Supplemental Binance Recovery Convergence
A final provenance audit found the shared raw-ingestor recovery fix had reached
OKX but not Binance. The parent rolled only ingestor_binance_usdm to the
already-tested image sha256:0f6876e16e51600419e1f172aae596bed9c2d369d1afcef93d1da30c18f262ec.
No new source/build or config/TLS/state/offset change. The other16 role images and
start times were unchanged during this supplemental step; three DL roles changed
across both steps in total. The earlier two-Query receipt remains historical evidence.
Five Binance symbols, two Query replicas, six read paths: 60/60 usable.
SDK-to-converter median/max milliseconds, n10 each: QUOTE28.37/99.89,
TRADE44.74/70.10, MARKsnapshot31.80/66.54, MARKreference23.48/67.91,
BOOKsnapshot85.89/109.36, BOOKdelta79.36/204.18. BOOKdelta uses snapshot RPC here,
not a new streaming/burst test; timing excludes Redis apply. No p99 claim.
Actual TS worker initially degraded to10/22session-ready despite these Query reads.
A parent-owned TS CPU trial (1->1.5vCPU) recorded18/18READY,22/22sessions,
18/22execution-ready over170.199s between samples in a planned180s window.
Average1.026CPU and5.64% throttled periods are a bounded trial, not capacity proof.
A separate TS adapter reconnect/ACK race was reproduced; SDK rejection is the
correct safety behavior. Its fix is not a DL code change and is not certified by
this Query result. Publication remains gated on parent consumer-recovery closure;
the final tag must link that receipt. Earlier TRADE refusals remain unchanged.
Supplemental Binance rollback is the same role/config to
sha256:658a9570c5fc23f4906413aa2460f4d82e21772cc63c9ed900363d4d13d54023.
This image also remains active for three Rust cores and must not be pruned.
Rollback
Only query_kn_2 and query_kn_1, same config/TLS/state, to
sha256:3af57ddf17642e2073e09e8d92450e3aca551ebd5d855462c9ce60f148e5e85c.
No Kafka offset reset, Redis flush, old SQLite reactivation or TS/order mutation.
Final Publication Attestation
The frozen certificate records REQUIRES_REMOTE_CI_AND_MAIN_ANCESTRY because it
was assembled before the final remote checks. Publication resolves that external
gate through the annotated v2.2.2 tag, not by rewriting historical evidence.
The annotation records the exact successful CI heads, run IDs and job conclusions,
feature/dev/main ancestry, and SHA-256 of this frozen certificate and endpoint report.
Inspect it with git fetch origin tag v2.2.2 followed by
git show --no-patch v2.2.2, or resolve the tag object via the
GitHub tag reference
and read its object.url annotation. The annotation must resolve all required
checks to PASS before the tag is pushed. The tag-triggered publication workflow
then verifies main ancestry and the certified endpoint-report and SDK-wheel hashes.
This resolution does not widen the scoped runtime acceptance or erase refusals.
Quant Data Layer v2.2.1
Quant Data Layer v2.2.1
Scope
Kafka-native execution-readiness repair, retaining V2 primary and policy-limited V1 fallback. No broker orders, TS/alpha rollout, Kafka offset reset, cache flush, or history deletion.
- Rust core flushes against an absolute batch deadline; arrivals cannot extend the deadline indefinitely.
- Existing transactional quarantine carries bounded, generation-fenced L2 resnapshot feedback to the existing venue ingestor. BOOK recovery does not restart TRADE/QUOTE lanes.
- Fair socket polling and coalesced control checkpoints prevent feedback pressure from starving live input. Genuine gaps still fail closed.
- Includes additive OKX inverse sandbox binding and previously integrated realm/diagnostic fixes; not a grant of new alpha execution rights.
- SDK2.0.6 gives existing typed refusal diagnostics a distinct reproducible artifact identity. API schema remains2.0.0; existing consumer pins are unchanged.
Evidence
64 entitlement products x two Query replicas:128 typed views,126 usable, two OKX TRADE refusals matched to latest public trade ID/time at observation. All MARK/QUOTE/BOOK/BAR preflight reads usable. Request-to-validated-usable observed maxima are below50ms in these small groups; they are not p99.
One300s production observation: paper60/60 and sandbox22/22 READY on11 distinct published heartbeats each. Four OKX transport disconnects produced12paper/8sandbox session refusals followed by recovery. This is fail-closed recovery, not continuous availability. Paper cache was measured176s after correcting the observer Redis selection; sandbox300s. Polling ages include up to1s observation delay and are not exact Redis commit latency.
Isolated authentic RF3 replay at unchanged core caps:330000 records across6partitions, bounded4k/s and5k/s windows, all final offsets caught up. Not indefinite5000/s per consumer or unlimited fanout. Whole runtime observation averaged4.70cores, sampled peak5.12, no cap increase.
Deployment And Rollback
Three Rust cores and two native ingestors use immutable658a9570c5fc...54023. Two Query readers additionally use b40b49c361c7...4a65 (sourcee366f75) with PyJWT2.15.1/urllib32.8.0 security fixes;32packagedauth/SDKtests pass. Query rollback is0ed57643...eb959 with identical mounts/config. Stream, KN market projectors, BAR edge, V1 and TS images stay unchanged. Certificate contains full per-role digests. Native rollback is coref2040ac9...ca79 and ingestor7fe34806...f69f with original per-role config;7fe remains active in KN market projectors. No reset needed. Historical authority handoff digest is provenance, not a false claim of current executable identity.
Limits
Binance3d and DNSE/VN V2 are excluded by owner. Quiet last trades can legitimately fail execution eligibility; no stale price is made fresh. Unchanged warmup/batch/reference/diagnostics capacity inherits dated v2.2.0 evidence, not a new full-catalogue benchmark. TS E lifecycle/order certification remains separate. See certificate and endpoint report for denominators, sampling limitations and observed reconnects.
Query Security Readback
After the separately approved twoQuery roll,128/128 typed views returned,125 usable and3 strict TRADE-age refusals; no auth/manifest failure. BinanceSOL andOKXBNB matched provider quiet-last-trade evidence. OKXSOL had a newer trade228ms before the request, present in raw/canonical; this is not called quiet. Canonical CreateTime is not commit visibility. Ten follow-up reads yielded7usable/3age-refused then a newer trade, retained separately rather than replacing the128-read denominator. No additional C2 or threshold relaxation. Latest per-feed observed request maximum8-12usable samples is in the endpoint report; no fabricated p99 or exact Redis commit latency.
Quant Data Layer v2.2.0
Quant Data Layer v2.2.0
Scope And Architecture
Kafka-native V2 is the active data path for the declared Binance USD-M and OKX
Swap consumer demand. Rust owns ingestion, canonical normalization/L2, Kafka
state projection and streaming. Python owns provider adapters and public Query
contracts. Bounded Redis market cache rebuilds from Kafka state; SQLite is not
in the active read path. Ten old reader/projector roles are stopped; historical
state is retained, not erased.
TS consumes 60 crypto routes for BTC/ETH/SOL/DOGE/BNB on both venues. Five-symbol
price/bar execution demand and the 255-member universe are different scopes:
510 universe venue-products use daily batch history, not 510 execution streams.
Warmup supports declared limits through 10,000 when authentic provider history
exists; initialize once, then append/deduplicate final BAR with bounded maxlen.
SDK 2.0.5 provides bounded off-loop durable cursor ACK. Apply data first, then
await acknowledgement; fsync, monotonic checkpoint and generation fencing remain.
Cancellation/global shutdown drains accepted I/O. Shared alpha source is tested
but this release does not start an alpha or place an order.
Measured Acceptance
Final actual TS window: 300.000849 seconds, 29/29 samples at 60/60 READY,
no fallback or V2 error. Native Stream delivered 82,522 events, no new overflow,
replay, reconnect or closure; final queue zero, both Stream RSS values unchanged,
no restart/OOM. This is bounded acceptance, not a multi-day leak or HA certificate.
Timer wakeup (29 samples, 10-second cadence): p50/p95/max 1.08/2.51/4.96 ms.
No p99 is claimed. Durable cursor counters: 29,995 acknowledged checkpoints,
7,805 batched commits, zero errors, pending peak since startup 28/64, final zero.
Mean batch completion 15.09 ms runs off-loop; maximum since startup 406.20 ms is
not an in-window percentile. Separate syscall probe found 0 main-thread fsync
calls; fsync was not removed. External TS Redis GET worst measured route p99
0.905 ms is cache-read time, NOT exchange-to-consumer delivery latency.
Inherited production workload: 50 logical alpha sessions (20 candle,15 realtime,
10 grid,5 multi-feed), 15,504 requests without errors/missed offers,90 streams,
12 reconnect probes. Full serving stack including Kafka/provider coordination
averaged4.669 vCPU over374.415 seconds. Limits and synthetic allocator capacity
are reported separately in certificate.json; allocator fill is not market data.
Consumer SDK call until validated usable result, milliseconds:
| Data | Binance p50/p95/p99 | OKX p50/p95/p99 | Samples/venue |
|---|---|---|---|
| QUOTE | 9.72/24.38/46.53 | 9.91/21.59/33.30 | 2750 |
| MARK/INDEX | 11.07/23.26/39.69 | 11.18/22.52/38.11 | 4125 |
| TRADE | 9.79/23.78/unavailable | 13.23/29.51/unavailable | 55 |
| BOOK snapshot | 25.76/44.75/unavailable | 28.92/44.53/unavailable | 54/55 |
| Final BAR latest | 15.31/29.55/unavailable | 18.58/36.15/unavailable | 55 |
Cold history under load:2500 rows3189-5144 ms;5000 rows4838-6956 ms, four reads,
not a percentile. Full per-binding source-age-to-TS-cache, GET measurements,
stream latency and sample counts are in endpoint-report.json. Quiet source age
is not wire latency and does not itself grant execution eligibility.
Consumer API
HTTP: instruments/list and identity lookup, per-feed status, snapshot, history,
warmup, warmup:batch, reference:batch, readiness/readiness:check, retained-window
gap diagnostics. Native gRPC: GetSnapshot, Subscribe, Replay, GetFeedStatus.
Use the authenticated paired targets in DATA_LAYER_SERVICE_ACCESS_GUIDE.md.
Final BAR drives signals. QUOTE and verified L2 support executable price/impact
context; MARK/INDEX requires the declared trigger/risk policy. Risk remains the
order-admission authority. Reference metrics include funding, OI, long/short,
taker flow, metadata and supported basis variants, subject to venue capability
and explicit identity entitlement. A wrapper is not implicit execution authority.
Boundaries And Rollback
- DNSE/Vietnam remain V1; new images exclude quarantined vnstock/vnai.
- Deferred Spot is not certified by this crypto KN release.
- Five Binance3d histories contain provider discontinuities. Strict complete
history is rejected, never interpolated or silently padded. - Gap scan covers retained windows:702 scanned,6excluded,4VN unavailable;
it does not prove listing-to-now completeness. - V1 fallback is allowed only by product policy; unsupported products stay
BLOCKED, with no cross-venue substitution or direct-provider bypass. - Reader rollback is the exact image/config in certificate.json. Old SQLite
rollback was rehearsed, but restarting its retired groups requires checking
current Kafka retention/offset floors. No reset/flush/delete to force recovery. - Same-host replicas do not certify an independent failure domain. Future
payload growth and arbitrary50-alpha workloads require capacity validation.
Provenance
Runtime acceptance is PASS in certificate.json, with15 hashed evidence files.
Source/SDK/image components are individually identified; rebuilding is not
assumed byte-identical. Publication requires green remote CI, feature->dev->main,
and the v2.2.0 tag/release. This note alone is not publication proof.
Quant Data Layer v2.1.0
Quant Data Layer v2.1.0
Scope
This release certifies the existing 299-product Binance USD-M/OKX Swap V2
data plane plus actual Trading System adoption of its 60 sealed routes:
BTC/ETH/SOL/DOGE/BNB on both venues, TRADE, QUOTE, final BAR 1m,
MARK_INDEX_PRICE, BOOK_SNAPSHOT and BOOK_DELTA. SDK 2.0.3 / consumer revision
10 / release routing revision 22. No public API/schema change.
Changes
- Avoid repeated 12,064-offset SQLite retention scans. Dense retained windows
use an indexed boundary, with foreign-commit/rollback invalidation and an
exact sparse-history fallback. Identity, retained rows, checksums, cursor
expiry and durability are unchanged. - Converge four reader/stream roles on source
1c0844f562ff419a98d4374a838acf7764569219.
This includes previously tested SQLite recovery/hydration contention fixes.
Projector recovery fixes remain on their already-deployede4a7377image. - Raise only the two existing Stream memory ceilings from 512MiB to 1GiB
after a measured A/B test. CPU limits remain unchanged. The actual writer
used about 677MiB, with no direct reclaim during final acceptance.
Evidence
- 207 selected source tests passed; 123 tests passed on the immutable image.
- Authentic read-plane preflight: all 299 products, both Query replicas.
- Real Trading System: 60/60 routes for 305.120 seconds after 30.081-second
convergence; 413 SDK reads validated, all 60 routes published by TS,
no test orders, direct-provider calls or fallback. - Independent same-projection SDK probe: 240/240 reads passed.
- Existing 299-product C2 cursor/reconnect/fallback proof is inherited by hash;
unrelated passing provider tests were not repeated after each patch. - Failed observer setup/API/strict-TRADE probes and the failed 512MiB load
window remain recorded in the implementation journal; they are not passes.
Consumer-Call Latency
Measured request start through SDK validation/projection during real TS load.
Each venue/feed bucket has 30-33 steady samples. Report p50/p95, not a p99 SLA.
| Feed | Binance p50 / p95 | OKX p50 / p95 |
|---|---|---|
| TRADE | 9.46 / 13.71ms | 8.91 / 15.36ms |
| QUOTE | 9.58 / 12.97ms | 9.39 / 19.75ms |
| MARK_INDEX_PRICE | 16.78 / 27.42ms | 15.85 / 28.08ms |
| BOOK_SNAPSHOT | 55.16 / 74.78ms | 50.52 / 77.71ms |
| BOOK_DELTA | 43.69 / 75.71ms | 35.64 / 57.10ms |
| Final BAR snapshot | 532.38 / 806.16ms | 530.93 / 812.70ms |
These are not venue-event ages or candle-close reaction times. Quiet TRADE
may be observable while still ineligible for execution. Risk retains that
distinction. BAR is for signal computation, not a substitute for current
QUOTE/L2/MARK execution context. Warm up once, then append/deduplicate stream
updates instead of polling a full history window.
Projector sampled weighted mean canonical age was 288.555ms (maximum
1,074.2ms); durable append mean 57.392ms (maximum 558.5ms). These stage
statistics are not request percentiles or an unlimited-capacity guarantee.
Deployment And Rollback
Reader/Stream immutable image:
qdl-v2-python:2.1.0-1c0844f /
sha256:579d578e30814192ae5d20c4f29b653b5c5bc173cfaad73f86e9c192dcb6aa6c.
Existing service names, mounts, TLS, Kafka offsets and data remain unchanged.
Exact per-role rollback digests are in certificate.json.
V1 remains legacy/fallback; DNSE/VN and dark Spot are not newly certified.
This release grants no order authority, starts no alpha and does not complete
Trading System P18. It certifies bounded same-host consumer load, not mainnet
execution or independent failure-domain HA. GitHub CI must pass before the
approved dev -> main -> v2.1.0 publication.
Quant Data Layer v2.0.27
Quant Data Layer v2.0.27
Certified Scope
v2.0.27 certifies the declared V2 consumer plane for Binance USD-M and OKX
Swap. The sealed no-order C2 receipt passed all 299 active products: 234
durable and 65 on-demand. It exercised both Query replicas, public V2
Query/Stream SDK paths, signed cursor replay/reconnect, final BAR/history,
reference batch, L2 status/snapshot and manifest-governed V1 fallback.
The final receipt observed for 300.1s, completed every opening and closing
product read, made zero order actions and zero provider connections, retained
no secrets or raw market payload, and removed its cursor directory. Seven
allowed TRADE routes passed V2_PRIMARY -> V1_FALLBACK -> V2_PRIMARY; all
292 blocked routes remained blocked.
Runtime Provenance
The active Query pair runs
qdl-v2-python:2.0.26-8232d1b@sha256:f671dcebfdca1b28fc2cb99f3129f8f13e78a9a5853d2be8adfd9ee00c6f31b9.
The release alias qdl-v2-python:2.0.27-8232d1b resolves to that same
immutable digest; no redundant rebuild or service recreation is required.
The active Stream pair remains on the separately attested
qdl-v2-python:2.0.26-657e86f@sha256:c8d7458e57d62d6fb2d6366f42911d86085918f6f6939d624c61494798dac2ab.
All four reader roles were healthy with restart count 0 and no OOM kill at
certification. Frozen V1 fallback remains available at
qdl-v1-fallback:v1.2.4-2b0dcf7@sha256:dbfb57844977513ae7ec0a4782e04da0213028a789753c6b991f26043b615d65.
This is a component-attested patch release. The final Query executable source
is 8232d1b; later source changes through the release tree are the journal and
public certificate only, not a serving-code change. No redundant Query rebuild
or runtime recreation was performed merely to change a version label.
Acceptance Timing
The C2 opening was 993.905s against a quota-derived 1015s budget, followed
by 300.1s observation and 28.171s closing readback. This is a full
concurrent evidence workload, not a normal alpha read latency measurement.
Per-binding endpoint latency and freshness retain their own semantics in the
sealed receipt and implementation journal.
Declared Boundaries
- DNSE/VN remains
V1_PRIMARYuntil its separate market-hours certificate. - Dark or unentitled Spot/catalog rows are not claimed as V2 execution scope.
- This data-plane release does not grant broker-order authority; Trading System
and alpha paper/live policy remain their own control plane.
The machine-readable certificate and
scope evidence are part of this release.
v2.0.20 — R1.31: the release gate, with four of this executor's own errors on the record
v2.0.20 — R1.31: the release gate, with four of this executor's own errors on the record
CERTIFICATE. Digests read with docker image inspect from the running
stack at 2026-09-19 09:20Z; numbers from ledger entry 45 and the R1.31
entries in DATA_LAYER_UNIFIED_IMPLEMENTATION_PLAN.md.
RUNNING IMAGES
projector_v2, projector_v2_2, projector_v2_3,
query_v2_1, query_v2_2, stream_v2_active, stream_v2_passive,
binance_bar_edge
qdl-v2-python:2.0.20-95d9595
sha256:9039236e7a8e570f2364b470b33386ab702bc1dde5ae9d5e7d90a4dda531e8f0
rust_core, rust_core_2, rust_core_3, ingestor_okx_swap
qdl-v2-rust:2.0.19-003b5f9
sha256:b7b9d153f0ed31892007639ca35f42642f9d41e713e4a06a9222fabdd82b81ec
ingestor_binance_usdm
qdl-v2-rust:2.0.18-3ecf0ac
sha256:eec6388421845ef570cb3878145d0c5e0398fc44cc2a662805298295cfb0976a
No image was built for this tag, and none needed to be: git diff from
each image's own commit to HEAD is empty for rust/, Cargo.*, generated/rust,
qdl/ and qdl_sdk/. Everything in this tag is documentation, tests and one
read-only script, so the running images are HEAD's runtime.
Catalog revision 9, source policy 1, authority 1 — none moved.
GATE
Python 1,675 tests, 0 failures, 7 skipped, run against /src in
qdl-v2-python:2.0.20-95d9595. Rust unchanged since 003b5f9, so R1.29's
fmt/clippy/186-test gate still stands.
HISTORY FOR PAST DATES, 14/14 Binance intervals, every one FULL and
compared field by field against the venue's own REST at both ends
1m 431ms/0.1d 5m 444ms/0.4d 15m 323ms/1.3d 1h 395ms/5.0d
4h 265ms/20.0d 6h 269ms/30.1d 12h 107ms/60.4d 1d 98ms/120.4d
3d 100ms/362.4d 1w 58ms/845.4d
DELIVERY TO A CONSUMER, 200 subscribes across five feeds, 0 refused
feed p50 p95 max budget
TRADE 665.5 1024.1 1187.4 3000
QUOTE 587.5 933.7 950.4 2000
MARK_INDEX_PRICE 719.1 1330.1 1373.9 2000
BOOK_SNAPSHOT 655.4 1096.3 1523.3 60000
BOOK_DELTA 847.7 1257.7 1407.5 2000
MARK_INDEX_PRICE holds the thinnest margin in the system, 626 ms against
a BLOCK policy. That is a 2,000 ms budget against a ~650 ms path, not a
defect, and it is named so it is not rediscovered as an incident.
ENDPOINT INVENTORY
216 catalog bindings: 206 live, 0 over their own stale_after_ms, 10 with
no event stored. The ten are the four DNSE bindings — owner decision,
served by V1, and live V2 refuses them with "required data is not
available" — and six spot bindings that no ingestor produces and no
consumer manifest requests. The 53 ingestor subscriptions are 24
binance-usdm and 29 okx, none spot.
CPU BUDGET 5 -> 6 VCORE, on the owner's decision
kafka2 and kafka3 1.25 -> 1.75, rust_core_2 0.50 -> 0.75, stable_redis
unchanged at 0.50. Net ceiling added +1.25. Applied with
docker update --cpus and no container recreated — which is what keeps
stable_redis away from ProjectionCacheMismatch. kafka1 and every role
throttling at roughly zero were left alone.
The raises were first justified on cumulative counters, which is the wrong
basis, and then re-measured with 60-second cpu.stat deltas holding kafka1
and kafka2 fixed as controls. rust_core_2 throttles 0.3% at 0.75 against
3.6% at 0.50 while drawing 0.183 either way — burst shape, so it is kept.
stable_redis throttles 0.0% at both and draws 0.034, 7% of a 0.50 ceiling,
so that raise was withdrawn. The controls moved 0.5-1.0 points on their own,
which sets the noise floor these numbers are read against.
Not proven beneficial and not proven harmful. Summed p95 across the five
feeds fell 6,234 -> 5,642 ms and summed max 7,294 -> 6,443 ms, while
summed p50 rose 3,292 -> 3,475 ms, all under higher ambient load (11.6
against 10.2) from live_data_executor outside this stack. Kept because
the tails improved, the binding margin widened and no consumer was
refused — not because throttling moved. R1.29's C3 cut kafka2 throttling
16.9% -> 4.2% and was still reverted; unlike C3, this raise took nothing
away from another role.
Memory was checked separately, because a cpus limit throttles and cannot
OOM: worst headroom is kafka2 at 50.2% of 2 GiB.
FOUR CORRECTIONS OF THIS EXECUTOR'S OWN REPORTS
- A regression I caused. The B1 roll recreated ingestor_okx_swap with a
chain carrying r125-rollout.override.yml but not
okx-ingestor-image.override.yml, dropping it from 2.0.19-003b5f9 to
2.0.17-1acf87a — the digest that override names as its own rollback.
Repaired; the rendered diff was exactly one line. - "Sixteen bindings are over budget" — no. I compared the spool's
committed_at_ns against stale_after_ms; the runtime reads a
PROVIDER_CONFIRMATION binding from received_at_ns. Correct: 0. - "TRADE stream p50 1530 ms, a regression" — it was the drain after a
projector cold start. Steady state 619 ms. - "The raise cut kafka2 throttling 14.4% -> 1.4%" — an invalid
comparison of a 47-hour cumulative counter against a 120 s window.
kafka1, untouched, read 1.2% in the same window. Withdrawn, and the
four raises were re-measured with 60-second deltas; one of them,
stable_redis, turned out to fix nothing and was reverted.
CLOSING STATE, 09:20Z
17/17 roles Up, 14 healthy; the three rust_core replicas carry no
healthcheck, which needs a Rust change and is recorded with B3.
206 bindings live, 0 over budget, 0 refused in 200 subscribes.
Data layer draw 3.76-4.29 vcore against the new 6.0 budget; per-role
60-second draw kafka2 0.796, kafka3 0.666, kafka1 0.481, rust_core_2 0.183,
stable_redis 0.034.
Images 33 -> 29, build cache 6.60 -> 5.61 GB, no volume and no container
removed — the three dangling volumes are the three kept deliberately and
the only two stopped containers are the stack's own one-shot inits.
RUNTIME HYGIENE CLOSED
This stack had no single recorded compose chain: each container stores the
chain used the last time it was created, and they disagreed — okx ingestor
19 files, the B1 roles 18, rust_core_2 15, kafka2 10, stable_redis 1 with a
base compose file in a worktree that no longer exists. Recreating a role
from its own label drops every override added since, which is exactly how
the OKX ingestor fell back an image today. canonical-chain.txt now records
the complete chain; rendering it and comparing image, cpus and healthcheck
against docker inspect for all seventeen roles gives 0 mismatch.
Two prohibitions are written with it: never up -d without --no-deps,
since every stored config hash came from a shorter chain; and never recreate
stable_redis, whose base file is gone and whose recreation freezes the spool.
NOT IN THIS RELEASE
Debt B as a block: B2 bar-edge active/passive, B3 Rust ingestor HA
(lease_epoch is a static fencing token, not leader election), B4 Redis HA.
One measured optimisation deferred with them: each projector posts every
canonical batch to both stream gateways and the non-holder answers 409,
about 36,000 rejected mTLS POSTs an hour across three.
Quant Data Layer v2.0.17
Quant Data Layer v2.0.17
Certified Patch Scope
v2.0.17 returns OKX native bars to the canonical stream, removes a physical
ceiling that failed every canonical write closed for two hours, and lifts a
throughput limit that made the resulting backlog unable to drain. Three defects,
each proven against the running stack before it was changed.
What was wrong
OKX bars never reached the canonical stream. Every closed candle was
quarantined StaleGeneration - 1,183 records across two windows on 2026-09-17 -
while provisional candles were filtered correctly and the raw stream carried all
70 candle channels from all five instruments. The spool's newest OKX bar-1m
record was 7,158 s old.
The cause was not another connection. It was the bar edge:
partition_key …/okx_bar/okx-swap-bnb-usdt-swap-bar-1m-primary-v2
frame_session okx-business-001-25-… frame_generation 25
tracked_session qdl-v2-stable-okx-rest-r1-g1789653808007759187
tracked_generation 1789653808007759187
The bar edge publishes REST bootstrap and repair rows carrying a nanosecond
timestamp where a connection generation belongs. One such row on a bar
partition fences every native candle behind it for the life of the core
process, because 25 will never exceed 1.79e18. That timestamp decodes to
14:03:28Z, the minute the bar edge was recreated in this session's rollout and
bootstrapped its history; the quarantines began at 14:20. The first outage has
the same shape - the bar edge was given a new state path at 07:53 and the
candles died at 09:16.
Two hypotheses fitted the partial evidence before this one and neither survived
contact with it, so the rejection was made self-describing rather than guessed
at a third time: a stale-generation rejection now names the tracked session and
generation it compared against. That diagnostic is part of this release.
The spool failed every write closed while inside its own retention policy.
From 10:09:06Z the gateway answered bridge physical storage bound would be violated: main file 2,252,414,976 B plus WAL 966,902,232 B plus shm 1,900,544 B
against a 3 GiB ceiling - 7,720 bytes of headroom. The WAL frames had already
been checkpointed, because PRAGMA wal_checkpoint(PASSIVE) recycles a WAL but
never shrinks the file. A single TRUNCATE reclaimed all 922 MB in 0.08 s with
no reader blocking it. The ceiling itself was also tighter than the rows
max_records permits: 1,841,712 rows at the payload size measured on the live
cache need roughly 3.3 GB on disk.
The backlog could not drain. Consumption had converged on production at
759 events/s with nothing saturated: projectors at 0.49 of a 2.00 CPU ceiling,
no throttling, and a benchmark on the spool's own volume under its own pragmas
returning 40,280 rows/s at synchronous=FULL. The projector asked Kafka for one
record at a time and paid a thread hop for each.
What changed
| Slice | Change |
|---|---|
| R1.25.1 | a producer the ordering fence cannot identify may not fence the live ingestor; generations are only comparable inside one identified lane |
| R1.25.2 | a stale-generation rejection logs both sides of the comparison |
| R1.25.3 | the projector fetches a bounded batch in one broker call; brokers without it keep the original fill loop |
| R1.25.4 | the spool reclaims a WAL that outgrew its declared journal_size_limit, and reclaims again before the physical bound refuses a write |
| R1.25.5 | the physical bound is sized from the rows retention permits, not chosen |
| R1.25.6 | the canonical sink retries a dead pooled connection once before failing over to the passive peer |
| R1.25.7 | Compose records the broker memory bound raised live after the kernel memcg killed brokers at 768m |
| R1.25.8 | the drift verifier seeks the spool by key instead of scanning it, and watches the bound that failed writes closed |
| R1.25.9 | an endpoint report that separates delivery lag, durable-cache age and the contract, per instrument, with the sample count behind every number |
The test helpers built synthetic session identities (s1, session-1) that the
running system never emits, which is why the suite stayed green while production
did not. They now use the production shape, and 1,106 missing bars were
republished through scripts/repair_stable_final_bar_history.py - 1,014 of them
OKX history the fence had been swallowing since 09:16Z.
What this release does not change
Binance MARK_INDEX_PRICE stays outside policy, and Binance BAR stays on
provider REST. Both were re-tested directly against the venue on 2026-09-17:
Binance USD-M acknowledges a subscription for @markPrice@1s and @kline_1m
({"result":null}) and then sends nothing - 80 s, zero frames - on the same
socket where btcusdt@trade delivered 353 frames in 12 s. Four subscription
shapes were tried for mark price and all returned zero. Our configuration is
correct on both sides. This is the condition production_catalog.py already
records for klines, and it holds for mark price too, so neither can be repaired
from the WebSocket path. They need the reference/REST pair path, which is a
catalog migration and is not in this tag.
Binance BAR delivery stays at roughly 7 s for the same reason and by design:
the bar edge holds a closed bar for 6 s and confirms it twice, because a closed
Binance 1m kline was still changing 5.04 s (BTCUSDT) and 3.96 s
(ETHUSDT) after its close boundary when measured on 2026-09-17. Cutting that
delay would publish bars the venue then revises.
Quant Data Layer v2.0.16
Quant Data Layer v2.0.16
Certified Patch Scope
v2.0.16 returns every realtime V2 feed to its freshness policy. The defect was
not capacity and not a venue: it was four costs on the delivery path that a
consumer paid for every record.
What was wrong
Measured on 2026-09-16 against a 2,000 ms policy, TRADE freshness was
355,952 ms. BOOK_SNAPSHOT and OKX MARK_INDEX_PRICE were rejected
outright. The projector backlog reached 3,002,343 records and the consumer
reported 27 of 60 slices ready.
Four causes, each in the module that carried it:
qdl/stream/gateway.py— the stream gateway took the spool's singleRLock
to do a durable read on the live delivery path, so every subscription's
delivery rate was bounded by disk, not by the feed.qdl/stream/gateway.py— a latest-state feed queued a FIFO of superseded
records. A reader that fell one buffer behind could never reach the newest
quote, because the newest one was always behind the stale ones.qdl/replay/handoff.pyandqdl/stream/grpc_service.py— a replay token
was re-signed on a worker thread once per record, paying a thread hop
for a watermark the loop already knew.qdl/query/contracts.pyandqdl/query/service.py—DATA_STALEdid not
say which predicate failed, so the consumer could not distinguish a slow
feed from a dead session and tore the slice down for both.
What changed
| Slice | Change |
|---|---|
| R1.1 | the delivery path reads a cached high watermark instead of taking the spool lock |
| R1.2 | the replay loop collapses token advances over unmatched runs |
| R1.3 | DATA_STALE names EVENT_AGE, SESSION_STATE or SESSION_LIVENESS |
| R1.8 | a latest-state feed keeps the newest record in a bounded buffer and drops the superseded ones |
| R1.9 | a cursor whose watermark is known is signed on the loop, not on a worker |
| R1.11 | the projector drain window is a configurable budget, not a hard-coded 10 ms |
| R1.12 | an overflowing subscription reports backpressure instead of discarding in silence |
Seven python roles were recreated on qdl-v2-python:2.0.16-df4b8aa:
both stream processes, both query readers, all three projectors.
Measured result
Consumer-side, through the real data plane with the production binding, eight
iterations at 2026-09-17T04:11:44Z, after the offset recovery:
| Feed | p50 event age at serve | Policy |
|---|---|---|
QUOTE |
437-700 ms | 2,000 ms |
TRADE |
804-1,711 ms | 2,000 ms |
BOOK_SNAPSHOT |
1,186-1,361 ms | was rejected |
OKX MARK_INDEX_PRICE |
785-892 ms | was rejected |
Request latency p50: snapshot reads 6.5-7.2 ms, book snapshot 76.6 ms,
feed status 6.9 ms, instrument lookup 4.1 ms, 1m warmup 119 ms, batched
warmup 649 ms.
OHLCV against the venues' own REST klines: 20/20 bars exact, all FINAL.
Projector steady state, 24 consumer-group snapshots over 5m56s: produced
373 records/s (132,880), consumed 373 records/s (132,894). The projector
consumed 14 records more than were produced: there is no backlog.
Thirty-one minute certification window, thirty samples: the consumer stayed
V2_PRIMARY with zero fallback to V1 on every sample, 60 slices demanded,
READY on 26 of 30, worst sample 3 of 60 slices transiently unhealthy. The
recovery lag gate passed in consecutive runs of 3, 5, 5, 8, 2 and 5 samples, so
the three-consecutive-sample acceptance is met six times over.
Also in this release
- The boundary v2.0.15 left open is closed. All five Rust roles run
qdl-v2-rust:2.0.15-c5a5be0withrustls 0.23.45, so RUSTSEC-2026-0285
is closed in the runtime and not only in source. That rollout was performed
on 2026-09-16 (ledger entry 18); this release did not recreate those roles,
it records the state they are in. - CPU ceilings are per service with the measurement that justified each one.
Declared total12.25 -> 13.35on a 16-core host, with eight services
reduced. - Canonical Kafka retention
24h -> 6h. Canonical is derived and can be
rebuilt from raw; raw stays at 24h because it cannot be refetched. Measured
across the three replicas: canonical55.6 GB -> 17.0 GB, raw
40.1 GB -> 50.7 GB(it is still 24h and the realtime volume is higher than
when the before figure was taken), total95.7 GB -> 67.7 GB. The host
filesystem is at 52%. Verified live:retention.ms=21600000is a dynamic
topic config onmd.canonical.v2.
Gates
- Python suite in the release image, network disabled, read-only source mount,
both log directories on tmpfs: 1,495 tests, 0 failures, 0 errors, 7 skipped
in 421 s. The four "pre-existing import errors" recorded against v2.0.15 were
a missing/app/logstmpfs, not a defect; with it, discovery is clean. - Rust gate inherited: no
.rs,Cargo.tomlorCargo.lockchange between
c5a5be0anddf4b8aa, and the running image is the one that passed
fmt,clippy -D warnings, 81 tests across 13 targets andcargo-deny. - CI run
35182153883ondev:contract-tests,sdk-python310and
unit-testsall green. The earlier run ondf4b8aafailed at the
"Set up Buf" download step, which is infrastructure rather than a
contract violation. - Buf contract checks verified locally in
bufbuild/buf:1.50.0:
format --diff --exit-code,lint, andbreaking --against baseline/qdl-v2-phase1.binpball exit 0.
Boundaries
- Binance
MARK_INDEX_PRICEstill answersDATA_NOT_READY. Pre-existing, not
touched here, open on its own slice. - One of sixteen OKX
MARK_INDEX_PRICEsamples was rejected onEVENT_AGE;
the mark-price/index-tickers pairing recorded in ledger entry 21 is still the
cause. - The 500-total/250-per-partition lag bound is a convergence gate, not a
health gate. It belongs to the projection-cache rebuild runbook, where its
job is to prove a replay drained. At 373 records/s a 500-record bound is
1.3 seconds of work, so an instantaneous sample of a perfectly healthy queue
crosses it — which is exactly what 5 of 30 window samples did while the
consumer stayed ready and never fell back. Steady-state health is the
produced-versus-consumed rate and the seconds-of-work figure. Correcting the
gate's use is a scheduled slice; the runbook's own use of it is correct and
was left alone. binance_bar_edgestays onqdl-v2-python:2.0.15-5130f6f: it does not use
the stream delivery path and keeps the certified BAR settlement build.- The OKX last closed 1m bar serves about 60 s behind the Binance one
(96.6 s vs 36.1 s age at measurement). Both are inside the 180 s consumer
contract; the gap is the OKX calendar boundary, not a regression. - A governed offset reset to latest was performed on
stable-projector-v1
during recovery, on a pre-production stack and with the owner's authorisation.
Canonical records between the old committed offset and latest were not
projected; raw still holds them. - The companion consumer change R1.4 lives in the
trading_systemrepository
and is not released by this tag. - This patch inherits the
v2.0.15certificate and does not recertify the full
product matrix.
Quant Data Layer v2.0.15
Quant Data Layer v2.0.15
Certified Patch Scope
v2.0.15 corrects how the stable V2 edge captures a closed Binance bar, makes
the stable stack recover itself after a host boot, and takes one published
dependency advisory.
Binance final-BAR settlement
Polling GET /fapi/v1/klines for a freshly closed 1m bar returns different
answers from different Binance replicas: measured on 2026-09-16, trade counts
cycled 3155 -> 3232 -> 3263 for about five seconds before every replica
converged. The edge read that bar once at close plus 0.10s and never revised
it, so whichever partial answer it drew became the durable bar permanently.
Measured against the venue's own REST klines before the fix, five consecutive
1m bars per Binance symbol were FINAL, revision=0 and wrong: trade count
always lower than the venue, volume short by 0.17-307 bps, close off by up to
0.46 bps. OKX matched exactly.
The edge now reads the same target bar until two consecutive reads return an
identical row and the bar is at least six seconds old, then publishes
exactly that venue row. It merges nothing, invents nothing, and fails closed
when the venue does not settle inside its read budget, leaving the existing
retry and coverage repair to own the outcome. OKX keeps its single read: its
confirm=1 candles are final on arrival.
Verification, two independent rounds over fresh bars: 40/40 bars exact on
open, high, low, close, volume, base volume and trade count across Binance
BTCUSDT/ETHUSDT and OKX BTC-USDT-SWAP/ETH-USDT-SWAP.
Cost, stated plainly: the final 1m bar now lands 6.7-8.2s after close
instead of about 2s. The consumer contract allows 180s. The previously
published close-to-final-BAR availability figure of p50 2.151s no longer
describes Binance and must be re-measured before it is quoted again.
Host-boot recovery
stable_redis is ephemeral by design, so after a boot the projectors stop
fail-closed on ProjectionCacheMismatch. Every runtime role now declares
unless-stopped, and scripts/v2_stable_boot_recovery.py with its systemd
oneshot performs exactly the governed projection-cache rebuild when, and only
when, it observes that post-boot state. Rehearsed end to end on the running
stack in 17m 44s, ending HEALTHY with the projector group inside its
bounded lag gate.
Dependency advisory
rustls moves from 0.23.43 to 0.23.45 for RUSTSEC-2026-0285, published
2026-09-14: TLS 1.3 handshake messages incorrectly accepted across encryption
level boundaries.
Gates
- Pinned
rust:1.82-slim:cargo fmt --all -- --check,
cargo clippy --workspace --all-targets --locked -- -D warnings,
cargo test --workspace --locked81 passed / 0 failed across 13 targets,
cargo-deny 0.20.2 checkadvisories, bans, licenses and sources ok. - Python discovery
1436tests with the four import errors that a pristine
devworktree also has. - CI run
35077114929ondev:unit-tests,sdk-python310,
contract-testsall green.
Boundaries
- Only
binance_bar_edgewas recreated, on
qdl-v2-python:2.0.15-5130f6f. V1, Kafka topology, Redis, SQLite, every
other role, Trading System and alpha were untouched. - The deployed
qdl-v2-rust:2.0.12-3f1c50estill containsrustls 0.23.43.
The source is fixed; that runtime rollout is a separate packet. - This patch inherits the
v2.0.14certificate and does not recertify the full
product matrix. The venue comparison covers the 1m interval for four
instruments; other intervals inherit the existing OHLCV certification.
Quant Data Layer v2.0.14
Quant Data Layer v2.0.14
Certified Patch Scope
v2.0.14 repairs an exact durable-replay optimization in the stable V2
projector. When a canonical Kafka record is already byte-identical and durable,
the projector skips a duplicate canonical sink write while retaining lineage and
idempotent compatibility-cache projection. It also corrects the release
acceptance client so its quiet-trade request exactly matches the governed
manifest entitlement.
This patch inherits the v2.0.13 certificate. Its bounded, real V2 no-order
acceptance verifies the changed runtime through two query replicas and two
stream aliases for:
- Binance USD-M
BTCUSDTandETHUSDT. - OKX Swap
BTC-USDT-SWAPandETH-USDT-SWAP. - Five closed 1m BAR warmup rows plus authoritative TRADE, durable cursor ACK
and contiguous reconnect/resume for every case.
The disposable client made no provider-direct connection and no order, signal
or sizing action. This is a single-host data-plane patch certification, not a
broker execution, independent HA/DR or VN market-hours certificate.
Runtime And Rollback
- Active V2 Python projector image:
sha256:3e062a3ba38d52d31718162bd21cd52a246e414ee117eae56481da00b8db7b4a. - Explicit V2 projector rollback image:
sha256:d190d7696f4ebe5c34f2b83bf690ac0027e2c356ca548952cf58b5a3293b134d. - Active Rust core image:
sha256:407a67131ca6567f803950aefd56c547306a20bec6992a42c44ae1719beccabd. - V1 fallback, Kafka, Redis, SQLite, query/stream, Trading System, alpha and
order paths were unchanged by this release packet.
The machine-readable scope and certificate are in
scope-evidence.json and
certificate.json. The implementation journal is
DATA_LAYER_UNIFIED_IMPLEMENTATION_PLAN.md.