v0.14.0

[0.14.0]

Added

• A latency-first logs fetch policy (ADR-0996 amendment, superseded by
  ADR-1196). Measured on a reference cold-cache corpus, the cost-based
  default resolves to whole-object reads and moves 3x the bytes of
  byte-minimal at a deployment where transfer and retrieval are free,
  because the derived per-request rate saturates. --logs-fetch-policy latency-first resolves the same byte quantities as byte-minimal. It is
  an intent, not a tuning constant: it carries no concurrency default of its
  own, and resolves --store-get-concurrency, --sql-partition-count, and
  --promql-fetch-fanout exactly as every other policy does. cost-based
  stays the default; latency-first is an operator opt-in for deployments
  where cold wall-clock matters more than the request bill, and it pays off
  only once the operator raises concurrency explicitly to the measured
  configuration (ravel_query::LATENCY_FIRST_MEASURED_CONCURRENCY, 256), at
  a measured cost of about 5.45x the GET requests for about 41% less cold
  time. Raising that concurrency also raises in-flight fetch memory, which
  is not yet bounded by a process-wide budget (issues #1170, #1007); the
  flag's own documentation and a startup log line under latency-first make
  that precondition operator-visible.
• The alerts and audit SQL tables (ADR-1101). POST /api/v1/sql and
  Flight SQL serve five tables, and Flight GetTables lists all five; naming
  two in one query is still rejected before any listing. alerts is alert
  history, one row per state transition, and each row carries the write
  identity of its record (writer_id, writer_epoch, writer_seq), so a
  ROW_NUMBER() fold ordered by time and write identity returns exactly one
  current row per alert even when two evaluators overlap at a lease handover.
  audit reads back a tenant's own legal-hold and reshard records, which the
  maintenance process writes directly. It also serves query-audit records, but
  no shipped startup path installs the pipeline those go through, so
  attrs['kind'] = 'query' selects nothing until a deployment attaches one.
• A read-side shard floor for fixed-shard signals (ADR-1101). Alert and
  audit writers pin their shards by constant and neither signal is provisioned,
  so the catalog's scan-set derivations now take the maximum of the
  provisioning history and the signal's fixed shard count. An audit query on
  a --shards 1 deployment reads the query-audit shard instead of silently
  omitting it, and a wider deployment scans exactly as before.
• ravel-memory, one process-wide memory budget (ADR-1170). The server
  derived four memory ceilings from the host and enforced each in a component
  that knew nothing of the others, so N tenants could each reserve half the
  box. ravel-memory is a leaf crate holding one counter every ledger draws
  from: a compare-and-swap reserve for the SQL adapter and an RAII reservation
  for the fetch layer.
• Bloom pruning for PromQL __body__ matchers (ADR-1103 follow-up). A
  __body__ equality, or an anchored regex with a token-bounded mandatory
  literal run, now pushes that literal onto the scan as a has_word
  predicate, so the RLOG block bloom skips blocks before decode. The
  per-record check still runs on every decoded record: the pushed word only
  prunes, it never decides. Negated matchers, unsupported metacharacters and
  +-quantified patterns are rejected by the extractor rather than pushed,
  because token matching is not a superset of substring matching.
• A gate against wall-clock waits in injected-clock tests.
  scripts/check-injected-clock-helpers.sh, run by gates.sh and CI, fails on
  thread::sleep, tokio::time::sleep, tokio::time::timeout, Instant::,
  SystemTime, a bare sleep(...) or .elapsed() inside a helper that takes
  a TestClock or FixedClock, unless the line carries
  // allow-wall-clock: <reason>. Its default scope is the loader's test
  module in ravel-cli; the one wait it found there was made clock-driven
  rather than exempted.
• scripts/verify-dispatch-gates.sh --with-gates runs gates.sh itself
  inside the cold worktree instead of a hand-listed command set, so a
  dispatched branch is checked against the same feature lanes CI runs and the
  run leaves a gate receipt the merge script can reuse.

Changed

• Cache warm-up keys off each tenant's latest ingest hour, not the current
  hour. On a tenant whose data is older than the warm-up window the previous
  pass issued about 1,900 small object reads from the first query's own path
  before warming nothing; those reads are gone, and the first query on a cold
  process is about 3 s faster on the reference corpus. The replacement probe
  costs about 1 s at startup, so the end-to-end saving on a cold start is
  about 2.5 s, not 3. The probe fans out on the configured resolve concurrency,
  asserts tenant isolation on every listed key, and is bounded per shard.

• Catalog resolve GET concurrency is configurable, default 128. Every
  record GET in Catalog::resolve_impl passed through a fixed bound of 16.
  Measured against S3 on a 10,000-record unsealed tail, one cold resolve each:
  23.2 s at 16, 4.4 s at 64, 2.3 s at 128, with the same 10,001 GETs at every
  level. Cold resolve is concurrency-bound; this was the lever.

• SQL reservations are charged to the process budget (ADR-1170). Every SQL
  reservation now flows query, then tenant, then process on the way up, with a
  process refusal rolling the tenant and query charges back before surfacing
  as ResourcesExhausted naming the process figures. The infallible grow path
  trips a ceiling breach when the process limit is exceeded, so a DataFusion
  overshoot still ends in a typed error on the stream's next poll rather than
  an unaccounted allocation.

• --fetch-concurrency unbundled into three flags (ADR-1195): the SQL
  scan partition count, the PromQL/analytics per-query fetch fan-out, and the
  object-store GET concurrency were one knob with three coupled effects; they
  are now --sql-partition-count, --promql-fetch-fanout, and
  --store-get-concurrency, each independently sizeable. --fetch-concurrency
  still sets all three together for a config that predates the split (source
  legacy-flag in the startup log). Combining it with any of the three new
  flags is a startup error naming both flags, and a value of 0 in any of the
  four is a startup error naming that flag, raised before any fetcher, engine,
  or SQL session exists.

• GET concurrency is process-wide, not per engine (ADR-1195): ravel-server
  now builds exactly one Arc<GetLimiter> where it assembles its shared state
  and hands that same Arc to every fetcher- and engine-construction site in
  the process (the PromQL query path, the SQL executor's RSEG/RLOG/RSPAN
  fetchers, the distributed fragment path, cache warming, exemplars, and
  alerting). Before this, each RSEG and RLOG fetcher held its own semaphore, so
  N fetchers each configured to "8 concurrent GETs" could together put 8N GETs
  in flight against the store. Two behaviour changes follow. RSEG fetchers now
  honour the configured limit instead of the compiled default of 16, so a host
  whose derived value is below 16 issues fewer concurrent GETs than before.
  RSPAN fetchers are bounded for the first time: span reads previously ran with
  no GET limit at all, so a span-heavy deployment can see lower read
  concurrency and should size --store-get-concurrency for it. No fetcher in
  the server process owns a private limiter anymore.

Fixed

• The RLOG plan phase's whole-object read is carried into the scan. On the
  whole-object fallback, plan_segment fetched each object to plan it and the
  scan fetched the same object again. The plan phase now hands its bytes to
  the scan, which short-circuits on them before any GET and charges them to a
  bytesReused figure rather than a cache hit. Retention is bounded: the
  first segments to complete their plan keep their buffer, up to the SQL
  partition count, and every later segment is re-fetched exactly as before, so
  peak retained bytes are the partition count times the object size, never
  the corpus. The saving is therefore about one duplicate read per unit of
  plan fan-out; removing the rest needs the carry to stream per partition
  instead of being held at the plan barrier, tracked separately. Correction. This change landed after the 0.14.0 tag and ships in 0.15.0, where the retention bound is the plan fan-out rather than the SQL partition count.
• The RLOG raw prefetch is gated on the cursor budget before try_join!
  (ADR-0979 decision 4). The merge cursor's refill fetched the next two
  row-group blocks before the budget had been checked, so up to twice the
  group size was allocated and only accounted for on the following iteration.
  The pending fetch window is now priced from resident metadata and reserved
  before the fetch is issued.

What's Changed

• feat(promql): pin evaluator contract for equal-timestamp samples by @pmoust in #1149
• docs(adr): amend ADR-0873 for the .cstat identity and STR rule by @pmoust in #1178
• feat(query): add the PromQL-over-logs series source by @pmoust in #1158
• fix(promql): compute rate-family intervals directly from nanoseconds by @pmoust in #1183
• feat(sql): register the alerts and audit tables on both SQL transports by @pmoust in #1186
• docs: describe the alerts and audit SQL tables by @pmoust in #1189
• feat(query): answer PromQL log selectors from the logs signal by @pmoust in #1188
• docs: mark ADR-1101 accepted by @pmoust in #1190
• fix(maintain): resolve the retention window like the fold does by @pmoust in #1179
• fix(ingest): stamp declared columns over the merged attribute view by @pmoust in #1167
• test(server): prove PromQL over logs end to end, and document it by @pmoust in #1193
• feat(server,cache): expose per-subsystem process memory diagnostics by @pmoust in #1205
• docs: correct two statements the TLA+ reconnaissance found false by @pmoust in #1212
• docs(ravel-query): scope the covering-read resident-memory redesign by @pmoust in #1208
• test(tla): add the online resharding TLA+ model by @pmoust in #1210
• test(formal): add the catalog fold, snapshot and MVCC model by @pmoust in #1211
• fix(server): derive the read cache from 25% of MemTotal, not 80% by @pmoust in #1213
• test(tla): add the maintenance ownership and compaction claim models by @pmoust in #1209
• test(formal): model commit publication, acknowledgement, retry and read-your-write by @pmoust in #1176
• docs: split the fetch epic into two decidable ADRs by @pmoust in #1197
• test(formal): add the lifecycle GC TLA+ model by @pmoust in #1206
• docs(lifecycle): label fenced code blocks with language identifiers by @pmoust in #1223
• fix(tla): enforce the TLC wall-clock budget with GNU timeout(1) by @pmoust in #1200
• docs(tla): add the verification suite report and accept ADR-1113 by @pmoust in #1225
• test(object-store): pin four contract claims the TLA+ suite named by @pmoust in #1226
• fix(cli): refuse writes to a marker-less bucket, add tenancy resolve by @pmoust in #1207
• test(maintain): pin eight lifecycle and catalog claims the TLA+ suite named by @pmoust in #1227
• test(failure-tests): pin four commit-protocol claims the TLA+ suite named by @pmoust in #1229
• fix(catalog): tolerate shard-count drift for provisioned tenants by @pmoust in #1181
• test: pin five catalog, ingest and fleet claims the TLA+ suite named by @pmoust in #1232
• feat(ravel-query): share one GET limiter across an engine's fetchers by @pmoust in #1234
• fix(ravel-query): permit every RLOG GET and harden the limiter tests by @pmoust in #1239
• test: pin the CLI lead check, CLI publication and memo write mode by @pmoust in #1240
• docs(tla): reconcile five figure and coverage inconsistencies by @pmoust in #1242
• docs(tla): record the regression tests in the traceability tables by @pmoust in #1241
• feat(ravel-query): record and report a query's I/O dependency shape by @pmoust in #1231
• test(ravel-query): pin the GET limiter's builder paths and RLOG permits by @pmoust in #1246
• docs: make the TLA+ verification suite a highlight and add its guide by @pmoust in #1249
• docs: add ADR-1170, one process-wide memory budget for ravel-server by @pmoust in #1245
• docs: add ADR-1199 on bounded query I/O and tail checkpoints by @pmoust in #1218
• feat(memory): add ravel-memory, the process-wide budget accountant by @pmoust in #1262
• test(ravel-cli): make max_flush_delay pair test clock-driven by @pmoust in #1265
• feat(ravel-server): split fetch concurrency into three flags, one limiter by @pmoust in #1261
• feat(ravel-catalog): make resolve GET concurrency configurable, default 128 by @pmoust in #1263
• fix(ravel-server): pass the resolve concurrency in the limiter test by @pmoust in #1268
• fix(rlog): gate raw prefetch on cursor budget before try_join! by @pmoust in #1269
• docs: scope the serviceBatches lower-bound claim to a single wave by @pmoust in #1264
• feat(scripts): add --with-gates mode to verify-dispatch-gates.sh by @pmoust in #1270
• fix(cache-warm): warm from each tenant's latest ingest hour, not now by @pmoust in #1271
• feat(ravel-query): bloom-prune body matchers via HasWord by @pmoust in #1266
• feat(sql): charge every SQL reservation to the process memory budget by @pmoust in #1274
• docs(adrs): name the current RLOG stamping symbols in ADR-0049 and ADR-0095 by @pmoust in #1277
• feat(ravel-server): add a latency-first logs fetch policy by @pmoust in #1280
• feat(scripts): gate wall-clock waits in injected-clock test helpers by @pmoust in #1285
• feat(catalog): count commit records served from the resolve record cache by @pmoust in #1283
• chore(release): prepare 0.14.0 by @pmoust in #1315

Full Changelog: v0.13.0...v0.14.0

Downloads

The attached binaries were extracted unmodified from the signed container
images this release published, so each one is byte-identical to the file inside
the image you can pull. They are glibc-dynamic, built against Debian 12
(the runtime image is gcr.io/distroless/cc-debian12), so they need glibc 2.36
or newer. On an older distribution, run the container image instead:

docker pull ghcr.io/nofireai/ravel-server:0.14.0

One binary per platform, named <binary>-<os>-<arch>:

• ravel-server-linux-amd64, ravel-server-linux-arm64
• ravel-cli-linux-amd64, ravel-cli-linux-arm64
• ravel-operator-linux-amd64, ravel-operator-linux-arm64
• ravel-ingest-router-linux-amd64, ravel-ingest-router-linux-arm64

Verifying the downloads

SHA256SUMS covers every attached binary and symbol archive. Check the files
you downloaded, then verify SHA256SUMS itself against the signature and
certificate beside it. The signature is keyless, bound to this release
workflow's identity, so there is no key to obtain:

sha256sum --check --ignore-missing SHA256SUMS

cosign verify-blob \
  --certificate SHA256SUMS.pem \
  --signature SHA256SUMS.sig \
  --certificate-identity 'https://github.com/NOFireAI/ravel/.github/workflows/publish-images.yml@refs/tags/v0.14.0' \
  --certificate-oidc-issuer 'https://token.actions.githubusercontent.com' \
  SHA256SUMS

Debug symbols

The shipped binaries are stripped of debug info but keep their symbol table,
so a backtrace resolves function names with nothing extra installed. Line
numbers come from the matching symbols archive. Each binary records the
basename of its symbol file, so unpacking the archive next to the binary is
enough and no symbol-file step is needed:

tar -xzf ravel-debug-symbols-linux-amd64.tar.gz
gdb ./ravel-server-linux-amd64

gdb looks for ravel-server.debug in the same directory (and in
.debug/), which is the name inside the archive. Do not rename the extracted
.debug files.