Repository navigation
v0.2.0
Operational release: a projection can now say how far behind it is, be
tested without a pipeline, be inspected after it dead-letters, be rebuilt,
and survive a rolling deploy. Upgrading from 0.1.x means one migration —
Scriba.Migrations.up(from: 1) — and nothing else breaks.
Changed
-
Documented the assumption
:positioncarries: a global, monotonic,
numeric ordinal over every event a projection consumes. Dedup, cursor
monotonicity, the watermark and:start_fromall rest on it, and
Commanded'sevent_numbersatisfies it — which is also why the cursor
carries over fromcommanded_ecto_projections. A store whose position is
not a single increasing integer (commit/prepare pairs, a vector clock) is
a design question rather than an adapter detail, and is left open rather
than guessed at. -
bench/gained a suite of experiments that need a real event store rather
than a real database: acknowledgement loss under a straggler, standby
takeover, the watermark end to end, subscription contention, and the
broadway_dashboardintegration. Each one refutes or confirms a claim the
documentation makes, which is why they are kept rather than deleted once
the question was answered. The contention experiment is the reason standby
exists: it disproved a predicted crash loop and found the real gap, which
was that nothing retried. -
Documented that the target is the transaction boundary — read-model rows,
cursor advances and dead letters commit together or not at all — and that
cursor and dead-letter storage will not be split into a separate
behaviour. Splitting them would either leave them in the same transaction
anyway or turn effectively-once delivery into at-least-once with a crash
window.Other sources and targets are not being prepared for speculatively. An
interface with one implementation behind it encodes that implementation's
assumptions, which is how the acknowledgement defect fixed in 0.1.3
survived a green test suite. The README andScriba.Targetnow invite an
issue instead: another adapter gets built with whoever needs it. -
A producer that cannot get its subscription now stands by instead of
failing. A persistent subscription admits one subscriber, so on a
multi-node deployment every node but one is refused. Previously the loser
raised duringinit, which meantDynamicSupervisornever adopted it and
start_projection/1returned an error — no crash loop, but no standby
either: nothing retried, so a failover had nobody to fail over to.The producer now starts, retries in the background, and acquires the
subscription when the holder releases it. The retry curve has three
phases, because two different failures share the path: five attempts in
milliseconds for the reap race after a deliberate producer death, then one
a second for half a minute — a killed tree holds its names until its
slowest in-flight handler returns — then about once a minute with jitter,
so standbys that started together do not retry in lockstep.
Configuration errors still raise; retrying those forever would hide them.Two new telemetry events make a takeover observable:
[:scriba, :source, :standby]per failed attempt, and
[:scriba, :source, :subscribed]when acquired. A standby's projection
reports:running, because its pipeline is up — the telemetry is what
distinguishes the node doing the work.This also covers a cutover from
commanded_ecto_projections: Scriba can
be started while the old projector still holds the subscription name, and
picks up when it stops.
Added
-
REBUILDING.mdandScriba.reset/2. Rebuilding a read model is a
procedure, not a function: declare the samenamewith a newversion,
start it from:originwith its own subscription name, watch:lag_ms
fall, point reads at the new table, then retire the old version.
reset/2is the last step — it clears a version's cursors and watermark
(and dead letters on request) so it can start over, refuses while the
projection is running, and does not touch the read model, which Scriba
does not know the shape of.The guide is explicit about what will bite: handler side effects replay
over the whole history, and dead letters are not replayed — a rebuild
that ends with the same error kinds fixed nothing. It also records what
Scriba will not do: shadow targets and an atomic swap, which are the
application's decision, and progress as a percentage, which the source
cannot supply because Commanded exposes no head position. -
Dead-letter inspection:
Scriba.dead_letters/2lists rows with
filters, paging and ordering;Scriba.dead_letter_stats/2summarises how
many, of what kind, over what span.Scriba.DeadLetter.list/3,count/3
andstats/3are the underlying API.They read the table rather than a running process, and the module form
takes the repo from the projection's own config, so a halted or stopped
projection can be inspected — which is when anyone actually looks.The
by_error_kinddistribution is the diagnosis: one kind on one stream
is a poison event, one kind across every stream is a schema problem that
dead-lettering is papering over.No replay function.
:event_datais stored serialized —__struct__
becomes a string — so a row records what failed rather than a value that
can be re-dispatched; replay has to read the event from the source by
position, and needs an ordering policy this does not yet have. -
README documents running
broadway_dashboardagainst a projection. It
works as-is: a projection is a Broadway topology, and the dashboard both
discovers it throughBroadway.all_running/0and accepts Scriba's
{:via, Registry, ...}pipeline names. Scriba will not ship a dashboard of
its own. -
:lag_intervalset inuse Scriba.Projectionwas accepted and then
ignored — the macro validated it but never passed it on, solag_interval: 0
did not disable anything and the Coordinator always used the 5s default.
Found by the release audit, between the option shipping and the release. -
[:scriba, :projection, :lag]telemetry. The Coordinator emits it on a
timer —:lag_interval, default 5s,0disables — carryinglag_msand
thewatermarkit was derived from, with the projection'sstatusin the
metadata.On a timer rather than on traffic, deliberately: a projection that has
stopped receiving events emits nothing else, which is exactly when an
operator wants to know how far behind it is. It stays silent until the
projection has committed something, because a projection with no watermark
reportinglag_ms: 0would read as caught-up when it has not started.Throughput gets no event of its own. Broadway's batch telemetry and the
per-event:stopevents already carry the rate, and a second number could
disagree with them. -
scriba_watermarks: the contiguous global position a projection has
reached, surfaced as:watermarkand:lag_msonScriba.info/2. Every
event at or below the watermark has been committed, skipped or
dead-lettered, with no gap below it — so it is a position a replica could
resume from, and the number that says how far a rebuild has got. Per-stream
cursors answer neither question: a minimum across them ignores streams the
projection never wrote to, a maximum counts work above an event still in
flight.:lag_msis measured from the event's ownoccurred_at, not from the
event store's head — Commanded's adapter behaviour exposes no head
position, so event-count lag is not obtainable, and time-lag is what
operators alert on anyway.The source already computed this number to acknowledge safely (0.1.3); this
persists it. Written outside the commit transaction and throttled to about
one write a second while events are in flight, flushed immediately once the
projection catches up — so it can trail what was applied but never runs
ahead of it, which is the only direction that is recoverable. -
Migration steps are idempotent (
create_if_not_exists). Without that, the
documented upgrade path broke fresh installs: an app's original migration
callsup(), which means "latest", so a new database got the version 2
table from it and then the upgrade migration collided. Found by running
examples/bankend to end as part of the release gate. -
Versioned migrations.
Scriba.Migrations.up/1takes:fromand:to,
so a schema change ships as a numbered step: version 1 is
scriba_positions+scriba_dead_letters, version 2 adds
scriba_watermarks. A fresh install still callsup(); an existing one
adds a migration callingup(from: 1). Scriba tracks no migration state of
its own — the version lives in the user's migration file, where Ecto
already records what ran. -
Scriba.Testing— run a projection's handlers in a test without a
pipeline.project/3applies events through the projection's configured
target and commits them, so a test asserts on read-model rows;handle/3
calls one clause with no database, for asserting the shape a handler
returns. Events run in one transaction, in order, with per-stream cursors
advancing as they do in production, and the result reports what committed,
skipped, failed, or came back in a shape the target cannot apply.It deliberately stops at the handler and the commit: no retries, no
dead-letter routing, no dedup, no telemetry. Those decisions live in the
pipeline, and a second implementation of them here would be the copy that
drifts.
Changed
- The release gate (architecture §14) now requires the CHANGELOG entry to
account for everything in the release, checked against
git log <previous tag>..HEAD, and records that a published tag is not
moved afterwards.