Repository navigation
v3.0.0-rc.1
Pre-releaseRelease date: 2026-10-06 (release candidate; tag v3.0.0-rc.1, published as a
GitHub pre-release).
This is the first release candidate of 3.0.0, the native Rust PRISM line. It
is the tree that #291's go/no-go evaluates on the PRISM production pair: two
frontends, each with its own qbitd, a PostgreSQL primary with one
asynchronous standby, and a public-read replica. It is not a production
release. Until #291 records a go, the production line stays 2.x.x (2.0.2).
A candidate runs in pre-production with block submission and broadcasting off,
so it offers the network no found block and no transaction. A rehearsal uses
both holds, as the rehearsal procedure requires:
- hold the ledger itself with
qbit-prism-server submission-hold set --reason ...(#664, migration 023) before any frontend starts on it, so that it
holds every frontend that connects; - and start every frontend with
PRISM_BLOCK_SUBMIT_ENABLED=0(#661), which
also keeps the CTV fanout broadcaster from starting. A frontend started
without it, on a ledger with no hold, offers everypendingrow it finds.
See the block submission kill switch
and the rehearsal procedure.
Never leave either in place on a production frontend.
Highlights
- Replaces the 2.x.x Python coordinator with
qbit-prism-server, a native
Rust executable that runs Stratum, job refresh, block submission, CTV
broadcasting and HTTP; no Python runtime or builder subprocess remains in
the mining path. Operator work moves to subcommands, among them
migrate,import-audits,self-check,candidates,fatal-state,
policy-transition,signing-transition,share-archiveand
submission-hold(#244, #360, #401, #419, #439, #464, #682). - Several frontends can serve miners at once against one PostgreSQL writer
endpoint. Every instance coordinates through PostgreSQL, and only the
canonical share order and each job's accounting boundary are serialized: a
short transaction lock gives accepted shares one globalshare_seqorder,
database constraints deduplicate proofs across instances, extranonces come
from a database sequence, expiring claims coordinate block candidates and
CTV broadcasts, and a database fingerprint refuses conflicting
configuration. A reconnecting miner can resume unexpired work on another
frontend (#244, #319, #397). - Payout vectors exported from 2.x.x replay identically except for three
differences approved under decision D2: bootstrap work, which pays the
solver, is used only while there are no historical shares (2.x.x also used
it below its default three-miner gate); a proof that meets the network
target but not its share target is credited at network difficulty once its
block confirms on the active chain (2.x.x credited its share difficulty at
node acceptance); and a carry-only balance at or above the payout floor
can be paid in a bootstrap block. The 2.x.x per-worker credited-difficulty
overrides are not supported (#269, #302, #310, #316). qbit_share_ledgeris partitioned byRANGE (share_seq)(migrations 016
and 017):migratemakes the existing table the first partition behind a
validated bound, copying no rows and rebuilding no index, and every
frontend keeps empty lead partitions attached. Migration 013 trims the
ledger's secondary indexes, built concurrently on a populated ledger
(#144, #153, #369, #419).share-archiveis the share retention path: it plans, seals, archives,
verifies, detaches, drops and restores whole partitions with the frontends
running, never deletes a share, and first seals each dependent audit's
canonical bytes, so blocks keep their published digests. A miner's
last_share_atreadsnullonce its last share is archived, and
/audit/share-windowreturns no rows for an archived anchor. Stored native
audit bodies no longer grow with the payout window (#144, #267, #353,
#419).- Migration 002's share-hash backfill runs after the migration transaction,
in short batches with a durable cursor, somigrateneeds no statement
timeout above the default 15 s and resumes where it stopped. No frontend
starts until the backfill finishes, and meanwhile the
share_hash_backfill_pendingcapability fences earlier native builds off
(#582, #663, #669, #681). Migration 016 attributes block solvers in
bounded statements for the same reason (#672, #680). - Found blocks are offered to the node before their audit lands, under one
durable reservation per block, so no frontend offers a block twice, even
after a crash. Unknown outcomes are reconciled from read-only chain
observations, a proven lost tip race settlesorphaned, and a
submitblockthat provably never ran (no connection, orRPC_IN_WARMUP)
is offered again (#391, #415, #424, #522, #526, #527, #566). - A block found on work whose payout revision was superseded on the current
tip is captured and offered within a per-frontend overpay ceiling, with any
overpay recorded (migration 020) and carried as debt. Candidate and CTV
fanout claim leases are timed on the taker's monotonic clock, so a
database clock step no longer hands a live claim to another frontend
(migrations 021 and 022; #478, #504, #581, #646, #654, #657, #667, #674). PRISM_BLOCK_SUBMIT_ENABLED=0holds one frontend for rehearsals: found
blocks are enqueued but staypending,submitblockis never called and
the CTV broadcaster does not start.submission-hold setholds every
frontend from the ledger, whatever its own setting (migration 023),
journaled with a reason, andPrismBlockSubmissionHeldpages when a
frontend holds submission (#291, #661, #664, #666, #676, #682).- CTV fanouts are broadcast by several frontends through database claims.
The claim lane reads only due fanouts through a partial index (migration
024), claims are handed back at shutdown, and a pass yields to its
frontend's pending template refresh for at most the replacement-build
budget. Every settlement mode now requires a pool fee (#433, #525, #534,
#535, #568, #573, #576, #668, #679). - The HA reference (decision D3) is one primary plus one dedicated
asynchronous failover standby, separate from the public-read replica, with
a two-frontend overlay,compose.prism-ha.yaml. Failover can lose
acknowledged shares in the replication gap. An optional bounded wait lets
the standby flush a found block's rows before its offer, and, given
synchronized database clocks, blocks on work from a lost gap are refused
after a promotion. Promotion, fencing and the TCP load balancer are
operator-supplied, the last against a documented/healthzprobe contract
(#281, #304, #331, #449, #529, #570, #619, #645). - Stratum keeps the 2.x.x miner contract, including username syntax, BIP310
version rolling, vardiff, the high-difficulty listener and the one-parent
stale grace (3 s by default). A per-source connection cap and per-session
budgets for malformed frames, unknown job IDs and authorize attempts are
new and off by default. A session opening during a same-tip rebuild storm
no longer waits for the whole fan-out, overtaken same-tip builds are still
delivered, and submits are answered while a session's rebuild is in flight.
A plain share the payout-revision fence refuses is answeredstale-job,
notledger-confirmation-failed(#244, #313, #468, #471, #598, #604, #616,
#621, #625, #642, #675, #698). - Template refresh keeps its incremental path across retargets, builds on a
per-process worker pool (PRISM_REFRESH_BUILD_THREADS), and renews
readiness from a guarded tip poll, so a slow rebuild no longer refuses
current work as stale. After each block it lands, a frontend returns the
freed heap to the kernel (PRISM_LANDING_MALLOC_TRIM_ENABLED, default 1)
(#275, #501, #502, #600, #622, #627, #643). - Coordinator metrics come from one native registry, with a generated
inventory covering both roles, and/metricsanswers 200 with freshness
headers. The repository specifies the native alert rules and a review-only
diff for the deployed rules that maps each retired Python alert. The
native rules cover a stalled work refresh, a PostgreSQL outage and held
block submission (#277, #278, #279, #299, #308, #320, #534, #646, #676). - The new
qbit-prism-loadharness drives real Stratum sessions against
real frontends, PostgreSQL with a streaming standby and a fake or real
regtest node, and reconciles the ledger exactly with what clients were
acknowledged. Presets model recipient weights, bursty arrival and rental
churn; afaultsphase injects faults under load;externalmode drives
deployed frontends through their load balancer; and #511's release
benchmark adds an A/B driver and a D1 comparison summarizer (#271, #342,
#521, #536, #539, #547, #554, #564, #590, #623, #635, #662). - CI runs PRISM end to end. Required CI fails unless every gated test, live
regtest included, runs without skipping. A nightly workflow runs load
presets, real-node scenarios and Stratum fuzzing, and therun-loadlabel
runs its preset set on a pull request; weekly jobs run the 2.x.x migration
lifecycle and a measured mainnet-shaped cutover on a real node, a 5.5 h
soak, and the shipped images under Compose with the HA overlay. The L3
production-window matrix runs on a version-bump pull request, the
release-candidatelabel andv*tags, and a reduced matrix weekly. None
of these lanes is a required check (#322, #487, #544, #545, #549, #550,
#553, #565, #575, #587, #588, #594, #596, #597, #608).
Upgrading from 2.0.x
Follow the migration guide. Native
migration is one-way, with no down-migrations (decision D5, #287, #366):
Before the first native share is acknowledged, restore the complete
pre-migration database and artifact backup and restart the pinned old image
in isolation from the migrated database. After the first native share is
acknowledged, restoring an older database loses those accepted records.
Any rollback that discards acknowledged history requires an explicit
accounting reconciliation and recovery decision; it is not an ordinary
image rollback. Keep every native frontend stopped while performing an
isolated restore/recovery operation against its replacement database.
- Drain pending candidates with a v2.0.1 or later image (v2.0.2 for a v2.0.2
database), then stop every 2.x.x process;migratetakes v2.0.0 to v2.0.2
sources and refuses an undrained one. Runcheck-config,migrateand
import-audits, whose two missing counts must be 0, after rehearsing on
an isolated restore. - On a populated ledger the outage includes 002's share-hash backfill (35 to
over 80 minutes for 65.5M shares) and the online steps of 013, 017 and 024,
which can take hours. Earlier native builds are refused while the backfill is
pending; never record migration 2 by hand (#582, #669). - The 102 retired 2.x.x settings are listed in
retired-settings.txt.
check-configandrunname any still
set, and production mode refuses them (#361, #462). - Production reads signing seeds from
_FILEmounts, and every settlement
mode needs a pool fee with one recipient, 0 bps allowed (#535). - Block totals, chart markers and the default
chain_state=activeview of
/public/v1/blockscount confirmed blocks only. Native block intents are
persisted before node acceptance and stay out of them until confirmed;
chain_state=allshows every candidate with its state. - This candidate requires every native schema migration through 024.
migrateapplies them, and every start refuses a database missing one. - It is tested against qbit 1.0.0, the release
.github/scripts/install-prism-qbit.shpins. - The coordinator listener on 3341 now also serves
/public/v1(2.0.x
answered 404); keep it private and send public traffic topublic-apion
3342, which needs no audit mount. Runbooks:
ledger operations,
HA,
alerts and
go-live checks.
Verification
A candidate is verified by the full suite (#557):
- CI (
ci.yml), on the version-bump pull request, and dispatched by hand
on the tag, sinceci.ymlhas no tag trigger: lint, compile and Compose
validation; the Python and Rust unit tests; the PostgreSQL contract shards
and the proof that every gated test executed; the dependency audit; and the
Docker and real-qbitdimage builds. - The L3 production-window matrix (
prism-load-l3.yml, suitel3-full:
every #473 cell, three repeats, on 32 vCPU runners). It runs by itself on
the version-bump pull request. Thev*tag promotes that run when the
tagged commit's tree is the one the run measured, and runs the suite again
otherwise (#550, #608). - The load harness (
prism-load-nightly.yml), dispatched by hand on the
version-bump pull request and on the tag, withpreset=all live=true weekly=true fuzz=true images=true bridging=true:- every nightly and manual preset, including #473's 400k and 500k window
cells at 1, 2 and 4 frontends, the 200k one-frontend cell and the
mainnet shapes; - the bridging pair (#552);
- the live regtest nightly and weekly scenario lists (#521, #553, #575);
- the Stratum fuzz targets;
- the shipped images under the Compose
prismprofile with the HA
overlay (L6, #544).
- every nightly and manual preset, including #473's 400k and 500k window
- The 5.5 h soak and the long fault set (
preset=weekly:soak-weekly
andfaults-long-real-node). For this candidate the soak runs once, at
3.x.xc95febb3. The only runtime change between that commit and this
candidate is #698 (#675).faults-long-real-noderuns there and again on
the version-bump pull request. Their verdicts are in the evidence bundle.
The evidence bundle on the v3.0.0-rc.1 GitHub pre-release lists:
- every run, with the commit and tree it tested;
- each job's verdict and its artifacts;
- any failure, recorded as a named exception with its issue.
#291 links to the bundle. Job and gate verdicts are correctness evidence for
#291. The measured rates are not D1 verdicts (#487 decision 5).
Known issues
Open on 3.x.x when this candidate was cut:
- #695: at the 20x growth shape (8,000 sessions at a 1,000/s mean), share
appends queue on the global order lock and reach 420 to 675 shares/s, short
of the shape's rate. Today's mainnet shape (130 addresses) and the 5x shape
(650) kept up with their offered rates, with zero shortfall; their headroom
was not measured. This is a capacity ceiling for D1 (#477), not a
correctness defect: no acknowledged share was lost and reconciliation was
exact. - #600: a 400k-window frontend's resident set ratchets up with block
landings, from allocator retention. #627 returns the freed heap after each
landing. Making jemalloc the allocator (#628) and judging the soak's memory
on per-cycle floors (#629) are open, and the weekly soak still records #600
as an expected failure. - #602: block landing and settlement hold the order lock, which can delay
share acknowledgements around a pool block. - #622: new-tip delivery missed a 30 s bound with about 3,000 sessions at a
1 s reanchor. #643 fixed the mechanism found; the issue stays open until
that scenario is re-run. Production reanchors every 60 s. - #630: a payout-revision race during the post-offer landing raises
"offered candidate could not be processed" about twice per 30 minutes
under load. The ledger runbook says when those ALERT lines are benign and
when to act on them (#696). - #670:
broadcast-ctvcannot take over a dead frontend's fanout claim
within one pass. - #688, #689 and #690: the public pending-fanouts query, the CTV claim lane
in a backlog, and fatal-state clearing each do work that grows with the
number of fanouts. - #582 (the share-hash backfill on a production-sized ledger) and #604
(first jobs queued behind rebuild storms) are fixed in code (#663, #680,
#681; #625) and stay open for #291's go/no-go sweep. - #701: in
faults-long-real-nodeatc95febb3, one public-API request
failed with a connection error during the block failover. It was the run's
only read-tier failure, and it fell outside the window in which the public
API refuses by design. #291 requires the public API to keep answering
through the failover drill.
Before a go, every acceptance criterion of #291 must be met. When this
candidate was cut, the open ones included:
- the production-sized migration rehearsal with its timings;
- the D4 key-rotation rehearsal and the D5 isolated-restore rehearsal;
- the failover drill and the load-balancer exercise under mining load, with
prism-public-apianswering/public/v1throughout; - the two-hour soak against its criteria (#556);
- D1 throughput on the rehearsal host (#477).
Found after the cut
- #708: the legacy carry-forward integrity rule partitions by a
case-sensitive label, soself-checkrefuses a production 2.x.x ledger
with false mismatches (the money is per payout program). The fix is a
runtime migration, 025, so production is expected to run a 3.0.0-rc.2
(this candidate plus 025), not rc.1.
Evidence (#557)
Tagged commit 67b73e87 (tree 5de7dff6), from version-bump PR #703 on 3.x.x b0627b45.
| Lane | Run | Result |
|---|---|---|
CI on #703 (merge ref, head 1370b090) |
37417900896 | pass, 20/20 jobs |
L3 l3-full on #703 (tree 5de7dff6) |
37417900875 | complete and passed: 33 of 33 runs reached and passed their gate |
| L3 on the tag | 37423477460 | promoted #703's run 37417900875 on the tree match (5de7dff6) and carried its verdict: complete and passed, no rerun |
| CI dispatched on the tag | 37423587920 | pass, 20/20 jobs, including the real-qbitd image smoke |
| CI on the 3.x.x push of the tagged commit | 37423439080 | pass |
Nightly set on #703 at 8c60ab23 (docs-only difference from the tag) |
37410760555 | 25 of 27 presets pass, plus the live regtest nightly and weekly lists, fuzzing, L6 and bridging. Two named exceptions, below |
Soak and long fault set at 3.x.x c95febb3 |
37403129136 | 5.5 h soak: every gate passed. That covers one server lifetime each, memory flat (peak 1.17x and 1.13x the warm-up), descriptors and connections flat, WAL peak 1040 MiB of 2048, 4 partition rollovers, 2 partitions archived and dropped, 0 payout divergences, zero lost shares and exact reconciliation. The run is marked failed only by the preset's designed #600 looks fixed tripwire, below. faults-long-real-node: named exception #701 |
First pass at 3.x.x c95febb3 |
37403125243, 37403127199 | CI pass. Nightly: 22 of 23 presets (#695), plus live, fuzzing, L6 and bridging |
The only runtime change between c95febb3 and this tag is #698 (#675): coordinator/miner_submit.rs, plus tests and a doc comment in ledger/window.rs.
Named exceptions (diagnosed; not release blockers):
soak-weekly(#600): its memory gate is set to expect #600's resident-set ratchet. Every memory row passed, so the soak reports#600 looks fixedand fails by design until the preset'srss_expected_failuregoes back to null. #627's post-landing heap release is the likely fix.mainnet-shape-2600-addresses(#695): at the 20x growth shape, share appends queue on the global order lock at 420 to 746 shares/s against a 1,000/s mean. It is a capacity ceiling with every correctness check passing, reproduced at both commits.faults-long-real-node(#701): every functional check of all 12 faults passed. One public-API request in the last fault failed because the harness's own read-tier stop killedpublic-apiunder it. Fixed in #702.
Job and gate verdicts are correctness evidence for #291; the measured rates are not D1 verdicts (#487 decision 5). The raw bundles of the nightly set, the live lists, fuzzing, L6, bridging and all 33 L3 runs are attached below as <lane>-<preset or job>-run<id>-attempt<n>.tar.gz. The nightly set ran at 8c60ab23, which differs from the tag only in doc/. The index is rc-evidence-v3.0.0-rc.1.md (and .json), generated from the Actions API by the attached rc_bundle.py with rc-evidence-v3.0.0-rc.1.config.json. It lists every run with its commit and tree, each job's verdict, every preset's gate and per-phase rates, and every artifact. The soak's and the first faults-long-real-node's bundles are attached too.