qbit-mining-bootstrap 3.0.0 Release Notes

Release date: 2026-10-22 (tag v3.0.0, cut from the 3.0.x branch).

Since 2.x.x

Native Rust mining path: qbit-prism-server replaces the Python coordinator and builder subprocesses, handling Stratum, job refresh, block submission, CTV broadcasting and HTTP while retaining the existing miner protocol.
Multiple serving frontends: PostgreSQL coordinates share ordering, duplicate detection, block candidates and broadcasts across instances. Miners can reconnect to another frontend and resume unexpired work; an HA deployment reference covers the database standby and failover.
Stronger block handling and operations: durable submission reservations, reconciliation of uncertain node responses, submission holds, native metrics and recovery commands support crashes, rehearsals and incident recovery.
Large-ledger maintenance: partition-based share archiving, bounded audit imports, parallel recovery exports and deferred share-hash backfills improve maintenance and shorten the migration outage.
Upgrade planning required: moving from 2.x.x requires a one-way database migration and configuration review. There are documented payout-accounting changes, and per-worker credited-difficulty overrides are no longer supported. Follow the migration guide and the upgrade instructions below.

3.0.0 is 3.0.0-rc.6 plus #770's fix for #765, in which import-audits held
SETTLEMENT_LOCK while it sent and stored each legacy audit body, so a large
body stalled settlement and template refresh on the frontends serving beside
it, and #760's fix for #759, in which the node RPC client reused an idle
connection past qbitd's own idle close. Those two pull requests were prepared
as 3.0.0-rc.7 (#772), which was never tagged. 3.0.x branches from 3.x.x
at #770's merge, 4c134913. 3.x.x continues as 3.1 from 3.1.0-rc.1, which
adds the dual writer and migrations 027, 028 and 031; 3.0.0 has none of them.
#291 records the cutover qualification and the go/no-go for the production
rollout.

Everything in the 3.0.0-rc.1,
3.0.0-rc.2,
3.0.0-rc.3,
3.0.0-rc.4,
3.0.0-rc.5 and
3.0.0-rc.6 notes applies, with the changes
below; see also the
development-line status. Like
rc.6, an upgrade from the previous candidate replaces the binary only: 3.0.0
adds no migration (Upgrading).

Changes since 3.0.0-rc.6

#770 (#765): import-audits writes each legacy audit before it takes
SETTLEMENT_LOCK, holds the lock only to check the write guard and
commit, and bounds every phase of a row's transaction.
- Before: for each row, rc.6 took SETTLEMENT_LOCK, checked the write
  guard, and only then encoded the row's UPDATE and sent it: the canonical
  body as bytea, beside the audit's commitment and witness Merkle leaves
  as JSON. The lock was held while the client encoded and sent the
  statement and while the server stored it. Nothing bounded the send: the
  session's statement_timeout (PRISM_DATABASE_STATEMENT_TIMEOUT_MS,
  15 s by default) starts only once the whole statement has arrived. In
  #765, a legacy audit with a ~112 MB canonical body held the lock for
  about 90 s, until it was killed, while the server waited for the rest
  of the statement (idle in transaction, waiting on ClientRead).
  Every other SETTLEMENT_LOCK user waited behind it: the frontends'
  advisory-lock waits timed out, template refresh was deferred, and
  settlement would have waited too. docker stop didn't stop it either:
  in a container the command runs as PID 1, which ignores a SIGTERM it
  has no handler for.
- Now, per row: the body is loaded, verified and canonicalized, and all
  nine UPDATE parameters are encoded, on one blocking thread before any
  transaction opens. The UPDATE is then sent and run on the row's own
  connection, holding only that row's lock. Only then does the importer
  poll for SETTLEMENT_LOCK with pg_try_advisory_xact_lock, every 25 ms
  and never queued, check the write guard again under it, and commit.
  Every other SETTLEMENT_LOCK holder still sees a row committed before
  it took the lock, or not at all. The write guard also runs before the
  write, so a halted cluster or a live legacy writer is refused before
  anything is sent.
- A lock holder that needs the row goes first. A poll that misses the
  lock also walks the lock holder's blockers (pg_blocking_pids, then
  theirs in turn). If the holder waits on this row, directly or through
  another session, the row is rolled back so the holder can proceed, and
  the command fails naming the row. Other sessions waiting on the row,
  such as a second import, just wait.
- Bounds: two new settings, read by import-audits only.
  - PRISM_IMPORT_AUDITS_WRITE_TIMEOUT_MS (60000; 1000 to 600000) is the
    write's transaction-local idle_in_transaction_session_timeout,
    statement_timeout and lock_timeout. So the server ends a session
    that stops sending mid-statement, and cancels an UPDATE that runs, or
    waits for the row, that long.
  - PRISM_IMPORT_AUDITS_LOCK_TIMEOUT_MS (5000; 100 to 600000) bounds the
    wait for SETTLEMENT_LOCK. Once the lock is held, it is also the
    transaction's idle and statement timeout, so a client or statement
    that stalls under the lock loses its session, and the lock, within
    that long.
  - The row's connection also sets client_connection_check_interval to
    1 s where the server supports it (PostgreSQL 14 or later, on Linux).
    An abandoned row is then rolled back within about a second, even while
    the server is storing it.
- A row that exceeds a bound is rolled back, and the command exits 1
  with an error naming the row, importing legacy audit <block hash>, and
  its cause, such as SETTLEMENT_LOCK stayed busy for 2000 ms; the row is rolled back and nothing was stored. Rows imported before it stay
  imported, and a re-run resumes from it.
- SIGTERM or SIGINT stops the command. It stops before the next row,
  or rolls back a row that has not been granted the lock, and exits 1 with
  import-audits stopped on request after importing <n> audit bodies; ….
  A row that holds the lock finishes its commit first, and a stop that
  lands once the last row is in ends as a finished import. A second signal
  exits at once.
- Each row it writes prints one line on stderr, imported, or
  unchanged for a row that changed after it was read, for example
  import-audits: block_hash=<hash> imported parameter_bytes=block_hash:64,…,canonical_audit_bytes:<n> total_parameter_bytes=<n> prepare_ms=… write_ms=… lock_wait_ms=… lock_held_ms=… total_ms=…. stdout keeps the completeness counts and
  Imported <n> audit bodies.
- Measured in a release build, over TLS through a TCP proxy, on a
  synthetic legacy row with a 112.2 MB canonical body and a 37.5 MB
  witness-leaves array. rc.6's order held SETTLEMENT_LOCK for 3.145 s:
  0.10 s encoding the statement, 3.01 s sending it and the server storing
  it, about 0.15 s of that the send, and 28 ms for the guard and the
  commit. 3.0.0 holds it for 13 ms. With the client's send
  slowed to 16 MiB/s, the hold was 14.9 s against 16 ms. A session taking
  the lock every 20 ms waited at most 3.13 s and 14.9 s beside rc.6's
  order, and 3 ms and 17 ms beside 3.0.0's.
- Why #765's send took so long is not proven: it did not reproduce
  here. The likely cause is #771, below. 3.0.0's import-audits
  doesn't depend on it: whatever slows or stops the send now happens
  before the lock and within the write timeout. The client also polls the
  statement every 100 ms, which un-sticks #771's lost wakeup if it occurs.
- It changes import-audits only: the command, its transaction and two
  setting names. There is no migration or schema change, and nothing the
  frontends or the public-api service run changes.
#760 (#759): every node RPC client keeps an idle connection for at most
20 s.
- Before: the frontends' qbitd RPC client kept an idle kept-alive
  connection for 60 s, and the public API's node client for reqwest's
  default 90 s, while qbitd closes one after its -rpcservertimeout, 30 s
  by default. A request written into a connection just as the node closes
  it is never read, and fails after the connection existed: the plain
  transport failed error, which nothing retries. On submitblock that
  is an unknown outcome, and the block is never offered again. One
  deployment, with PRISM_CTV_BROADCASTER_POLL_SECONDS=30 (the default is
  10), logged the error from the CTV broadcaster, whose passes ran in step
  with the 2 s refresh, on about one pass in eight to fifteen.
- Now every node client is built by node_client_builder(): the
  frontends', the public API's and the load harness's. It keeps an idle
  connection for at most 20 s (POOL_IDLE_TIMEOUT), so the client drops
  it before the node can. A compile-time assert keeps the bound below
  qbitd's 30 s default. Relay calls, submitblock included, share the
  bounded pool as before.
- Operator rule: keep qbitd's -rpcservertimeout, and the idle
  timeout of any proxy between a PRISM process and its node, above 20 s.
  qbitd's default, 30 s, is.
TLS dependencies, for the RustSec advisories cargo deny reports against
rc.6's lockfile: rustls 0.23.45 to 0.23.46 (RUSTSEC-2026-0346, -0347,
-0348 and -0349), rustls-webpki 0.103.15 to 0.103.16 (RUSTSEC-2026-0382)
and webpki-roots 1.0.9 to 1.0.10 (RUSTSEC-2026-0381), in the root and fuzz
lockfiles. They are patch releases within the versions rc.6 already
resolved, with no other dependency change. The serving binary's
PostgreSQL (sqlx) and HTTPS (reqwest) clients use them.

Upgrading

From 2.0.x, or from 3.0.0-rc.4: as in the
rc.5 notes, with 3.0.0
in rc.5's place. That covers migrate for 026, which every start of
3.0.0 requires, as rc.5's does; the cutover's
migrate --defer-share-hashes --offline-indexes, and finishing the
backfill after go-live with backfill-share-hashes; rolling back to rc.4
at fence 2; and the operating rules still in force: #737's statement
timeout for rc.4's tools, #738's snapshot rule and
fatal-state clear --timeout-seconds. From an earlier candidate, the
upgrade sections of each later candidate's notes apply in turn.
From 3.0.0-rc.5: replace the binary, as the
rc.6 notes describe for rc.6;
everything below about rc.6 applies too.
From 3.0.0-rc.6: no migration; replace the binary on every frontend and
tool, and on the public-api service. 3.0.0 requires the same
native schema migrations as rc.6, 2 to 26, declares the same capabilities
and understands the same share-hash fence values, 1 and 2. The same
settings give its frontends the same configuration fingerprint, so they
can replace rc.6's one at a time. It changes no evidence or admission-gate
script.
- The two new settings are accepted by 3.0.0 binaries only. rc.6's
  run and check-config refuse a PRISM_* name they don't know in
  production mode (QBIT_CHAIN main or mainnet, QBIT_PRODUCTION or
  QBIT_TOOLS_PRODUCTION), and warn about it otherwise. So pass them only
  to a 3.0.0 import-audits run, never through an environment that an
  rc.6 frontend reads.
- #760's node rule: check that qbitd's -rpcservertimeout, and the
  idle timeout of any proxy between a PRISM process (a frontend, a tool or
  the public-api service) and its node, are above 20 s.
- import-audits beside serving frontends no longer needs the guard
  #765 describes, one that kills it with SIGKILL once a settlement waiter
  has waited a few seconds. Its hold is now the guard read and the commit,
  and docker stop stops it.
Rolling back to rc.6: first remove the two settings from every
environment an rc.6 binary will read, then replace the binary; nothing
else changes. With rc.6, #765 returns: run its import-audits beside
serving frontends only under the guard #765 describes. #759 returns too:
the node clients keep an idle connection for 60 s again, 90 s on the
public API.

Verification

The runtime changes between rc.6 and 3.0.0 are in import-audits, in how
long the node clients keep an idle connection, and in the TLS crates' patch
releases above:

#770, merged as 4c134913: import-audits only, in
ledger/migration.rs (Ledger::import_legacy_audits_with and its per-row
transaction) and tools.rs (the command, its two settings, its stop
signal and its stderr line), with the two names in
config/native-settings.txt and a re-export in ledger.rs. It adds
tests/import_audits_settlement_lock.rs and its entries in
test/prism-gated-tests.txt, and updates the server's README, the ledger
runbook and the pool-acquire census.
#760, merged as 12f1ae8e: the node client's idle bound in rpc.rs, used
by the public API's node client (api.rs) and the load harness's, with
the node requirement in the migration guide.

Neither touches a migration, a capability, the share append
(ledger/window.rs), the order lock or settlement. #760 changes only how
long each node client keeps an idle connection. So the serving evidence
rc.3's notes cite still applies: the D1 peak-second gate, the 750/s ceiling
and the failover drill under load, run 1, all measured on #719's build.

In CI, the changes are proved by their tests:

#770 by import_audits_settlement_lock's 13 gated tests. They put the
importer's connections behind a relay that slows or stalls what the client
sends, which leaves the server in #765's state, and check:
- that a session taking SETTLEMENT_LOCK every 20 ms beside the import of
  a 107 MB body with a 37.5 MB witness-leaves array waits less than 3 s,
  while the throttled write takes longer than that (rc.6's order fails
  it);
- that a stall mid-send and a stall under the lock each end within their
  bound, and that a busy lock fails the row within its bound;
- that a lock holder waiting on the row goes first, directly or behind
  another session;
- that a stop mid-send, mid-execution, before the grant, between rows and
  after the last row each behaves as above;
- and, on the real command, its SIGTERM exit and its busy-lock output.
#760 by rpc_idle_close: the real client against a loopback node that
drops a request arriving after its idle limit, as the race does. With a
bound below the node's, the next call opens a new connection and is
answered; with a bound past it, the call fails with the production error.

Otherwise the evidence is the same set as rc.6's (#557):

CI on the version-bump pull request, and dispatched on the tag;
the L3 production-window matrix (l3-full) on the v3.0.0 tag push. The
version-bump pull request targets 3.0.x, where the matrix doesn't run
on pull requests, so the tag runs it rather than promoting a pull
request's run (#608);
the targeted nightly batch: short-plan-fake-node, short-plan-real-node,
soak-smoke, faults-short-real-node and faults-failover-fake-node;
the live regtest weekly scenarios.

rc.1's full nightly set and 5.5 h soak carry over. The evidence bundle on the
v3.0.0 GitHub release lists every run, and #291 links to it.

Known issues

As in the rc.6 notes, with these changes:

Fixed since rc.6:
- #765, by #770: rc.6's import-audits held SETTLEMENT_LOCK while
  it sent and stored each body, with nothing bounding the send, and ignored
  docker stop (above).
- #759, by #760: the node clients kept an idle connection past qbitd's
  idle close, so a call could fail in the close (above).
Open since rc.6, not fixed in 3.0.0:
- #771: sqlx 0.8.6's rustls stream can lose the wakeup of a large write.
  A large statement sent over TLS can then stall, with the server waiting
  for the rest of it, until something closes the connection. It is the
  likely cause of #765's hold. 3.0.0's import-audits works
  around it (above), but other large statements from the serving binary
  don't. The fix is upstream, transact-rs/sqlx#4251, which no 0.8.x
  release carries; #771 tracks backporting it.
- #762: a submitblock whose deadline,
  PRISM_BLOCK_SUBMIT_RPC_TIMEOUT_SECONDS (1 s by default), passes while
  its connection is still opening is recorded as an unknown offer, not an
  unsent one, so that block is never offered again. It needs no free
  pooled connection when the call starts and a connect slower than the
  deadline, which the 2 s refresh calls make unlikely. #760 leaves relay
  calls on the bounded pool for this reason.
- #757: five alert-rule UIDs in docs/prism-native-alert-rules.json
  exceed Grafana's 40-character limit, so provisioning the rule set as
  published stops Grafana from starting. Until it's fixed, map the five
  to short forms when rendering the rules; #757 lists them.
- #758: the two accepted-block revision-work warnings,
  PrismAcceptedRevisionWorkPending and
  PrismAcceptedRevisionWorkTrackingUnknown, fire after every pool block
  on a frontend that has no miners and doesn't commit the settlements, a
  backup frontend say, and the second latches until that frontend
  restarts. Until it's fixed, silence the two on idle frontends.
Still open from rc.6, as the rc.6 notes describe them: #752, #738,
#739, #724's other answer, #737's open part, #741, #731, #734, #716's open
part, #711 and #712.

Migration and recovery (D5)

Native schema migration is one-way. No down-migrations are provided. Prefer
forward repair or a compatible native image retaining the latest durable state.
Both migration and
ledger operations
runbooks contain the numbered reconciliation procedure and exact queries.

Before the first native share is acknowledged, restore the complete
pre-migration database and artifact backup and restart the pinned old image
in isolation from the migrated database. After the first native share is
acknowledged, restoring an older database loses those accepted records.
Any rollback that discards acknowledged history requires an explicit
accounting reconciliation and recovery decision; it is not an ordinary
image rollback. Keep every native frontend stopped while performing an
isolated restore/recovery operation against its replacement database.

Native import-audits is the canonical backfill for recoverable legacy bodies,
including those without a canonical sidecar. It verifies the original audit
digest, trusted ledger key and recorded coinbase. Rehearse it on an isolated
source restore before migrating the source. Production self-check fails when
audits lack shared stored bodies or canonical availability; native immutable
snapshots provide canonical reconstruction without a duplicate full byte copy.

The one-shot commands self-check, import-audits, backfill-ctv and
broadcast-ctv no longer register a frontend heartbeat (#381). Earlier 3.x.x
builds left a starting row under a generated instance ID after every
invocation, which fatal-state clear then refused, and overwrote the live
frontend's status when the ID was configured. Rows those builds left behind are
not removed automatically; the fatal-state runbook's investigation still applies
to them.

Navigation Menu

Repository navigation

v3.0.0

Choose a tag to compare

Sorry, something went wrong.