Skip to content

Reconcile the recorded state with production and with upstream - #39

Merged
bitcoinuniverseadmin merged 5 commits into
developfrom
docs/protocol-coverage-reconcile
Aug 31, 2026
Merged

Reconcile the recorded state with production and with upstream#39
bitcoinuniverseadmin merged 5 commits into
developfrom
docs/protocol-coverage-reconcile

Conversation

@bitcoinuniverseadmin

@bitcoinuniverseadmin bitcoinuniverseadmin commented Aug 30, 2026

Copy link
Copy Markdown

Two records that had drifted from reality, corrected from evidence rather than
by hand.

1. Protocol coverage

The committed table claimed 3 of 36 protocols were readable. The manifest
the overlay serves at /api/v1/universe/protocols says 6 of 38.

Regenerated from the live manifest:

node scripts/universe/generate-protocol-coverage.mjs --from <live manifest>
node scripts/universe/generate-protocol-coverage.mjs --check
Protocol coverage table matches the recorded manifest (38 protocols).
protocol was is
zerdinals BLOCKED / unknown VERIFIED READ ONLY / complete
zrunes BLOCKED / unknown VERIFIED READ ONLY / complete
zrc20 absent from the registry VERIFIED READ ONLY / complete
dunes absent from the registry BLOCKED / unknown

The other 32 stay BLOCKED, because that is what they are. Most are not missing
code: their indexer repositories exist with real implementations, and the gap
is that the service is not configured in the overlay's source list on the
indexer host.

2. Upstream

upstream-base.json carried an empty modifiedSubsystems and an empty
knownUpstreamConflicts while UPSTREAM.md described eleven modified
subsystems. The two disagreed, and the machine-readable one was wrong. They now
carry the same eleven, plus the three files where an upstream sync predictably
conflicts, plus the mirror sha measured against.

upstream/master is 40 commits ahead of the mirror recorded at fork time.
All 40 were read and classified:

class count
Dependency bumps 36
Toolchain (Angular 20.3.29) 2
Infrastructure (CI provenance check) 2
Product features 0
Correctness or security fixes to product code 0

Nothing upstream added changes product behaviour or fixes a defect this fork
carries.

Exposure, measured rather than inferred

Upstream's bumps do not describe this fork's exposure, because this fork has
its own lockfiles. So it was measured:

  • backend: zero advisories at any severity.
  • frontend: eleven, ten of them in build tooling that never reaches a
    browser (browser-sync, less, image-size, webpack-dev-server,
    immutable, sockjs, uuid, the two @angular-devkit packages).

The one that ships

echarts 5.4.3 carries GHSA-fgmj-fm8m-jvvx,
a moderate XSS advisory, CVSS 6.1. echarts draws every chart in the explorer.

The fix is echarts 6.1.0, a major version. Upstream is still on 5.x, so
there is nothing upstream to take. Moving a charting library across a major
version touches every chart on every chain and needs the full visual matrix
behind it. It is recorded with its exact identity and left as the next
dependency task rather than slipped into a release unreviewed.

This closes the release criterion that UPSTREAM.md and upstream-base.json
must agree. They now do, and both agree with what upstream and npm audit
actually say.

🤖 Generated with Claude Code

bitcoinuniverseadmin and others added 2 commits August 30, 2026 18:22
The committed table said 3 of 36 protocols were readable. The manifest the
overlay serves says 6 of 38: dunes and zrc20 joined the registry, and
zerdinals, zrunes and zrc20 are now verified read only with complete
coverage.

Regenerated from the live manifest with
generate-protocol-coverage.mjs --from, so the table and the recorded JSON
agree and the --check gate passes. Nothing here is hand edited.

This is a documentation change and a small one, but a stale coverage table is
the kind of stale that gets quoted: it undercounts what works and it hides
which authorities moved. The remaining 32 are recorded as BLOCKED because
that is what they are.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…ly say

upstream-base.json carried an empty modifiedSubsystems list and an empty
knownUpstreamConflicts list while UPSTREAM.md described eleven modified
subsystems. The machine-readable file and the human-readable one disagreed,
and the machine-readable one was the one that was wrong.

Both now carry the same eleven subsystems, plus the three files where an
upstream sync predictably conflicts, plus the mirror sha this fork was
measured against.

UPSTREAM.md gains the reconciliation itself. The 40 commits upstream has
added since the recorded mirror were read and classified: 36 dependency
bumps, 2 toolchain, 2 infrastructure, and nothing that changes product
behaviour or fixes a defect this fork carries.

Upstream's bumps do not describe this fork's exposure, because this fork has
its own lockfiles, so the exposure was measured rather than inferred. The
backend has zero advisories. The frontend has eleven, ten of them in build
tooling that never reaches a browser.

The eleventh ships: echarts 5.4.3 carries GHSA-fgmj-fm8m-jvvx, a moderate
XSS advisory fixed only in echarts 6.1.0, a major version upstream has not
taken either. Moving a charting library across a major version touches every
chart on every chain, so it is recorded with its exact identity and left as
the next dependency task rather than slipped into a release unreviewed.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@bitcoinuniverseadmin bitcoinuniverseadmin changed the title Record the protocol coverage production actually serves Reconcile the recorded state with production and with upstream Aug 30, 2026
@bitcoinuniverseadmin

Copy link
Copy Markdown
Author

Heads up on an overlap, and a suggestion for the order.

#40 rewrites both files this touches. It found the same drift you did, from the other direction: /api/v1/universe/protocols served 38 while the recorded roster said 36, dunes and zrc20 were the two, and the gate stayed green because it rendered the table from the recorded JSON and compared it against the committed table. Both sides came from the same file, so re-recording fixes today and the gate still cannot see tomorrow.

That PR replaces the generator with one that pins the manifest by provenance, holds this repository's own surfaces to it, and compares the pin against what a deployment serves in the production smoke run. It also regenerates the README readable list, which was naming three readable protocols while six were readable.

Your UPSTREAM.md and upstream-base.json work does not overlap with any of it.

Suggested order: merge this one first. Re-recording the roster is a single command in mine (node scripts/universe/protocol-contract.mjs --record --from <origin>), so I resolve the conflict by re-running it rather than by hand-merging two generated files, and nothing is lost either way.

One thing to know if this merges alone: the manifest it records comes from the live endpoint, which does not yet carry the identity fields (schemaVersion, sourceRepository, sourceSha) that bitcoinuniverseio/backend-apis#101 adds. Once that overlay is deployed the pin has to be re-recorded from it, which is the same single command.

bitcoinuniverseadmin and others added 2 commits August 30, 2026 19:05
A release record of readings rather than intentions: the fee arithmetic
verified against a live node, the Dogecoin coverage figures, the throughput
and disk measurements that show the inscription reindex cannot finish, the
lag defect and what it was reporting, the listener that was public and now is
not, the origin's headers, the dependency exposure measured rather than
inferred, and the protocol coverage production actually serves.

It ends with what was not reached and why, because a release record that only
lists successes is the kind that gets quoted later by someone who needed the
other half.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The record said the volume exhausts before the reindex could finish. Measured
across 90 minutes of indexing, free space did not move: nearly all of that
I/O rewrites pages in place rather than growing the file. Throughput alone is
the blocker.

The second reading also refines the rate to about 33 blocks an hour rather
than 16. Both numbers put the remaining work in years, so the conclusion is
unchanged, but a record that overstates one of its own measurements is worth
less than one that does not.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@bitcoinuniverseadmin

Copy link
Copy Markdown
Author

Cancelled this PR's in-flight run to free two ultra runners, because it could not have passed.

Its Frontend build started at 19:54Z and was 73 minutes into a 100 minute cap. Even if the passes had finished, the job ends on Keep the contrast report and the route screenshots, and that step fails right now: Failed to CreateArtifact: Artifact storage quota has been hit. My run 33330398257 passed every gate in that job and then failed exactly there.

#40 fixes both halves: the deep pass is split into three shards so the job finishes in about 23 minutes instead of 95, and the screenshot uploads only run on failure so a green run writes nothing. Re-run this against develop once that lands and it should go green without any change on your side.

@bitcoinuniverseadmin

Copy link
Copy Markdown
Author

Second correction on top of my earlier note, and this one was my fault.

#40 split the deep visual pass into three shards that each built their own copy, which took a run from two concurrent Angular production builds to five, on a fleet that runs more than one runner per host. Everything that started a build in the same ninety second window was killed with exit code 137, out of memory, including on pull requests that touched nothing but scripts. That is what put Frontend build, Mobile and adaptive layout and the shards red on this PR after I told you #40 would clear them.

#48 is merged and fixes it: the frontend job keeps its build under a commit-keyed cache and the shards take it, so a run is back to two concurrent builds. Verified on its own run, 33340340913: all eight jobs green, every shard's fallback build step skipped because the cache hit, whole run 48 minutes.

Merge develop into this branch now and it should go green. Still expected to fail and still not required: Analyze (actions), Analyze (javascript-typescript), Analyze (rust), which need GitHub-hosted runners the current plan does not provide.

Sorry for the churn. Say the word and I will do the develop merge for you.

@bitcoinuniverseadmin

Copy link
Copy Markdown
Author

The CI repairs are all merged now, so this should go green on a develop merge with no change to your work.

What landed since your last run:

#42 went through all eight jobs green after its author merged develop, which is the shape to expect here.

Still failing and still not required: Analyze (actions), Analyze (javascript-typescript), Analyze (rust). CodeQL needs GitHub-hosted runners that the organization's current plan does not provide, so those three will stay red on every PR until the plan is restored. They do not block a merge.

I have left this branch alone because it is yours. Merge develop into it when you are ready, or say the word and I will do it.

The two coverage files conflicted because develop re-recorded them from
the deployed overlay after the roster contract landed: seven readable
protocols rather than six, with the schema, registry version, source
repository and source commit the pin now carries. Those take develop's
version.

Nothing else on this branch is touched. The upstream reconciliation, the
pinned upstream base and the release evidence record are the work this
branch exists for and they carry through unchanged.

Verified on the merged tree: the roster gate passes, its twenty-five
cases pass, and the text gate passes.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@bitcoinuniverseadmin

Copy link
Copy Markdown
Author

Merged develop into this branch, since I offered twice and it had not moved in six and a half hours. A merge commit, no rebase, no force push.

The two PROTOCOL-COVERAGE files conflicted and take develop version: it re-recorded them from the deployed overlay after the roster contract landed, so the pin now carries seven readable protocols rather than six, plus the schema, registry version, source repository and source commit. Your UPSTREAM.md, upstream-base.json and the 172 line RELEASE-EVIDENCE-2026-08-30.md carry through untouched, which is the work this branch exists for.

Verified on the merged tree before pushing: the roster gate passes with 38 protocols and 7 readable, its 25 cases pass, and the text gate passes.

CI is running. The three Analyze jobs will stay red, which is CodeQL needing GitHub-hosted runners the plan does not provide, and which does not block a merge.

If it comes back green I intend to merge, because the frontend deployment is blocked on the remaining PRs and the hourly production smoke has been red since 00:39Z purely because develop is ahead of what is deployed. Say so if you would rather land it yourself.

@bitcoinuniverseadmin
bitcoinuniverseadmin merged commit ad8b8f4 into develop Aug 31, 2026
15 of 19 checks passed
@bitcoinuniverseadmin
bitcoinuniverseadmin deleted the docs/protocol-coverage-reconcile branch August 31, 2026 03:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant