release: v1.11.0 - #413
Merged
Merged
Conversation
chore: sync main back into development after v1.10.0
…C-335) (#403) Asked whether views-faoapi's Better Stack setup generalises here. The answer is "partly", but investigating it surfaced something worth more than the answer: we monitor one of our two systems. The heartbeat answers "did the pipeline run?". Nothing answers "can a consumer read anything?". The pipeline writes on the host; Caddy serves over HTTP; they fail independently: host down -> no ping -> alerts after 30d period + 48h grace (~32 days worst case; slow, but bounded) Caddy down, host up-> pipeline runs, succeeds, PINGS. Check stays green. Consumers get nothing. Detected never. Registered as C-335 at Tier 2, not Tier 3, because the system does not merely fail to report a problem — it actively reports health while broken, and a green light gets used as evidence. views-faoapi hit this exact shape (their C-50/C-170: a 139-day-old artifact behind green health), which is why their ADR-032 has a content check on top of liveness. Better to learn it from their incident than from ours. ADR-051 keeps the push heartbeat where it is. That is not inertia: I checked Better Stack's heartbeat docs, and there is no /start equivalent — the documented signals are success, /fail, and /$? for exit codes. /start is what PR #359 shipped for C-317, where a SIGKILL bypasses the ERR/EXIT traps so no /fail is ever sent. Migrating would have silently discarded a mitigation we shipped deliberately, and closed C-317 by deletion rather than by fixing it. The poll targets status.html, which ADR-038 makes public. So unlike views-faoapi — which must store an X-API-Key in its vendor — we hand the monitoring provider no credential at all. Recorded at the operator's explicit request that this not be stamped as the forever solution: the two-vendor split is a compromise, not an architecture, and ADR-051 carries five concrete revisit triggers, including "C-317 solved without /start" — which would make consolidation free. ADR-018 gains a "what it does NOT detect" section, because its existing text reads as though the monitoring question were settled. Header trimmed: two superseded Prior blocks retired (both now recorded in pyproject comments, ADR-050, the publishing guide, and the v1.10 plan addendum) to keep the open-concerns index inside the 8000-char search window the guard enforces. My first draft broke that invariant — 9201 — which is the second time this week I have run the guards only after assuming they would pass. Verified: all 13 register guards green, ruff clean, index 6758 < 8000, entries 51 == rows 51, header 335 IDs / 41 open = 0+2+10+23+6. Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
The register header was doing two jobs. It was the index that readers and six other guards search, and it was also two append-only logs: a narrative of every session, and a list of every audit ever run. The index does not grow. Both logs do. test_falsification_merge_readiness.py requires "open concerns" within the first 8000 characters. Every time the growing halves hit that limit, the resolution was the same: delete history to reclaim bytes. Two days ago I retired two narrative blocks to free 1200 characters — not because they had stopped being true. That is byte pressure making editorial decisions about what history to keep. Both growth vectors moved to reports/register_changelog.md: narrative 3014 chars, 5 entries, newest first Source: line 2783 chars, 71 audits -> "Where the findings came from" The second one was already known. The 8000-char guard's own docstring says it must be "wide enough to find 'N open concerns' despite Source-line growth" — so the header was being defended against that list rather than relieved of it. Widening a window to accommodate an append-only list postpones the collision; it does not remove it. index position 9201 (broken) -> 1670 headroom none -> 6300 chars The deleted history is restored. Both retired blocks were recovered from git (8bdd11c) and are back in the changelog, with a test asserting they are present. Fixing the mechanism while keeping the damage would have missed the actual complaint. tests/test_register_changelog.py (7 tests) makes the rule enforceable instead of a comment: the changelog exists and is not a stub, the register links to it, the header stays under 3500 chars, the Source line cannot become a list again, and Last update cannot grow a Prior: chain. The 3500 budget is a smoke alarm well below the 8000 contract, so drift is caught while it is one sentence rather than at the cliff where the only remedy is deletion. Drilled, not assumed: re-added a Prior: chain, deleted a restored block, broke the changelog link. Each failed on the expected assertion and recovered. Verified: full suite exit 0, ruff clean, all 13 register guards green. Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
…pped (#402) (#406) I said nothing in the runbook documented the back-merge. That was wrong. hetzner_deployment_guide.md documents it twice, at :15-20 and :815-835. The real defect is worse. Both of those blocks prescribe `git push origin development` and a local `git merge development --ff-only` on `main`. Branch protection has forbidden BOTH since 2026-07-31 — PR required, admins included, force-push refused. The more detailed of our two release guides is entirely unexecutable, while publishing_to_pypi.md documents the PR-based flow and omits the back-merge. Two guides, contradictory, neither matching reality. So the fix is not "add a missing step". It is one procedure, one home. publishing_to_pypi.md becomes that home: back-merge added as step 5, with a section on why it exists and why squashing it defeats it (a squash creates another commit that is not main, so main still is not an ancestor — the step looks done and achieves nothing). The server guide's git sequences are deleted, replaced by an explanation of why they are gone; a reader who remembers those commands needs to be told, or they will reinstate them. Test docstrings reframed. The assertions did not change because the invariant did not — what changed is why it matters: not "so ff-only works" but so `git log main..development` counts only unreleased work. TestF2FastForwardMerge is renamed TestF2ReleaseTopology; a class name is documentation too, and that one described a procedure that no longer exists. release-topology.yml closes the detection gap. The deploy gates could only ever run locally — the default shallow checkout makes `git merge-base` unanswerable, so they skip (C-320) — which is why the v1.10.0 divergence surfaced four hours late by accident, having been silently true after every prior release. The workflow runs on release, daily, and on demand with fetch-depth 0. Deliberately NOT in the PR test job. Between a release and its back-merge the branches ARE diverged, correctly; gating pull requests on that would redden every PR for reasons unrelated to the change under review. Same reasoning ADR-051 used for the serving-path poll. It opens one reused issue rather than a PR because Actions cannot create PRs here (can_approve_pull_request_reviews: false). The PAT alternative was rejected: a stored credential with a rotation burden, days after a sprint spent removing exactly that. Two things testing caught that reasoning had not: - Running the workflow's shell with gh stubbed showed a line continuation collapsing into `main \ --title` on one line. - My own new guard failed on my own incomplete edit — but the remaining matches were in prose explaining the commands are forbidden. The guard could not tell "prescribes" from "warns against", so the GUARD was fixed to inspect fenced code blocks only. Prose saying "do not do this" is the part worth keeping. Drilled, not assumed: reinstating a runnable copy fails the server-guide guard; removing the never-squash warning fails the release-guide guard; and replaying the pre-#400 state makes the detector fire on exactly one commit, d72602b — the real incident. Verified: full suite exit 0, ruff clean, validate_docs.sh exit 0. Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
docs/validate_docs.sh has existed since the base-docs set was created and ran NOWHERE — not in CI, not in pytest. It only ever executed when somebody happened to type it, which is to say almost never. views-frames found the identical defect in their copy and wired theirs in (their C-74). A check that nothing runs is not a check. It is a file that looks like assurance. Blocking rather than advisory, and that is safe here in a way the credential review-date check was not (#392). This script only inspects internal consistency of the documentation set — dangling contributor_protocols references, unfilled template placeholders, cross-ADR links. Every failure it can produce is something the author of a pull request can fix inside that same pull request. It cannot go red because a date passed, so it cannot become the permanently-red CI that C-320 warns about. An advisory check would also reproduce the original failure: a check that cannot fail is a check people stop reading, which is how this one came to be wired into nothing. Verified it actually fails, rather than assuming: added a dangling `contributor_protocols/this_protocol_does_not_exist.md` reference to docs/CICs/README.md and the script exited 1 with "FAILED: 1 issue(s) found"; restored, exit 0. Worth checking, because a blocking gate that cannot fail is worse than no gate — it manufactures confidence. No Python or uv in this job; the script is bash, so the job is a checkout and one command. Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
… is (#402) (#408) Three drift mechanisms, all of the same family: documentation that was true when written and stopped being true without anything failing. 1. LINE NUMBERS ROT SILENTLY. Ten `file.py:NNN` citations across the ADRs. Three were already wrong — ADR-026 cited ucdp_annual.py:132-142 for get_ucdp_token() and line 132 is now blank; ADR-040 cited a line in grid_compilation.py that is now empty. Nothing detects this, because a line number stays syntactically valid when it points at whitespace. All ten now cite the symbol. views-frames reached the same conclusion independently (their #212). 2. A SIBLING REPOSITORY DELETED A PACKAGE WE CITE. ADR-006 and ADR-010 describe GridConfig/TemporalConfig/SpatioTemporalGrid as coming "from lab_grid/config.py". That package lived at views-metric-lab/src/ lab_grid/ and was deleted in their commit 6e1a34d. ADR-010 goes further and asserts it "remains as-is for the lab's own use" — a claim about another repo's state, false for months, invisible here. Annotated rather than deleted: an ADR that quietly drops its own premise stops being a record of a decision. 3. A CITATION THAT EXCUSES ITS OWN ABSENCE IS NEVER FIXED. grid_to_country_month.md read "Tests in tests/test_grid_to_country_ month.py (if present)." That file has never existed. The hedge is why it survived twenty months: a claim qualified into unfalsifiability cannot be wrong, so nobody corrects it. Also written: docs/CICs/load_dataset.md. Thirty-two contracts existed, all for config dataclasses, while the one surface ADR-050 declares a public contract — the function every downstream model calls — had none. The CICs had been written where writing was cheap, not where dependency was heavy. Includes the storage_options seam from v1.10.0 and the non-obvious rule that local paths must NOT receive storage_options. THE NEAR-MISS, RECORDED BECAUSE IT NEARLY WENT THE OTHER WAY: audit_data_parity.py and config_queryset.py were on my stale list until I opened views-models and found them under models/bright_starship/. Deleting them would have been C-330's error — absence in this repo read as absence in the world — committed inside the audit written to catch that error. The new guard therefore does NOT assert that every referenced path exists: a guard that cannot distinguish "deleted" from "lives elsewhere" teaches people to delete true references. tests/test_docs_citations.py (7 tests): no line-number citations; cited test files exist; cited symbols are defined where the doc says; the public entry point has a substantive CIC that knows about storage_options; lab_grid references stay annotated. Drilled: reintroducing a line citation, citing a nonexistent symbol, and stripping the deletion note each fail on the expected assertion. The test-file guard proved itself immediately by catching the stale CIC citation it was written for — and then caught my own explanatory prose quoting the dead filename, which I reworded rather than loosening the guard (unlike the release-guide case, where the dead commands must be shown verbatim for a reader to recognise them). An existing guard I had not seen — TestF2CicRegistryCompleteness — caught that a new CIC must be listed in docs/CICs/README.md. C-336 registered at Tier 4: no correctness impact, but the fourth instance in a week of the same failure (C-164's phantom "9 sources", C-330's logrotate path, ADR-026's "Public GitHub is safe"), which makes it a pattern worth a standing entry rather than four one-off fixes. Verified: full suite exit 0, ruff clean, validate_docs.sh exit 0, all register invariants (index 1723 < 8000, header 1723 <= 3500, entries 52 == rows 52, 336 IDs / 42 open = 0+2+10+24+6). Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
…0.0 (C-337) (#409) The operator asked for this bump. I audited it, concluded it was unjustified, and was wrong. Recording how, because the failure is more useful than the fix. MY AUDIT. I asked "what does this package import from views-frames?" Four symbols — FeatureFrame, FrameMetadata, SpatialLevel, SpatioTemporalIndex — none of them estimators. I installed views-frames==1.0.0 in a clean venv, constructed a frame with our exact kwargs, ran assert_frame_contract, and it passed. Every step true. The conclusion — "the floor can stay at >=1.0" — was wrong twice over. WRONG ONCE: a floor constrains the RESOLVER, not our import list. views_frames_summarize ships in the same wheel, and views-frames changed how the statistics are computed three times, all MINOR so none of it reads as breaking: 1.2.0 outside-in HDI tower + mass-aware tip ("fixes a silent" error) 1.3.0 no magnitude-based zeroing by default 1.9.0 tower-tip MAP: tip_mass 0.5 -> 0.25. Their words: "Behavior change to tower_point/summarize_tower outputs, shipped MINOR." Two systems on different versions produce different numbers from the same posterior, with no error. We are widely installed; if we are the loosest constraint, we are the one admitting old semantics. WRONG TWICE, AND WORSE: I never opened uv.lock. It had pinned views-frames 1.0.0 since June, because `uv lock` keeps an existing pin while it still satisfies the constraint and >=1.0 satisfies 1.0.0 forever. So roughly six weeks of CI and every local run executed against pre-amendment MAP/HDI. The loose floor did not merely permit stale semantics — it froze them. That was the actual finding and it was one command away the whole time. This is the same error as C-336, committed inside the audit written about C-336, one day later. Caught by the operator, who knew the estimator history. C-337 records the recurrence rather than presenting the fix as a win. CHANGES - pyproject: views-frames>=1.10.2,<2, with the three estimator changes named and why the import-surface argument was insufficient. 1.9.0 is the strict minimum; 1.10.2 is current and the gap is docs/test releases. - uv.lock: 1.0.0 -> 1.10.2. Full suite run against the new version — a combination that had genuinely never been exercised here. Zero failures from the dependency. - tests/test_views_frames_floor.py: fails if the floor drops below the audited value, naming what a lower floor re-admits; separately fails if any estimator symbol is imported, because that reopens the question. Both drilled. - views-frames#237 filed upstream. NOT as "your CONFORMANCE_FLOOR value is wrong" — it is correct by their governance rule, which bumps only on breaking changes to the conformance surface, and the 1.9.0 MAP-containment law was additive. Filed as the real defect: that number reads as a safe dependency floor and is not one, with this incident as evidence. C-337 left OPEN at Tier 2. The fix covers views-frames; nothing checks our other floors the same way. numpy>=1.26, pyarrow>=14, zarr>=2.16 have the identical shape and the identical freeze is available to all of them. Verified: full suite exit 0 against 1.10.2, ruff clean, validate_docs.sh exit 0, register invariants (index 1764 < 8000, header <= 3500, entries 53 == rows 53, 337 IDs / 43 open = 0+3+10+24+6). Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
…ap for free (#410) C-335 is resolved. A Better Stack monitor now polls the public status page every 3 minutes: verified Up at ~27ms from Europe, with a test alert delivered and read. The unbounded failure — Caddy stops while the host stays up, pipeline keeps succeeding and pinging, nothing ever notices — is closed. But ADR-051 specified TWO checks and only one was buildable. Better Stack gates keyword matching behind a paid plan. Rather than pay, or quietly ship an ADR describing a monitor that does not exist, the content half moved to .github/workflows/serving-freshness.yml — daily, running on GitHub, so genuinely external: it does not share fate with the host it watches, which is the objection that ruled out an on-host cron. THE DRILL FOUND THE SPEC WAS WRONG, NOT JUST THE IMPLEMENTATION. My first version searched the page text for "stale" and "missing". Against the live, healthy server it reported one of each: CASE 1: live healthy page (expect degraded=false) degraded=true - 1 stage(s) report **stale** - 1 stage(s) report **missing** The page carries a legend — "● OK ● Stale ● Missing" — explaining the dot colours, so those words appear on every healthy page. An earlier count said zero because it was case-sensitive. Shipped, this would have opened an issue every day until somebody muted it: a monitor that cries wolf on day one and is ignored by day three. ADR-051 specified the PAID check the same way — "alerts when the body does not contain the healthy marker". Identical bug. Buying the feature would not have saved us; we would have configured a keyword monitor with the same false positive and trusted it MORE, because it came from a vendor. The workflow parses per-cell title="<status>" attributes instead (ok x38, not_applicable x10 on a healthy page). Four paths drilled: healthy, aged past 40 days, one stage genuinely stale, and the page reshaped so the check would inspect nothing — the last treated as a problem, because a check that silently inspects nothing is worse than no check. ALSO - ADR-051 gains a dated amendment: what was built, what was not, why, and that its own specification was wrong. Not a silent rewrite — the gap between what was decided and what was buildable is the useful record. - docs/guides/monitoring.md is the setup of record: the monitor exactly as configured, the three-way division of labour (heartbeat = did it run, Better Stack = is it reachable, workflow = is it current), what we deliberately do not use, how to leave, and — at the operator's explicit request — §5 listing what to configure if the plan is ever upgraded. - C-338 registered for the residual: freshness detection is daily and by issue rather than 3-minute and by e-mail, and GitHub may delay cron. Tier 4 — late notice of stale data, not wrong data. - The workflow stays silent when the page is unreachable. That is Better Stack's job and it has already alerted; two alarms for one event is how people learn to ignore both. Verified: full suite exit 0, ruff clean, validate_docs.sh exit 0, all register guards green (index 1741 < 8000, header <= 3500, 338 IDs / 292 resolved / 43 open = 0+2+10+25+6, 108 struck-through). Live serving path re-checked HTTP 200 before closing C-335 — the claim being closed is a claim about the world, which is C-330's lesson. Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
…ixing it (#411) C-330 WAS RIGHT, THEN I CORRECTED IT INTO BEING WRONG, THEN IT WAS RIGHT. Observed on the server, which is the only way this entry could ever have been settled: - The logrotate config rotated /root/views-datafactory/logs/refresh.log. The log has been under /home/views-deploy/ since the pipeline moved to the service account. `missingok` made logrotate find nothing and exit SUCCESSFULLY every night for four months. - The file mode was 644 — world-readable, on a box with four shell accounts. Exactly as originally claimed, and later downgraded by me to "inferred, not observed". The 2026-07-31 retraction found an archived plan saying "logrotate configured on server — Done 2026-03-31", which was true when written, and concluded the rotation half was a false alarm. Existence was never the question; EFFICACY was. Checking that a config exists is not checking that it works. Fixed on the server: correct path; size 100M -> monthly (a size trigger on a 130 KB/month log bounds nothing, and the point is bounding time); create 0640 views-deploy views-deploy; su views-deploy views-deploy, without which logrotate refuses to touch a file in a non-root directory and we would have swapped one silent failure for another; and missingok REMOVED, because that option is what hid the bug. Verified by `logrotate --debug`: "Handling 1 logs" against the real path. C-339 — THE INCIDENT, WHICH IS MINE. Fixing the rotation, I handed the operator a multi-line `sudo tee <<EOF` heredoc to paste. Their terminal joined the first two lines, so tee received the log path as a second output file and, running as root, overwrote it. 528 KB -> 150 bytes. Unrecoverable: no backup, and no rotated copy — because the rotation this work was fixing had never run. The fix for the missing rotation destroyed the thing the rotation would have preserved. Lost: four months of run output. Not lost: provenance ledgers under data/, the status page, healthchecks ping history, git. Nothing reads refresh.log; the pipeline appends to it. Registered at Tier 3, not Tier 4, and the entry says why: the consequence was mild, the mechanism was not. A root-privileged command whose failure mode is writing to an unintended path, handed to someone to paste blind, lands differently against a data directory or a zarr store. The register should carry the mechanism, not the luck. Three failures named in the entry: wrong format, when single-line commands had been used all session precisely because pasting was already causing trouble; no dry run, when the command could have printed to stdout first; and a backup that protected the config I was replacing rather than the log I could reach. Standing rule adopted: commands given to a human to paste are one line. Multi-line content goes in an editor — which is what worked afterwards. ALSO - ADR-026 no longer says the GDL token "is still in use". It was rotated and revoked 2026-08-01, verified with a live request. The paragraph stays, because the mechanism it describes — a credential with no expiry that nothing prompts anyone to rotate — is unchanged. - hetzner_deployment_guide.md documents the rotation, flagging the two load-bearing lines so nobody tidies away `su views-deploy` or re-adds `missingok`. That closes C-330's surviving half: it was undocumented anywhere an operator looks. Verified: full suite exit 0, ruff clean, validate_docs.sh exit 0, all register guards green (339 IDs / 293 resolved / 43 open = 0+2+11+24+6, 109 struck-through, index 1764). Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
…ndency floor (#412) src/ is untouched. The wheel is byte-identical to v1.10.0 apart from metadata: eight commits, none of them a line of shipped code. Docs, CI, tests, and the register. Minor rather than patch for exactly one reason. views-frames>=1.0 became >=1.10.2, and raising a floor can break a consumer's dependency resolution — which a patch must never do. Same call as 1.10.0, which went minor for the pandas extra. That floor is also the entire reason this release exists. Without it there would be nothing to deploy; the server would re-checkout identical code. What it actually delivers is a CORRECTED ENVIRONMENT: uv.lock had pinned views-frames 1.0.0 since June, because `uv lock` keeps an existing pin while it still satisfies the constraint and >=1.0 satisfies 1.0.0 forever. Six weeks of CI, and the server's venv, ran against pre-amendment MAP/HDI semantics (C-337). The v1.11 addendum in the product plan records all eight commits' worth — including C-339, where I destroyed refresh.log with a pasted multi-line command. Release notes that only list wins are not a record. Verified: full suite exit 0, ruff clean, validate_docs.sh exit 0, plan-staleness gate green, uv.lock diff is the version line and nothing else (checked, not assumed). Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Promotes
developmenttomainfor v1.11.0. 10 commits sinced72602b.What this release is
Governance repair.
src/is untouched — the wheel is byte-identical to v1.10.0 apart from metadata.The single consumer-visible change is
views-frames>=1.0→>=1.10.2, and that is the whole reason the release exists. Without it there'd be nothing to deploy; the server would re-checkout identical code. What it delivers is a corrected environment.Why the floor moved (C-337)
uv.lockhad pinned views-frames 1.0.0 since June.uv lockkeeps an existing pin while it still satisfies the constraint, and>=1.0satisfies1.0.0forever — so nothing ever pulled it forward. Six weeks of CI, and the server's venv, ran against pre-amendment MAP/HDI semantics.views-frames changed how the statistics are computed three times, all MINOR: 1.2.0 (outside-in HDI tower), 1.3.0 (no magnitude-based zeroing), 1.9.0 (tower-tip MAP
tip_mass0.5→0.25). My audit initially cleared the floor by asking "what does this package import?" — the wrong question. A floor constrains the resolver, and I never opened the lockfile. Caught by the operator. views-frames#237 filed upstream.Also in this release
publishing_to_pypi.md, with the back-merge that had been skipped after every release before v1.10.0.release-topology.ymlnow detects divergence daily.serving-freshness.ymlbecause keyword matching is paid — and the drill found ADR-051's spec for that check was itself wrong, so buying it wouldn't have helped. C-338 registered.validate_docs.shnow runs in CI as a required check, having previously run nowhere.lab_gridannotated (views-metric-lab deleted it); a CIC written forload_dataset, the public contract, which had none while 32 config dataclasses did. C-336.refresh.logwith a pasted multi-line command while fixing that rotation. Registered at Tier 3 for the mechanism, not the damage.Pre-flight
ruff check .validate_docs.shuv.lockdiffv1.11.0tagMerge method
Merge commit, not squash. Squash rewrites the release SHAs and recreates the divergence the back-merge exists to heal. Being armed via GraphQL and read back, because
gh pr merge --auto --mergesilently refuses to change the method on an already-armed PR — that nearly put a squash onmainduring v1.10.0.