Skip to content

Releases: RMI-Valuation/docket-yard

v2026.10.3 — decided dates, quoted

Choose a tag to compare

@camrex camrex released this 03 Oct 20:33
342f409

Migration 0034 (ADR 0023 addendum of 2026-09-16, accepted 2026-10-03): decision_decided_date keys on the page, and a machine quotation names the document_text row it read. docketyard citator decided quotes every Decided: line a decision's document displays — a held table; nothing a reader sees changes, and no line is called a decision's date. A migrating release: deploy behind the wall (infra/deploy/README.md); 0034 refuses a store whose decision_decided_date is not empty. Rollback is a Litestream restore, not a tag.

v2026.10.2 — the published schema loads

Choose a tag to compare

@camrex camrex released this 03 Oct 19:31

The nightly snapshot's schema.sql now loads into an empty database: it no longer repeats the full-text index's shadow tables, which made it stop at "table already exists". A forward pass logs its duration and says when it overruns its interval; the MCP recent_activity description states its 366-day window; a failed migration script rolls back. No migration, no index rebuild: deploy by tag.

v2026.10.1 — what a scheduled brief needs

Choose a tag to compare

@camrex camrex released this 03 Oct 17:24
881278e

The machine surface gains what an assistant running a daily or weekly brief needs to find what is new without guessing search words or diffing sheets (PR #43).

  • recent_activity (new MCP tool): what arrived in a window, across every proceeding or a list of dockets the caller passes (never kept). By default it is windowed on when the forward watch observed each record, so late postings are not missed; by: board_date uses the Board's own dates instead. Filters: prefix, prefixes to leave out, record type, the Board's type, words in the Filed For cell as printed, and deciding body. Outages inside the window are named. A window spans at most 366 days.
  • count_decisions (new MCP tool): counts by prefix, served dates, decision type and deciding body.
  • list_proceedings (new MCP tool, from 2026-09-18): the proceedings behind a count_filings count.
  • search_the_record takes filters (prefix, prefixes or dockets to leave out, a date range, record type, Board type, sort, page) and returns one list with a total; with no filters, the answer is unchanged. get_docket_sheet takes a range on the Board's dates.
  • /coverage and the coverage tool now say that the Board's activity outside its dockets (voting conferences, hearings as events, press releases, Federal Register notices) is not held.
  • Pages: prose pages get section headings; the home page's week figures and /dockets' figures are sentences, not stat rows; a sheet's type filters sit behind a disclosure ordered by count; the subscribe field has its border back.

No migration, no index rebuild: deploy by tag.

v2026.09.28 — a Search link in the masthead

Choose a tag to compare

@camrex camrex released this 17 Sep 16:32

The masthead links to /search, where the filters, order and grouped results are; the box beside it stays a quick lookup. No migration, no index rebuild: deploy by tag.

v2026.09.27 — search built out: grouped by proceeding, filings, filters

Choose a tag to compare

@camrex camrex released this 17 Sep 15:19
bf8c58d

A migrating release (0033): deploy behind the wall, and run search rebuild before the wall comes down. Migration 0033 empties the record search index.

  • /search grouped by proceeding, each with its matched filings, decisions, comments and pages, and "N more matches in this proceeding"; a flat list of every match.
  • Filings are searchable by the Board's type and Filed For; every decision by its type.
  • Filters: docket type, dates the Board printed, what to search, filing and decision type; best match or newest first; paging with totals. Filters are in the address.
  • A record entered in several proceedings is placed in each (migration 0033: search_place, search_document).
  • Bounded for one web worker: page text ranks the best 5,000 pages unfiltered and filters every page under 1.5 s; the page says when it was bounded.
  • A family's index row carries its own caption only; /suggest and MCP otherwise unchanged.

Rehearsed in v2026.09.26's image on the 2026-09-17 restore: migration 29→33 28 s, rebuild 11 s (write lock 2.2 s), all checks passed. PR #41.

v2026.09.26 — text pages keep their place; connect an AI assistant

Choose a tag to compare

@camrex camrex released this 17 Sep 12:21

No migration and no search index rebuild: deploy by tag.

  • Text pages: the sidebar stays beside a long document's text, so the docket stays in view at page 100.
  • Search: a result in the text of documents links its filing, decision or comment and its docket, not only the page.
  • /ai: connecting Claude, ChatGPT, Claude Code, VS Code, Cursor or Gemini CLI to the MCP server, for readers who never have.
  • MCP: every tool declares readOnlyHint, so clients stop asking a reader to confirm each lookup.
  • Home and week pages: "What this is" becomes "Use the record" (follow, connect an assistant, data, coverage), placed last on a phone.
  • Footer: names the release that served the page.
  • README: brought up to date.

v2026.09.25 — counts and page text for assistants; the independent graders' fixes; JSON shape 3

Choose a tag to compare

@camrex camrex released this 17 Sep 09:39

Two new MCP tools, the independent graders' fixes, and migrations 0030–0032 (PRs #34, #35, #36, #37, #38, #39, #40).

A migrating release (schema 29 → 32): deploy behind the wall (infra/deploy/README.md § Deploying a migrating release). Roll back with a Litestream restore, not a tag. JSON shape 3: a sheet's last_checked now means the last poll, and last_new_entry holds what last_checked used to mean. It is announced on /data.

What changes

  • MCP count_filings (#38) counts filings by the Board's own filing type, prefix and filed-date range, and how many of those proceedings also hold a second type.
    • It counts distinct filings and proceedings as entered.
    • It names unfinished months, including months no wave has begun.
    • It says what it did not count.
  • MCP read_page, and a matched passage on each page search hit (#38, ADR 0022 addendum). An assistant may read page text on a user's request.
    • Each answer carries who read the page, the Board's file, the scan, the operator's caveat and the licence line.
    • Engine-read pages say they carry OCR errors.
  • The independent graders' first fixes (#39):
    • Decision summaries and deciding bodies are passed through MCP.
    • A .json miss answers with JSON.
    • The /api example is held to the live shape by a test.
    • Wording an assistant repeats is corrected: 1 filing, last filed …, the note at the 100-entry cap, the hedge on a comment miss.
    • README and CONTRIBUTING describe the live project.
  • The operator's nine decisions (#40):
    • robots.txt: ChatGPT-User, Claude-User and Perplexity-User may read party and text pages. Indexers and training crawlers stay refused, and /search stays refused to every named agent.
    • Decision dates are labelled served, JSON entries gain date_kind, and the cite line reads (STB served Sept. 3, 2026).
    • "Last checked" means the last poll (JSON shape 3).
    • Webhooks add a timestamped signature alongside the unchanged one, and the payload carries shape_version.
    • Cite blocks carry an access date and the newest kept archive.
    • /coverage, /stats and early sheets say the early years are thin because the Board's own table is.
    • The CC0 label covers the compilation and the Board's own fields; filings and comments are reproduced as filed.
    • The methodology's repeated-filer sentence matches the rule.
  • Migrations 0030–0032 (#35, #37, #36): the veto trigger, per-page OCR failure records, and the router's verdict (page_route). Their addenda were accepted 2026-09-16.
  • The tabular fleet pass (#34): HunyuanOCR-1.5 workers, plus the worker fixes. Fleet code only; the running pass picks it up when the fleet is redeployed.

Verified before release

  • Rehearsed 2026-09-17 in v2026.09.24's image (SQLite 3.46.1) on a Litestream restore of production taken at 09:35 UTC, with main mounted.
    • The 0030 pre-check returned 0 rows.
    • 29 → 32 in 11.4 s: foreign keys clean, quick_check ok, row counts unchanged across capture, event, docket, filing, decision_record, enviro_comment, document and document_text (1,685,851).
    • The site served from the migrated store, every page checked in under 350 ms.
    • On real data: JSON shape 3 with the true last poll, the early-years counts (1996: 4 … 2001: 456 · 2002: 2,382), the UP-SP sheet's record-begins line, the served cite line, the JSON 404 and the robots split.
    • All six MCP tools answered. count_filings gave 201 AB proceedings holding a Consummation Notice and a trail-use filing (59 ms), and read_page read Decision 46314 from a "trackage rights Peoria" search.
  • Reviews:
    • /code-review at medium, high and low levels on each PR, with every finding fixed or dated in docs/deferred.md.
    • /security-review on the MCP tools and the webhook signing: no findings.
    • Copilot's findings on #34, #38 and #40 fixed. Codex was at its usage limit throughout.
  • Tests and CI: 998 tests pass; CI green on every PR.

Deployed 2026-09-17 behind the wall, 09:41:23–09:44:52 UTC (restore point 09:41:23Z, Litestream generation 073494ca664fa87f).

  • The 0030 pre-check returned 0 rows on the live store at schema 29.
  • migrate took the store 29 → 32. /health reports v2026.09.25, schema 32, and ingest polled on the new release at 09:44:01.
  • Checked against the live store before the wall came down: foreign keys clean; JSON shape 3; the sheet, coverage and served-cite changes; all six MCP tools, including count_filings (201) and read_page.
  • From outside after: every page checked returns 200, /filing/nope.json returns a JSON 404, and MCP lists six tools.

v2026.09.24

Choose a tag to compare

@camrex camrex released this 15 Sep 11:57
b847338

A footnote digit fused onto a decision's own docket keys as that docket (PR #33; ADR 0018 addendum of 2026-09-14, Accepted by the operator).

The Board prints a footnote marker straight after a docket number, and the text layer fuses the two: STB Finance Docket No. 340071 in a decision filed in FD 34007. 346 such findings in 331 documents were stored as a six-digit number the record does not hold, unresolved, though they name the decision's own proceeding. They now key as that docket, and the span test decides whether each is a caption or a citation. cited_raw keeps the number as printed.

What changes:

  • One rule, keys.own_key, used by the finder, the span check, load and the served-date anchor. The finding carries its key; load rebuilds the decision's docket family from the record and refuses a finding whose key the rule does not give, and a document whose re-keyed number is itself a held docket. Both refusals are counted apart from failures and exit 3.
  • Each reading the rule shaped records it: source_location.key_rule names the rule, the six-digit keys it re-keyed and the family they were checked against.
  • The finder moves to 2026-09-14b, the resolver's rules to rule-1-2026-09-14b / rule-2-repair-2026-09-14b, the rank to v7. No migration; KEY_VERSION does not move.

Not a migrating release. Deployed as routine: set DY_TAG, pull, bring up. Schema stays 29. The re-load follows (docs/runbook.md § The own-fused re-load).

Verified before merge:

  • Reviewed by schema-critic, the ingest specialist and /code-review; every finding verified and fixed or recorded in docs/deferred.md. 867 tests; CI green; Copilot: no comments.

  • Cards: text layer 223/227, 217/221, 217/221, work 140/140; OCR 36/59, 36/43, 36/43.
    Deployed 2026-09-15 11:58:59–11:59:15 UTC, routine (no wall); re-loaded 12:01:07–12:08:12 UTC (restore point 12:01:07Z). /health reports v2026.09.24, schema 29; pages return 200. Every figure matched the rehearsal: 346 readings re-keyed, 346 six-digit keys retracted onto the own key, 0 refused, 0 documents changed or lost; projection 26,205 rows, edges 22,547 / 332, queues 438 / 2 / 900 unchanged with no edge identity added or removed; the older-rule gate and both drift queries at 0.

  • Rehearsed in v2026.09.23's image on a restore taken after the v6 re-load: rows 26,205, edges 22,547 / 332 and queues 438 / 2 / 900 unchanged, no edge identity added or removed; 346 readings re-keyed (61 citations), 346 six-digit keys retracted onto the own key; 0 refused, 0 documents changed or lost; the older-rule gate and both drift queries at 0.

v2026.09.23 — the family is a fallback anchor, not a peer

Choose a tag to compare

@camrex camrex released this 14 Sep 12:10
1120c60

Finder 2026-09-14, rank v6: the family is a fallback anchor, not a peer (PR #32).

A citation's served-date window now anchors on the target's own printed occurrences first. Its family's (sub-docket or suffixed forms) count only when those print no service date. This is the operator's decision. Before, a parent printed beside its sub-docket took every date both printed:

  • Decision 41393 prints FD 34554 (STB served Oct. 7, 2004) … FD 34554 (Sub-No. 2) (STB served February 11, 2005). Under v5 the parent named nothing. Now it names notice 35093.
  • Decision 39033 prints EP 575 and EP 575 (Sub-No. 1) (STB served Oct. 30, 2007). The parent has no date of its own, so it still reaches 36758.

The resolver's rules carry the version (Codex's P1 on the PR): rule-1-2026-09-14 and rule-2-repair-2026-09-14, ranked under v6.

  • load retires every older-version resolution onto the new one. A human answer is never touched.
  • stamp refuses a card measured on another rule.
  • FINDER_VERSION moves to 2026-09-14 with them, although the finder emits exactly what it did.

Not a migrating release. Schema stays 29. Deployed as routine, then the citator is re-declared and re-loaded on both channels (docs/runbook.md § The fallback re-load).

Cards, rebuilt on this finder and these rules, identical to 2026-09-13b's but for version and date:

  • text layer: truth 226, citation 223/230, resolution 217/221, projection 217/221, work 140/140;
  • OCR: truth 57, 36/59, 36/43, 36/43.

Rehearsed inside v2026.09.22's image on a 2026-09-14 restore, against an untouched copy at v5:

  • the gate reads 104,765 old-rule resolutions on live keys before and 0 after;
  • projection 26,205 rows, edges 22,547 text-layer / 332 OCR, and queues 438 / 2 / 900 are all unchanged;
  • one edge identity changes: decision 41393's FD 34554 names 35093.

Reviews:

  • schema-critic scoped the rule-version change;
  • /code-review low found no findings (one candidate was verified unreachable);
  • Codex's code review found the one P1, fixed and replied to, but its re-review of the fix hit the usage limit;
  • Codex security: no findings;
  • Copilot on the final head: 0 comments.

Not in this release: the six-digit own-docket rule (346 findings). It changes citation identity, so an ADR 0018 addendum comes first.

Deployed 2026-09-14 12:11 UTC, routine (no wall); re-loaded 12:13–12:17 UTC (restore point 12:11:55Z). /health reports v2026.09.23, schema 29. Both cards were declared, then find produced 104,765 findings, load ran on the text layer (0 failed, work_gained 1) and OCR (0 failed, 0 refused), and restamp re-stamped 0 rows on each channel.

Measured on production with the shipped code. Every figure matches the rehearsal:

Before (v5) After (v6)
Projection rows 26,205 26,205
Edges, text layer / OCR 22,547 / 332 22,547 / 332
citation_exposed / citation_repaired / citation_unresolved 438 / 2 / 900 438 / 2 / 900
Live resolutions by rule rule-1 105,666, rule-2-repair 2 new rules 104,763 + 2, rule-1 903 (retracted keys only)
Old-rule resolutions on live keys (the gate) — 0

One edge identity changed, the decided one: decision 41393's FD 34554 names 35093.

857 tests.

v2026.09.22 — the image takes libsqlite3-0 deb13u2

Choose a tag to compare

@camrex camrex released this 14 Sep 10:03
6ac87d9

The image takes Debian's libsqlite3-0 security update (PR #31).

v2026.09.21's image carried libsqlite3-0 3.46.1-7+deb13u1. Its base, python:3.12-slim of 2026-09-01, had not picked up Debian's 3.46.1-7+deb13u2 (2026-06-14), which backports two fixes in FTS5, the full-text engine search is built on:

A rebuild alone does not take the fix, because the Dockerfile used the base as it is.

What changes: one RUN in the Dockerfile's final stage upgrades libsqlite3-0 alone, unpinned: a pinned version leaves Debian's mirror at the next update and would break the build. SQLite stays 3.46.1. There is no code change, no schema change and no migration.

Not a migrating release. Deployed as routine: set DY_TAG, pull, bring up. Schema stays 29.

Verified before merge:

  • Local build: libsqlite3-0 deb13u1 → deb13u2.
  • The suite inside the built image, against its installed code: 853 passed, 1 skipped.
  • CI green; Codex code and security reviews: no findings.

Deployed 2026-09-14 10:04–10:05 UTC, routine (no wall). DY_TAG v2026.09.22; /health reports v2026.09.22, schema 29. Both the web and ingest containers carry libsqlite3-0 3.46.1-7+deb13u2, and SQLite reports 3.46.1. Every service is running and web is healthy; the poller captured at 10:05:17 UTC. /, a search, a docket sheet and /methodology return 200.

The operator's decision, 2026-09-13. Moving past SQLite 3.46 (the WAL-reset fix, 3.53's ALTER for constraints) stays its own later decision: docs/deferred.md § SQLite 3.53.4.