feat: dmtri fix — automated repair of consistency-report inconsistencies - #34
Merged
Conversation
…tency reports `dmtri fix <report>` reads an eida-consistency report (local file or URL), finds the inconsistent streams and their day-by-day boundaries, and runs the repair chain for each: wfcatalog_refresh (re-run the collector) then availability_rebuild (rebuild the availability view for the exact window). - fix.py: locate eida-consistency, parse `explore --json`, validate every report value against strict allow-lists (the report is untrusted URL input), map fixes to playbook vars, drive the chain with continue-on-error. - playbooks/fix/availability_rebuild.yml: scoped rebuild via `docker exec fdsnws-availability-cacher avail-rebuild`, using command/argv (no shell) for injection safety. - hooks: run_hook now returns the rc and takes exit_on_error, so one failed stream doesn't abort the batch. - --verify-only re-checks a report without changing anything. - README: "Fix Inconsistencies" section + eida-consistency prerequisite. - tests: 21 tests (parsing, schema guard, date-boundary mapping, dependency missing/too-old messages, command-injection rejection).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds
dmtri fix, which turns an eida-consistency report into automated repairs.Point it at a report (file or URL) and, for each inconsistent stream, it re-runs
the WFCatalog collector and rebuilds the availability view for the exact broken
window — closing the gap where
/availabilityfailed to show data that/dataselectactually returns.How it works
eida-consistency explore --jsonto get structured fixes(NSLC + true window + direction + status).
wfcatalog_refresh→availability_rebuild, continue-on-error(one failed stream is recorded and skipped, not fatal).
/availabilitycaches answers ~20 min;--verify-onlyconfirmslater.
Companion changes in other EIDA repos (required for this to work)
This feature depends on two changes made in sibling repos:
explore --json, a machine-readable output of theinconsistencies and their boundaries (schema v1.0: NSLC, start, end, direction,
status).
dmtri fixconsumes this instead of scraping human logs.avail-rebuildconsole script, which rebuildsthe availability materialized view for a specific NSLC + date range. dmtri
invokes it over SSH via
docker exec fdsnws-availability-cacher avail-rebuild.No public mutate route was added to the FDSN service (kept off the public API
on purpose).
Files
src/dmtri/fix.py(new) — command logic, dependency resolution, inputvalidation, fix→playbook-vars mapping, chain driver, verify-only.
src/dmtri/data/playbooks/fix/availability_rebuild.yml(new) — scoped rebuild,command/argv (no shell) for injection safety, with job tracking.
tests/test_fix.py(new) — 21 tests.src/dmtri/cli.py,src/dmtri/hooks/hooks.py,src/dmtri/paths.py— wiring(subcommand,
run_hookrc +exit_on_error, playbook path).README.md— "Fix Inconsistencies" section + eida-consistency prerequisite..gitignore— ignore demo recordings and AI-assistant folders.Direction handling (refresh vs clean)
Inconsistencies have a direction, and they are NOT handled the same way:
availability view is stale → rebuild it. This is the safely-automatable case
and the only one this command acts on.
skipped. This pattern is predominantly false positives — restricted data
(eida-consistency#42) and transient service errors (eida-consistency#36) —
where auto-removing availability records would destroy correct metadata. It is
reported, not actioned.
Security
The report is fetched from a URL (untrusted) and its values flow into
docker exec/ shell-touching playbooks. Defense in depth:_validate_fixrejects anything not matching strict allow-lists (FDSN codes,ISO dates) before it can reach a command line.
command:/argv:rather thanshell:.Validation
Run against the live NOA node on a real report: all 8 inconsistencies repaired,
every Ansible run reported
failed=0/unreachable=0, and an independent--verify-onlypass afterward reported 8 fixed, 0 still inconsistent.Full test suite: 61 passing.
Notes / scope
dmtri fixrequires the eida-consistency CLI; it prints exact install/upgradeinstructions if missing or too old, and stops before touching anything.
Closes #33.