EMRY-023: File watch + v1 JSONL tail - #65
Merged
Merged
Conversation
JsonlTailer offset-polls a metrics.jsonl file and parses appended rows into Event::MetricsBatch, leniently (epoch/phase default, so minimal third-party files parse). Holds partial trailing lines until complete, restarts on truncation/rotation, skips+counts garbage lines, exposes labels() for the TUI. run_watch is the thin poll loop. Polling (not inotify) is deliberate: robust over NFS on HPC. Live TUI wiring lands in EMRY-025.
- poll() decodes each line independently; an invalid-UTF-8 line is skipped and counted instead of aborting the whole poll (third-party files mustn't take down the watcher). - run_watch swallows transient poll errors and retries rather than terminating on the first I/O hiccup (NFS/HPC robustness).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #18 (EMRY-023). Branched off
main. First emry-ingest work.What
crates/emry-ingest/src/watch.rs—JsonlTaileroffset-polls ametrics.jsonlfile and parses appended rows intoEvent::MetricsBatch:epoch/phasedefault) so minimal third-party / v1 files with juststep+valuesparse.labels()exposes discovered metric names so the TUI can resolve them from the file.run_watchis the thin, resilient poll loop.Design: polling, not inotify
Deliberate — the target is HPC (SLURM / NFS), where inotify is unreliable over network filesystems. Offset polling is robust there and dep-free; latency is bounded by the poll interval (fine for a human-watched dashboard).
Verification
./scripts/pre-commit-rust.shgreen: clippy-D warnings, 10 ingest tests, watch.rs 97.8% lines, total 96.78%Review-driven fixes
run_watchretries transient I/O errors instead of dying (NFS robustness).Scope
Live
emry watch→ TUI wiring is EMRY-025.