pi/omp: discover transcripts whose session record follows a title slot - #859
Conversation
The shared Pi/OMP discovery gate only checked the first physical line of a transcript for a `type: "session"` record. Oh My Pi writes a fixed-width `type: "title"` metadata line before that header (upstream can1357/oh-my-pi@0ce330a, 2026-06-27), so valid OMP transcripts were rejected and omitted from `codeburn sessions --provider omp`. readFirstEntry now scans up to MAX_HEADER_LINES_SCANNED (20) leading lines via the existing streaming readSessionLines helper, skipping blank lines and malformed JSON, until it finds a session record. This keeps discovery bounded for message-only files (and pathological blank/junk-line runs) instead of reading the whole file, and Pi and OMP continue to share the exact same discovery path. Fixes getagentseal#845
iamtoruk
left a comment
There was a problem hiding this comment.
Landing this for the one difference you flagged, now that #846 has merged the correctness fix. Rebased onto current main and reduced to exactly the perf delta: discovery streams through readSessionLines and stops after MAX_HEADER_LINES_SCANNED (20) leading lines, instead of #846's readSessionFile which reads each transcript in full (up to the 128MB cap) before looking at line 0. On the discovery hot path (walked for every file on every run), that turns a message-only file with no session record from a full read into a 20-line scan.
One reconciliation on top of your branch: the auto-merge reverted #846's readFirstEntry → readSessionEntry rename, so I re-applied it (and dropped the now-redundant first.type !== 'session' caller guard, since readSessionEntry only ever returns a session entry or null). Net effect on main is purely "bounded streamed scan," with #846's naming preserved.
Verified on the merged tree: tsc clean, omp + pi provider suites 37/37 (both #846's title-first omp case and your bounded-scan cases, including the "session record beyond the 20-line bound is not found" test that proves the scan is bounded rather than full-file), full suite green apart from the two pre-existing parser.test.ts failures. The 20-line bound is safe for the real OMP/Pi format (title slot + header at the top). Thanks for the analysis on the read-cost difference — it was the right call to keep it. Merging.
# Conflicts: # src/providers/pi.ts
29f9071 to
62f6eb2
Compare
…he all-provider view buildDurablePeriod derived the today slice of the multi-day, all-provider headline from the unsliced whole-range parse, so a turn spanning local midnight kept its category and turn count anchored on its yesterday start. The per-call cost and calls bucketed onto today correctly, but By Activity and the JSON daily turn count lost the post-midnight half — categories summed to only the pre-midnight cost while the headline, By Model and By Project were right. Slice the today parse with filterProjectsByDays first, which re-anchors the straddling turn to its surviving today calls, so today's category cost lands on today. Category cost is the sum of the slice's own calls, so day-N + day-N+1 still equals the whole-range total (no over-count); the per-day turn-count split matches the cache side and the documented per-day semantics. Adds a regression test in the straddling-turn conservation suite (mutation-checked: fails on the pre-fix code). Also fills in the CHANGELOG Unreleased entries for the batch (#853, #856, #872, #846/#859, #866/#867, #833).
Relates to #845.
The defect
The shared Pi/OMP JSONL discovery gate inspected only the first physical line of a transcript for a
type: "session"record. Oh My Pi writes a fixed-widthtype: "title"metadata line ahead of that header, so valid OMP transcripts were excluded fromcodeburn sessions --provider omp. Both this branch and #846 scan past leading non-session lines to fix it, and both keep Pi and OMP on the identical shared path.Difference from #846
Read cost. #846 keeps
readSessionFile, which reads the entire transcript into memory (up to the 128MB cap) and then iterates its lines. This branch streams throughreadSessionLines— the helpercodex.ts,droid.tsand others already use — and stops after 20 leading lines.The distinction matters most for the negative case. A message-only file with no session record is read in full under both the old code and #846; under a bounded scan it costs 20 lines. Session directories accumulate such files, and discovery walks all of them on every run.
Worth noting the pre-existing code only looked bounded: it read the whole file and then used line 0. So this is not a regression #846 introduces, it is an existing cost neither the old code nor #846 avoids.
One semantic change here:
readFirstEntrynow returns only session entries rather than whatever line 0 parsed to. The caller'sfirst.type !== 'session'guard becomes redundant but is left in place rather than widening the diff.Acceptance criteria
Testing
Fixture tests in
tests/providers/pi.test.tsreproducing a title-slot-first OMP transcript, confirmed failing before the change. Also covered: blank-line tolerance, malformed leading JSON handled without throwing, message-only exclusion, a session record placed beyond the scan bound (which must not be found, proving the scan is bounded rather than full-file), and a large message-only file completing well inside a generous time ceiling.Full suite 2471 passed,
tsc --noEmitclean.Credit
@jbspeakr reported #845 and traced the format to its upstream origin (
can1357/oh-my-pi@0ce330ab, 2026-06-27), which is what made this straightforward to scope.