fix(phi-scan): corroborate every v2 field number against a published HL7 v2.5.1 (PHI-SCAN-RESIDUALS) - #37
Merged
Merged
Conversation
…HL7 v2.5.1, and read seven fields it disclosed as unread (PHI-SCAN-RESIDUALS) The segment field list IS the detector in this package: the cross-cutting SSN/email floor finds zero in the tracked files carrying inline `PID|` literals, so a wrong field number is a MISSED LEAK or a FALSE POSITIVE ON A CLINICAL FIELD. Fifteen numbers had never been checked against any published source: the whole GT1 row, plus PID-6/9/19/20, NK1-30/33 and IN1-18/19. All fifteen are now checked against the v2.5.1 SEGMENT ATTRIBUTE TABLES in Chapter 3 (PID 3.4.2, NK1 3.4.5) and Chapter 6 (GT1 6.5.5, IN1 6.5.6), and cross-checked field by field against a second version-pinned publication. NONE OF THE FIFTEEN WAS WRONG. One citation was: the source said GT1 was clause 6.5.4. Three of these rows read differently in a later v2 (PID-9, PID-19 and PID-20 are withdrawn from v2.7 onward), so the version is load-bearing rather than decorative. Every row now carries its v2.5.1 ITEM NUMBER, the standard's own stable identifier for an element. Seven fields a refuter had measured as reported-clean are read now, because the same published tables ground them: NK1-26, NK1-31, NK1-32, NK1-37, GT1-2, GT1-4 and IN1-49. A UNION, never a replacement, and the superset is pinned cell by cell: every field that reported before still reports, every deliberate non-field (IN1-17, IN1-7, PID-10, PID-18, NK1-3, GT1-11, PV1-19) still reports nothing, and the tracked corpus stays clean on `pnpm phi-scan`. `PHI_SEGMENTS` is derived from the union of all five field tables instead of one of them. Read from one table, a segment added to another alone would never be located and nothing would report: no error, no warning. Both expressions answer identically today, so it guards the next edit rather than changing this one. Grounding, sources, the version trap and what is still NOT ground truth here: documentation/agent-notes.md, under the PHI scanner section.
…ative-control rows the first draft asserted from recall (PHI-SCAN-RESIDUALS) A refuter measured `PV1-7` written into the suite's negative-control list as v2.5.1 item `00147`. PV1-7 is `00137` (Attending Doctor); `00147` is PV1-17 (Admitting Doctor). The 35 READ rows were extracted from the published attribute tables mechanically. The nine negative-control rows in the same case were not: they were written from recall. Eight of the nine happened to be right, which is the finding rather than a mitigation, because being right was luck and being wrong was invisible: PV1 is read by nothing and the case asserts a clean result, so no detection changed either way. A wrong citation costs nothing at all until the next reader re-checks a row against it, lands on a DIFFERENT element, and comes away confirmed. That is the failure the item numbers were added to prevent, reproduced by the change that added them. All 44 rows are extracted and verified now, with a negative control that re-checks every row against the WRONG segment's table and mismatches 43 of 44. The rule is recorded where the table lives: extract an item number or do not write one. Also corrected: a sentence in the notes said no tracked file carries a GT1 segment literal, and wrote the literal to say so. This file is inside the scan's own corpus, so the sentence became the only occurrence of the thing it said did not occur. It parsed to an empty segment and reported nothing, so it was false rather than dangerous, but a note about a detector, inside the corpus that detector reads, is itself an input.
…rrect two loose sentences about it (PHI-SCAN-RESIDUALS) Prose only. No code, no test, no detection change. A second refuter pass could not reproduce "44 of 44 against the published tables, with a negative control that re-checks every row against the WRONG segment's table and mismatches 43 of 44", and it was right that it could not. 43 was the count taken BEFORE `PV1-7` was corrected, when the wrong row mismatched its own segment too, and the sentence never said WHICH table the rows were re-checked against, so neither number was reproducible. Re-measured and stated with its pairing: 44 of 44 rows match their own segment's table, and re-running the same comparison with every row looked up in the PV1 table mismatches 42 of 44, the only two matches being PV1-7 and PV1-19, which genuinely are PV1 rows. A control whose comparand is unstated is not a control. It is also a scratch measurement rather than a test, and is now recorded as a citation to repeat rather than as a gate that fires. Also corrected: the note said the stray GT1 literal "parsed to an empty segment". It opened a segment whose only field is GT1-1, which no table reads, so it reported nothing. The conclusion was right and the mechanism was not.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What and why
The segment field list is the detector in this package. The cross-cutting SSN/email floor finds zero in the tracked files carrying inline
PID|literals, so a wrong field number is not cosmetic: it is either a missed leak (a PHI-bearing field never read) or a false positive on a clinical field.IN1-17shipped once as a telephone field and is Insured's Relationship To Patient, so a coded relationship was reported as a phone number.Fifteen numbers had never been checked against any published source: the whole GT1 row, plus PID-6/9/19/20, NK1-30/33 and IN1-18/19.
What was grounded, against what
All fifteen (and every other row) are checked against the HL7 v2.5.1 segment attribute tables: Chapter 3, Patient Administration (PID 3.4.2, NK1 3.4.5, PV1 3.4.3) and Chapter 6, Financial Management (GT1 6.5.5, IN1 6.5.6), parsed out of the published chapter HTML rather than read through a summarizer. Every row was then cross-checked field by field against a second, independently published, version-pinned artifact: the HAPI HL7 v2 generated structures for v2.5.1. The two agree on the number, the name and the data type of every row, and on the segment lengths (PID 39, NK1 39, GT1 57, IN1 53).
Every row now carries its v2.5.1 item number, the standard's own stable identifier for an element.
What it bought as detection
Seven fields a review had measured as reported-clean are read now, because the same tables ground them: NK1-26, NK1-31, NK1-32, NK1-37, GT1-2, GT1-4, IN1-49.
A union, never a replacement, and the superset is proved cell by cell rather than asserted: every one of the 28 cells that reported before still reports, and every deliberate non-cell (IN1-17, IN1-7, PID-10, PID-18, NK1-3, GT1-11, PV1-7, PV1-19) still reports nothing.
pnpm phi-scanexits 0 over the tracked corpus before and after, so nothing was traded for it. NK1-37 is the sharpest of the seven: an undashed contact SSN the floor is structurally blind to, pinned in both polarities in one case.PHI_SEGMENTSis derived from the union of all five field tables instead of one of them. Read from one table, a segment added to another alone would never be located and nothing would report: no error, no warning. Both expressions answer identically today, so it guards the next edit.Both polarities, everywhere
The coverage case is 35 positives and 9 negative controls in the same case, so a wholesale detector failure cannot produce the zeros. The positive locator assertion is the exact
segment=<SEG>-<N>token, becausePID-3alone is a prefix ofPID-33and would let a renumbering pass. Bite was checked by mutation, not assumed: dropping NK1-37 reds 2 cases, renumbering PID-6 to PID-8 reds the coverage case, and re-adding IN1-17 as a phone reds it with the original defect's own message.Reviews
Two conformance-refuter passes. Pass 1 returned
REFUTEDon anINTRODUCEDmajor of exactly the class this change is about: the nine negative-control rows had been written from recall rather than extracted, andPV1-7was given as item00147(it is00137;00147is PV1-17). Eight of nine were right, which is the finding, because being right was luck and being wrong was invisible. Fixed in a follow-up commit, with all 44 rows extracted and verified. Pass 2 re-verified 44/44 independently and returnedNOT REFUTED; its one remaining minor (a control sentence that did not name its comparand) is corrected in the last commit.Two
PRE-EXISTINGminors are for the backlog and are not touched here: the coverage case is durable in the narrowing direction only, and the test file's assemble-at-runtime banner is falsified by literals already present at base.Verification
scripts/verify.sh transformgreen. Steps that actually ran: audit, licenses, typecheck, lint, format:check, phi-scan, check:no-emdash, check:no-internal-refs, test:coverage (26 files, 428 tests, coverage 97.63/92.09/100/98.69), build, attw.check:agent-notesis gate-shaped and not on the umbrella's ladder, so it was run by hand: exit 0.CLAUDE.mdis untouched at 21,891 bytes. The narrative went todocumentation/agent-notes.mdunder the existing PHI-scanner section, as a###subsection, so the two-file agent-instruction contract still holds and no trap was deleted or relocated away.