Let the evidence leave the screen (re-targeted at main) - #38
Merged
Conversation
Closes the first half of #33. Until now the thing this project exists to produce — the named tells, the hidden characters at their line and column, the citations a document makes that its own bibliography contradicts — died when the tab closed. A teacher could be shown all of it and forward none of it. That is a hole in the argument, not a missing feature: a percentage is not evidence, and evidence you cannot hand to anyone is not much better. The report comes out as Markdown for pasting into an email or an LMS, or as a self-contained HTML page that opens by double-click and prints to PDF. No stylesheet, no script, no external request — it has to open from a downloads folder years later on a machine that has never heard of this project, and one that fetched anything would leak which documents were read and when. Three things it always does. It prints this build's own false-positive rate, quoting the upper bound of the interval rather than the observed rate, and names the rules known to misfire so a reader can weigh evidence that leans on one — the sample report counts `rhet.in-terms-of` as a signal and says on the same page that it fires on 10% of human academic writing. It separates checkable facts from the judgement, because a DOI on two different works is not an opinion about prose. And it states that it holds the document and was never uploaded, since the share card is built to be posted in public and confusing the two would publish a student's work. Those numbers come from a calibration snapshot embedded at build time and written by the same tool run that regenerates Docs/CALIBRATION.md, so the page cannot quote a threshold three versions old. A fork that has never measured itself ships no snapshot and the report says so, rather than inheriting a figure it did not earn. CLI: `--report <file>`, written after everything else so a failing --max-score gate still leaves the document behind for whoever has to look at it. The Markdown-to-HTML conversion handles exactly what this file emits and nothing more. The input is produced twenty lines above the converter, so anything it cannot parse is a bug here rather than in someone's document, and a real parser would be a dependency this library refuses to take. Refs #33 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Finishes #33. The web app, the desktop app and the folder triage can now each save the evidence as a document, using the download path the rule-pack template already used. The button sits beside the share card on purpose. The card is a picture of the score, carries no text and is made for posting in public; the report holds the document itself and is made for keeping, forwarding or printing. Putting them next to each other is what makes the difference visible at the moment somebody chooses, and the button's tooltip says the report contains your text and is saved to your device. The folder report is not the single-document one repeated. What a teacher has is a stack, and the honest thing a triage list can do is say where to look first — so the page states, above the table, that it is a reading order and not a ranking, and that nothing on it establishes that anyone did anything. A table of students sorted by score with no such sentence reads as a ranking of guilt, and it would be this project's own argument turned inside out. Files that could not be read are listed rather than dropped, because a scan that quietly skips the encrypted PDF is worse than one that admits it. Also fixes the last host that was still counting demoted findings as signals: the batch page reported Findings.Count per file, which would have put a paper at the top of a teacher's reading order for using "furthermore" the way people use it. Closes #33 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The committee read the report as the artefact it is — something forwarded to a nineteen-year-old and read by a committee that will not read the caveats — and found six things worth stopping for. All six verified by hand before touching anything. **It asserted a contradiction it had not checked.** "None of this needed the internet: the document contradicts itself" printed whenever there was anything at all to say about sources, including the case where the report had just said, one line above, that no reference list was found and the cross-checks were not run. A student who simply omitted a bibliography got a page accusing them, in the report's own voice, under a heading promising checkable facts. That sentence is now conditional on an actual disagreement, and the other case says plainly that nothing here is a finding. **A heading claimed something false about half its own table.** "Characters that writing does not produce" covers soft hyphens and unusual spaces, which Word inserts on its own and any copy-paste carries. The table now says what is in the file and names the innocent explanations, because the reader's inference — that a soft hyphen implies a rewriting tool — is exactly the judgement the "checkable facts" framing invites and cannot support. **User content could break the page structure.** A pipe in a filename opened an extra table cell and shifted every number one column right, in the table a teacher reads scores against student names; verified as five cells in a four-column row. A newline in an extractor's error message closed the list and let the remainder become report prose — a line beginning "## " arrived as a heading in the report's own voice. Both go through Cell() now, and the table splitter honours an escaped pipe. **"Nought out of ninety" was hardcoded** two lines under the same figures read from the calibration snapshot, so the next corpus would have had the page contradict itself — the precise failure PublishedCalibration exists to prevent, committed inside it. **The tool wrote the embedded snapshot relative to the working directory**, so running it from anywhere but the repository root left the real one stale while Docs/CALIBRATION.md updated correctly, and the build would embed last month's threshold under this build's name. Anchored to the manifest, and it now prints where it wrote. **PublishedCalibration.Current could throw from a property getter**, taking down every report over a malformed resource. It fails to null, which the page already knows how to say out loud. Its two null branches also contradicted each other: one called the build uncalibrated while the other went on to print the corpus and the date. Also: the folder report lists every file rather than the first forty — it is the document a teacher keeps, and silently dropping 160 of 200, every low scorer among them, would be worse than not writing it. Files are named by their path under the scanned folder, since one directory per student makes "essay.docx" ambiguous exactly when it matters. A file with both a score and an error no longer appears in both sections. And the caveat no longer says "at most", which promised a guarantee a 95% interval does not give. Six regression tests, one per defect. The tests no longer assert that a calibration snapshot exists, because shipping without one is a state the design blesses and a suite that failed for it would push a fork toward quoting somebody else's number. Refs #33 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The architecture review found the worst thing this project has produced, and Docs/CALIBRATION.md had already written the sentence condemning it, one line above its own table: "a rate that holds in English and fails in Spanish is not one number, and reporting it as one would hide exactly the failure that matters here." The report quoted the aggregate. On the current corpus English bounds at 5.6% and Spanish at 13.3%, and neither group is large enough to support the 5% target on its own — both show a dash on that page. A report about a Spanish essay was handing its author a bound three times better than anything measured for their language, under the heading "A score is not proof". The snapshot now carries the strata, and the caveat reads the one for the language actually analysed, saying plainly when that language supports no threshold at all and that the overall figure is not a substitute for it. The second thing it found is the same failure in a different place: the page opened with a verdict it then spent four lines disowning. "Reads mostly human" printed above "treat the score as saying nothing" is a document arguing with itself, and a reader keeps whichever half suits them. A low score is not evidence a person wrote something — a detector that detects nothing also returns zero, and this project deliberately never measured how much machine writing it catches. Below the threshold the build can support, the number now stands alone and the page says why. And the checkable facts are named in the headline rather than left below the score. The document this project keeps writing about scores near zero and has an invented bibliography; a page that opened "0/100 — reads mostly human" and buried four source contradictions further down had chosen the wrong thing to make salient, on the one page where it matters most. Refs #33 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The report existed in Core and stopped there. Every other consumer got it — the CLI, the web app, the desktop app, the folder triage — while the MCP server, the one interface where something else is doing the work on somebody's behalf, could still only describe what it found. A description in a chat window is exactly what a teacher cannot forward to a student or take to an integrity committee, which is the gap the report was built to close in the first place. `write_report` returns the finished Markdown: the score, what counted and what was found at a rate people write at, the characters with their line and column, the citation disagreements, the checkable facts named at the top, and the error rate measured for the language actually analysed. Markdown rather than the HTML variant on purpose. It survives being pasted into an email or a comment box and stays legible when an agent quotes part of it back, while pushing tens of kilobytes of markup through a tool result to a model that will not read it would be waste. The apps and the CLI still produce the printable page. Its description tells the agent the result carries material from the document and is to be handed to the person who asked rather than posted anywhere — the same thing the report says on its own face, in the one place where the reader is not a person. Refs #33 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
#35 was merged into
rules-that-misfireat 21:59:48, twenty-two seconds after that branch had itself been merged intomain. Both PRs show green and both are honest — but the report never reachedmain. This re-targets the same five commits at the right base.Nothing here is new work.
maincurrently has the genre gating from #34 and none of the report: noEvidenceReport, no buttons in the web or desktop apps, no folder report for/batch, and no tenth MCP tool.Contents, unchanged from #35:
EvidenceReportrenders an analysis as Markdown or as a self-contained HTML page — no script, no stylesheet, no external request, so it opens years later from a downloads folder and leaks nothing about which documents were read.--report <file>in the CLI;write_reportas the tenth MCP tool, returning Markdown so it survives being pasted.PublishedCalibration, a build-time snapshot with per-language strata, so the printed error rate names the population it was measured on.Known and tracked rather than hidden: #36 (the report is English-only) and #37 (the Markdown round-trip, the two-phase build, custom-pack provenance).
🤖 Generated with Claude Code