Evidence

Evidence

A manuscript's quality shows in how its author checked the claims and in what the author admits no one has checked. The checks here fall into three kinds: those the papers describe about themselves, those that this wiki repeated against the code repositories, and the gaps that no check has yet closed. Because all four papers are unreviewed manuscripts, the first kind carries the author's word, the second carries a repeatable run, and the third stays in plain view so that a reader does not mistake the first for the second.

How the papers checked their own claims

Paper	Stated check	Where it is stated
Guardrail (2026)	The author verified every claim in Results by direct execution, compiling specifications, running the resulting code, and, where relevant, calling a live external model, and reproduced each defect before and after its fix	Methods
Guardrail (2026)	The author resolved each of the fourteen references in the earlier review programmatically against Crossref, arXiv, and OpenReview, replaced a wrong identifier after a bibliographic search, and withdrew a source with no matching record	Methods and Results
Watermark synthesis (2026)	The author verified substantive claims in the primary papers or official provider documentation, and used search indexes only for discovery and for reconciling bibliographic records	Section 2.1
Watermark synthesis (2026)	The author did not pool performance figures, and each quoted figure carries its experimental context, since model, tokenizer, text length, attack, and false-positive threshold differ across studies	Section 2.3
White paper (2026)	The recommendations rest on the same 20-paper corpus, and the paper says that cross-paper headline values are not directly comparable	Sections 2 and 3
Pseudocode review (2025)	The author summarized sources in an annotated bibliography and compared them in a research matrix; the paper describes no reference check, and the 2026 paper later found one was needed	Methods

The reference audit, in numbers

The audit is the clearest piece of evidence in the set, because it tests the author's own earlier work and finds it faulty in part.

Outcome	Count	Sources
Identifier confirmed as originally cited	4	Bashir (2025); Michaelsen and Santos (2024); Toksoz and coauthors (2024); Kwon and Kwon (2022)
Identifier corrected	9	Two papers by Liu and coauthors (2025), Lim and coauthors (2022), Mornar and coauthors (2014), Kulkarni and coauthors (2025), Rioux (2022), Imam and Alnsour (2019), Neal and coauthors (2023), Safder and coauthors (2018)
Unverifiable, excluded	1	The entry "Overturning and Moving (2024)"

For one entry, this wiki repeated the check on 2026-11-06. The 2025 list gives 10.1515/jisys-2018-0012 for Imam and Alnsour (2019); the Crossref API returned no record for that identifier, and returned the Imam and Alnsour paper in the Journal of Intelligent Systems for 10.1515/jisys-2018-0291, which is the identifier in the 2026 list. This one case confirms the method and says nothing about the other eight.

Cross-checks against the code repositories

The 2026 pseudocode paper quotes numbers from a code repository, which makes them testable. The runs below took place on 2026-11-06, in fresh virtual environments, and the wikis for the two repositories give the full commands and output.

Quoted in the paper	Rerun result
Appendix C: 39 passed, 1 skipped for the full test suite	39 passed, 1 skipped
Appendix C: five specifications, twenty-two vectors, pass rate 1.0 and zero unexpected failures for each	Five records with 5, 4, 3, 6, and 4 vectors, pass rate 1.0, and zero in both unexpected columns
Appendix D: eight defects	docs/DEFECT_LOG.md in the repository holds entries D6, D7, D9, D12, and D13, which together describe eight

The two watermark papers say that the application was designed and not yet validated. The lab that followed passed its own suites on the same date, 679 Python tests with 8 skips and 20 Rust tests, and its measurements are pilots labeled non-promotable. Passing tests show that the code behaves as written, and they do not validate the papers' scientific claims; the papers never made that claim.

Check it yourself in 5 minutes

1. Open the guardrail paper's Table 1 and choose two rows that read "Identifier corrected".
2. Open each citation in the 2026 reference list, follow its DOI at https://doi.org/, and confirm that the title and authors match the citation.
3. Find the same author in the 2025 list and follow that older identifier. The paper reports that such identifiers either failed to resolve or led to an unrelated work.
4. Open the guardrail paper's Appendix C, then run python evals/run_evals.py in the pseudocode-ai-workflows repository and compare the pass rates and vector counts.
5. Open the watermark synthesis at section 4.5 and read what the study says it is not.

Gaps that no check has closed

• No manuscript has been through peer review, and none has been published in a journal.
• The guardrail study has no comparison condition, so it cannot say how many of its eight defects review or conventional tests would also have found.
• The watermark papers rest on a 20-paper core corpus that the synthesis itself calls a high-relevance selection and not an exhaustive review.
• Empirical validation of the watermark application was pending at the papers' date.