Limits and Next Steps

Limits and Next Steps

The papers support modest claims, and they say so. The pseudocode study shows that a contract-checked, reviewable pipeline can be built and that building it exposed defects; it does not show how often the approach succeeds or that it beats other review methods. The watermark papers show how the research literature constrains what a detector may claim and how software could report that; they do not show that any detector works in practice. The limits below follow from those boundaries.

Limits

Status. All four are manuscripts by one author. They have had no peer review and no journal acceptance, and the repository records no citations. Their dates, 2025 and the summer of 2026, matter, because the code repositories have moved on since then and the papers describe the state at their dates.

Scale and attribution in the guardrail study. One maintainer, one working session, one commercial model queried once in the central demonstration, and five specifications in the benchmark. The pass rate of 1.0 describes five specifications and is not a measured rate. With no comparison condition, the attribution of the eight defects to the contract-and-execution method is suggestive and not demonstrated.

The central claim awaits measurement. The study's practical claim is that reviewing a specification beats reviewing generated code. Its own Limitations section names the missing cost-benefit comparison, and the set holds no measure of reviewer effort or accuracy.

Corpus limits in the watermark papers. The synthesis covers 20 papers released between 2023 and August 2026, chosen for relevance, and it excludes training-data and model-weight watermarks, image watermarking, and simple Unicode tricks unless they clarified a security property. Its categories describe that corpus and not the field. One method it discusses was a preprint at the time and could change.

Validation pending. Both watermark papers are design and evidence drafts. They propose adapters, a decision vocabulary, and controls, and they state that empirical validation of the application was pending at August 14, 2026.

A known-faulty reference list. The 2025 review keeps its reference list as first written, with nine wrong identifiers and one unlocatable source. A reader who cites it should use the 2026 list.

Figures from other studies. The watermark synthesis quotes results from other authors, such as true-positive rates under different paraphrasing conditions, and it warns that these cannot be ranked against each other. They are context and not findings of these papers.

Next steps

The sequence follows what the papers themselves recommend, and none of it carries a date.

1. Issue a corrected edition of the 2025 review, with the audited reference list and the unverifiable source removed.
2. Run the guardrail across several models and several independent tasks before treating the single live run as typical.
3. Measure reviewer accuracy and effort when the same defect appears in a specification and in the generated code, which would supply the comparison the 2026 study lacks.
4. Complete the watermark programme's empirical step: a reference implementation and a multilingual benchmark with matched human and model text, calibrated before any claim of detection.
5. Submit the manuscripts for independent review, so that the statements about their status can eventually change.