Tested with Panoptes workbench results vary by model family/adapter/harness

[Encryptic1]

Result (signed backend/artifacts/cards/external-repo-watermark-remover.json): the tool's Unicode hygiene drives the zero-width watermark from 100% present to 0%, while leaving the statistical KGW watermark fully detectable (100% → 100%) — a concrete demonstration that removers are family-specific. The repo's network-dependent "Layer B" statistical rewrite is not exercised by the reference adapter.

We are seeing similar issues with other removers and have tried to create a corpus of model family tests given the residue is very model family and even harness/adapter specific. Would be happy to work on this mutually to run more tests as you add features we have already implemented the test for your repo natively into the Panoptes system https://github.com/marketstandard/Panoptes/blob/main/docs/testing-external-repos.md

[guillaumemeyer]

Absolutely. Good feedback.
I am working on adding a benchmark that covers multiple specific techniques.
Stay tuned.
You can also submit a PR ;-)

[Encryptic1]

@guillaumemeyer Thanks! I pulled the latest tree — the multi-scheme surface in detect_text_watermark.py (KGW / SynthID / EXP / Unigram / SIR) and bench_synthid_text.py are exactly the right shape. The same-config detection caveat in your benchmark docs is precisely where an external workbench adds value, so before opening a PR I'd rather align on what's actually useful to you.

What we could contribute

#	Proposal	What you get	Effort
1	Native Panoptes adapter in your repo	panoptes.adapter.json + small panoptes_adapter.py at the root implementing our adapter contract (transform(text) -> str) — Layer A deterministic, opt-in Layer B. Anyone running python -m bench evaluate-repo gets a signed before/after card (Unicode present-rate, KGW detection rate, mean z) with no injected adapter.	Small diff, no CI cost
2	Cross-detector verification tier	Optional second-opinion pass in the bench: each cleaned variant is also scored by Panoptes' independent stack (calibrated detector + our own KGW/Unicode checks), so the report shows cleared under same-config detection vs. cleared under an independent detector. In our testing those diverge, especially across model families.	Medium — co-design
3	Multi-family watermarked corpus	Fixture packs (watermarked samples + expected pre/post detection outcomes per scheme) spanning watermark families × model families × harnesses, consumable by bench_synthid_text.py as additional --scheme tiers beyond SynthID.	Medium — co-design

Questions so I PR the right thing

• Which techniques is your benchmark targeting first — MarkLLM schemes only, or Unicode/steganographic carriers as first-class tiers too?
• Adapter location preference: in your repo (option 1) vs. kept on our side?
• CI conventions: Layer B needs a rewrite model, so I'd gate anything network/model-dependent behind an opt-in flag like your existing harnesses — does that match?

Happy to start with whichever slice you want — option 1 is a day; options 2–3 we'd co-design with you so the fixtures match your report format.

[guillaumemeyer]

100% @Encryptic1 !
Feel free to create a PRs for these 3 great suggestions.
@bentalay please have a look at this.