Repository navigation
docmutant: shareable reproducers and exact-ground-truth test files from unshareable PDFs #845
wittjeff
started this conversation in
Show and tell
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Hi opendataloader team,
We have started an open-source tool, docmutant (MIT), that may be useful for opendataloader-pdf and opendataloader-bench, and we would like to invite you to use it and contribute.
What it does. docmutant changes a PDF at the level of its structure and checks every change with an oracle: any command with
{pdf}and{page}placeholders that says whether a problem still shows. The opendataloader CLI works as a target as it is. The reduce mode turns a document that shows a problem into a small file that still shows it but keeps none of the original content: one page, only the operators the problem needs, new image samples, text swapped for codes of equal width, no font programs. Each output file comes with a manifest that records the source hash, every change, the tool version and seed, the oracle results, what remains from the source, and the file's license.First run. A scanned manual page that crashes the Blend2D renderer in docling-parse reduces, in 13 oracle runs, to a 2 KB file: one sheared image placement with flat grey samples. It still crashes the buggy build and renders with the fix, and its manifest records that no words, images or font programs of the source remain.
Why it may matter to you.
/ActualTextand form fields change together with the visible text. That fits opendataloader's accessibility work.Invitation. It is early: a manifest, a validator and a first reduce mode. We would welcome you trying it on a problem document, telling us what you would need from it, and contributions: oracles for your Java pipeline, mutators, or test documents.
Design notes: docs/DESIGN.md
All reactions