Skip to content

Repository files navigation

Accessibilizer

Accessibilizer turns visually readable Source PDFs into documents whose Visual Layer is preserved and whose Semantic Layer can be consumed by assistive technology.

The current implementation converts a whole Source PDF one page at a time, reconstructs each page's Semantic Layer from the Source PDF itself, imports every native source page as artifact content into one PDF/UA-1 document, then gates the result on internal semantic checks, full-page visual comparison across every page, and veraPDF's PDF/UA-1 profile. It resolves and verifies an exact OpenAI-compatible vision provider before Source PDF work begins. By default the whole document is converted; pass --page to convert a subset.

The output carries a document title and language, a heading hierarchy (H1–H6), a bookmark outline built from that hierarchy, the ordered Semantic Layer in document Logical Reading Order, and PDF/UA representations for reconstructable links. It additionally produces reproducible, source-linked recognition evidence for each page: a pinned CPU-only PaddleOCR pass detects text or handwriting, Formula, table, figure, and Document Structure candidates, and the existing PDF text layer is retained with its geometry only as non-authoritative evidence. Every candidate receives a stable identifier and a source-region crop.

A single strict, versioned page-level vision call then reconstructs the page's meaning — its title, language, and the ordered heading, paragraph, Formula, Informative Figure, and Semantic Table Semantic Layer in Logical Reading Order — and required high-resolution crop calls verify the Formula, table, and Figure regions. The PDF text layer and the PaddleOCR candidates are reconciled against that reconstruction without silently replacing it: disagreement, ambiguity, unsupported static inputs, suspected source errors, and suspected prompt injection each raise a non-bypassable Conversion Warning. Source document content is treated as untrusted data — requests never expose tools, the reply is constrained to a strict JSON Schema, and no field of the source is interpreted as a control instruction.

Each Formula keeps a faithful normalized representation and a concise Spoken Math Alternative; its fractions, superscripts, subscripts, symbols, and units survive the complete conversion and PDF/UA authoring path because the exact mathematical text is carried on the tagged structure's ActualText and Alt independently of the authoring font. The high-resolution Formula reconstruction is reconciled against the specialized recognition of the same region, and a formula-recognition-disagreement warning is raised when they diverge or a formula-spoken-fidelity warning when the Spoken Math Alternative reads as raw markup rather than spoken English. Apparent source mistakes are preserved rather than corrected, each with its own warning, so Source Fidelity is never quietly overridden.

Two macOS Preview and VoiceOver validation sessions (second session) found clipped text and missing Figure Alternative and Detailed Figure Description content even though the Visual Layer and automated PDF/UA checks passed, because Preview derived accessibility text from the glyphs laid out inside the one-point-wide, zero-opacity overlay. ADR 0026 rejected that overlay. The Semantic Layer is now authored as full-size text drawn with text rendering mode 3, which produces no marks on screen or in the print path (ADR 0027), so the glyphs Preview reads spell the complete heading, paragraph, and Formula strings in Logical Reading Order; the Figure carries its Alternative in the image alternate text and its Detailed Figure Description on a sibling caption. A recorded macOS Preview and VoiceOver session confirmed VoiceOver reads all four nodes — heading, paragraph, Formula, and Figure with both its Alternative and Detailed Figure Description — in the intended Logical Reading Order, satisfying the acceptance gate for issue #3.

Build and convert

Build the canonical runtime:

docker build --tag accessibilizer:0.1.0 .

Run the public host launcher:

./accessibilizer convert \
  "testdata/Chapter 20_ Electric Current Resistance and Ohms Law.pdf" \
  --bundle electric-current.accessibilizer \
  --provider-base-url http://localhost:11434/v1 \
  --provider-model exact-model-identifier \
  --provider-data-location local \
  --json

The whole document is converted by default. Pass --page to convert a subset: a single page (--page 3), a range (--page 1-11), or a comma list (--page 1,3,5).

The launcher supports macOS and Linux paths and keeps Docker as an implementation detail. It refuses to overwrite an existing Conversion Bundle. Pass --replace to explicitly authorize replacement. Accessibilizer builds the replacement in a protected staging directory and leaves the existing bundle, including Reviewer edits, untouched if conversion fails.

Interrupted or paused work remains in a protected sibling directory named .BUNDLE.in-progress. Repeat the same command with --resume to continue it. Completed source, per-page region, recognition, and page-semantics stages, and the whole-document authoring and validation stages, are reused only when their dependency key and artifact hashes remain valid. Changing a Source PDF, tool version, schema, prompt, model, or relevant rendering setting invalidates the stages that depend on it; changing the provider model or a prompt or schema version re-runs page-semantics reconstruction and everything downstream.

Before conversion, Accessibilizer rejects encrypted or digitally signed Source PDFs and PDFs containing forms, JavaScript, embedded files or media, links, or other interactive actions that this version cannot preserve safely.

With --json, every outcome is machine-readable. Exit 0 reports accessible, exit 2 reports review_required when unresolved Conversion Warnings remain, and exit 1 reports operational_failure.

Progress and interruption

Conversion reports concise, stable progress to stderr while stdout is reserved for the single final result — so a --json run still emits exactly one machine-readable value on stdout. Each major stage (provider capability check, Source PDF preflight, per-page rendering and recognition, provider reconstruction and verification requests, Review Record assembly, PDF authoring, internal, visual, and veraPDF validation, and Conversion Bundle publication) prints a start and a completion line with elapsed time. Per-page lines name the current page and the selected page count; provider lines name the request's purpose, its number and estimated total, and — immediately before a document-bearing request is sent — the configured endpoint and model. Reused checkpoints are reported as reused, so a --resume run does not look as though stages were skipped by accident. A long recognition or provider operation prints a plain elapsed-time heartbeat line every ten seconds. There are no spinners, colors, or carriage-return rewriting. Pass --verbose for finer technical detail (model, endpoint, reported token usage, and retry specifics); no flag is required for ordinary progress.

Every run also appends durable, metadata-only events to conversion-events.jsonl in the Conversion Bundle (schema schemas/conversion-events-1.0.schema.json). The log survives --resume and is published with the finished bundle. It records timestamps, stage, state, elapsed time, page/request context, and safe operational metadata only — never credentials, authorization headers, images, Source PDF text, prompts, raw request or response bodies, model-produced Semantic Layer content, or hidden reasoning. Heartbeats are terminal feedback and are not persisted, and no telemetry ever leaves the machine.

Interrupting a conversion with Ctrl-C is treated as an intentional pause, not an error: Accessibilizer records the active stage, page, and provider request, preserves the in-progress Conversion Bundle and its checkpoints and events, prints the exact command needed to resume, and exits 130 without a traceback. Under --json, it emits one interrupted result on stdout while all progress and diagnostics stay on stderr.

Provider configuration and consent

Provider settings resolve in this order, from lowest to highest precedence:

  1. User configuration at ${XDG_CONFIG_HOME:-~/.config}/accessibilizer/config.toml
  2. Project configuration at ./accessibilizer.toml
  3. CLI flags

Both TOML files use the same shape:

[provider]
base_url = "https://api.openai.com/v1"
model = "gpt-5.6-sol"
api_key_env = "OPENAI_API_KEY"
data_location = "remote"

[conversion]
max_requests = 100
provider_max_retries = 3
provider_retry_base_seconds = 0.5
provider_retry_max_seconds = 8.0

base_url and model are required after the layers are merged. The corresponding flags are --provider-base-url, --provider-model, --provider-api-key-env, and --provider-data-location. The model must be an explicit, exact identifier; explicit latest aliases are rejected. URL credentials, queries, and fragments are rejected so the recorded endpoint cannot capture a query-string secret.

For the initial hosted OpenAI quality baseline, use gpt-5.6-sol, the official identifier for the flagship GPT-5.6 variant. Do not configure the shorter gpt-5.6 routing alias when exact model selection and reproducible Conversion Provenance are required. See OpenAI's GPT-5.6 model guidance.

api_key_env names an environment variable; it does not contain the secret. The host launcher forwards only that named variable into the canonical container. Credentials are never written to TOML or Conversion Provenance. Providers that do not require a key may omit it.

data_location is local or remote. When omitted, loopback URLs default to local and every other or uncertain endpoint defaults to remote. For the canonical Docker runtime, loopback provider requests are routed to the host through Docker's host.docker.internal gateway while provenance retains the configured loopback URL. A remote run prompts for per-run confirmation in an interactive terminal. Automation must pass --allow-remote; configuration and credentials alone never authorize transmission. There is no provider fallback.

After authorization, Accessibilizer sends a small base64 capability image to POST /chat/completions beneath the configured base URL and requires a strict JSON-Schema answer derived from its visible content. A provider that cannot use the image and satisfy the response schema fails before the Source PDF is copied, rendered, inspected, or converted. Conversion Provenance records the resolved base URL, exact model, and data location, but not credentials, environment-variable names, request dumps, or hidden reasoning.

Transient timeouts, connection failures, rate limits, and server errors receive bounded exponential-backoff retries. The pipeline estimates the provider requests it will make — the capability check when needed, plus one page-level reconstruction call and one call per verified Formula, table, and Figure crop — counting zero for any stage whose checkpoint can be reused, then enforces max_requests before every attempt, including retries. Use --max-requests, --provider-max-retries, --provider-retry-base-seconds, and --provider-retry-max-seconds to override the layered settings. Reaching the request ceiling pauses the Conversion Bundle instead of exceeding it; raise the ceiling and pass --resume to continue. Conversion Provenance retains estimated and actual request counts and any prompt, completion, and total token usage reported by the provider. Accessibilizer does not estimate dollar cost.

Conversion Bundle

The generated protected directory contains:

  • source.pdf: immutable copy of the Source PDF
  • output.pdf: whole-document PDF/UA-1 output
  • review-record.yaml: the human-editable, whole-document Review Record — canonical Source Regions and displayed-page dimensions, the flat Semantic Layer with stable node identities and explicit evidence references, distinct Recognition Candidates, Conversion Warnings with node/region references and resolution history, and reconstruction provenance. It validates against schemas/review-record-3.0.schema.json; derived crops live at regions/<source-region-id>.png and are not stored as record paths.
  • review-baseline.json: the last tool-committed snapshot of the Review Record, used to detect changed resolutions and preserve history; not meant for editing
  • review-report.html: WCAG 2.2 AA presentation of the Review Record — a JavaScript-required, split-screen Component navigator that steps through every Semantic Layer node in Logical Reading Order while keeping its full Source PDF page and coordinate-aligned Source Region boxes in view. Loads only its sibling relative assets review-report.css, review-report.js, and regions/; no network access at view time
  • review-report.css, review-report.js: the report's local, offline, dependency-free stylesheet and behavior, referenced only through relative paths
  • page-semantics/page-N.json: each page's reconstructed Semantic Layer and warnings the Review Record is built from
  • recognition/page-N.json: per-page non-authoritative recognition candidates and existing PDF text evidence
  • regions/page-N.png: stable rendered source context for review
  • regions/page-N-recognition.png: full-page render used for recognition
  • regions/page-N-rNNNN.png: stable per-candidate source-region crops
  • authoring.json: versioned Python-to-Java contract (schemas/authoring-2.0.schema.json)
  • provenance.json: source hash, converted pages, authoring versions, and resolved provider identity
  • request-usage.json: resumable request ceiling, count, estimate, and reported token totals
  • conversion-events.jsonl: durable, metadata-only progress event log (schemas/conversion-events-1.0.schema.json), preserved across --resume
  • checkpoints/*.json: atomic dependency keys and hashes for completed stages
  • validation/preflight.json: Source PDF preflight result
  • validation/internal.json: internal semantic-check categories (source-region coverage, alternatives, table relationships, reading order, recognition agreement, and Review Record consistency)
  • validation/visual.json: full-page pixel-difference result per page and tolerance
  • validation/verapdf.xml: independent PDF/UA-1 validation report

Review and finalize

When a conversion exits 2, its Conversion Bundle is Review-Required: one or more Conversion Warnings remain unresolved. A Reviewer resolves them by hand-editing the YAML Review Record and finalizing without repeating OCR or provider calls. None of these commands touch the network; the launcher runs them with container networking disabled.

Open review-record.yaml and, for each warning, set its resolution to exactly one of corrected, accepted, or not_applicable. A not_applicable resolution requires a reason. You may also correct the Semantic Layer text (heading, paragraph, Spoken Math Alternative, Figure Alternative, and Detailed Figure Description) directly; those edits are preserved.

# Check the record against the canonical schema and report finalizability.
./accessibilizer validate --bundle electric-current.accessibilizer --json

# Stamp the edited resolutions with your identifier, move any superseded
# resolution into history, and regenerate the Review Report.
./accessibilizer review --bundle electric-current.accessibilizer --reviewer jdoe

# Rebuild the Accessible PDF deterministically from the corrected record.
./accessibilizer finalize --bundle electric-current.accessibilizer --reviewer jdoe --json

Each resolution carries the Reviewer's non-secret identifier, taken from --reviewer or from a [review] table (reviewer = "jdoe") in the user or project configuration. finalize verifies the immutable Source PDF hash, then refuses (exit 2) while any warning is unresolved. Once every warning is resolved it re-authors the PDF, re-runs the internal semantic checks, the visual comparison, and veraPDF's PDF/UA-1 profile, and exits 0 with an Accessible PDF. Superseded resolutions are retained in each warning's history, and the original recognition candidates are never discarded.

Recognition evidence

For each converted page, Accessibilizer renders the source at a deterministic recognition resolution and runs a pinned, CPU-only PaddleOCR pass. It records text or handwriting, Formula, table, figure, and Document Structure candidates, each with a stable identifier (page-N-rNNNN), a bounding box, a confidence value used only as evidence, and a source-region crop under regions/. The existing Source PDF text layer is extracted with its geometry and stored as pdf_text_evidence marked "authoritative": false, so it can inform later reconciliation without ever contaminating the Semantic Layer. Recognition is a checkpointed stage: it is reused when its source, tool versions, resolution, backend, and weights are unchanged. The candidate contract is documented by schemas/recognition-1.0.schema.json.

PaddleOCR code and weights are pinned in the canonical image, so recognition runs offline with no runtime model downloads. Set ACCESSIBILIZER_RECOGNITION_BACKEND=fake to select a deterministic backend that fabricates one candidate per type without running OCR; this is intended for fast, credential-free tests, never for a real conversion.

Verify

make test
make typecheck

The acceptance test invokes the public launcher and therefore requires Docker. The fast suite selects the deterministic fake recognition backend. Set ACCESSIBILIZER_RUN_REAL_OCR=1 to additionally run the opt-in check that pinned PaddleOCR produces schema-valid candidates for all 11 sample pages offline.

The authoring boundary is documented by schemas/authoring-2.0.schema.json, and the recognition-evidence contract by schemas/recognition-1.0.schema.json.

Vision-only full-document prototype

Issue #73's isolated experiment is intentionally separate from the production convert command. Its highest-level Python entry point processes every page of the gold Source PDF with one logical full-page request per page:

from pathlib import Path

from accessibilizer.provider import ProviderConfig
from accessibilizer.vision_prototype import PrototypePricing, reconstruct_prototype_document

reconstruct_prototype_document(
    ProviderConfig(
        base_url="https://api.openai.com/v1",
        model="gpt-5.6-sol",
        api_key_env="OPENAI_API_KEY",
        data_location="remote",
    ),
    source_pdf=Path(
        "testdata/Chapter 20_ Electric Current Resistance and Ohms Law.pdf"
    ),
    artifacts_root=Path("prototype-runs"),
    pricing=PrototypePricing(
        as_of="2026-07-27",  # Example only: supply the dated baseline rates used.
        input_per_million_tokens=1.0,
        output_per_million_tokens=10.0,
    ),
)

Each invocation creates a fresh run-<UUID>/ directory and refuses to reuse an existing identity. manifest.json records the exact model and versioned prompt, schema, Source PDF identity, normalized pages, token usage, latency, and request counts. It also records every provider attempt's purpose, page, elapsed time, and reported usage, plus pass/fail checks for the 11-call target, 22-call ceiling, complete usage reporting, and $2 cost ceiling. Cost uses the explicit dated OpenAI pricing supplied to the prototype; the provider-neutral production CLI does not infer cost. The prompt/ directory stores the exact instructions and strict schema; each pages/page-N/ directory independently stores its rendered input, native PDF text and geometry, schema-valid response, and normalized result. These artifacts exclude API credentials, authorization headers, hidden reasoning, and provider HTTP traces. Pass a completed run directory to replay_prototype_document() to validate its recorded responses against the stored schema and reproduce every normalized page without provider access.

Issue #75's deterministic evaluator compares that replay with the approved gold Review Record. Source-derived semantics and Logical Reading Order must match exactly, while wording differences in Spoken Math Alternatives and Informative Figure descriptions are queued for a Reviewer decision:

from accessibilizer.prototype_evaluation import evaluate_prototype_fidelity
from accessibilizer.review import load_yaml
from accessibilizer.vision_prototype import replay_prototype_document

replayed = replay_prototype_document(Path("prototype-runs/run-UUID"))
gold = load_yaml(Path("testdata/gold-review-record.yaml").read_text())
result = evaluate_prototype_fidelity(
    replayed,
    gold,
    visual_review_dir=Path("prototype-runs/run-UUID/geometry-review"),
)

The result reports semantic fidelity failures separately from Conversion Warning recall and precision failures and reviewer-localization geometry failures. Geometry accepts coarse and shared Source Regions without IoU or proposal-area matching, but requires finite in-page boxes, deterministic identity, coverage of the corresponding gold content's center, and justification for regions covering at least 80% of a page. The optional review directory remains empty on a geometry pass; each failed check creates one focused SVG comparing the produced and gold locations. Warning identity, wording, and geometry do not affect warning comparison; the page/code pairs must match the approved gold Review Record exactly.

About

make scanned PDFs readable by a screen reader

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages