Skip to content

a11y(desktop/pdf): struct-tree text layer + annotation tool shortcuts (#316)#342

Merged
physercoe merged 2 commits into
physercoe:mainfrom
agentfleets:desktop/316-a11y-tail
Jul 20, 2026
Merged

a11y(desktop/pdf): struct-tree text layer + annotation tool shortcuts (#316)#342
physercoe merged 2 commits into
physercoe:mainfrom
agentfleets:desktop/316-a11y-tail

Conversation

@agentfleets

Copy link
Copy Markdown
Collaborator

Closes #316

The last two open items from the issue's remaining tail: expose PDF prose to screen readers, and single-key annotation-tool shortcuts.

What

  • Struct-tree accessible text layer (PdfCanvas.tsx): pdfjs-dist 4.10's API-layer TextLayer has no struct-tree wiring (upstream that lives in the viewer's StructTreeLayerBuilder, which we don't bundle). This replicates its minimal core:
    • Text is now streamed with includeMarkedContent: true, so each struct element's run is wrapped in an id'd span.markedContent (zero-size anchors; layout, selection, copy, and the search-highlight <mark>s are unchanged — textLayer.textDivs still yields only the text spans).
    • page.getStructTree() is walked into a hidden span tree (faithful port of upstream's #walk: PDF role → ARIA role map, H<n>heading/aria-level, altaria-label, lang, single-content-child collapse, aria-owns onto the marked-content ids) and appended inside the <canvas> as fallback content — never painted, but exposed to assistive tech in reading order.
    • The page wrapper changes role="img"role="region" with a localized "Page {n}" label (like the upstream viewer): img would mask all descendants as presentational, hiding the prose.
    • Untagged PDFs resolve a null tree → nothing appended (same as upstream). The split-view mirror pane skips its tree to avoid duplicate-id prose exposure.
  • Tool shortcuts: window-level keydown (V/Escape = select, H highlight, U underline, N note, A area, I ink), inert in inputs/textareas/contenteditable and with modifiers held, so the find box, page input, comment textarea, and Ctrl/Cmd zoom/undo keep their keys. Each tool button's tooltip now shows its key, localized via a new read.annToolTitle format key (en (H), zh full-width (H)).

Why

  • Screen readers previously got only "Page N, image" — no prose. Tagged PDFs now read structurally (headings, lists, tables, figures with alt text).
  • Keyboard users couldn't switch annotation tools without the mouse.

Trade-offs / notes

  • This is the pragmatic-equivalent path the issue allows: the struct-tree layer is hand-wired against pdfjs-dist 4.10's getStructTree()/includeMarkedContent contract rather than importing the upstream viewer bundle (which would drag ~430 kB of pdf_viewer into the chunk for one class). Upstream's #addImageInTextLayer (positioned role=img spans for figures inside the text layer) is not ported — figure alt text is still exposed via aria-label on the struct node.
  • Untagged PDFs remain prose-inaccessible to AT (as in the upstream viewer); the prior role="img" page label is superseded by the role="region" label.

How verified

  • npm run build (sync:tokens + tsc --noEmit + vite build) — green.
  • bash ../scripts/lint-desktop-tokens.sh — clean (no new off-token/phantom tokens; CSS additions are color-free).
  • git diff origin/main --stat — only the 3 intended files.
  • Walk logic smoke-tested against a stub DOM (heading levels, role map, single-child collapse, null-tree); the marked-content id ↔ struct-tree aria-owns contract verified against the bundled pdfjs-dist 4.10.38 sources (worker emits ${pageObjId}_mc${mcid} ids on both sides). No Rust changes.

@physercoe physercoe left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

APPROVE. Struct-tree/markedContent positioning faithfully mirrors pdf.js, overlay-bleed guard preserved, keydown listener uses the ref-latch pattern (no stale closure), and it fixes a pre-existing hardcoded aria-label. Rebased onto main and resolved the i18n + drilled-t→local-useT() merge (aligned PageView with main's new local-useT idiom). Minor to fold in later: includeMarkedContent: !readOnly so the split-view mirror pane does not emit duplicate DOM ids.

@physercoe
physercoe merged commit 6c7e440 into physercoe:main Jul 20, 2026
6 checks passed
physercoe pushed a commit that referenced this pull request Jul 20, 2026
…ic-tail merges

Since v0.3.79:
- fix(terminal): tell xterm the pty is ConPTY on Windows so a repainting TUI's
  intermediate frames no longer pile up in the scrollback (2b26a16).
- 8 epic-tail PRs merged: #341 terminal connect-phase + SSH split-duplicate,
  #342 PDF struct-tree a11y + annotation shortcuts, #343 token burn-down /
  plural / responsive, #344 PDF fit-page / rotation / hand-pan / ink,
  #345 perf (page memo, debounce, chart cap, sync dirty-tracking),
  #346 modal migration + dirty-close guards, #347 vault TOTP / ed25519 /
  coded errors, #348 read reconcile / AuthorNav / table nav / frame check.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(desktop): accessibility pass — PDF viewer, tab semantics, aria-sort, live regions, keyboard-operable resize

2 participants