feat(annotate): hit-test pinpoint targeting for raw-HTML sessions - #1251
Merged
Conversation
…list Pinpoint hover in raw-HTML sessions now resolves the real element under the cursor via deep elementFromPoint (piercing open shadow roots) instead of the SEMANTIC_* tag-whitelist graph, so styled div/span prototypes (chips, icon buttons, cards) are individually targetable. Scope control stays mouse-only and geometric, matching the markdown surface: the deepest element under the pointer wins; pointing at a container's padding or any area not covered by a child selects the container. - Exclusions are identity-based: html/body, script/style, and an identity Set of viewer overlay nodes (pin badges, pinpoint box/label, vim UI) so page markup cannot spoof its way out of targeting. Annotation marks are transparent (resolve to their parent). - Tiny-element promotion: a leaf under 16px on both axes climbs to the nearest ancestor at least 16px on one axis (MIN_CAPTURE_SIZE floor, not a whitelist). SVG shape primitives still promote to their <g>. - Hover no longer builds the semantic target graph per pointer frame (graphForPointerFrame removed); it is per-event hit-testing with a 2px/16ms last-position cache that the scroll reconcile invalidates. The graph survives untouched as the vim-navigation vocabulary. - Hover labels for generic containers use a cascade: aria-label -> role -> 1-2 meaningful class tokens (hash-ish tokens stripped) -> own text when <= 40 chars -> 'container'. Known semantic tags keep their names. - Click annotates exactly what the hover box shows; text-less elements post '[element: <label>]' instead of being untargetable. - Anchors stay fail-closed: data-test-id/data-cy/data-qa join the author-controlled identity attrs, and text-less elements store a '[[pn-shape]]' signature (tag + sorted classes + child count + 16px size buckets) verified whole-string on restore.
New DOM regression tests modeled on the signoff prototype pages: chip/button targeting on styled div/span markup, container selection via uncovered-area pointing, tiny-element promotion (mocked 8x8 dot promotes to its 200x100 card), the generic-container label cascade, the text-less shape-signature anchor round trip (verify + fail-closed rejection on shape drift), data-cy identity-attr trust, and an instrumented assertion that hover hit-testing performs zero document-wide querySelectorAll calls across 30 mousemoves. Also reattaches the cached hover label element when the body was replaced (parity with the hover box), and accepts the shape-prefixed snapshot through the parent-side DTO test.
… weak selectors Review findings on the pinpoint hit-testing branch: - D1 (blocker): structure-derived shape signatures are identical across identical siblings by construction, so a positional selector whose sibling was removed resolved the WRONG element while the signature still verified — the exact wrong-binding failure the anchor system exists to prevent. The mechanism is removed entirely: text-less elements now anchor ONLY when the element itself carries a stable-identity rung (#id or the data-* identity attrs); otherwise no anchor ships and the pin simply does not restore (same as before the branch, fail closed). A wrong-binding anchor is worse than no anchor. D2-D4 (in-band prefix discriminator, uncapped class-list component, viewport-dependent size bucket) die with it. - D5: the pinpoint click handler gated pin-badge ownership on a [data-plannotator-pin-badge] selector match, contradicting the identity-only overlay rule. It now gates on the overlay identity set: a page element spoofing the attribute hovers and annotates like any other element, while real badges keep owning their clicks. - D6: class-token hover labels now route through the 40-char label cap (two long tokens could reach 73 chars), and the '[element: ...]' posted text goes through capSelectionText for consistency. - D7: the pointer-hit cache invalidation moved to the top of the rAF reconcile, ahead of its guards, so every reconcile pass (scroll, resize, and body ResizeObserver — i.e. layout-changing mutations) clears it; the 2px/16ms TTL bounds any remaining staleness to one frame. Tests updated to the new contract, with regressions for D1 (identical text-less siblings ship no anchor; crafted positional anchors with empty snapshots resolve nothing after sibling removal), D5 (spoofed badge is annotatable, real badge posts mark-click), and the D6 label cap.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
TLDR: Pinpoint mode in raw-HTML sessions now targets whatever is actually under the cursor instead of a hardcoded tag whitelist. Chips, icon buttons, and styled spans on div-based prototype pages become individually annotatable; containers select by pointing at their padding; everything is mouse-only. Twice adversarially reviewed; final verdict merge-safe.
AI-assisted (research, implementation, and two adversarial review rounds).
The old semantic whitelist (headings/paragraphs/tables/sections) made real prototype pages built from divs and spans resolve every hover to the enclosing section. After studying agentation and react-grab, hover now uses deep hit-testing (elementFromPoint through open shadow roots) with an identity-based exclusion set for viewer overlays, a 16px tiny-element promotion floor, and smart container labels (aria-label, then role, then meaningful class tokens). The interaction model matches markdown pinpoint: the deepest element under the pointer wins, and parents are selected via their uncovered area. No keyboard involvement anywhere.
The semantic graph survives untouched as the vim vocabulary and click-time parent resolver. The hover path no longer rebuilds it per frame: zero document-wide queries per mousemove (instrumented in tests), retiring the perf debt deferred from #1243.
Anchors keep every fail-closed property. data-test-id/data-cy/data-qa join the trusted identity attributes. Text-less elements are annotatable in-session and their pins restore only via stable identity; a shape-signature mechanism from the first iteration was removed after review proved it could rebind pins to identical siblings, and no anchor beats a wrong one.
Verified: 942 DOM tests pass (+10 net new covering chip targeting, container selection, tiny promotion, label caps, fail-closed identical-sibling anchors, badge-spoof clicks), typecheck clean, builds clean, and both real-world signoff pages validated end-to-end through live-served annotate sessions.