Skip to content

Fix external monitor geometry conversion#206

Merged
Jam-Cai merged 1 commit into
mainfrom
fix/external-monitor-geometry
May 25, 2026
Merged

Fix external monitor geometry conversion#206
Jam-Cai merged 1 commit into
mainfrom
fix/external-monitor-geometry

Conversation

@Jam-Cai
Copy link
Copy Markdown
Collaborator

@Jam-Cai Jam-Cai commented May 25, 2026

Summary

  • convert Accessibility/CoreGraphics rectangles by matching the owning display before flipping into AppKit coordinates
  • preserve Retina text-range fallback behavior while scaling relative to the display origin instead of the main screen
  • choose overlay visible frames by caret midpoint before falling back to intersection order
  • add pure display-coordinate tests for above-primary, negative-X, and Retina-origin monitor layouts

Validation

  • xcodebuild -project tabby.xcodeproj -scheme tabby -destination 'platform=macOS' -only-testing:tabbyTests/DisplayCoordinateConverterTests test CODE_SIGNING_ALLOWED=NO
  • xcodebuild -project tabby.xcodeproj -scheme tabby -destination 'platform=macOS' build CODE_SIGNING_ALLOWED=NO
  • xcodebuild -project tabby.xcodeproj -scheme tabby -destination 'platform=macOS' build-for-testing CODE_SIGNING_ALLOWED=NO

Greptile Summary

This PR fixes AX/CG rectangle conversion for multi-monitor setups by replacing a single desktop-union Y-flip with per-display geometry. A new DisplayCoordinateConverter enum handles all coordinate math against injected DisplayGeometry snapshots, keeping it fully testable without NSScreen.

  • Adds DisplayCoordinateConverter and DisplayGeometry to tabby/Support/, correctly flipping CG rects within their owning display and handling Retina pixel-to-point scaling anchored at each display's own origin rather than the global origin.
  • Updates AXHelper.cocoaRect and validatedCocoaTextRect to use the new per-display converter with legacyDesktopUnionFlip as a last-resort fallback, and refactors OverlayController.targetScreenVisibleFrame to prefer caret-midpoint containment in visibleFrame before the frame-intersection fallback.
  • Adds DisplayCoordinateConverterTests with three pure-geometry tests covering above-primary, negative-X boundary-crossing, and Retina-origin display layouts.

Confidence Score: 5/5

Safe to merge; the per-display Y-flip math is correct for primary, side-by-side, above-primary, and Retina configurations, and the legacy fallback is preserved for virtual/mirrored displays without an NSScreenNumber.

The coordinate arithmetic has been verified against all three test scenarios and the fallback paths (legacyDesktopUnionFlip, displays.isEmpty early-return) correctly mirror the pre-existing behavior. No new crash paths, data loss, or security concerns were introduced.

No files require special attention.

Important Files Changed

Filename Overview
tabby/Support/DisplayCoordinateConverter.swift New pure-math type for CG→AppKit rect conversion; bestDisplay uses midpoint containment then intersection-area fallback; appKitRectsFromPixelRect correctly anchors pixel-to-point scaling at each display's own CG origin.
tabby/Support/AXHelper.swift Replaces desktop-union Y-flip in cocoaRect and validatedCocoaTextRect with per-display converter calls; legacyDesktopUnionFlip kept as fallback for displays without an NSScreenNumber; logic is clean and symmetric between both methods.
tabby/Services/UI/OverlayController.swift Adds midpoint-in-visibleFrame priority check before the existing frame-intersection fallback, correctly routing the overlay to the screen that owns the caret rather than any intersecting screen.
tabbyTests/DisplayCoordinateConverterTests.swift Three new deterministic geometry tests; expected values are arithmetically verified for above-primary, negative-X boundary crossing, and Retina pixel scaling on a right-side 2x display.
tabby.xcodeproj/project.pbxproj Registers DisplayCoordinateConverterTests.swift in the test target; mechanical Xcode project file change.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[AX/CG rect from Accessibility API] --> B{cocoaRect or validatedCocoaTextRect?}
    B -->|cocoaRect| C[displayGeometries]
    B -->|validatedCocoaTextRect| D[displayGeometries]
    C --> E{displays empty?}
    E -->|no| F[DisplayCoordinateConverter.appKitRect]
    E -->|yes| G[legacyDesktopUnionFlip]
    F -->|nil| G
    F -->|converted| H[Return AppKit rect]
    G --> H
    D --> I{displays empty?}
    I -->|yes| J[Return raw textRect no Y-flip]
    I -->|no| K[Candidate A: appKitRect ?? legacyDesktopUnionFlip]
    K --> L{anchor available?}
    L -->|no| M[Return Candidate A]
    L -->|yes| N{Candidate A midpoint inside expandedAnchor?}
    N -->|yes| M
    N -->|no| O[Candidate B: appKitRectsFromPixelRect]
    O --> P{Any Candidate B inside expandedAnchor?}
    P -->|yes| Q[Return first matching Candidate B]
    P -->|no| R[Return Candidate A best-effort]
Loading

Reviews (2): Last reviewed commit: "Fix external monitor geometry conversion" | Re-trigger Greptile

Comment thread tabby/Support/AXHelper.swift
Comment thread tabbyTests/DisplayCoordinateConverterTests.swift
@Jam-Cai Jam-Cai force-pushed the fix/external-monitor-geometry branch from 023588a to 2076102 Compare May 25, 2026 01:34
@Jam-Cai Jam-Cai merged commit 8ade904 into main May 25, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant