Skip to content

Fix Windows capsule native container/input contract#499

Closed
Cooper-X-Oak wants to merge 1 commit into
Open-Less:betafrom
Cooper-X-Oak:codex/windows-capsule-contract
Closed

Fix Windows capsule native container/input contract#499
Cooper-X-Oak wants to merge 1 commit into
Open-Less:betafrom
Cooper-X-Oak:codex/windows-capsule-contract

Conversation

@Cooper-X-Oak
Copy link
Copy Markdown
Contributor

@Cooper-X-Oak Cooper-X-Oak commented May 19, 2026

Fixes #498.

Windows capsule contract PR.

What changed:

  • Windows capsule DOM pill owns the visible surface.
  • Windows native HWND is a transparent carrier and input envelope only.
  • Removed Acrylic/Mica/DWM material from the capsule host to avoid gray carrier margins.
  • Added rounded native SetWindowRgn envelope for the pill and translation badge.
  • Transparent host margins pass through; visible pill and badge consume input.
  • macOS/Linux behavior and macOS vibrancy remain untouched.

Evidence screenshots:

Recording capsule:
Recording capsule host crop

Translation badge capsule:
Translation capsule host crop

Accepted reference comparison:
Accepted/reference/current comparison

Evidence JSON:

Recording HITL summary:

  • DPI 120, scale 1.25x
  • Expected logical size 220x84, observed physical size 275x105
  • Transparent top-left, left-middle, bottom-right all pass through
  • Pill center hits OpenLess Capsule
  • Screen pill captured
  • allPassed true

Translation HITL summary:

  • DPI 120, scale 1.25x
  • Expected logical size 220x118, observed physical size 275x148
  • Transparent top-left, left-middle, bottom-right all pass through
  • Pill center hits OpenLess Capsule
  • Badge center hits OpenLess Capsule
  • Screen pill and badge captured
  • allPassed true

Validation passed:

  • node ./src/lib/capsuleLayout.test.ts
  • node ./scripts/windows-capsule-contract.test.mjs
  • PowerShell parse check for openless-all/app/scripts/windows-capsule-hitl.ps1
  • npm run build
  • cargo check --manifest-path ./openless-all/app/src-tauri/Cargo.toml --bin openless
  • cargo build --manifest-path ./openless-all/app/src-tauri/Cargo.toml --bin openless
  • cargo test --manifest-path ./openless-all/app/src-tauri/Cargo.toml capsule_container_region --no-run
  • HITL recording pass with ExpectedWidth 220 ExpectedHeight 84
  • HITL translation pass with ExpectedWidth 220 ExpectedHeight 118 TranslationActive

Known existing red outside this PR:

@github-actions
Copy link
Copy Markdown

PR Reviewer Guide 🔍

Here are some key observations to aid the review process:

🎫 Ticket compliance analysis 🔶

498 - Partially compliant

Compliant requirements:

  • Windows capsule host no longer applies Acrylic/Mica/DWM material.
  • Static contract tests reject Acrylic/Mica/DWM and dynamic WebView child-HWND click-through styles.
  • Separate host, pill, and native input envelope metrics are introduced.
  • The translation badge is included in the Windows native region contract.
  • DPI-aware HITL runner logic is added with explicit physical-pixel conversion.
  • macOS-specific backdrop behavior is left unchanged in the code paths shown.

Non-compliant requirements:

  • The native input envelope is still defined as an exact rounded-rect union, so it can clip shadow/antialiasing bleed outside the pill and badge boxes.
  • The HITL runner does not robustly prove a full non-clipped visible capsule in all cases; it relies on center-pixel checks and a PrintWindow fallback.

Requires further human verification:

  • Windows Rust build output for the app binary.
  • End-to-end HITL verification on real Windows hardware across DPI and multi-monitor setups.
  • State-transition behavior during recording -> transcribing/polishing -> done/error -> idle.
⏱️ Estimated effort to review: 4 🔵🔵🔵🔵⚪
🧪 PR contains tests
🔒 No security concerns identified
⚡ Recommended focus areas for review

Clipped edges

The Windows region is built from the pill and badge rectangles with no extra bleed for shadow or antialiasing, so SetWindowRgn can clip the visible capsule edge on high-DPI displays even if hit-testing still works.

fn windows_capsule_pill_region_rect(bounds: CapsuleWindowBounds) -> WindowsCapsuleRegionRect {
    const PILL_WIDTH: f64 = 196.0;
    const PILL_HEIGHT: f64 = 52.0;
    const BOTTOM_INSET: f64 = 12.0;

    WindowsCapsuleRegionRect {
        x: (bounds.width - PILL_WIDTH) / 2.0,
        y: bounds.height - BOTTOM_INSET - PILL_HEIGHT,
        width: PILL_WIDTH,
        height: PILL_HEIGHT,
        radius: PILL_HEIGHT / 2.0,
    }
}

#[cfg(target_os = "windows")]
fn windows_capsule_translation_badge_region_rect(
    bounds: CapsuleWindowBounds,
) -> WindowsCapsuleRegionRect {
    const BADGE_WIDTH: f64 = 132.0;
    const BADGE_HEIGHT: f64 = 24.0;
    const BADGE_GAP: f64 = 8.0;

    let pill = windows_capsule_pill_region_rect(bounds);
    WindowsCapsuleRegionRect {
        x: (bounds.width - BADGE_WIDTH) / 2.0,
        y: (pill.y - BADGE_GAP - BADGE_HEIGHT).max(0.0),
        width: BADGE_WIDTH,
        height: BADGE_HEIGHT,
        radius: BADGE_HEIGHT / 2.0,
    }
}

#[cfg(target_os = "windows")]
fn windows_capsule_region_rects(
    bounds: CapsuleWindowBounds,
    translation_active: bool,
) -> Vec<WindowsCapsuleRegionRect> {
    let mut rects = Vec::with_capacity(if translation_active { 2 } else { 1 });
    if translation_active {
        rects.push(windows_capsule_translation_badge_region_rect(bounds));
    }
    rects.push(windows_capsule_pill_region_rect(bounds));
    rects
}

#[cfg(all(target_os = "windows", test))]
fn windows_capsule_rounded_rect_contains_point(
    rect: WindowsCapsuleRegionRect,
    x: f64,
    y: f64,
) -> bool {
    if x < rect.x || x > rect.x + rect.width || y < rect.y || y > rect.y + rect.height {
        return false;
    }

    let radius = rect.radius.min(rect.width / 2.0).min(rect.height / 2.0);
    let nearest_x = if x < rect.x + radius {
        rect.x + radius
    } else if x > rect.x + rect.width - radius {
        rect.x + rect.width - radius
    } else {
        x
    };
    let nearest_y = if y < rect.y + radius {
        rect.y + radius
    } else if y > rect.y + rect.height - radius {
        rect.y + rect.height - radius
    } else {
        y
    };
    let dx = x - nearest_x;
    let dy = y - nearest_y;
    dx * dx + dy * dy <= radius * radius
}

#[cfg(all(target_os = "windows", test))]
fn windows_capsule_region_contains_point(
    bounds: CapsuleWindowBounds,
    translation_active: bool,
    x: f64,
    y: f64,
) -> bool {
    windows_capsule_region_rects(bounds, translation_active)
        .into_iter()
        .any(|rect| windows_capsule_rounded_rect_contains_point(rect, x, y))
}

#[cfg(target_os = "windows")]
fn windows_capsule_region_px(value: f64, scale: f64, outward: bool) -> i32 {
    let scaled = value * scale;
    if outward {
        scaled.ceil() as i32
    } else {
        scaled.floor() as i32
    }
}

#[cfg(target_os = "windows")]
fn windows_capsule_create_region(
    bounds: CapsuleWindowBounds,
    translation_active: bool,
    scale_x: f64,
    scale_y: f64,
) -> windows::Win32::Graphics::Gdi::HRGN {
    use windows::Win32::Graphics::Gdi::{
        CombineRgn, CreateRoundRectRgn, DeleteObject, HRGN, RGN_ERROR, RGN_OR,
    };

    let mut combined = HRGN::default();
    for rect in windows_capsule_region_rects(bounds, translation_active) {
        let x1 = windows_capsule_region_px(rect.x, scale_x, false);
        let y1 = windows_capsule_region_px(rect.y, scale_y, false);
        let x2 = windows_capsule_region_px(rect.x + rect.width, scale_x, true);
        let y2 = windows_capsule_region_px(rect.y + rect.height, scale_y, true);
        let round_w = ((rect.radius * 2.0 * scale_x).round() as i32).max(1);
        let round_h = ((rect.radius * 2.0 * scale_y).round() as i32).max(1);
        let region = unsafe { CreateRoundRectRgn(x1, y1, x2, y2, round_w, round_h) };
Single monitor

The HITL runner hard-codes PrimaryScreen for the backdrop and screen capture. If the capsule opens on a different monitor, the green backdrop and pixel sampling will miss the actual window and produce false failures or misleading evidence.

$form = [Windows.Forms.Form]::new()
$form.Text = "OpenLess Hit Test Backdrop"
$form.FormBorderStyle = [Windows.Forms.FormBorderStyle]::None
$form.StartPosition = [Windows.Forms.FormStartPosition]::Manual
$form.Bounds = [Windows.Forms.Screen]::PrimaryScreen.Bounds
$form.BackColor = [Drawing.Color]::FromArgb(37, 200, 81)
$form.ShowInTaskbar = $false
$form.TopMost = $false
$form.Show()
$form.Activate()
[Windows.Forms.Application]::DoEvents()
Start-Sleep -Milliseconds 400

$triggerAttempts = 1
$lastTriggerAt = Get-Date
Invoke-ToggleDictation $ExePath

$capsuleRow = $null
$deadline = (Get-Date).AddSeconds($WaitSeconds)
while ((Get-Date) -lt $deadline) {
  Start-Sleep -Milliseconds 150
  [Windows.Forms.Application]::DoEvents()
  Hide-OpenLessMainWindow
  $capsuleRow = Get-VisibleWindows | Where-Object {
    $_.title -eq "OpenLess Capsule" -and $_.className -eq "Tauri Window"
  } | Select-Object -First 1
  if ($null -ne $capsuleRow) { break }
  if ($triggerAttempts -lt 3 -and ((Get-Date) - $lastTriggerAt).TotalSeconds -ge 3) {
    $triggerAttempts += 1
    $lastTriggerAt = Get-Date
    Invoke-ToggleDictation $ExePath
  }
}

if ($null -eq $capsuleRow) {
  throw "OpenLess Capsule window was not visible within $WaitSeconds seconds."
}

$capsule = [IntPtr]$capsuleRow.hwnd
$rect = [OpenLessCapsuleHitlNative+RECT]::new()
[void][OpenLessCapsuleHitlNative]::GetWindowRect($capsule, [ref]$rect)
$width = $rect.Right - $rect.Left
$height = $rect.Bottom - $rect.Top
$dpi = [OpenLessCapsuleHitlNative]::GetDpiForWindow($capsule)
if ($dpi -le 0) { $dpi = 96 }
$dpiScale = [double]$dpi / 96.0
$expectedPhysicalWidth = Convert-LogicalToPhysicalDimension $ExpectedWidth $dpiScale
$expectedPhysicalHeight = Convert-LogicalToPhysicalDimension $ExpectedHeight $dpiScale

$hits = [ordered]@{
  transparentTopLeft = Hit-Point "transparentTopLeft" ($rect.Left + 5) ($rect.Top + 5) $capsule
  transparentLeftMiddle = Hit-Point "transparentLeftMiddle" ($rect.Left + 5) ($rect.Top + [int]($height / 2)) $capsule
  transparentBottomRight = Hit-Point "transparentBottomRight" ($rect.Right - 5) ($rect.Bottom - 5) $capsule
  pillCenter = Hit-Point "pillCenter" ($rect.Left + [int]($width / 2)) ($rect.Top + [int]($height / 2)) $capsule
}
if ($TranslationActive) {
  $badgeCenterY = $rect.Top + [int][Math]::Round(34 * $dpiScale)
  $hits.badgeCenter = Hit-Point "badgeCenter" ($rect.Left + [int]($width / 2)) $badgeCenterY $capsule
}

$screenBounds = [Windows.Forms.Screen]::PrimaryScreen.Bounds
$screenBmp = [Drawing.Bitmap]::new($screenBounds.Width, $screenBounds.Height)
$screenGraphics = [Drawing.Graphics]::FromImage($screenBmp)
$screenGraphics.CopyFromScreen($screenBounds.Location, [Drawing.Point]::Empty, $screenBounds.Size)
$fullPath = Join-Path $outDir "screen-green-bg.png"
$screenBmp.Save($fullPath, [Drawing.Imaging.ImageFormat]::Png)

@Cooper-X-Oak
Copy link
Copy Markdown
Contributor Author

Manual HITL gate request for PR #499.

Hot branch:

  • Cooper-X-Oak/openless:codex/windows-capsule-contract
  • Commit: 74e4631

What to manually verify on Windows:

  1. Recording capsule has no gray native host edge/rectangle behind it.
  2. Capsule shape remains a normal rounded pill, not clipped or deformed.
  3. Transparent host margins do not block clicks behind the capsule.
  4. Visible pill buttons remain clickable.
  5. Translation badge mode still shows the badge above the pill and remains clickable in its visible region.
  6. macOS behavior should be reviewed separately by maintainer; this PR intentionally does not touch macOS vibrancy/backdrop paths.

Automated evidence already attached in the PR body:

  • recording HITL: allPassed=true, 220x84 logical -> 275x105 physical at 125% DPI
  • translation HITL: allPassed=true, 220x118 logical -> 275x148 physical at 125% DPI
  • recording/translation screenshots and accepted-reference comparison are embedded above.

@Cooper-X-Oak
Copy link
Copy Markdown
Contributor Author

Withdrawing this PR per maintainer/user direction. We will not continue the capsule contract approach in this branch.

@Cooper-X-Oak Cooper-X-Oak deleted the codex/windows-capsule-contract branch May 19, 2026 05:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Windows capsule needs a stable native container/input contract

1 participant