Linux Inspector: window-id capture accuracy and geometry alignment

[mahbd]

Summary

• Add optional window_id input to capture_screenshot and get_ui_tree so a specific Linux window can be captured/queried directly.
• Add _get_frame_geometry_for_window and crop composite screenshots to the matching AT-SPI frame bounds to remove the window-frame misalignment.
• Preserve existing behavior when window_id is not provided (fallback remains app-name based).

Why this change

• Composite screenshots were visually offset by ~40px due to title bar/frame geometry mismatch.
• Targeted window identification by app name is brittle when multiple windows have similar names or titles.
• This change makes screenshot/UI-tree capture more reliable and precise for automation flows.

Files changed

• Framework/Built_In_Automation/Desktop/Linux/BuiltInFunctions.py
• node_cli.py

Validation

• Verified this branch is clean and pushed after sync:
  • Branch: linux-inspector-installer
  • Head commit: e841933a
  • Rebases: origin/dev and origin/linux-inspector-installer

Reproduction (what to run)

# On repo root
git checkout linux-inspector-installer
git pull --rebase origin linux-inspector-installer
git log --oneline -n 3

git status --short --branch

# Push updates if needed

git push origin linux-inspector-installer

Test plan

• Verified composite capture for HampsonRussell window crops correctly to 1896x1043.
• Verified geoview window capture renders as 700x997.
• Run Linux Inspector end-to-end against the corresponding server PR.

[riz-hossain]

🔎 ZeuZ PR Review

Open the full report in ZeuZ: Review findings and apply suggestions

Overview	Value
Agents	✅ 4 completed
Suggestions	💡 4

Agent breakdown

→ General Review

Status: ✅ Completed
Suggestions: 1 suggestion

The PR is directionally solid, but there is a notable correctness gap around window_id handling that can make the DOM and screenshot refer to different windows.

→ Security Review

Status: ✅ Completed
Suggestions: 0 suggestions

No high-signal security issues found in the PR diff. The changes add Linux window enumeration and screenshot alignment logic, plus pinned dependencies, without introducing a clear injection, authz, or secret-handling flaw.

→ Performance Review

Status: ✅ Completed
Suggestions: 1 suggestion

I found one performance risk: the new Linux apps/window discovery path now shells out to xdotool repeatedly per PID/window, which can get expensive as the number of visible windows grows.

→ Testing Review

Status: ✅ Completed
Suggestions: 2 suggestions

The Linux inspector change adds new window-aware behavior and a new /linux/apps payload shape, but I found no corresponding tests covering those code paths or the regression scenarios they’re meant to fix.

Open ZeuZ to inspect full findings, continue an agent conversation, or apply safe patch suggestions.

[riz-hossain]

🔎 ZeuZ PR Review

Open the full report in ZeuZ: Review findings and apply suggestions

Overview	Value
Agents	✅ 4 completed
Suggestions	💡 4

Agent breakdown

→ General Review

Status: ✅ Completed
Suggestions: 2 suggestions

I found one correctness issue in the new window-selection fallback, plus one maintainability issue in the API model definition.

→ Security Review

Status: ✅ Completed
Suggestions: 0 suggestions

No high-signal security issues found in this PR diff. The changes add window selection and geometry alignment without introducing obvious injection, authz, or secret-handling regressions.

→ Performance Review

Status: ✅ Completed
Suggestions: 1 suggestion

The PR adds useful multi-window support, but it also introduces several extra xdotool round-trips on the hot paths for app lookup and /apps, which can get expensive as the number of windows grows.

→ Testing Review

Status: ✅ Completed
Suggestions: 1 suggestion

The PR adds important Linux window-selection and geometry-alignment logic, but there are no tests covering the new code paths in server/linux.py or Framework/Built_In_Automation/Desktop/Linux/BuiltInFunctions.py yet.

Open ZeuZ to inspect full findings, continue an agent conversation, or apply safe patch suggestions.

[riz-hossain]

🔎 ZeuZ PR Review

Open the full report in ZeuZ: Review findings and apply suggestions

Overview	Value
Agents	✅ 4 completed
Suggestions	💡 3

Agent breakdown

→ General Review

Status: ✅ Completed
Suggestions: 1 suggestion

I found one high-impact issue in the new screenshot alignment logic: the crop rectangle is computed from AT-SPI desktop coordinates but compared against the captured window's local geometry, so the alignment path will often never apply. I also noted a smaller maintainability issue around a mutable model default.

→ Security Review

Status: ✅ Completed
Suggestions: 0 suggestions

No security-relevant issues found in the PR diff; the changes are focused on window selection and screenshot alignment without introducing obvious injection, auth, or data-exposure risks.

→ Performance Review

Status: ✅ Completed
Suggestions: 1 suggestion

One performance concern: /linux/apps now does a per-process window enumeration that can balloon into many xdotool subprocesses, and the screenshot lookup path also repeats window queries per candidate. No correctness issues stood out beyond that.

→ Testing Review

Status: ✅ Completed
Suggestions: 1 suggestion

The PR adds new window-selection and frame-alignment behavior, but I don’t see any tests covering the new Linux inspector paths or the regression scenario they fix.

Open ZeuZ to inspect full findings, continue an agent conversation, or apply safe patch suggestions.

[riz-hossain]

🔎 ZeuZ PR Review

Open the full report in ZeuZ: Review findings and apply suggestions

Overview	Value
Agents	✅ 4 completed
Suggestions	💡 5

Agent breakdown

→ General Review

Status: ✅ Completed
Suggestions: 2 suggestions

I found one behavior regression in the web click path; the rest of the diff looks broadly reasonable from a review pass.

→ Security Review

Status: ✅ Completed
Suggestions: 0 suggestions

No high-confidence security issues were introduced by this PR diff. The changes mainly add window scoping and geometry alignment for Linux inspection, without exposing a clear injection, authz, or secret-handling regression.

→ Performance Review

Status: ✅ Completed
Suggestions: 0 suggestions

No high-signal performance regressions stand out in this diff. The added window targeting and geometry alignment look correctness-driven, and the new executor usage in node_cli.py is a net improvement for event-loop responsiveness.

→ Testing Review

Status: ✅ Completed
Suggestions: 3 suggestions

The PR adds several Linux-inspector code paths but I couldn’t find corresponding tests in the repo, so the main gap is missing coverage for the new window-aware inspection and app/window discovery behavior.

Open ZeuZ to inspect full findings, continue an agent conversation, or apply safe patch suggestions.