feat: harden tart-gui-automation + context attachments UX - #19
Merged
Conversation
…ixes Add batch command for multi-step VNC operations in a single connection, eliminating the Twisted ReactorNotRestartable limitation. Add missing VNC operations (double-click, scroll, drag), screenshot resize via Pillow, coordinate scaling, and landmark resolution from .tart/target.yaml. Includes target manifest system for project-specific UI coordinates, flow recipes for agent-readable workflows, and comprehensive reference docs (keyboard mappings, image matrix, troubleshooting). Bug fixes: pass button param to vnc_click, validate --y with --x, clear errors for missing landmarks/fields in batch steps, cache YAML per batch instead of per step. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Scripts have shebangs and are executable, so docs now show direct invocation (e.g., `scripts/tart_vm_harness.py start`) instead of `uv run --script scripts/tart_vm_harness.py start`. Easier to allowlist and cleaner to read. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Fix sidebar_first_repo/sidebar_first_ws y-coordinates in target.yaml: old values (68, 92) were above the toolbar (y=125), clearly wrong. Updated to plausible estimates (250, 280) and marked uncalibrated. - Add button value to double-click log message (consistent with click). - Note send_keys vs send-keys naming difference in flow recipe schema. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
vncdotool uses short lowercase names (esc, bsp, del, pgup, pgdn) but the harness was lowercasing user input without mapping to these names, causing keys like "escape" to fail with a TypeError. Add KEYSYM_ALIASES mapping and normalize_key() for both standalone and batch send-keys. Recalibrate menu bar landmarks — "WorkspaceManager" is long enough to push File to x=400 and Edit to x=510 (previously 165 and 210). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
fairchild
added a commit
that referenced
this pull request
Mar 2, 2026
* feat: harden tart-gui-automation with batch ops, landmarks, and bug fixes Add batch command for multi-step VNC operations in a single connection, eliminating the Twisted ReactorNotRestartable limitation. Add missing VNC operations (double-click, scroll, drag), screenshot resize via Pillow, coordinate scaling, and landmark resolution from .tart/target.yaml. Includes target manifest system for project-specific UI coordinates, flow recipes for agent-readable workflows, and comprehensive reference docs (keyboard mappings, image matrix, troubleshooting). Bug fixes: pass button param to vnc_click, validate --y with --x, clear errors for missing landmarks/fields in batch steps, cache YAML per batch instead of per step. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * chore: use direct execution instead of uv run --script in docs Scripts have shebangs and are executable, so docs now show direct invocation (e.g., `scripts/tart_vm_harness.py start`) instead of `uv run --script scripts/tart_vm_harness.py start`. Easier to allowlist and cleaner to read. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: consistency pass — sidebar coords, log message, schema note - Fix sidebar_first_repo/sidebar_first_ws y-coordinates in target.yaml: old values (68, 92) were above the toolbar (y=125), clearly wrong. Updated to plausible estimates (250, 280) and marked uncalibrated. - Add button value to double-click log message (consistent with click). - Note send_keys vs send-keys naming difference in flow recipe schema. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat: context attachments UX improvements * fix: keysym aliasing for send-keys + recalibrate menu bar landmarks vncdotool uses short lowercase names (esc, bsp, del, pgup, pgdn) but the harness was lowercasing user input without mapping to these names, causing keys like "escape" to fail with a TypeError. Add KEYSYM_ALIASES mapping and normalize_key() for both standalone and batch send-keys. Recalibrate menu bar landmarks — "WorkspaceManager" is long enough to push File to x=400 and Edit to x=510 (previously 165 and 210). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
4 tasks
fairchild
added a commit
that referenced
this pull request
Apr 26, 2026
Three small clarifications surfaced by a fresh-session readiness audit: - P0 #1 (workspace creation hang): add an explicit "Gate: live repro" note. In-vitro investigation has gone as far as it can — PR #372's WorkspaceCreationRaceTests rules out a basic deadlock, and code reading shows the MainActor-serialized save path can't deadlock without an external factor. A fresh autonomous session shouldn't pick this as a first reach; it needs a deliberate interactive repro at the keyboard. - P0 #2 (main-window + Ghostty maintainability): split the band entry into two parallel sub-tracks (2a structural, 2b narrow appearance hardening) so a fresh session can pick by time/risk appetite rather than guessing which file to start with. - ghostty-appearance-hardening_followup.md: refresh the stale "Roadmap Position" section. Old text referenced "the same daily-driver hardening band as PR #18/#19 outcomes" — a framing the ROADMAP no longer uses. New text matches its current placement as P0 #2 sub-track 2b. Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
4 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Hardens the tart-gui-automation skill with batch operations, target manifests for project-specific landmarks, and missing VNC operations (double-click, scroll, drag). Adds coordinate scaling for resolution independence and comprehensive documentation including a human-oriented README. Also includes context attachments UX improvements for ExternalEditorService integration.
Highlights
tart-gui-automation improvements:
Context attachments work:
Testing: All 139 Swift tests passing.