Skip to content

OSC 133 support#88

Open
nindanaoto wants to merge 7 commits intoconnectbot:mainfrom
nindanaoto:OSC133
Open

OSC 133 support#88
nindanaoto wants to merge 7 commits intoconnectbot:mainfrom
nindanaoto:OSC133

Conversation

@nindanaoto
Copy link
Contributor

Fix #28.
Add the ability to retrieve the output of the last completed command using OSC 133 (FinalTerm) shell integration semantic markers. This is exposed through:

  • A TalkBack custom accessibility action ("Read Last Output") that announces the output via announceForAccessibility
  • A public API (TerminalEmulator.getLastCommandOutput()) that ConnectBot uses to expose this in a menu item

How it works

The implementation scans backward through terminal lines (scrollback + visible) to find:

  1. The most recent COMMAND_FINISHED marker (OSC 133;D)
  2. The matching COMMAND_INPUT marker (OSC 133;B) with the same promptId
  3. All lines between them, which are the command output

Changes

AccessibilityOverlay.kt

  • Add getLastCommandOutput(lines) internal function that extracts the last command's output text from semantic segments
  • Add "Read Last Output" custom accessibility action that calls getLastCommandOutput() and announces the result via TalkBack

TerminalEmulator.kt

  • Add getLastCommandOutput(): String? to the public TerminalEmulator sealed interface
  • Implement in TerminalEmulatorImpl by reading the current snapshot's scrollback + visible lines and delegating to getLastCommandOutput(lines)

OscParser.kt

  • Move COMMAND_INPUT segment creation from C time (OSC 133;C) to B time (OSC 133;B). In real shell integration, the user presses Enter before C is sent, which scrolls the terminal. Since libvterm uses screen-relative cursor positions, the cursor row stays the same after scroll while the column resets to 0, causing the old column comparison to always fail. Creating the segment at B time ensures it is placed on the correct screen row and gets properly shifted by pushScrollbackLine along with the text content.

ReadLastOutputTest.kt (new)

  • 8 unit tests covering: basic output, no finished marker, no input marker, empty output, multiple commands, multi-line output, trailing
    whitespace, and scrollback scenarios

OscParserTest.kt

  • Update testOsc133PromptFlow for new behavior (B now emits both PROMPT and COMMAND_INPUT)
  • Add testOsc133PromptFlowCrossRow covering the realistic scenario where scroll keeps the cursor at the same screen-relative row

Unit Tests

  • ReadLastOutputTest — 8 tests covering getLastCommandOutput() logic
  • OscParserTest.testOsc133PromptFlow — same-row OSC 133 flow
  • OscParserTest.testOsc133PromptFlowCrossRow — cross-scroll OSC 133 flow (the real-world case)
  • Manual test on device with bash OSC 133 shell integration: run commands, verify TalkBack action and public API both return correct output

nindanaoto and others added 2 commits January 31, 2026 11:36
Adds a custom accessibility action that extracts the last command's output
using OSC 133 shell integration semantic segments and announces it via
TalkBack, enabling screen reader users to review command output.

Closes connectbot#28

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
…iming

Expose getLastCommandOutput() on the TerminalEmulator sealed interface so
ConnectBot can access it from the menu without reaching into internal types.
The implementation reads the current snapshot's scrollback + visible lines
and delegates to the existing getLastCommandOutput(lines) function.

Fix a bug in OscParser where COMMAND_INPUT segments were never created in
real shell integration scenarios. Previously, COMMAND_INPUT was created at
C time (command output start), but after the user presses Enter the screen
scrolls while the cursor stays at the same screen-relative row. This caused
the column comparison to fail (startCol > cursorCol) so no segment was
created. Move COMMAND_INPUT creation to B time (command input start) where
the segment is placed on the correct screen row and will be properly shifted
by pushScrollbackLine along with the text content.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
nindanaoto and others added 5 commits January 31, 2026 21:36
The View.announceForAccessibility() method was deprecated in API 36.
The replacement AccessibilityManager.announce() is only available on
API 36+, and our minSdk is 24, so suppress the warning for now.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Use a Compose live region Box instead of the deprecated
View.announceForAccessibility() to announce text to TalkBack.
A hidden Box with liveRegion semantics is conditionally shown
when there is text to announce, causing TalkBack to read the
updated contentDescription automatically.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Add the new getLastCommandOutput() method to the Metalava API
signature file so the compatibility check passes.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The test was checking all semanticSegments and expecting 1, but
COMMAND_INPUT marker added at OSC 133;B time caused count to be 2.
Use getSegmentsOfType(SemanticType.PROMPT) to match test intent.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
buildSnapshot() read currentLines without holding damageLock, so
when the handler-posted processPendingUpdates() ran on the main
thread concurrently with the test thread, the snapshot could miss
segments written by addSemanticSegment on the JNI callback thread.

Move currentLines and scrollbackSnapshot reads under damageLock in
buildSnapshot() to ensure cross-thread visibility.

Also add waitForIdleSync() in test helper to drain the main looper
before taking a snapshot, preventing the handler from consuming
pending state before the test's own processPendingUpdates() call.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

a11y: Add custom action for "Read Last Output"

1 participant