Skip to content

feat(go): composer + attachment UX overhaul#373

Merged
zomux merged 2 commits into
openagents-org:developfrom
baryhuang:fix/go-composer-bubble-ime-paste
May 12, 2026
Merged

feat(go): composer + attachment UX overhaul#373
zomux merged 2 commits into
openagents-org:developfrom
baryhuang:fix/go-composer-bubble-ime-paste

Conversation

@baryhuang
Copy link
Copy Markdown
Collaborator

Summary

Rebuild of the OpenAgents Go composer (macOS + iOS) plus a top-to-bottom attachment-pipeline overhaul. Cleared a stack of dogfooding issues in one cohesive change. v0.2.1 → 0.2.2.

Composer

  • Replace the SwiftUI TextEditor with a platform-native NSTextView / UITextView representable so we own the responder.
  • IME-safe Return-to-send: gates on hasMarkedText() (macOS) / markedTextRange (iOS) so pressing Enter mid-Pinyin or mid-Kana confirms the candidate instead of false-sending.
  • macOS image paste: extend readablePasteboardTypes + acceptableDragTypes so Cmd+V actually validates for image-only pasteboards (the system bonk before this change came from menu-validation, not from our paste(_:) override). Override handles PNG → JPEG → TIFF → NSImage fallback, plus file URLs from Finder.
  • iOS image paste: subclass UITextView; canPerformAction + paste(_:) read UIPasteboard.images. Hardware Cmd+V works; soft Return inserts newline to match iMessage.

Bubble layout

  • MessageBubble drops maxWidth: .infinity on the prose-only path so short user messages hug their text. Code / table content still claims the full row.

Toolbar titles

  • macOS: detail column's navigationTitle set to the current thread title (replaces the default "OpenAgents Go" in the window toolbar). Subtitle shows participating agents.
  • ThreadListView gets a visible workspace name + slug header at the top of the sidebar on macOS (NavigationSplitView doesn't surface a sidebar .navigationTitle anywhere visible).

Stop button

  • New WorkspaceAPI.sendAgentControl mirrors the React app.
  • WorkspaceStore.stopAllAgents fans out workspace.agent.control events with optimistic "Stopping…" status, retries once after 3s, clears the flag on terminal status from the message poll.
  • ChatGPT-style: send-arrow flips to a red stop circle while an agent is working; grays out while a stop is in flight.

Attachment UX

  • Dedup: optimistic message content is rewritten in-place to finalContent after api.sendMessage returns so the existing poll dedup matches. Fallback matcher compares the set of attached filenames for the mid-upload race. Fixes the "two bubbles per upload" symptom.
  • Auto-downsample to ≤ 2000px: new ImageDownsampler (ImageIO-based) wired into every ingest site (paste, photo picker, file import). Targets the exact Anthropic many-image-request limit so a Retina screenshot can't poison the conversation downstream.
  • Thumbnail chip: image attachments render as a 48×48 thumbnail with X overlay (decoded via ImageIO, cached). Non-image keeps the Slack-style filename + size row.
  • Long-text paste → file: paste ≥ 5,000 chars converts to a Pasted-text-<ts>.txt attachment instead of dumping into the input. Threshold matches ChatGPT's published behavior.

Test plan

  • Builds clean for OpenAgentsGo_macOS and OpenAgentsGo_iOS (Debug + Release).
  • macOS: short message hugs the bubble; long paragraphs wrap at row width; code blocks still claim full row.
  • macOS: Pinyin → space → Enter confirms candidate; second Enter sends. No mid-composition false sends.
  • macOS: Cmd+V on a screenshot adds an attachment chip (no bonk); pasted Retina screenshot is downsampled to ≤ 2000px before upload; downstream agent's Read no longer trips Anthropic's 2000px limit.
  • macOS: long-text paste (≥ 5000 chars) creates Pasted-text-…txt chip; the text view stays empty.
  • macOS: single bubble per send (no duplicate "uploading…" + final pair); chip remove (X) clears the pending attachment.
  • macOS: Stop button appears while an agent is working; tapping sends a stop control event; flips back to send arrow on terminal status.
  • macOS: window toolbar shows the thread title + agent names; sidebar shows workspace name.
  • iOS regression sweep on simulator (paste, photo picker, IME, Stop) — not run in this PR.

Files

New:

  • packages/go/OpenAgents/Views/ComposerTextView.swift
  • packages/go/OpenAgents/Helpers/ImageDownsampler.swift

Modified:

  • packages/go/OpenAgents/Views/ChatView.swift — bubble layout, composer wiring, send/stop button, chip redesign, downsample at ingest sites.
  • packages/go/OpenAgents/Views/ThreadListView.swift — workspace header.
  • packages/go/OpenAgents/State/WorkspaceStore.swift — stop state + dedup.
  • packages/go/OpenAgents/Networking/WorkspaceAPI.swiftsendAgentControl.
  • packages/go/OpenAgents/Models/Attachment.swiftmakeThumbnail.
  • packages/go/OpenAgents/Models/Message.swiftlocalStoppingStatus helper.
  • packages/go/project.yml + regenerated project.pbxproj — version bump.

Deferred

  • Sent-bubble attachment cards (render [file.png](url) lines as proper thumbnail cards inside the message bubble) — explicitly punted to keep the diff bounded; markdown link rendering is acceptable now that dedup makes it a single line.

Native Swift composer rebuild that addresses a stack of macOS/iOS issues
surfaced during dogfooding. Single round of work covering input, send,
and attachment paths.

Composer (ComposerTextView)
- Replace SwiftUI TextEditor with NSViewRepresentable / UIViewRepresentable
  wrapping NSTextView / UITextView so we can intercept Return-to-send
  around IME composition (hasMarkedText / markedTextRange) — fixes false
  sends mid-Pinyin / Kana composition.
- macOS: extend readablePasteboardTypes + acceptableDragTypes to include
  PNG / JPEG / TIFF / public.image / fileURL so Cmd+V validates and our
  paste(_:) override actually runs (was previously rejected at the menu
  validator → system bonk).
- macOS: paste(_:) handles PNG → JPEG → TIFF → NSImage fallback; file URL
  paste from Finder.
- iOS: subclass UITextView; canPerformAction / paste(_:) read
  UIPasteboard.images. Hardware Cmd+V works; soft Return inserts newline
  to match iMessage.

Bubble layout (Issue: empty padding on user bubbles)
- MessageBubble drops maxWidth: .infinity on the prose-only path so short
  messages hug their text; code/table content still claims full row.

Toolbar titles
- macOS: ChatView sets navigationTitle = current session title (replaces
  default "OpenAgents Go" in the window toolbar) with agent names as
  subtitle.
- ThreadListView gets a visible workspace name + slug header at the top
  on macOS (split-view sidebar doesn't surface .navigationTitle).

Stop button
- Add WorkspaceAPI.sendAgentControl mirroring the React app's pattern.
- WorkspaceStore.stopAllAgents fans out workspace.agent.control events
  with optimistic "Stopping..." status; retries once after 3s; clears the
  flag on terminal status from the message poll.
- ChatGPT-style: send arrow swaps to a red stop circle while an agent is
  working; grays out while a stop is in flight.

Attachment UX
- Dedup optimistic vs real attachment messages: rewrite optimistic
  content to finalContent in-place after api.sendMessage returns so the
  poll's content-equality dedup matches. Fallback matcher compares the
  set of attached filenames if exact-equality misses (mid-upload race).
  Fixes the "two bubbles per upload" symptom.
- ImageDownsampler (ImageIO-based): any pasted/picked/imported image
  whose longest side is > 2000px gets resized to 2000 and re-encoded as
  PNG. Targets the exact Anthropic many-image-request limit so a Retina
  screenshot can't poison the conversation.
- AttachmentChip: image variant shows a 48x48 thumbnail with X overlay
  (decoded via ImageIO, cached on the chip). Non-image keeps the
  Slack-style filename + size row.
- Long text paste >= 5000 chars converts to a Pasted-text-<ts>.txt
  attachment chip instead of dumping into the input field
  (matches ChatGPT's published threshold).

Bumps MARKETING_VERSION 0.2.1 -> 0.2.2.
@vercel
Copy link
Copy Markdown

vercel Bot commented May 9, 2026

@baryhuang is attempting to deploy a commit to the Raphael's projects Team on Vercel.

A member of the Team first needs to authorize it.

@baryhuang
Copy link
Copy Markdown
Collaborator Author

Ready to review. This is package version 0.2.2

Copy link
Copy Markdown
Contributor

@zomux zomux left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Excellent overhaul of the composer and attachment pipeline. Good use of platform-native text views for IME-safe input, proper image downsampling, and clean separation of concerns. Minor nit: iOS force-unwrap in insertHardwareNewline (textRange(from: endOfDocument, to: endOfDocument)!) — low risk but could be guarded.

@zomux zomux merged commit 1184760 into openagents-org:develop May 12, 2026
2 of 3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants