Skip to content

Improve autocomplete UX (inline + external overlay), add serve mode, and add macOS ARM64 release workflow#66

Merged
senamakel merged 21 commits into
tinyhumansai:mainfrom
senamakel:fix/tool-calls
Mar 30, 2026
Merged

Improve autocomplete UX (inline + external overlay), add serve mode, and add macOS ARM64 release workflow#66
senamakel merged 21 commits into
tinyhumansai:mainfrom
senamakel:fix/tool-calls

Conversation

@senamakel
Copy link
Copy Markdown
Member

Summary

  • Adds inline autocomplete ghost text rendering in the Conversations composer and Tab-accept behavior in the app UI.
  • Extends core autocomplete UX for external macOS apps by adding focused-element bounds capture and a persistent native overlay helper process.
  • Adds CLI autocomplete start --serve and --spawn modes for running autocomplete as a foreground/background process.
  • Adds a dedicated macOS ARM64 build workflow and updates release scripts/workflow wiring for that target.

Problem

  • Autocomplete suggestions were generated/accepted but not consistently visible as ghost text in UI and external apps.
  • The CLI path did not expose a process-style serve mode (--serve) even though helper logic existed.
  • Release automation did not include the new macOS ARM64 build lane needed for packaging consistency.

Solution

  • Implemented suffix-based inline ghost rendering in app/src/pages/Conversations.tsx with debounce polling of openhuman.autocomplete_current, plus Tab acceptance.
  • Upgraded src/openhuman/autocomplete/core.rs to:
    • capture focused element bounds (AXPosition/AXSize),
    • drive a persistent Swift/AppKit overlay helper process for external-app ghost hints,
    • retain notification fallback for non-ready/error/accept/reject states,
    • shut down helper lifecycle on stop/disable.
  • Wired CLI autocomplete start options in src/core/cli.rs to support:
    • --serve (foreground service loop),
    • --spawn (background process),
    • --debounce-ms passthrough.
  • Added macOS ARM64 workflow/release script updates in .github/workflows/* and scripts/*.

Testing

  • yarn -s compile
  • cargo check --manifest-path app/src-tauri/Cargo.toml
  • Other checks run (list commands)
  • cargo check --manifest-path Cargo.toml
  • yarn workspace openhuman-app format:check
  • yarn workspace openhuman-app lint
  • Manual validation completed (list scenarios)
  • Ran cargo run --bin openhuman -- autocomplete start --serve --debounce-ms 120 and verified phase/suggestion streaming.
  • Verified cargo run --bin openhuman -- autocomplete start --help includes --serve|--spawn.
  • Validated runtime suggestions are emitted while focusing external apps (including TextEdit).

Impact

  • Platform impact: macOS-focused autocomplete UX improvements; app UI impact in Conversations text composer.
  • CLI impact: new serve/spawn flags for autocomplete process management.
  • Release/CI impact: additional macOS ARM64 build lane.
  • No data model or config migration required.

Breaking Changes

  • None
  • Yes (describe clearly, including migration steps)

Related

  • Issue(s): N/A
  • Follow-up PR(s)/TODOs:
  • Add helper stderr/file logging for overlay diagnostics on machines where AppKit overlay still fails.
  • Add dedicated tests for CLI autocomplete start --serve|--spawn argument parsing.

…tation

- Added a note to emphasize the importance of opening pull requests against the upstream repository instead of a fork's default remote, ensuring better collaboration and code management.
- Improved the overall clarity of the Git workflow section, contributing to a more streamlined development process.
- Removed the `repl_session_chat` functionality from the schema and related files, streamlining the REPL session management.
- Introduced a new `start_chat` method in the web channel provider for handling chat interactions, enhancing the chat system's architecture.
- Updated socket handling to improve event emission with alias support, ensuring better communication across the application.
- Enhanced error handling and logging for chat operations, providing clearer feedback during interactions.
- Consolidated chat-related logic into the new web channel module, improving maintainability and organization.
… dependencies

- Renamed `_getToken` to `authTokenGetterRef` for clarity and to avoid clashing with transpiled private method names.
- Updated the `setStoreForApiClient` function to use the new `authTokenGetterRef` variable, ensuring the `apiClient` remains free of direct imports from `store/index`.
- Enhanced the `ApiClient` class by renaming the `getToken` method to `resolveAuthToken`, improving readability and consistency in token retrieval logic.
- Adjusted comments in `apiClient.ts` and `store/index.ts` to reflect the changes and clarify the purpose of the lazy store accessor.
- Introduced `setStoreForApiClient` in `main.tsx` to bind the Redux store's auth token for API requests, ensuring seamless token access.
- Updated comments in `apiClient.ts` to clarify the lazy token accessor's purpose and prevent circular dependencies.
- Removed the previous direct call to `setStoreForApiClient` from `store/index.ts`, centralizing the setup in `main.tsx` for better organization.
- Enhanced test setup to include the new store binding, ensuring consistent token retrieval during testing.
- Updated `WebChatSseEvent` structure to include new fields: `tool_name`, `skill_id`, `args`, `output`, `success`, and `round`, improving the granularity of event data.
- Refactored event emission logic in `emit_web_chat_event` to utilize the new fields, ensuring more informative payloads for tool calls and results.
- Enhanced the `start_chat` and `cancel_chat` functions to accommodate the new event structure, improving chat session management.
- Introduced a new function `publish_tool_events_from_history` to streamline the emission of tool call and result events from chat history, enhancing the overall chat experience.
- Updated references from the deprecated `web_channel` module to the new `channels::providers::web` structure, improving code organization and maintainability.
- Refactored the `build_registered_controllers` and `build_declared_controller_schemas` functions to utilize the new web channel controller methods, ensuring consistency across the application.
- Removed the obsolete `web_channel` module and its associated files, streamlining the codebase and enhancing clarity in the channel management system.
- Introduced new functions for managing web channel events and schemas, improving the overall architecture of the web channel integration.
- Reordered imports in `main.tsx` to enhance clarity and maintainability, ensuring that `setStoreForApiClient` is called with the correct store reference.
- Removed unnecessary whitespace in `setup.ts`, streamlining the test setup file for better readability.
- Introduced `tracing-log` as a dependency to bridge `log` and `tracing` for improved logging capabilities.
- Added a `--verbose` flag to the CLI for enhanced logging detail, initializing the logging level based on this flag.
- Implemented an HTTP request logging middleware to capture and log request details.
- Updated CLI help output to reflect the new `--verbose` option and improved logging messages throughout the application.
- Added `nu-ansi-term` dependency for colored terminal output in logs.
- Implemented a custom logging format for better readability in CLI.
- Updated logging initialization to conditionally use ANSI colors based on terminal support.
- Added debug logging for agent responses to aid in troubleshooting.
- Added functionality for inline suggestions based on user input in the Conversations component.
- Introduced debounce logic for autocomplete requests to optimize performance.
- Enhanced user experience by allowing suggestions to be accepted via the Tab key.
- Updated UI to display inline completion suffixes in the input area.
- Modified backend to support new autocomplete features and improved error handling.
- Introduced a new GitHub Actions workflow for building signed macOS ARM64 bundles.
- Updated the release workflow to validate signing prerequisites for macOS builds.
- Enhanced the Tauri configuration preparation to include updater settings.
- Added necessary secrets to the example secrets file for macOS signing.
- Implemented CLI autocomplete functionality for macOS, including options for debounce and process management.
@senamakel senamakel merged commit 652bb6e into tinyhumansai:main Mar 30, 2026
2 of 5 checks passed
@senamakel senamakel deleted the fix/tool-calls branch March 31, 2026 00:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant