Add inline ghost text completions and fix NFA wildcard completion#1933
Merged
Add inline ghost text completions and fix NFA wildcard completion#1933
Conversation
- Add terminalUI module with EnhancedSpinner, TerminalLayout, InputBox, CompletionMenu classes - EnhancedSpinner supports output-above-spinner pattern and streaming text - InputBox provides bordered input areas with emoji-aware width handling - CompletionMenu adds trigger-based autocompletion (@ for agents, / for commands) - Add ESM module support and string-width dependency for Unicode handling - Include demo script and design documentation Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add --testUI flag to CLI interactive command for enhanced terminal UI - Convert debug output to use 'debug' npm package across grammar/cache modules - Restore async grammar generation mode (fire-and-forget) for faster command responses - Add grammar result display when new rules are added to cache - Fix terminal input double-echo and cursor position issues - Fix prompt bracket overlap with emoji icons Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Fix manifest grammarFile paths to be manifest-relative (for patchPaths resolution) - Add NFA type extraction in AgentGrammar.validateEntityReferences to allow generated rules to reference types like TrackName, ArtistName from base grammar - Remove diagnostic console.log statements, keep debug() logging Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add single console.log showing generated rule or rejection reason - Remove verbose startup logging from setupGrammarGeneration - Keep debug() logging for detailed tracing when needed Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Include the generated rule in rejection messages to help diagnose entity validation failures and other rule addition errors. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Call registerBuiltInEntities() in setupGrammarGeneration to register Ordinal, Cardinal, CalendarDate converters - Make entity registry check case-insensitive (ordinal matches Ordinal) - Include generated rule in rejection messages for debugging Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
…itive matching - Revert case-insensitive entity validation (too dangerous for all symbols) - Register lowercase aliases (ordinal, cardinal, calendarDate) alongside PascalCase versions to match paramSpec convention from .pas.json schemas Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Use ClientIO notify for grammar rule notifications instead of console.log - Log grammar notifications to ~/.typeagent/grammar.log in CLI - Suppress startup agent errors (use debug logging instead of error notifications) - Fix entityWildcardPropertyNames: exclude basic wildcard types (wildcard, word, number, string) from being treated as entity wildcards The entity wildcard fix resolves the smoke test failure where $(listName:wildcard) was incorrectly added to entityWildcardPropertyNames, causing "Invalid entity wildcard property name" errors. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The first getList rule was too greedy - "what's on the shopping list?" was matching with wildcard capturing "on the shopping" instead of just "shopping". Added a more specific rule for "what's on (my)? $(listName) (list)?" that takes precedence over the generic pattern. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
…rminal rendering - Fix smoke test failure: "what's on the shopping list" was parsing "the shopping" as list name instead of "shopping" - added (the)? as optional article in grammar rules - Add marked and marked-terminal dependencies for CLI markdown rendering - Add convertMarkdownToTerminal function for styled terminal markdown output - Enhance HTML-to-text conversion with better formatting for track lists - Support DisplayType "markdown" in CLI content rendering Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Actions like pause, resume, next, previous, status don't have parameters
in their schema. The evaluators in dfaMatcher.ts and environment.ts were
always adding `parameters: {}` which caused validation to fail with:
"Action schema parameter type mismatch: player.pause"
Now only include parameters if there are any to include.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Add grammar rules for: - next -> nextTrack - skip -> nextTrack - skip track -> nextTrack Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Remove custom selectors using invalid format names (orderedList, unorderedList, listItem, dataTable) that caused "format is not a function" errors. Use only valid built-in formats (block, inline, skip). Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The rule (what's)? (show)? (me)? (the)? (my)? $(listName:wildcard) list could match with ALL optionals empty, becoming just $(listName:wildcard) list which captured everything before "list" as the list name. Split into specific rules that each require at least one fixed word: - what's (the)? (my)? $(listName:wildcard) list - show (me)? (the)? $(listName:wildcard) list - the $(listName:wildcard) list - my $(listName:wildcard) list This prevents "add bread, milk, flour to the shopping list" from being incorrectly parsed as getList with listName="add bread, milk, flour to the shopping" Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Fix issue where "list?" wouldn't match "list" in grammar rules. Also adds local test file for faster grammar testing. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Replace regex-based punctuation stripping with a simple loop that runs in O(n) time where n is the number of trailing punctuation chars. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add grammar normalization for passthrough rules before NFA construction Passthrough rules like `@ <S> = <C>` are converted to explicit form `@ <S> = $(_result:<C>) -> $(_result)` during preprocessing - Add popEnvironment flag for exiting nested rules without parent capture When rules like `(<Item>)?` create environments but the parent doesn't capture their result, we now properly pop the environment stack - Track anyRuleCreatedEnvironment to avoid unnecessary pops Rules like `(the)?` that don't create environments don't trigger a pop - Skip CalendarDate test pending grammar imports work Fixes "play the second track" returning trackNumber: undefined instead of 2 Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Implement comprehensive Windows settings control through natural language: - 47 new settings actions across 14 categories (Network, Display, Taskbar, Mouse, Privacy, Power, Accessibility, etc.) - C# handlers in AutoShell_Settings.cs using Win32 APIs, Registry, WMI, and COM - Grammar patterns supporting 300+ natural language variations - Full TypeScript integration with action schema, connector mapping, and grammar compilation - 100% test pass rate (32/32 test cases) This enables users to control Windows settings like "turn on bluetooth", "increase brightness", "center taskbar", "set mouse speed to 12" through TypeAgent. Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
The TypeScript test file was causing compilation errors: - Missing Jest configuration and type definitions - Invalid import path for 'action-grammar' package - Unused variables flagged by compiler Functional testing is already provided by test-grammar-matching.mjs which successfully tests all 32 grammar patterns with 100% pass rate. Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Follow the code agent pattern by organizing Windows settings actions into focused sub-groups for better discoverability and maintainability. **Changes:** - Keep 26 common actions at top level (window management, volume, wifi, bluetooth, brightness) - Organize 44 specialized actions into 7 sub-categories: - desktop-display: Night light, scaling, orientation (5 actions) - desktop-personalization: Transparency, title bars, contrast (3 actions) - desktop-taskbar: Auto-hide, alignment, widgets, clock (7 actions) - desktop-input: Mouse/touchpad settings (8 actions) - desktop-privacy: Camera, microphone, location access (3 actions) - desktop-power: Battery saver, power modes (3 actions) - desktop-system: Accessibility, file explorer, time, focus, etc. (15 actions) **Technical:** - Created `src/windows/` subdirectory with 7 schema files - Updated `manifest.json` with `subActionManifests` section - Created `allActionsSchema.ts` for internal type unions - Main `actionsSchema.ts` reduced from 704 to 274 lines - All actions remain fully functional via connector.ts - Build passes: tsc ✅ asc ✅ agc ✅ Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Keep refactored schema structure with sub-categories.
Include smoke tests, full test scenarios, debugging tips, and success criteria. Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
- Update package.json to compile each sub-schema separately with asc:* scripts - Add compiledSchemaFile to all sub-action manifests (without grammarFile) - Update actionHandler to use AllDesktopActions type - Each sub-schema now generates its own .pas.json file for action resolution Note: Grammar system doesn't yet support schemaName field passthrough Need to implement post-processing or NFA matcher fix for sub-schema routing
Brings in 38+ commits with: - Enhanced terminal UI features - Grammar generation improvements - NFA matching fixes - CLI prompt and completion functionality # Conflicts: # ts/pnpm-lock.yaml
…refactoring
- Add kp (Knowledge Processor) package: keyword indexing, dictionary enrichment,
LLM answer generation with aiclient/GPT-5-Mini primary and agent SDK fallback
- Email agent: online RAG search (provider API -> chunks -> LLM answer) as
primary search path, offline kp index as async background enrichment
- Self-reference expansion ("me", "myself") to logged-in user email in
send/find actions, body placeholder resolution for [Your Name]
- User identity injection into answer generation prompts
- Calendar/email provider abstraction: Google Calendar and Gmail providers
alongside existing MS Graph providers
- Calendar action handler enhancements
- Built-in entity and grammar matching improvements
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Windows 11 TaskbarAl registry: 0 = left, 1 = center (was swapped). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Merges Curtis's grammar import system (#1920), undefined variable validation (#1921), and duplicate variable detection (#1922) with our normalization and entity changes. Conflict resolution in grammarCompiler.ts: kept both main's validateVariableReferences + duplicate var detection AND our hasValue logic for single-literal and passthrough rules. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…improvements - New chat-ui package: shared rendering (setContent, ChatPanel, CSS) used by both Electron shell and Chrome extension side panel - Chrome extension chat panel: connects to dispatcher via service worker WebSocket, full send/receive with agent messages - Per-request browser routing: shell commands use embedded browser, extension commands use Chrome tabs - Follow link improvements: navigable href filtering, window.location.href for proper history (goBack works), ancestor/sibling link discovery - Chat panel UX: collapsible action details, status message cleanup, input box fix, connection banner with auto-reconnect - Email RAG search integration with knowledge processor Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add license/author/homepage/repository to chat-ui package.json - Sort package.json files (chat-ui, browser, kp) per sort-package-json - Remove accidentally committed scratch files - Fix XSS in OAuth error display (googleCalendarClient, googleEmailClient) - Fix URL substring check in browserViewManager (use hostname comparison) - Add data: and vbscript: protocol checks in domUtils hasNavigableHref - Apply prettier formatting to action-grammar, graph-utils, browser, chat-ui Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Use Node.js base64url encoding instead of manual regex replacement - Use bounded quantifiers in calendar time range regex - Strip residual angle brackets in HTML-to-plaintext conversion - Bound email pattern and sentence splitter quantifiers in kp Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Loop the tag-removal regex until stable to handle nested constructs like <scr<script>ipt> before removing residual angle brackets. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
When types imported from .ts files (@import { Foo } from "schema.ts") are used as variable types in grammar rules, they are now automatically added to the grammar's entities list. This bridges the new import system with the existing entity validation infrastructure, ensuring the NFA interpreter validates matched tokens against registered entity validators. Also fixes the player grammar to require the "device" keyword in "use" rules, preventing "use <anything>" from matching as selectDevice. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…ice validation Custom typed wildcards (e.g. MusicDevice) now fail in the NFA interpreter when no synchronous validator is registered, preventing greedy matches. Player grammar changed to use checked_wildcard pattern for device names (like trackName/artistName) with proper validation and completion support. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Modernize chat message styling: softer colors, rounded bubbles, subtle shadows - Add rich HTML now-playing card with album art, progress bar, device info - Consolidate track list rendering into single HTML path with flexbox layout - Simplify window title to just app name (remove agent emojis, zoom, version) - Remove shade tabs (Settings/Metrics/Help banner) from chat view - Remove dark mode toggle button (still accessible via chat command) - Remove chat-title overlay div - Add comprehensive dark mode theme with matching color palette - Polish chat input with rounded corners and purple focus ring Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add [GRAMMAR] console.log tracing for grammar rule generation and NFA matching - Fix command backstack to exclude previous session history (flaky CI test) - Fix speech recognition null safety when microphone selector not in DOM Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…ovements - NFA completion engine: accepts token arrays, token-over-wildcard preference, minimal (next-token) completions with property completions for checked wildcards - Shell requests completions only at token boundaries (spaces), filters locally between boundaries, backspace re-requests at last boundary - Grammar cache: strip optional LLM-inferred parameters instead of rejecting, add schema optionality info to ParameterValidationInfo - Calendar events open in embedded browser (target="_blank") like email links - Player agent: fix artist property name matching for NFA completions - Completion groups: add kind field (literal/entity), sorted groups support Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
# Conflicts: # ts/packages/cache/src/cache/grammarStore.ts # ts/packages/shell/src/renderer/assets/styles.less # ts/pnpm-lock.yaml
…ams from cache - Sort browser package.json devDependencies and scripts for policy compliance - Strip optional LLM-inferred parameters from grammar cache instead of rejecting - Apply prettier formatting fixes across actionGrammar and shell packages - Open calendar event links in embedded browser instead of system browser - Add backspace recovery for mid-word completion filtering Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…ocal manifest paths - Remove console.log [GRAMMAR] diagnostics from grammarStore.ts (debug() calls remain) - Fix Spotify setVolume URL: remove stray ?volume_percent from base URL that caused double query param after getUrlWithParams was rewritten in PR #1923 - Fix playerLocal manifest to use agents/playerLocal/dist/... path convention matching getPackageFilePath expectations Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…adata - Add InlineSearchMenuUI: ghost text completions (like VS Code) as alternative to dropdown menu, with arrow key cycling and Tab accept - Add inlineCompletions setting toggle (ui.inlineCompletions, default true) controllable via @shell set ui.inlineCompletions true/false - Fix completion sort order to preserve backend group ordering (grammar completions like "by" appear before entity completions like song titles) - Fix NFA wildcard self-loop: don't emit property completions from loopState, only from entry state — wildcard is token+ so completions after consumed tokens come from following grammar, not the entity list again - Wire inline mode through SearchMenu, PartialCompletion, ChatView, SettingsView Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
steveluc
temporarily deployed
to
development-fork
GitHub Actions
Inactive
steveluc
temporarily deployed
to
development-fork
GitHub Actions
Inactive
curtisman
added a commit
to curtisman/TypeAgent
that referenced
this pull request
Apr 5, 2026
In Electron, use RemoteSearchMenuUI (rendered in a separate WebContentsView) so the completion menu can overlay beyond the chat view bounds. Falls back to LocalSearchMenuUI in non-Electron environments. This was removed in microsoft#1933 when inline ghost text completions were added.
github-merge-queue bot
pushed a commit
that referenced
this pull request
Apr 5, 2026
## Summary Restore `RemoteSearchMenuUI` usage in Electron so the completion menu renders in a separate `WebContentsView` that can overlay beyond the chat view bounds. When PR #1933 added inline ghost text completions, it replaced the `RemoteSearchMenuUI`/`LocalSearchMenuUI` branching with `InlineSearchMenuUI`/`LocalSearchMenuUI`, leaving `RemoteSearchMenuUI` as dead code. The remote search menu infrastructure (IPC handlers, `electronSearchMenuUI.ts`, `searchMenuView.ts`, preload bindings) was never removed and remains fully intact. ## Changes - **`packages/shell/src/renderer/src/search.ts`**: When not in inline mode, use `RemoteSearchMenuUI` in Electron (via `isElectron()` check) and fall back to `LocalSearchMenuUI` in non-Electron environments. This matches the original behavior before #1933.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
InlineSearchMenuUIrenders completions as grayed-out ghost text after the cursor (like VS Code), with arrow key cycling and Tab accept. Toggleable via@shell set ui.inlineCompletions true/falseTest plan
play→ ghost text shows first completion with counter (e.g., "1/N")play Shake It Off→ ghost shows "by" (not "Shake It Off" again)@shell set ui.inlineCompletions false→ switches to dropdown menu mode@shell set ui.inlineCompletions true→ switches back to inline ghost@configcommands also show inline completions🤖 Generated with Claude Code