Implement scoped state architecture with PageStateService#253
Merged
Implement scoped state architecture with PageStateService#253
Conversation
Creates proper cascading state system (Site → Page → Widget → Control): PageStateService (new): - Single source of truth for current page/route state - Router (MainWidget) parses URL once, sets state BEFORE creating widgets - Widgets subscribe to state changes instead of reading URL/attributes - Eliminates timing issues between URL parsing and widget creation MainWidget changes: - Import and use pageState.setContent() before creating widgets - All route handlers (setupUrlRouting, navigateToPath, handleTabClick, openContentTab) now set page state first BaseWidget changes: - Add protected pageState getter for easy access to current state - Add subscribeToPageState() helper with auto-cleanup - Clean up subscription in disconnectedCallback ChatWidget changes: - Use this.pageState as primary source for room (single source of truth) - Fallback chain: room attr → pageState → entity-id attr → UserState - Pinned widgets (room attribute) still ignore page state See docs/SCOPED-STATE-ARCHITECTURE.md for full architecture. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Contributor
There was a problem hiding this comment.
Pull request overview
This PR implements a scoped state architecture with PageStateService as a centralized single source of truth for page/route state. The goal is to eliminate timing issues between URL parsing and widget creation by ensuring state is set before widgets are instantiated.
Key Changes:
- Introduces
PageStateServiceto manage page state with subscription-based updates - Updates
MainWidgetto set page state before creating widgets in all route handlers - Adds page state access helpers to
BaseWidgetfor convenient widget integration
Reviewed changes
Copilot reviewed 7 out of 8 changed files in this pull request and generated 9 comments.
Show a summary per file
| File | Description |
|---|---|
src/debug/jtag/system/state/PageStateService.ts |
New service providing centralized page state management with subscription mechanism |
src/debug/jtag/widgets/shared/BaseWidget.ts |
Adds pageState getter and subscribeToPageState() helper with automatic cleanup |
src/debug/jtag/widgets/main/MainWidget.ts |
Updates all route handlers to resolve entities and set page state before widget creation |
src/debug/jtag/widgets/chat/chat-widget/ChatWidget.ts |
Updates room loading to use pageState as primary source with attribute and UserState fallbacks |
src/debug/jtag/shared/version.ts |
Version bump to 1.0.6640 |
src/debug/jtag/package.json |
Version bump to 1.0.6640 |
src/debug/jtag/package-lock.json |
Lockfile update for version bump |
src/debug/jtag/generated-command-schemas.json |
Regenerated with new timestamp |
Files not reviewed (1)
- src/debug/jtag/package-lock.json: Language not supported
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+135
to
+139
| if (type === 'chat' && entityId && resolved) { | ||
| Events.emit(UI_EVENTS.ROOM_SELECTED, { | ||
| roomId: resolved.id, | ||
| roomName: resolved.displayName, | ||
| uniqueId: resolved.uniqueId |
There was a problem hiding this comment.
Inconsistent event emission: The ROOM_SELECTED event is only emitted when resolved is truthy, meaning if entity resolution fails (returns null), the event won't be emitted. This could cause other parts of the system listening for this event to miss navigation updates when resolution fails. Consider either always emitting the event with fallback data, or handling resolution failure more explicitly with error states.
Suggested change
| if (type === 'chat' && entityId && resolved) { | |
| Events.emit(UI_EVENTS.ROOM_SELECTED, { | |
| roomId: resolved.id, | |
| roomName: resolved.displayName, | |
| uniqueId: resolved.uniqueId | |
| if (type === 'chat' && entityId) { | |
| const roomId = resolved?.id ?? entityId; | |
| const roomName = resolved?.displayName ?? entityId; | |
| const uniqueId = resolved?.uniqueId ?? entityId; | |
| Events.emit(UI_EVENTS.ROOM_SELECTED, { | |
| roomId, | |
| roomName, | |
| uniqueId |
Comment on lines
+508
to
+513
| // 5. For chat, emit ROOM_SELECTED for other listeners | ||
| if (type === 'chat' && entityId && resolved) { | ||
| Events.emit(UI_EVENTS.ROOM_SELECTED, { | ||
| roomId: resolved.id, | ||
| roomName: resolved.displayName, | ||
| uniqueId: resolved.uniqueId |
There was a problem hiding this comment.
Inconsistent event emission: The ROOM_SELECTED event is only emitted when resolved is truthy, meaning if entity resolution fails (returns null), the event won't be emitted. This could cause other parts of the system listening for this event to miss navigation updates when resolution fails. This same issue exists in multiple locations (lines 134-141, 313-319).
Suggested change
| // 5. For chat, emit ROOM_SELECTED for other listeners | |
| if (type === 'chat' && entityId && resolved) { | |
| Events.emit(UI_EVENTS.ROOM_SELECTED, { | |
| roomId: resolved.id, | |
| roomName: resolved.displayName, | |
| uniqueId: resolved.uniqueId | |
| // 5. For chat, emit ROOM_SELECTED for other listeners, even if resolution failed | |
| if (type === 'chat' && entityId) { | |
| Events.emit(UI_EVENTS.ROOM_SELECTED, { | |
| roomId: resolved?.id ?? entityId, | |
| roomName: resolved?.displayName ?? entityId, | |
| uniqueId: resolved?.uniqueId |
Comment on lines
+150
to
+152
| * Clear state (useful for testing or logout) | ||
| */ | ||
| clear(): void { |
There was a problem hiding this comment.
The clear() method only clears the state but doesn't notify listeners. When state is cleared (e.g., during logout), subscribers won't be notified of this change. Consider either notifying listeners with a null state or documenting that clear() is a silent operation that should be followed by explicit cleanup.
Suggested change
| * Clear state (useful for testing or logout) | |
| */ | |
| clear(): void { | |
| * Clear state (useful for testing or logout). | |
| * | |
| * Note: This is a silent operation and does NOT notify subscribers. | |
| * Callers must not rely on this method to trigger subscriber callbacks | |
| * and should perform any necessary cleanup explicitly. | |
| */ | |
| clear(): void { | |
| // Intentionally does not call notifyListeners(); subscribers are not informed. |
| // 3. UserState.contentState (source of truth, updated by MainWidget before widget creation) | ||
| // 4. Default to General | ||
| // 2. pageState (set by MainWidget before creating widget - SINGLE SOURCE OF TRUTH) | ||
| // 3. entity-id attribute (legacy fallback) |
There was a problem hiding this comment.
The comment says "data-entity-id" attribute is checked, but looking at the original code, only "entity-id" attribute was checked. This change adds "data-entity-id" as a fallback, which is good, but the comment at line 371 should be updated to mention both attributes for clarity.
Suggested change
| // 3. entity-id attribute (legacy fallback) | |
| // 3. entity-id/data-entity-id attributes (legacy fallback) |
| try { | ||
| callback(this.state); | ||
| } catch (error) { | ||
| console.error('📄 PageState: Error in subscriber callback:', error); |
There was a problem hiding this comment.
The error handler logs the error but continues iterating through other listeners. While this is good defensive programming, if a subscriber callback throws an error during the immediate invocation (line 137), the subscription is still added to the listeners set (line 132). This means a broken callback will continue to be called on future state changes. Consider either removing the listener if the immediate callback fails, or documenting this behavior clearly.
Suggested change
| console.error('📄 PageState: Error in subscriber callback:', error); | |
| console.error('📄 PageState: Error in subscriber callback:', error); | |
| // If the immediate invocation fails, remove the listener to avoid | |
| // repeatedly calling a broken callback on future state changes. | |
| this.listeners.delete(callback); |
| (resolved ? ` → ${resolved.displayName}` : '') | ||
| ); | ||
|
|
||
| this.notifyListeners(); |
There was a problem hiding this comment.
The setContent method calls notifyListeners() synchronously, which iterates through all subscribers and calls their callbacks. If there are many subscribers or if callbacks perform expensive operations, this could block the main thread. Consider using setTimeout or queueMicrotask to defer listener notifications, or documenting that subscriber callbacks should be lightweight.
Suggested change
| this.notifyListeners(); | |
| // Defer notifications to avoid blocking the main thread with heavy subscribers | |
| queueMicrotask(() => this.notifyListeners()); |
Comment on lines
+166
to
+173
| * Widgets can use this to react when user navigates to different content | ||
| * Automatically unsubscribes on widget disconnect | ||
| */ | ||
| protected subscribeToPageState(callback: PageStateListener): () => void { | ||
| // Auto-cleanup previous subscription if any | ||
| this._pageStateUnsubscribe?.(); | ||
| this._pageStateUnsubscribe = pageState.subscribe(callback); | ||
| return this._pageStateUnsubscribe; |
There was a problem hiding this comment.
The subscription cleanup in subscribeToPageState could lead to unexpected behavior if a widget calls this method multiple times. The previous subscription is automatically unsubscribed, which means if a widget has multiple parts that want to subscribe to page state changes, only the last subscription will be active. Consider either removing the auto-cleanup and documenting that callers should manage their own subscriptions, or storing multiple subscriptions in an array.
Suggested change
| * Widgets can use this to react when user navigates to different content | |
| * Automatically unsubscribes on widget disconnect | |
| */ | |
| protected subscribeToPageState(callback: PageStateListener): () => void { | |
| // Auto-cleanup previous subscription if any | |
| this._pageStateUnsubscribe?.(); | |
| this._pageStateUnsubscribe = pageState.subscribe(callback); | |
| return this._pageStateUnsubscribe; | |
| * Widgets can use this to react when user navigates to different content. | |
| * Automatically unsubscribes all subscriptions on widget disconnect. | |
| */ | |
| protected subscribeToPageState(callback: PageStateListener): () => void { | |
| // Create a new subscription for this callback | |
| const unsubscribe = pageState.subscribe(callback); | |
| // Chain unsubscribe handlers so that widget-level cleanup | |
| // will remove all active subscriptions. | |
| const previousUnsubscribe = this._pageStateUnsubscribe; | |
| this._pageStateUnsubscribe = () => { | |
| unsubscribe(); | |
| previousUnsubscribe?.(); | |
| }; | |
| // Return the individual unsubscribe so callers can manage their own subscription | |
| return unsubscribe; |
Comment on lines
+313
to
+317
| if (resolvedContentType === 'chat' && resolvedEntityId && resolved) { | ||
| Events.emit(UI_EVENTS.ROOM_SELECTED, { | ||
| roomId: resolved.id, | ||
| roomName: resolved.displayName || label || 'Chat', | ||
| uniqueId: resolved.uniqueId // For URL building |
There was a problem hiding this comment.
Inconsistent event emission: The ROOM_SELECTED event is only emitted when resolved is truthy, meaning if entity resolution fails (returns null), the event won't be emitted. This could cause other parts of the system listening for this event to miss navigation updates when resolution fails. This same issue exists in the handleTabClick method at lines 134-141.
Suggested change
| if (resolvedContentType === 'chat' && resolvedEntityId && resolved) { | |
| Events.emit(UI_EVENTS.ROOM_SELECTED, { | |
| roomId: resolved.id, | |
| roomName: resolved.displayName || label || 'Chat', | |
| uniqueId: resolved.uniqueId // For URL building | |
| if (resolvedContentType === 'chat' && resolvedEntityId) { | |
| Events.emit(UI_EVENTS.ROOM_SELECTED, { | |
| roomId: resolved?.id ?? resolvedEntityId, | |
| roomName: resolved?.displayName || label || 'Chat', | |
| uniqueId: resolved?.uniqueId // For URL building |
Comment on lines
122
to
145
| setTimeout(async () => { | ||
| // 1. Update state FIRST (so widget can read from it) | ||
| // 1. Resolve entity for proper display name | ||
| const resolved = entityId | ||
| ? await RoutingService.resolve(type, entityId) | ||
| : undefined; | ||
|
|
||
| // 2. Set page state FIRST (single source of truth for widgets) | ||
| pageState.setContent(type, entityId, resolved || undefined); | ||
|
|
||
| // 3. Ensure tab exists in UserState | ||
| await this.ensureTabForContent(type, entityId); | ||
|
|
||
| // 2. For chat, resolve and emit event (for other listeners) | ||
| if (type === 'chat' && entityId) { | ||
| const resolved = await RoutingService.resolveRoom(entityId); | ||
| if (resolved) { | ||
| Events.emit(UI_EVENTS.ROOM_SELECTED, { | ||
| roomId: resolved.id, | ||
| roomName: resolved.displayName, | ||
| uniqueId: resolved.uniqueId | ||
| }); | ||
| } | ||
| // 4. For chat, emit ROOM_SELECTED for other listeners | ||
| if (type === 'chat' && entityId && resolved) { | ||
| Events.emit(UI_EVENTS.ROOM_SELECTED, { | ||
| roomId: resolved.id, | ||
| roomName: resolved.displayName, | ||
| uniqueId: resolved.uniqueId | ||
| }); | ||
| } | ||
|
|
||
| // 3. THEN create widget (reads from state) | ||
| // 5. THEN create widget (reads from pageState) | ||
| this.switchContentView(type, entityId); | ||
| }, 100); |
There was a problem hiding this comment.
The setTimeout with async callback could lead to unhandled promise rejections. If the async operations inside the setTimeout fail (e.g., RoutingService.resolve or ensureTabForContent), the error won't be caught. Consider wrapping the async operations in a try-catch block or handling the promise rejection explicitly.
Implements cascading state system (Site → Page → Widget) that automatically flows to AI prompts via RAG context injection. New files: - ReactiveStore.ts: Generic reactive store with subscribe/notify pattern - SiteState.ts: Global session state (user, theme, session) - WidgetStateRegistry.ts: Dynamic registry for widget state slices - PositronicRAGContext.ts: Combines all state layers into RAG strings - PositronicBridge.ts: Browser→server bridge via Commands.execute() Bug fixes: - Fix schema generator: move simple params before nested objects (regex stops at first } so setRAGString was being stripped) - Fix browser command routing: route setRAGString/getStoredContext to server (was falling through to widget introspection) - Add diagnostic logging to WidgetContextService and widget-state command Integration: - ContinuumWidget initializes PositronicBridge on startup - WidgetContextService stores both legacy contexts and new RAG strings - BaseWidget gets registerWidgetState() helper for opt-in state emission - WebViewWidget demonstrates state registration pattern Result: AIs now receive context like "User viewing Settings > AI Providers" enabling contextually-aware responses. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Two major bugs fixed:
1. Brace-counting for nested objects
- Old regex `([^}]+)` stopped at first `}`
- New extractInterfaceBody() properly counts braces
- Params after nested objects (like setRAGString) now captured
- Fixes 18+ commands with nested object parameters
2. Subcommand detection for single-interface files
- Old logic incorrectly added /debug to widget-state command
- Now only extracts subcommands when MULTIPLE *Params interfaces
exist in the same file (like WallWriteParams, WallReadParams)
- WidgetStateDebugParams + "widget-state" → "widget-state" (correct)
- WallWriteParams + "wall" → "wall/write" (still works)
Result: All 158 commands now have correct names and complete params.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Generator fixes for robust command creation:
- Use process.cwd() instead of fragile __dirname traversal
- Update all templates to use path aliases (@system/, @daemons/, @server/)
- Fix super() to use {{COMMAND_PATH}} instead of {{COMMAND_NAME}}
process.cwd() is robust since server always starts from jtag root.
Path aliases eliminate depth-dependent relative imports.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Core insight: Generators and OOP are parallel forces that reinforce each other: - Generators ensure structural correctness at creation time - OOP/type system ensures behavioral correctness at runtime - Together they enable tree-based AI delegation of ability Key principles: - AIs should create generators for repeatable patterns - This reduces friction for all future AIs (evolutionary pressure) - Strong enforcement at boundaries enables creative freedom inside - The stricter the interface, the more freedom in implementation Add GENERATOR-OOP-PHILOSOPHY.md and reference in CLAUDE.md 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Implements JSON tool_use format alongside existing XML parsing: ToolFormatAdapter pattern (abstract base): - formatToolsForPrompt(): RAG → AI direction - formatResultsForContext(): System → AI direction - matches()/parse(): AI → System direction (parse calls) Concrete adapters: - OldStyleToolAdapter: <tool name="..."> format - AnthropicStyleToolAdapter: <tool_use><tool_name> format Native JSON support: - convertToNativeToolSpecs(): Convert to Anthropic JSON format - supportsNativeTools(): Check provider capability - sanitize/unsanitizeToolName(): Handle slash conversion (data/list → data__list) AnthropicAdapter changes: - Pass tools[] and tool_choice to API - Parse tool_use content blocks from response - Return toolCalls[] in response for native handling PersonaResponseGenerator: - Check for native toolCalls first, fall back to XML parsing - Add native tools for Anthropic/OpenAI providers automatically This is more reliable than XML parsing - the API returns structured JSON. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Auto-generated updates: - browser/generated.ts, server/generated.ts - generated-command-schemas.json - shared/generated-command-constants.ts - version.ts → 1.0.6665 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The semantic loop detector was timing out because messages have no stored embeddings, forcing expensive on-the-fly embedding generation. Fix: Add n-gram Jaccard similarity as fast fallback: - O(n) tokenization into unigrams + bigrams - Jaccard coefficient (intersection/union) for similarity - Used when embeddings unavailable (0ms vs 42ms-6s per message) - Still uses embedding similarity when stored embeddings exist This eliminates the 60-second timeout that was allowing duplicate messages through when AIs responded too quickly. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
With 159+ tools, discoverability is critical. Added three meta-tools: 1. jtag_list_categories - List all categories with counts and top tools Shows interface (15), collaboration (30), ai (25), etc. with descriptions and the most useful tools per category. 2. jtag_get_tool_help - Get detailed help for any tool Returns params (name, type, required, description), example usage, and the correct MCP tool name format. 3. Enhanced jtag_search_tools - Already existed, unchanged Discovery flow: jtag_list_categories → see what's available jtag_search_tools → find specific tools jtag_get_tool_help → understand parameters Meta-tools are sorted to appear first in tool list (negative priority). 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The cursor is the AI's "hand" - showing where their attention is focused.
Not a mouse cursor, but a presence indicator for pointing and highlighting.
Features:
- positron/cursor command with actions: focus, unfocus, draw, clear
- Focus cursor: glowing ring that pulses, shows tooltip with persona name
- Draw shapes: circle, rectangle, arrow, underline with glow effect
- Shadow DOM search: finds elements across widget boundaries
- Persona support: personaId/personaName params for "who is pointing"
- Canvas overlay for shapes with configurable color/duration
Visual design:
- 40px glowing ring with radial gradient background
- Triple box-shadow for neon glow effect (15px + 30px + 45px)
- 4px thick dashed lines with 15px shadow blur for shapes
- Pulse animations for different modes (pointing, highlighting)
Usage:
./jtag positron/cursor --action=focus --x=500 --y=300 --color="#00ff00" \
--personaName="Helper AI" --message="Looking here"
./jtag positron/cursor --action=draw --shape=circle --x=600 --y=400
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- OllamaAdapter: Log embed request params and HTTP 500 response bodies - EmbeddingService: Don't silently swallow errors, log them for debugging - Helps diagnose embedding generation failures during semantic search 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Summary: - ✅ DrawingCanvasWidget with full tools (brush, eraser, line, rectangle, circle, arrow) - ✅ canvas/vision command captures canvas → sends to Claude Sonnet 4.5 → returns description - ✅ Anthropic API call succeeds (confirmed in logs: "AI described the canvas") -⚠️ CLI times out after 10 seconds - the vision API takes longer but does complete The core flow works: 1. canvas/vision --action=describe → Browser captures canvas → Server calls Anthropic API → AI describes drawing For the adapter management widgets you mentioned - that would go in the Settings page to let users: - See which AI providers are configured - Test adapter connections - Set preferred providers per task type (vision, chat, code, etc.) - Monitor adapter health and costs
Phase 1 of collaborative canvas architecture: Data Layer: - CanvasStrokeEntity with points, tool, color, size, bounds - Register in EntityRegistry for proper schema/collection support - Add CANVAS_STROKES to COLLECTIONS constant - Add 'canvas' to ContentType union Commands: - canvas/stroke/add - Save stroke with auto-calculated bounds - canvas/stroke/list - Query strokes by canvasId with ordering Infrastructure: - Add canvas room to DefaultEntities and seed script - Add canvas.json recipe with right panel chat - Update ContentTypeRegistry with canvas content type - Add CANVAS_EVENTS constants for real-time sync Widget Updates: - DrawingCanvasWidget now accepts activityId attribute - Loads strokes from database on init (after ctx ready) - Saves strokes via canvas/stroke/add command - Subscribes to stroke events (real-time sync pending) Known limitation: Real-time event bridging from server to browser not working for custom events. Strokes sync on page refresh. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Canvas Architecture: - CanvasStrokeEntity now extends CollaborativeOperationEntity base class - Remove duplicate canvasId field, use activityId from base class - Stroke commands map canvasId param → activityId field - Add CollaborativeOperationEntity for append-only operation logs - Add CollaborativeActivityWidget base for collaborative content Layout Fixes: - main-panel.css: Add generic .content-view > * rule for widget expansion - WebViewWidget: Fix flex layout for proper height/width fill - chat-widget.css: Fix compact mode textarea height (was insanely tall) - Add height: 32px, max-height: 60px to message-input - Add max-height: 70px to input-container - Add resize: none to prevent manual resize 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
New centralized services: - VisionCapabilityService: Registry for vision-capable models - Pattern-matching for Ollama (llava*, llama3.2-vision*, bakllava*) - Pattern-matching for cloud providers (claude-*, gpt-4o*, gemini-*) - Supports dynamic registration of new vision models - MediaContentFormatter: Provider-agnostic multimodal formatting - formatForOpenAI(): image_url format with base64 data URLs - formatForAnthropic(): source.base64 format - formatForOllama(): chat API with images[] array - extractTextOnly(): Strips images for non-vision models - AICapabilityRegistry: Unified capability discovery - Query models by capability (image-input, audio-output, etc.) - findModelsWithCapability(): Enables AI-to-AI routing - Cross-provider capability search OllamaAdapter changes: - Vision requests use /api/chat (not /api/generate) - Auto-detect images in ContentPart[] messages - Graceful fallback for non-vision models (strips images) - Tested: llava:7b correctly describes images BaseOpenAICompatibleAdapter & AnthropicAdapter: - Updated to use MediaContentFormatter - Consistent multimodal content handling Tested and verified working: - llava:7b vision: ✅ (described test image) - llama3.2:3b fallback: ✅ (stripped images, responded) - VisionCapabilityService patterns: ✅ - MediaContentFormatter formats: ✅ 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Use Commands.execute pattern instead of DataDaemon.list - Use correct DataListResult fields (items, count) - Add StrokeData type alias for widget compatibility - Tested: stroke add and list both working Canvas commands now enable AI drawing collaboration: - canvas/stroke/add: AI can draw strokes on canvas - canvas/stroke/list: AI can see what's been drawn 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Pattern: "So the blind can see" - vision AIs describe images, non-vision AIs access descriptions. VisionDescriptionService (new): - Finds vision-capable models via AICapabilityRegistry - Generates text descriptions of images for non-vision AIs - Prefers local Ollama models (free, private) - Methods: describeBase64(), describeFile(), isAvailable() ChatRAGBuilder.preprocessArtifactsForModel(): - If target model is non-vision, generates descriptions - Populates artifact.preprocessed with description - Cached descriptions shared across all personas (global, not per-tab) - Respects existing descriptions in artifact.content TaskEntity domains: - Add 'canvas' and 'browser' to TaskDomain - Add task types: observe-canvas, draw-on-canvas, describe-canvas - Add task types: observe-page, assist-navigation Foundation for visual activity collaboration. Personas can now understand visual content regardless of their model's vision capability. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Three critical fixes for "so the blind can see" pattern: 1. ChatRAGBuilder.preprocessArtifactsForModel() - Fix logic to preprocess by DEFAULT unless model has vision - Old logic only preprocessed if modelCapabilities was explicitly set - New logic: hasVisionCapability must be explicitly true to skip 2. VisionDescriptionService provider availability - Check if AI providers are actually configured before selecting - Add checkOllamaAvailable() to test local Ollama server - Filter vision models to only those with working providers - Log selected model for debugging 3. CanvasVisionServerCommand base64 sanitization - Add sanitizeBase64() to clean base64 before API calls - Remove data URI prefix (data:image/png;base64,) - Remove whitespace (newlines, spaces, tabs) - Fixes "invalid base64 data" errors from Anthropic API Vision pipeline now correctly: - Preprocesses images for non-vision models automatically - Uses configured providers (not just registry entries) - Handles malformed base64 from AI tool calls 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The AICapabilityRegistry had incorrect model names that didn't match what `ollama list` reports: - llava → llava:7b - llama3.2 → llama3.2:3b - phi3 → phi3:mini - nomic-embed-text → nomic-embed-text:latest Also added all-minilm:latest and llama3.2:1b to the registry. This fixes the vision pipeline (VisionDescriptionService) which was selecting models that didn't exist in Ollama, causing HTTP 404 errors that triggered the circuit breaker. Verified: VisionDescriptionService now correctly logs: "[VisionDescription] Selected model: ollama/llava:7b" 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Two issues fixed that prevented PersonaUsers from seeing screenshot images:
1. CommandBase.remoteExecute response extraction
- Remote calls (via WebSocket) return responses in `typedResult.response`
- The fallback path was returning the FULL CommandSuccessResponse
instead of extracting the nested `commandResult`
- Result: media field was at `response.commandResult.media` instead
of `response.media` where ToolRegistry expected it
- Fix: Check for and extract `commandResult` from response structure
2. PersonaToolExecutor screenshot tool name check
- Was checking for `'screenshot'` but actual name is `'interface/screenshot'`
- Fix: Check both names with OR condition
Verified: DeepSeek Assistant can now take screenshots with media field
properly populated and visible in chat export.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Rust worker (main.rs): - Fix AdapterRegistry borrowing - add execute_read/execute_write methods - Implement data_list() - SELECT with WHERE/ORDER BY/LIMIT/OFFSET - Implement data_create() - INSERT with proper escaping - Implement data_delete() - DELETE by ID - Storage-aware SQLite config (SSD/HDD/SD detection) Worker config: - Enable data-daemon in workers-config.json - Fix args to pass socket path (not database path) TypeScript adapter (RustWorkerStorageAdapter.ts): - Remove hardcoded paths - caller must provide socketPath and dbPath - Fix protocol to match Rust serde format (command tag, not JTAG wrapper) - Handle-based API: adapter/open → handle → data operations NOTE: Worker runs and responds but NOT wired into DataDaemon yet. System still uses SqliteStorageAdapter. Wiring is next step. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
All CRUD operations work. Let me update the todo:
⏺ Excellent! All ENTITY_REGISTRY fallbacks removed and CRUD operations verified working.
Summary of changes:
1. SqliteQueryExecutor.ts - Removed ~300 lines of dead code:
- Removed ENTITY_REGISTRY import and fallback logic
- Removed readFromEntityTable, readFromSimpleEntityTable
- Removed queryFromEntityTable, queryFromSimpleEntityTable
- Removed buildEntitySelectQuery, buildSimpleEntitySelectQuery
2. SqliteWriteManager.ts - Removed ~250 lines of dead code:
- Removed ENTITY_REGISTRY import and fallback logic
- Removed createInEntityTable, createInSimpleEntityTable
- Removed updateInEntityTable, updateInSimpleEntityTable
- Removed deleteFromEntityTable, deleteFromSimpleEntityTable
3. SqliteSchemaManager.ts - Removed ~100 lines:
- Removed ENTITY_REGISTRY fallback in ensureSchema()
- Removed migrateTableSchema() (used entityClass)
- Removed logEntitySchemas() (used ENTITY_REGISTRY directly)
Architecture is now clean:
- Daemon extracts schema from ENTITY_REGISTRY + field decorators
- Daemon passes schema to adapter via ensureSchema(collection, schema)
- Adapter caches schema, uses it for all CRUD operations
- If schema not cached → operation fails (no fallback)
This makes the adapter layer a pure SQL executor that doesn't know about entities or decorators - exactly what's needed for a Rust drop-in replacement.
DatabaseHandleRegistry now uses RustWorkerStorageAdapter exclusively: - Remove SqliteStorageAdapter import and fallback logic - Route all 'sqlite' adapter requests through Rust worker - No TypeScript SQLite path remains - Rust is the only backend Rust worker fixes (clean build, no warnings): - Remove unused Value import from archive-worker - Add #[allow(dead_code)] to planned-but-unused code: - get_row_count method (archive) - WriteOperation struct (data-daemon) - SearchAlgorithm trait methods (search) - Test structs for serde deserialization Verified: All CRUD operations, chat messages, and AI responses work through Rust-only path. Noticeably faster. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Timing Infrastructure (Rust): - Add timing.rs module with nanosecond-precision instrumentation - Track all request phases: socket_read, parse, route, query_build, lock_wait, execute, serialize, socket_write - Log to /tmp/jtag-data-daemon-timing.jsonl for analysis - Track concurrent requests and queue depth Auto-Reconnection (TypeScript): - RustWorkerStorageAdapter now auto-reconnects on worker restart - Added ensureConnected() method to all CRUD operations - Socket close handler clears state for reconnection - sendCommand() auto-reopens adapter after reconnecting Cleanup: - Delete ghost continuum.db (0-byte artifact at wrong location) Verified: All 14 personas reconnect and save memories successfully. 32 operations at 100% success rate after schema migration. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Semantic memory recall now uses Rust for vector similarity: 1. CosineAlgorithm (workers/search/src/algorithms/cosine.rs) - SIMD-friendly cosine similarity computation - L2 normalization support - Threshold filtering - Unit tests for 384-dim embeddings 2. SearchWorkerClient (workers/search/SearchWorkerClient.ts) - TypeScript client for Rust search worker - Auto-reconnect on connection loss - Singleton pattern for connection reuse 3. RustWorkerStorageAdapter.vectorSearch() - Fetches memories with embeddings from persona DB - Sends to Rust search worker for ranking - Returns top-k results above threshold Performance: ~3.7s for semantic search (includes 2s embedding generation, database query, and Rust cosine on 1000 vectors). Previously: Fell back to filter-based recall (no semantic similarity). Now: Uses actual cosine similarity via Rust worker. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Root cause: RustWorkerStorageAdapter.read() returns DataRecord where id is in the wrapper but not in data.data. Code extracting userResult.data.data as UserEntity lost the id, breaking BaseUser.id getter and causing "User has no id" errors across all widgets. Two-layer fix: 1. RustWorkerStorageAdapter.read() - ensure item.id is set before return 2. SessionDaemonServer.getUserById() - fallback to assign id if missing Verified: Tabs restored (General, Settings, Together Assistant, Browser, Canvas all visible and functional). 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Same pattern as read() fix - callers that do .data.data.id need the id in the entity data object, not just in the DataRecord wrapper. This affects data/list results, ensuring all returned entities have their id accessible via entity.id. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
…t IPC Performance benchmarks showed Rust IPC path was 6.5x SLOWER: - Before (Rust IPC): 490ms for semantic memory search - After (in-process TS): 75ms for same search Root cause: JSON serialization overhead - 10K vectors × 384 dims = 3.84M floats → ~50MB JSON text over socket - V8's JIT-compiled JavaScript is faster than IPC + JSON parse overhead The Rust search worker remains useful for BM25 text search (where text payload is much smaller than float arrays). Changes: - VectorSearchAdapterBase now uses SimilarityMetrics.cosine() directly - Removed SearchWorkerClient import for vector search path 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Move vector similarity computation ENTIRELY to Rust to avoid IPC overhead: Rust data-daemon-worker (main.rs): - Add vector/search command that reads vectors from SQLite directly - Compute cosine similarity with rayon parallel iteration - SIMD-friendly 8-way loop unrolling for auto-vectorization - Return only top-k IDs and scores (small response) - Add blob_to_f64_vec() for BLOB → f64 vector deserialization TypeScript (RustWorkerStorageAdapter.ts): - Send ONLY query vector to Rust (3KB for 384 dims) - Rust reads corpus vectors from SQLite, computes similarity - Fetch full records only for top-k returned IDs - Remove SearchWorkerClient dependency Key insight: Previous approach sent 50K × 384 floats (~50MB JSON) over IPC. New approach: Query vector (3KB) → Rust → top-k IDs and scores (tiny). Rust rayon parallelizes similarity computation across CPU cores. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The Rust data-daemon-worker now handles both storage formats: - BLOB: Binary f64 vector (optimal, used for new embeddings) - TEXT: JSON array string (legacy format in persona DBs) Fix: Try reading as BLOB first, fallback to JSON parsing if TEXT. This enables vector search on persona longterm.db databases. Performance (3,422 vectors): - Rust execute: 63ms (SQLite read + JSON parse + cosine + fetch) - Include_data: Returns full records, eliminating k IPC round trips 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Implements proper cascading state system as documented in
docs/SCOPED-STATE-ARCHITECTURE.md:Each level inherits from above with override capability.
Changes
PageStateService (new)
MainWidget
pageState.setContent()firstBaseWidget
protected pageStategetter for easy accesssubscribeToPageState()helper with auto-cleanupChatWidget
this.pageStateas primary source for roomTest plan
/chat/general- loads General room correctly/chat/academy- loads Academy room correctly/- redirects to/chat/general/persona/grok) works🤖 Generated with Claude Code