diff --git a/README.md b/README.md
index c50c62d79c3..91c32587319 100644
--- a/README.md
+++ b/README.md
@@ -1,115 +1,229 @@
-# Browser Operator [Chromium DevTools with Agentic Framework]
+# Browser Operator - Open Source Agentic Browser
-Chromium browser with an user interface to run multi-agent workflows directly on the browser using a stateful, orchestration framework.
+
+
+[](LICENSE)
+**The first open-source, privacy-friendly AI browser that transforms how you work on the web. Your intelligent partner for research, analysis, and automation - all running locally in your browser.**
+## 🚀 Download & Get Started
-## Quick Start
+**[⬇️ Download Browser Operator for macOS](https://github.com/tysonthomas9/browser-operator-devtools-frontend/releases)**
-[Download the Agentic Browser for MacOS](https://github.com/tysonthomas9/browser-operator-devtools-frontend/releases)
+Or build from source: [Developer Setup Guide](front_end/panels/ai_chat/Readme.md)
-Or
+## 🎬 See It In Action
-[Set up the chromium dev tools with Agent framework on your system](front_end/panels/ai_chat/Readme.md)
+### Deep Research & Analysis
+Watch Browser Operator synthesize information from multiple sources, creating comprehensive research reports without manual copying and pasting.
-Note: Read this document to know more about the [build](front_end/panels/ai_chat/docs/PreBuilt.md)
-## Key Capabilities
+https://github.com/user-attachments/assets/225319db-c5a0-4834-9f37-5787fb646d16
-* Built in Agent Framework for running tasks / workflows.
-* Ability to perform actions such as: Navigate URLs, Perform Actions (Clicks, Fill Form, Scroll).
-* The agent can autonmously plan and execute tasks on behalf of user, such as
- * Summarize content
- * Deep research topics
- * Literature reviews
- * Product comparisons
- * Shopping assistance
- * Advanced search
- * And many more
-* Integrates with 100+ LLM Models
- * OpenAI GPT-4.1, O4-Mini
- * Claude 4, 3.7, 3.5
- * Google Gemeni
- * Llama
- * Deepseek
- * Qwen
- * And many more
-* Integrates with LiteLLM which supports ability to use multiple providers
- * Huggingface
- * Groq
- * Azure
- * AWS
- * OpenRouter
- * vLLM
- * Ollama
- * And many more
-* Customize workflows or agent behavior with config changes.
-
+### Smart Shopping Assistant
+See how it automatically compares products, analyzes reviews, and helps you make informed purchasing decisions.
-## Demos
+https://github.com/user-attachments/assets/c478b18e-0342-400d-98ab-222c93eecd7a
-Watch Browser Operator in action with our demo videos:
+### Professional Research
+Discover how businesses use Browser Operator for talent search, competitive analysis, and market research.
-#### Deep Research
-Browser Operator seamlessly integrates public web data with your private documents and knowledge bases, creating comprehensive research without switching between tools.
+https://github.com/user-attachments/assets/90150f0e-e8c8-4b53-b6a6-c739f143f4a0
-https://github.com/user-attachments/assets/225319db-c5a0-4834-9f37-5787fb646d16
+## ✨ Key Features
+
+### 🤖 Intelligent Automation
+- **Multi-Agent Framework**: Specialized agents work together to handle complex tasks
+- **Autonomous Navigation**: Understands and interacts with any website
+- **Smart Actions**: Click, fill forms, extract data, and navigate without manual scripting
+- **Adaptive Learning**: Improves task execution based on patterns and feedback
+
+### 🔒 Privacy First (Use local LLM)
+- **Local Processing**: Your data never leaves your machine
+- **No Cloud Dependencies**: Full functionality without sending data to external servers
+- **Secure Sessions**: Works with your existing browser authentication
+- **Open Source**: Complete transparency in how your data is handled
-#### Product Discovery & Comparison
-Streamline your shopping research by automatically gathering specifications, user ratings, and availability across retailers, to help you make confident purchasing decisions.
+### 🧩 Extensible Platform
+- **100+ AI Models**: Support for OpenAI, Claude, Gemini, Llama, and more
+- **Custom Workflows**: Build your own automation sequences
+- **Plugin Architecture**: Extend functionality with custom agents
+- **API Integration**: Connect with your existing tools and services
-https://github.com/user-attachments/assets/c478b18e-0342-400d-98ab-222c93eecd7a
+## 💡 What Can You Build?
-#### Professional Talent Search
-Efficiently discover and evaluate potential candidates based on skills, experience, and portfolio quality, creating detailed profiles for recruitment decision-making.
+
+
+
-https://github.com/user-attachments/assets/90150f0e-e8c8-4b53-b6a6-c739f143f4a0
+**Personal Productivity**
+- 📚 Literature reviews and research papers
+- 🛍️ Price tracking and comparison shopping
+- 📰 News aggregation and summarization
+- 📊 Data collection and analysis
+- ✈️ Travel planning and booking research
+
+
+
+
+**Business Intelligence**
+- 🔍 Competitive analysis and monitoring
+- 👥 Talent sourcing and recruitment
+- 📈 Market research and trends
+- 🏢 Lead generation and qualification
+- 📋 Compliance and audit automation
+
+
+
+
+
+## 🛠️ Technical Architecture
+
+Browser Operator combines a Chromium-based browser with an advanced agentic framework:
+
+```
+┌─────────────────────────────────────────────────┐
+│ Browser Operator UI │
+├─────────────────────────────────────────────────┤
+│ Multi-Agent Orchestrator │
+├──────────────┬────────────────┬─────────────────┤
+│ Research │ Navigation │ Analysis │
+│ Agent │ Agent │ Agent │
+├──────────────┴────────────────┴─────────────────┤
+│ Chromium Browser Engine │
+└─────────────────────────────────────────────────┘
+```
+
+### Core Components
+- **Orchestrator Agent**: Coordinates multi-agent workflows and task distribution
+- **Navigation Engine**: Handles web interactions and page understanding
+- **Tool Registry**: Extensible system for adding new capabilities
+- **State Management**: Maintains context across complex workflows
+
+[Full Technical Documentation →](front_end/panels/ai_chat/Readme.md)
+
+## ⚙️ Quick Setup
+
+### For Users: Pre-built Application
+
+1. [Download the latest release](https://github.com/tysonthomas9/browser-operator-devtools-frontend/releases)
+2. Open Browser Operator
+3. Configure your AI provider (see below)
+4. Start automating!
+
+### For Developers: Build from Source
+
+```bash
+# Clone the repository
+git clone https://github.com/tysonthomas9/browser-operator-devtools-frontend.git
+
+# Follow the detailed build instructions
+cd browser-operator-devtools-frontend
+# See front_end/panels/ai_chat/Readme.md for complete setup
+```
-### Quick Roadmap
+### AI Provider Configuration
-|Features| Status |
-|--|--|
-| Multi-Agent Workflow | Completed (Initial Release) |
-| OpenAI LLM | Completed (Initial Release) |
-| Local LLM | Completed (May 22) |
-| MCP | Planned |
-| Customize System Prompts in UI| Planned |
-| Customize Agents in UI| Planned |
-| Customize Workflow Graphs in UI| Planned |
-| Eval Management | Planned |
-| Memory | Planned |
-| A2A Protocol | Planned |
+
+Option 1: OpenAI (Recommended for beginners)
-### DevTools Documentation
+1. Get an API key from [OpenAI Platform](https://platform.openai.com)
+2. Open Browser Operator settings
+3. Select "OpenAI" as provider
+4. Enter your API key
+5. Choose a model (GPT-4.1 recommended)
+6. Save and start using!
-- [Agentic Framework Documentation](front_end/panels/ai_chat/Readme.md)
-- [Chromium Devtools Original Documentation](https://chromium.googlesource.com/devtools/devtools-frontend/+/main/docs/README.md)
+
-### Agentic Framework Documentation
+
+Option 2: LiteLLM (For multiple providers)
-* [`front_end/panels/ai_chat/core/Readme.md`](front_end/panels/ai_chat/core/Readme.md): Explains how to customize the `BaseOrchestratorAgent` to add new top-level agent types and UI buttons, and details its graph-based workflow.
-* [`front_end/panels/ai_chat/agent_framework/Readme.md`](front_end/panels/ai_chat/agent_framework/Readme.md): Describes the AI Agent Framework, its core components (`ConfigurableAgentTool`, `AgentRunner`, `ToolRegistry`), and how to create, configure, and register new custom agents, including agent handoff mechanisms.
+Perfect for using multiple AI providers or self-hosted models:
-### Setup LiteLLM Configuration
+1. Set up your LiteLLM proxy server
+2. Select "LiteLLM Provider" in settings
+3. Enter proxy URL and API key
+4. Click "Fetch Models" to verify connection
+5. Select your preferred model
-https://github.com/user-attachments/assets/579dcfdc-71c8-4664-87b8-c2b68cc5c1ce
+[LiteLLM Setup Video →](https://github.com/user-attachments/assets/579dcfdc-71c8-4664-87b8-c2b68cc5c1ce)
-1. Click on the setting config inside the chat panel
-2. Select LiteLLM Provider
-3. Input the LiteLLM URL and API key
-4. Click on fetch models to test the configuration
-5. Click save to update the configuration
+
+
+
+Option 3: Local Models (Maximum privacy)
+
+Run completely offline with Ollama:
+
+1. Install Ollama on your system
+2. Pull your preferred model (e.g., `ollama pull llama3`)
+3. Configure Browser Operator to use local endpoint
+4. Enjoy private, offline automation
+
+
+
+## 🗺️ Roadmap
+
+### ✅ Released
+- Multi-agent workflow engine
+- Support for 100+ AI models
+- macOS application
+- Core automation capabilities
+
+### 🚧 In Development
+- Windows and Linux support
+- Enhanced memory system
+- Custom agent builder
+
+### 🔮 Planned Features
+- MCP (Model Context Protocol) support
+- Visual workflow designer
+- Team collaboration features
+- Advanced scheduling system
+
+## 👥 Community & Support
+
+### Get Help
+- 📖 [Documentation](front_end/panels/ai_chat/Readme.md)
+- 💬 [Discord Community](https://discord.gg/fp7ryHYBSY)
+- 🐛 [Report Issues](https://github.com/tysonthomas9/browser-operator-devtools-frontend/issues)
+- 🐦 [Follow Updates](https://x.com/BrowserOperator)
### Contributing
-Found a bug 🐛 or have a feature idea ✨? Please create issues [here](https://github.com/tysonthomas9/browser-operator-devtools-frontend/issues)
+We welcome contributions! Here's how you can help:
+
+- **🐛 Report Bugs**: Help us identify and fix issues
+- **✨ Request Features**: Share your ideas for new capabilities
+- **📝 Improve Docs**: Help others get started
+- **💻 Submit PRs**: Contribute code improvements
+
+See our [Contributing Guide](CONTRIBUTING.md) for details.
+
+## 📚 Documentation
+
+- [Getting Started Guide](front_end/panels/ai_chat/docs/GettingStarted.md)
+- [Agent Framework](front_end/panels/ai_chat/agent_framework/Readme.md)
+- [Creating Custom Agents](front_end/panels/ai_chat/core/Readme.md)
+- [Architecture Overview](front_end/panels/ai_chat/docs/Architecture.md)
+- [Build Instructions](front_end/panels/ai_chat/docs/PreBuilt.md)
+
+## 🙏 Acknowledgments
+
+Browser Operator is built on top of Chromium and integrates with numerous open-source projects. Special thanks to all contributors and the open-source community.
+
+## 📄 License
+
+Browser Operator is released under the [BSD-3-Clause License](LICENSE).
+
+---
-### Join Us
+
-If you like this project, don't hesitate to ⭐ star this repository. For those who'd like to contribute code or just hang out with the community please join our Discord.
+**⭐ Star this repo to support open-source AI development!**
-[](https://discord.gg/JKYuuubr)
-[-000000?style=for-the-badge&logo=x&logoColor=white)](https://x.com/BrowserOperator)
+
\ No newline at end of file
diff --git a/front_end/panels/ai_chat/ui/ChatView.ts b/front_end/panels/ai_chat/ui/ChatView.ts
index c4cf9606c1a..988e3cff019 100644
--- a/front_end/panels/ai_chat/ui/ChatView.ts
+++ b/front_end/panels/ai_chat/ui/ChatView.ts
@@ -269,6 +269,9 @@ export class ChatView extends HTMLElement {
#isInputDisabled = false;
#inputPlaceholder = '';
+ // Add state tracking for AI Assistant operations
+ #aiAssistantStates = new Map();
+ #lastProcessedMessageKey: string | null = null;
connectedCallback(): void {
const sheet = new CSSStyleSheet();
@@ -289,6 +292,10 @@ export class ChatView extends HTMLElement {
disconnectedCallback(): void {
// Cleanup resize observer
this.#messagesContainerResizeObserver.disconnect();
+
+ // Clear state maps to prevent memory leaks
+ this.#aiAssistantStates.clear();
+ this.#lastProcessedMessageKey = null;
}
// Add method to scroll to bottom
@@ -345,6 +352,98 @@ export class ChatView extends HTMLElement {
}
};
+ // Helper methods for AI Assistant state management
+ #getMessageStateKey(structuredResponse: {reasoning: string, markdownReport: string}): string {
+ // Create stable hash from content - Unicode safe
+ const content = structuredResponse.reasoning + structuredResponse.markdownReport;
+
+ // Unicode-safe hash function using TextEncoder
+ const encoder = new TextEncoder();
+ const bytes = encoder.encode(content);
+
+ let hash = 0;
+ for (let i = 0; i < bytes.length; i++) {
+ hash = ((hash << 5) - hash) + bytes[i];
+ hash = hash & hash; // Convert to 32-bit integer
+ }
+
+ // Convert to hex for consistent 8-character length
+ const key = Math.abs(hash).toString(16).padStart(8, '0');
+
+ return key;
+ }
+
+ #getMessageAIAssistantState(messageKey: string): 'pending' | 'opened' | 'failed' | 'not-attempted' {
+ return this.#aiAssistantStates.get(messageKey) || 'not-attempted';
+ }
+
+ #setMessageAIAssistantState(messageKey: string, state: 'pending' | 'opened' | 'failed'): void {
+ this.#aiAssistantStates.set(messageKey, state);
+ }
+
+
+ #isLastStructuredMessage(currentCombinedIndex: number): boolean {
+ // We need to work with the combined messages logic to properly identify the last structured message
+ // The currentCombinedIndex is from the combined array, but we need to check against the original array
+
+ // Recreate the combined messages logic to understand the mapping
+ let combinedIndex = 0;
+ let lastStructuredCombinedIndex = -1;
+
+ for (let originalIndex = 0; originalIndex < this.#messages.length; originalIndex++) {
+ const message = this.#messages[originalIndex];
+
+ // Keep User messages and Final Model answers
+ if (message.entity === ChatMessageEntity.USER ||
+ (message.entity === ChatMessageEntity.MODEL && message.action === 'final')) {
+
+ // Check if this is a structured final answer
+ if (message.entity === ChatMessageEntity.MODEL && message.action === 'final') {
+ const structuredResponse = this.#parseStructuredResponse((message as any).answer || '');
+ if (structuredResponse) {
+ lastStructuredCombinedIndex = combinedIndex;
+ }
+ }
+
+ combinedIndex++;
+ continue;
+ }
+
+ // Handle Model Tool Call message
+ if (message.entity === ChatMessageEntity.MODEL && message.action === 'tool') {
+ const nextMessage = this.#messages[originalIndex + 1];
+
+ // Check if the next message is the corresponding result
+ if (nextMessage && nextMessage.entity === ChatMessageEntity.TOOL_RESULT && nextMessage.toolName === (message as any).toolName) {
+ // Combined representation: tool call + result = 1 entry in combined array
+ combinedIndex++;
+ } else {
+ // Tool call is still running (no result yet)
+ combinedIndex++;
+ }
+ continue;
+ }
+
+ // Handle Tool Result message - skip if it was combined previously
+ if (message.entity === ChatMessageEntity.TOOL_RESULT) {
+ const prevMessage = this.#messages[originalIndex - 1];
+ // Check if the previous message was the corresponding model call
+ if (!(prevMessage && prevMessage.entity === ChatMessageEntity.MODEL && prevMessage.action === 'tool' && prevMessage.toolName === (message as any).toolName)) {
+ // Orphaned tool result - add it directly
+ combinedIndex++;
+ }
+ // Otherwise, it was handled by the MODEL case above, so we skip this result message
+ continue;
+ }
+
+ // Fallback for any unexpected message types
+ combinedIndex++;
+ }
+
+ return lastStructuredCombinedIndex === currentCombinedIndex;
+ }
+
+
// Update the prompt button click handler when props/state changes
#updatePromptButtonClickHandler(): void {
this.#handlePromptButtonClickBound = BaseOrchestratorAgent.createAgentTypeSelectionHandler(
@@ -417,6 +516,34 @@ export class ChatView extends HTMLElement {
const willHaveMoreMessages = data.messages?.length > previousMessageCount;
const wasInputDisabled = this.#isInputDisabled;
+ // Handle AI Assistant state cleanup for last-message-only approach
+ if (willHaveMoreMessages && this.#messages) {
+ // When new messages are added, reset states for previous final messages
+ // so that only the last message can attempt to open AI Assistant
+ const previousLastFinalIndex = this.#messages.findLastIndex(msg =>
+ msg.entity === ChatMessageEntity.MODEL &&
+ (msg as ModelChatMessage).action === 'final'
+ );
+
+ if (previousLastFinalIndex >= 0) {
+ const previousLastMessage = this.#messages[previousLastFinalIndex] as ModelChatMessage;
+ if (previousLastMessage.answer) {
+ const structuredResponse = this.#parseStructuredResponse(previousLastMessage.answer);
+ if (structuredResponse) {
+ const messageKey = this.#getMessageStateKey(structuredResponse);
+ const currentState = this.#getMessageAIAssistantState(messageKey);
+
+ // If the previous last message was pending, mark it as failed
+ // But keep 'opened' state to preserve successfully opened reports
+ if (currentState === 'pending') {
+ this.#setMessageAIAssistantState(messageKey, 'failed');
+ }
+ // If it was 'opened', keep it that way to show button only
+ }
+ }
+ }
+ }
+
this.#messages = data.messages;
this.#state = data.state;
this.#isTextInputEmpty = data.isTextInputEmpty;
@@ -473,11 +600,6 @@ export class ChatView extends HTMLElement {
#handleSendMessage(): void {
// Check if textInputElement, onSendMessage callback, or input is disabled
if (!this.#textInputElement || !this.#onSendMessage || this.#isInputDisabled) {
- logger.info("Send prevented: ", {
- hasTextInput: Boolean(this.#textInputElement),
- hasCallback: Boolean(this.#onSendMessage),
- isDisabled: this.#isInputDisabled
- });
return;
}
@@ -492,7 +614,6 @@ export class ChatView extends HTMLElement {
// Always scroll to bottom after sending message
this.#pinScrollToBottom = true;
- logger.info("Sending message:", text);
this.#onSendMessage(text, this.#imageInput);
this.#textInputElement.value = '';
this.#textInputElement.style.height = 'auto';
@@ -511,12 +632,20 @@ export class ChatView extends HTMLElement {
const textarea = event.target as HTMLTextAreaElement;
textarea.style.height = 'auto'; // Reset height to shrink if needed
textarea.style.height = `${textarea.scrollHeight}px`;
- this.#isTextInputEmpty = textarea.value.trim().length === 0;
- void ComponentHelpers.ScheduledRender.scheduleRender(this, this.#boundRender);
+
+ const newIsEmpty = textarea.value.trim().length === 0;
+
+ // Only trigger re-render if empty state actually changed
+ if (this.#isTextInputEmpty !== newIsEmpty) {
+ this.#isTextInputEmpty = newIsEmpty;
+ void ComponentHelpers.ScheduledRender.scheduleRender(this, this.#boundRender);
+ } else {
+ this.#isTextInputEmpty = newIsEmpty;
+ }
}
// Render messages based on the combined structure
- #renderMessage(message: ChatMessage | (ModelChatMessage & { resultText?: string, isError?: boolean, resultError?: string, combined?: boolean }) | (ToolResultMessage & { orphaned?: boolean }), index?: number ): Lit.TemplateResult {
+ #renderMessage(message: ChatMessage | (ModelChatMessage & { resultText?: string, isError?: boolean, resultError?: string, combined?: boolean }) | (ToolResultMessage & { orphaned?: boolean }), combinedIndex?: number ): Lit.TemplateResult {
try {
switch (message.entity) {
case ChatMessageEntity.USER:
@@ -534,7 +663,6 @@ export class ChatView extends HTMLElement {
{
const toolResultMessage = message as (ToolResultMessage & { orphaned?: boolean });
if (toolResultMessage.orphaned) {
- logger.warn('Rendering orphaned ToolResultMessage:', toolResultMessage);
return html`
@@ -568,7 +696,7 @@ export class ChatView extends HTMLElement {
const structuredResponse = this.#parseStructuredResponse(modelMessage.answer || '');
if (structuredResponse) {
- return this.#renderStructuredResponse(structuredResponse, index);
+ return this.#renderStructuredResponse(structuredResponse, combinedIndex);
} else {
// Regular response - use the old logic
const isDeepResearch = isDeepResearchContent(modelMessage.answer || '');
@@ -583,7 +711,7 @@ export class ChatView extends HTMLElement {
@@ -621,33 +749,10 @@ export class ChatView extends HTMLElement {
const toolArgs = modelMessage.toolArgs || {};
const filteredArgs = Object.fromEntries(Object.entries(toolArgs).filter(([key]) => key !== 'reasoning'));
- // --- Styling and Icons ---
- const blockStyles = (bgColor: string) => Lit.Directives.styleMap({
- padding: '10px',
- borderRadius: '8px',
- marginBottom: '5px', // Reduced margin between blocks if they are separate
- border: '1px solid var(--sys-color-outline)',
- backgroundColor: bgColor,
- });
-
- const headerStyles = Lit.Directives.styleMap({
- display: 'flex',
- alignItems: 'center',
- gap: '8px',
- marginBottom: '8px',
- fontWeight: '500',
- });
-
- const contentStyles = Lit.Directives.styleMap({
- marginLeft: '24px', // Indent content under the header icon
- paddingTop: '5px',
- });
-
- // Icons
+ // Icons for tool status
const spinnerIcon = html``;
const checkIcon = html``;
const errorIcon = html``;
- // --- End Styling and Icons ---
return html`
@@ -728,7 +833,6 @@ export class ChatView extends HTMLElement {
}
default:
// Should not happen, but render a fallback
- logger.warn('Unhandled message entity type in renderMessage:', (message as any).entity);
return html`
Unknown message type
`;
}
} catch (error) {
@@ -789,9 +893,9 @@ export class ChatView extends HTMLElement {
const prevMessage = allMessages[index - 1];
// Check if the previous message was the corresponding model call
if (!(prevMessage && prevMessage.entity === ChatMessageEntity.MODEL && prevMessage.action === 'tool' && prevMessage.toolName === message.toolName)) {
- // Orphaned tool result - add it directly (maybe mark it?)
+ // Orphaned tool result - add it directly
logger.warn('Orphaned tool result found:', message);
- acc.push({...message, orphaned: true }); // Add marker if needed for rendering
+ acc.push({...message, orphaned: true }); // Add marker for rendering
}
// Otherwise, it was handled by the MODEL case above, so we skip this result message
return acc;
@@ -805,6 +909,7 @@ export class ChatView extends HTMLElement {
// Allow ToolResultMessage to potentially have an 'orphaned' flag
}, [] as Array);
+
// General loading state (before any model response or after tool result)
const showGeneralLoading = this.#state === State.LOADING && !isModelRunningTool;
@@ -912,7 +1017,7 @@ export class ChatView extends HTMLElement {