Skip to content

Releases: GoldenLoaf24h/browserclaw

v2.9.1 - Hardened Deep Shadow DOM Piercing, Contextual Grep & Visual Drift Compensation

Choose a tag to compare

@GoldenLoaf24h GoldenLoaf24h released this 19 Sep 18:13

BrowserClaw v2.9.1 — Hardened Deep Shadow DOM Piercing, Contextual Grep & Visual Drift Compensation

TL;DR: Version 2.9.1 delivers production-grade hardening for deep multi-level Shadow DOM piercing (tested on complex Web Components like Reddit Shreddit / Faceplate), contextual Grep matching snippets, zero-drift visual fallback coordinate alignment with real-time scroll offset compensation, and unified documentation for all 48 canonical tools.

Key Architectural Fixes & Enhancements

1. Deep Shadow DOM Piercing & Web Component Accessibility Recovery

  • Multi-Level Shadow Root Descent (deepElementFromPoint): Descends through nested open and closed Shadow DOM boundaries across arbitrary depths (e.g. <shreddit-comment> -> <faceplate-tracker> -> <button>), ensuring pointer hit tests resolve directly to the active interactive target rather than the outer custom element host.
  • Accessible Name & Semantic Recovery: Enhanced extractCleanElementText to resolve aria-labelledby, aria-label, title, aria-description, inner <svg><title>, svg[aria-label], and parent custom element wrapper attributes (data-action, action), completely eliminating false positive decorative icon button pruning.
  • Pointer-Events Gating Fix: Removed container-level pointer-events: none early-exit that previously halted traversal into child elements that re-enable pointer-events: auto.
  • Closed Shadow Host Strategy: Candidate custom elements without accessible open shadow roots are tagged with isClosedShadowHost and rendered with [closed-shadow-host] in both DOM tree and compact perception formats, routing click interactions to the host bounding box where native CDP events bubble via composed: true.

2. Deep Text Extraction & Contextual Snippets in chrome_grep

  • Contextual Centered Match Snippets: Fixed chrome_grep in page_text mode to extract centered contextual windows (...prefix [match] suffix...) around the matched substring instead of slicing from the beginning of the line.
  • Deep Shadow Text Traversal (inPageExtractDeepPageText): Recursively extracts deep text across light DOM and shadow roots, resolving <slot> projections via assignedNodes({ flatten: true }) and including accessible [aria-label] attributes.
  • Automatic Fallback for Interactive Queries: When searchType: 'interactive_only' yields 0 matches, chrome_grep automatically falls back to searching deep page text, returning matching text snippets with guidance on coordinate interaction.
  • Extended Grep Key Coverage: Added matching for data-testid, data-action, aria-description, and data-click-id.

3. Visual Fallback Coordinate Alignment & Drift Elimination

  • Document-Space Preservation: scaleCoordinates preserves absolute document space (isDocumentSpace: true) when processing fullpage screenshots (captureMode: 'fullpage'), avoiding coordinate compression into the 800px viewport.
  • Real-Time Scroll Drift Compensation (alignVisualCoordinate): Converts visual targets to absolute document space and applies real-time scroll offset correction. If the target has scrolled out of view, BrowserClaw instantly auto-scrolls (inPageInstantScrollTo) to center the target before CDP click dispatch.
  • Scroll Racing Lock (inPageLockScroll): Locks smooth scrolling to auto during CDP mouse press/release bursts, eliminating race conditions with inertia scrolling.
  • Magnetic Snapping Edge-Case Fix: inPageSnapCoordinate auto-scrolls clipped or edge-straddling elements comfortably into view before computing the safe click center.

4. Full-Spectrum Docs & Skill Parity

  • Re-verified all 48 canonical tools (including chrome_insert_media) across all documentation and references.
  • Synchronized all 6 skill directories via scripts/sync-skills.mjs with verified byte-for-byte SHA-256 equivalence.

Verification Record

  • TypeScript Compilation: 0 errors across monorepo (pnpm typecheck).
  • Chrome Extension Tests: 44/44 test files passed, 344/344 tests passed (vitest).
  • Native Bridge Tests: 4/4 test suites passed, 89/89 tests passed (jest).
  • End-to-End Tests: 4/4 tiers passed, 153/153 tests passed (runner.ts).
  • Total Automated Tests: 586/586 tests passing with 100% compliance.

Release Assets

  • browserclaw-extension-v2.9.1.zip — Production Chrome MV3 extension package
  • browserclaw-skill-v2.9.1.zip — Complete Agent Skill pack with reference guides

v2.9.0 - Deep Shadow DOM Piercing, Visual Coordinate Drift Compensation & 48-Tool Catalog

Choose a tag to compare

@GoldenLoaf24h GoldenLoaf24h released this 19 Sep 17:30

BrowserClaw v2.9.0 — Deep Shadow DOM Piercing, Visual Coordinate Drift Compensation & 48-Tool Catalog

TL;DR: Version 2.9.0 brings architectural solutions to two core automation challenges identified in complex modern Web Components (e.g. Reddit Shreddit, YouTube, X): Deep Shadow DOM penetration with semantic accessibility property extraction, and zero-drift visual fallback coordinate alignment with real-time scroll offset compensation. Additionally, all project documentation, skills, and schemas have been unified to 48 canonical tools.

Key Enhancements

1. Deep Shadow DOM Piercing & Web Component Accessibility Recovery

  • Multi-Layer Shadow DOM Traversal: Implemented recursive querySelectorAllDeep and querySelectorDeep functions that penetrate arbitrary depths of open and closed Shadow DOM roots (including <shreddit-comment>, <faceplate-tracker>, <faceplate-button>), plus flattened <slot> element resolution via assignedElements({ flatten: true }).
  • Semantic Name Recovery for Icon-Only Buttons: Enhanced extractCleanElementText to recover accessible text from aria-label, title, aria-description, and inner <svg><title> or svg[aria-label], completely preventing interactive icon buttons from being pruned as decorative noise.
  • Closed Shadow Host Candidate Discovery: Detects custom elements with closed/null shadow roots (isCustomElement(node) && getShadowRoot(node) === null), retains them as indexed candidates, and leverages CDP native event dispatch at the host's bounding box where events bubble up through composed: true.
  • Composed Tree Ancestry in Hit Testing: Exported composedParent and composedContains helpers, ensuring occlusion grid testing and magnetic snapping correctly traverse across shadow boundaries without misreporting shadow children as occluded by their own host elements.
  • Deep Text Search in chrome_grep: Recursive shadow tree text extraction in page_text and interactive modes, expanding query matches across title, id, and name attributes.

2. Visual Fallback Coordinate Alignment & Real-Time Scroll Drift Compensation

  • True Document Space for Full-Page Screenshots: Added isDocumentSpace to ScreenshotContext. Fullpage screenshots (captureMode: 'fullpage') now map coordinates directly to absolute document space without compressing long pages into the 800px viewport.
  • Dynamic Scroll Delta Compensation (alignVisualCoordinate): Click execution measures page scroll position in real time via inPageGetScrollState and compensates for any scroll movement occurring during the network RTT window between screenshot capture and click execution.
  • Automatic Scroll Centering: Document-space coordinates automatically trigger instant auto-scroll (inPageInstantScrollTo) to center the target coordinate in the viewport before dispatching physical CDP mouse events.
  • Scroll Racing Lock (inPageLockScroll): Temporarily disables smooth/inertia scrolling during CDP mouse down/up sequences to eliminate race conditions between page motion and click delivery.

3. Full-Spectrum Project Documentation & 48 Canonical Tools Unification

  • Updated all project documentation (README.md, README.zh-CN.md, PROJECT.md, PROJECT.zh-CN.md, docs/MAP.md, docs/TOOLS.md, docs/mcp-cli-config.md, docs/TROUBLESHOOTING.zh-CN.md, GitHub repository description, and extension manifests) to reflect all 48 canonical tools (including chrome_insert_media).
  • Executed scripts/sync-skills.mjs and verified byte-for-byte SHA-256 equivalence across all 6 skill directories.

Test & Verification Record

  • TypeScript Typecheck: Zero errors across monorepo (pnpm typecheck).
  • Chrome Extension Vitest Suite: 44/44 test files passed, 336/336 tests passed (100% compliance, including new deep-shadow-and-visual-drift.test.ts).
  • Native Bridge Jest Suite: 4/4 test suites passed, 89/89 tests passed (100% compliance).
  • End-to-End Suite: 4/4 tiers passed, 153/153 tests passed (100% compliance).
  • Total Tests Passed: 578 automated tests passing across all layers.

Release Assets

  • browserclaw-extension-v2.9.0.zip — Production Chrome MV3 extension package
  • browserclaw-skill-v2.9.0.zip — Complete Agent Skill pack with reference guides

v2.8.3 - Priority Scroll, Safety Breakpoints, Multiline Preservation & Media Injection

Choose a tag to compare

@GoldenLoaf24h GoldenLoaf24h released this 19 Sep 15:13

BrowserClaw v2.8.3 — Priority Scroll Engine, Safety Breakpoints, Multiline Preservation & Media Injection

TL;DR: Version 2.8.3 addresses critical user-testing feedback and production edge cases in real-world agent automation (e.g. Reddit, X/Twitter, complex rich-text editors), introducing an intelligent Priority Scroll Engine that eliminates sidebar hijacking, fine-grained Safety Breakpoints in System 1 autonomous loops, robust multiline newline preservation for modern rich-text composers, and native high-resolution media/diagram injection (chrome_insert_media) up to 50MB.

Key Enhancements

1. Priority Scroll Engine in chrome_smart_scroll (Sidebar Hijacking Fix)

  • Problem: When attempting to scroll through main feeds or comment sections (e.g. Reddit), narrow sidebars (such as reddit-sidebar-nav) were frequently misidentified as the scroll target and scrolled to 100% bottom while the primary content remained motionless.
  • Solution: Implemented an intelligent multi-signal scoring algorithm in inPageFindSmartScrollTarget:
    • Horizontal Center-Proximity Weight: Prioritizes containers positioned along the central reading axis of the viewport.
    • Sidebar Penalty: Heavily penalizes narrow vertical columns (width < 300px) and semantic navigation tags (<nav>, <aside>, role="navigation").
    • Semantic Main Boost: Adds positive weight to <main>, <article>, and role="main" containers.
    • Center Probe Fallback: Probes document.elementFromPoint(window.innerWidth / 2, window.innerHeight / 2) to discover scrollable containers directly under the central viewing area.
    • Default to Root Window: Falls back smoothly to window scroll (window.scrollBy) if no main container meets scrollability thresholds.

2. Safety Breakpoints in chrome_act_toward_goal (pauseBeforeKeywords: string[])

  • Problem: Users need System 1 (Jev/heuristic) to autonomously navigate forms, select tags, and type draft text without human intervention, but require the agent to stop immediately before triggering irreversible commit actions (e.g. "Post", "Submit", "Pay", "Delete").
  • Solution: Added pauseBeforeKeywords?: string[] to chrome_act_toward_goal:
    • When the next intended element matches any keyword in pauseBeforeKeywords, execution halts prior to action dispatch.
    • Returns status: "paused", pausedBeforeAction: { action, target: { index, text, role } }, and fresh currentElements.
    • Prioritized ahead of destructive escalations, enabling System 2 (Macro Supervisor) or the human user to review draft state and commit via chrome_interact_index with zero DOM re-reads.

3. Multiline Newline Preservation in chrome_fill_index

  • Problem: Modern rich-text composers and frameworks (Draft.js, Lexical, ProseMirror, Slate, Quill, CodeMirror) frequently collapse \n characters into single spaces or ignore them when filled via standard value assignment or single input events.
  • Solution:
    • Detects multiline input strings containing \n.
    • Splits text by paragraphs and dispatches physical CDP Enter events (rawKeyDown + keyUp, keyCode 13) between segments.
    • Dispatches beforeinput events with inputType: 'insertParagraph' and in-page DOM paragraph mutations.
    • Normalized newline verification in inPageVerifyInputCommitment to guarantee robust commit detection across both plain textareas and rich-text nodes.

4. Native Asset & Image Injection (chrome_insert_media — 48 Canonical Tools)

  • New Tool: Introduced canonical tool #48: chrome_insert_media (INSERT_MEDIA).
  • Input Sources: Supports local disk file paths (filePath), local file URLs (fileUrl), remote URLs (mediaUrl), and raw base64 payloads (base64Data).
  • High-Capacity Streaming: Native Server streams files larger than 650KB (up to 50MB, including high-res 4K diagrams and screen captures) via an in-memory loopback HTTP endpoint (/media-asset/:assetId), cleanly bypassing the 1MB Chrome Native Messaging IPC ceiling.
  • Native DOM Event Synthesis: Synthesizes authentic ClipboardEvent('paste') and DragEvent('drop') containing native File and DataTransfer objects, seamlessly accepted by drag-and-drop zones, file paste handlers, and image upload dropboxes.

5. Skill & Documentation Synchronization

  • Synchronized tool documentation across all repositories and guides to reflect 48 canonical tools.
  • Updated skill/SKILL.md, skill/references/dual-brain-jev.md, and all mirror skill registries (~/.gemini/config/skills/browserclaw/, ~/.gemini/config/skills/mcp-chrome/).
  • Updated .browserclaw-managed.json content hashes.

Test & Verification Record

  • TypeScript Typecheck: Zero errors across monorepo (pnpm typecheck).
  • Chrome Extension Test Suite: 43/43 test files passed, 325/325 tests passed (vitest run, including new comprehensive tests for priority scroll, multiline newline preservation, and media injection).
  • Native Bridge Jest Suite: 4/4 test suites passed, 89/89 tests passed.
  • E2E Test Runner: 4/4 tiers passed, 153/153 tests passed (100% compliance).

Release Assets

  • browserclaw-extension-v2.8.3.zip — Production Chrome MV3 extension package
  • browserclaw-skill-v2.8.3.zip — Complete Agent Skill pack with reference guides

BrowserClaw v2.8.2 - Progressive Disclosure Skill Architecture & Dual-Brain Optimization

Choose a tag to compare

@GoldenLoaf24h GoldenLoaf24h released this 19 Sep 11:45

BrowserClaw v2.8.2 — Progressive Disclosure Skill Architecture & Dual-Brain Optimization

TL;DR: Version 2.8.2 delivers a major architectural refactoring and deep optimization of the Agent Skill system, establishing the Hierarchical Dual-Brain execution workflow (Macro Planner System 2 + Fast Semantic Micro-Loop System 1 Jev) with an industry-standard Progressive Disclosure architecture (~2,000 tokens), zero-redundancy trigger matrix, complete elimination of ghost parameters, and byte-level synchronization across all skill mirrors.

Key Enhancements

1. Progressive Disclosure Agent Skill Architecture

  • High Information Density: Main SKILL.md (~2,000 tokens) provides complete coverage of trigger scenarios, tool selection matrix, explicit input/output contracts, and hard operational constraints without attention dilution or token bloat.
  • Hierarchical Dual-Brain Execution:
    • Macro Planner (System 2 / Caller LLM): Focuses on high-level goal formulation, multi-page strategy, navigation, and supervisory escalation handling.
    • Semantic Micro-Loop (System 1 / Fast Brain / Jev): Operates locally in Native Server at 200–400ms/step via chrome_act_toward_goal, eliminating 80%+ remote network roundtrips.
  • Explicit 5-Step Execution Workflow:
    • Step 1: Navigate & Orient (chrome_navigate, chrome_get_markdown)
    • Step 2: Delegate Sub-Goal to System 1 (chrome_act_toward_goal)
    • Step 3: Handle Status (done vs. escalate vs. stuck vs. blocked)
    • Step 4: Precision Macro Intervention & Recovery (Zero-Read candidate reuse from currentElements)
    • Step 5: Verify & Stale Index Recovery (includeDelta: true, mutated, STALE_ELEMENT_INDEX)
  • 6-Tier Routing Hierarchy:
    • Tier 1: Semantic Micro-Loop (chrome_act_toward_goal — default 70% for on-page action goals)
    • Tier 0: Deterministic Primitives (chrome_get_markdown, chrome_batch_actions, chrome_interact_index, chrome_fill_index — 20%)
    • Tier 2: In-Page Scripting & API Bypass (chrome_javascript, chrome_network_request — 5%)
    • Tier 3: Visual Fallback (chrome_screenshot, chrome_computer — 3%)
    • Tier 4: Human Handoff (chrome_request_human_intervention — 1%)
    • Tier 5: Raw CDP Escape Hatch (chrome_cdp_execute — <0.1%)
  • Specialized Reference Guides (skill/references/):
    • references/dual-brain-jev.md: Local Jev System 1 scoring contracts, heuristic fallback, escalation guards, and Macro Supervisor Recovery Protocol.
    • references/batch-pipeline.md: Atomic interaction pipelines, assertions (assert), extractions (extract), inline network capture, and autonomous wizard filling (chrome_form_pipeline).
    • references/visual-fallback.md: DPR 1:1 viewport normalization, calibrated grid rulers, Set-of-Mark 2.0, GoFullPage captures, and PCIE polymorphic coordinates.
    • config/TROUBLESHOOTING.md: Port conflicts, token authorization, and automated self-repair scripts.

2. Strict Operational Constraints & Parameter Integrity

  • 1-Based Index Integrity: Strict adherence to 1-based indices ([1], [2], [3]) from chrome_read_dom and currentElements.
  • Parameter Invariants: Complete elimination of ghost parameters; exact schema alignment (index, grid: true, action: 'accept', code: in chrome_javascript, coordinates in chrome_computer, query in chrome_form_pipeline).
  • Background Tab Isolation: All background tabs enforce background: true (creating tabs with active: false and windows with focused: false) to safeguard user foreground focus.
  • Native Event Fidelity: Dispatches physical CDP events (isTrusted: true) natively compatible with modern web frameworks (React 18/19, Vue, Angular, Shadow DOM).
  • Active Tab Closure Protection: chrome_close_tabs requires confirm: true or explicit IDs.
  • Zero-RTT Commits: pressEnter: true on chrome_fill_index enables 1-turn search and submission.

3. Complete Multi-Platform Mirror Synchronization

  • Byte-for-byte exact synchronization across all authoritative repositories and user configuration registries:
    • skill/
    • plugins/browserclaw/skills/browserclaw/
    • Production distribution mirror (D:\workspace\browserclaw\)
    • Global Agent skill registries (~/.gemini/config/skills/browserclaw/, ~/.gemini/config/skills/mcp-chrome/)

Test & Quality Verification

  • TypeScript Typecheck: 0 errors across all monorepo packages (pnpm typecheck)
  • Vitest Extension Suite: 42/42 test files passed, 320/320 tests passed (vitest run, +14 schema guard tests)
  • Native Bridge Jest Suite: 4/4 test suites passed, 86/86 tests passed
  • E2E Test Suites: 153/153 tests passed (100% compliance across Tier 1–4 suites)
  • Monorepo Build: Complete clean production build for packages/shared, app/chrome-extension, and app/native-server

Release Assets

  • browserclaw-extension-v2.8.2.zip — Production Chrome MV3 extension build
  • browserclaw-skill-v2.8.2.zip — Optimized Progressive Disclosure Agent Skill pack with reference guides

BrowserClaw v2.8.1 - Comprehensive Audit Hardening, Jev Optimization & Hermes Bridge Token

Choose a tag to compare

@GoldenLoaf24h GoldenLoaf24h released this 19 Sep 10:23

BrowserClaw v2.8.1 — Comprehensive Audit Hardening, Jev Optimization & Hermes Bridge Token

TL;DR: Version 2.8.1 delivers an exhaustive audit hardening across the entire BrowserClaw stack, deep semantic optimizations to the Jev System One micro-loop, official integration of Hermes bridge authentication tokens (PR #2), and 100% synchronized skills and documentation.

Highlights

1. Core Extension & API Hardening

  • parseOutcome Structured Error Unwrapping: Fixed execution edge cases where tool execution failures ({ success: false, error: ... }) were improperly unwrapped in nested error paths.
  • intercept-api Glob Boundary Strictness: Tightened path glob and wildcard matching to eliminate false-positive URL route intercepts across complex single-page apps.
  • Favicon Cache Isolation: Guarded tab-favicon cache from malformed data URI / non-HTTP self-poisoning scenarios with robust lifecycle cleanups.
  • Diagnostic Doctor Enhancements: Streamlined chrome_doctor diagnostics across Windows and Unix native-messaging paths.

2. Jev Dual-Brain Semantic Engine Enhancements

  • Multi-Quoted Placeholder & Token Matching: Enhanced heuristic-engine with resilient matching across English and Chinese quotation marks ("...", '...', “...”, ‘...’) and complex button labels.
  • Adaptive Stuck Detection: fast-decision-engine dynamically tracks state hashes and action repetition, escalating intelligently with prefetched DOM rather than burning retry quota on unclickable elements.
  • Grounding & Candidate Deduplication: jev-client prevents duplicate candidate indices in prompts and enforces strict none semantic grounding when target elements are absent from the active DOM.

3. Hermes Native Bridge Token Support (PR #2)

  • Merged upstream PR #2 by @teknium1.
  • Hermes MCP client plugin now provides seamless Bearer / bridge token authentication support, securing remote and multi-tenant native bridge configurations.
  • Integrated automated test suite (plugins/browserclaw/tests/test_bridge_token.py) verifying token dispatch and fallback behaviors.

4. Documentation & Skill Parity

  • Byte-for-byte synchronization across all 4 skill mirrors (skill/SKILL.md, plugins/browserclaw/skills/browserclaw/SKILL.md, user config skills, and release mirror).
  • Complete reference updates for the 47 canonical MCP tools, dual-brain fallback ladder, and in-page execution helpers.

Test & Quality Verification

  • Jest (Native Server): 86/86 tests passed (including Jev fast decision, heuristic, and client suites)
  • Vitest (Chrome Extension): 306/306 tests passed across 42 suites
  • E2E Compliance: 153/153 tests passed
  • Pytest (Hermes Bridge Token): 8/8 tests passed
  • TypeScript Typecheck: 0 errors across all monorepo packages

Release Assets

  • browserclaw-extension-v2.8.1.zip — Chrome MV3 production extension (unpacked / side-load)
  • browserclaw-skill-v2.8.1.zip — Complete agent skill pack with 47-tool contract, dual-brain routing, and recipes

Getting Started & Upgrading

For existing installations, pull the latest release, unpack browserclaw-extension-v2.8.1.zip into your Chrome extensions directory, and reload via chrome://extensions. If using the Jev System One micro-loop, ensure TYPESAFE_API_KEY is set in your environment.

BrowserClaw v2.8.0 - Hierarchical Dual-Brain with Jev Semantic Micro-Loop

Choose a tag to compare

@GoldenLoaf24h GoldenLoaf24h released this 19 Sep 04:34

BrowserClaw v2.8.0 — Hierarchical Dual-Brain with Jev Semantic Micro-Loop

TL;DR: Agent 浏览器操控的「感知→决策→动作」微闭环从 3 次 MCP 往返 + 远端大模型推理(6~10s/步)压缩到 Native Server 本地循环(200400ms/步)。宏观规划仍归大模型,微观执行交给快速决策引擎。

Highlights

  • 47th canonical tool chrome_act_toward_goal: semantic micro-loop that perceives, decides and acts locally at ~300ms/step with zero MCP round-trips.
  • Three-engine fallback ladder: TypeSafe Jev (System One) → zero-dependency heuristic scorer → structured escalate back to the macro planner with prefetched DOM. Never hard-fails.
  • Zero extension changes: all new logic lives in the Native Server; the Chrome MV3 extension's 46 existing tools are untouched and fully regression-tested.
  • Two-stage <select> + Score primitive: relevance shortlisting for large dropdowns.
  • Full decision observability: every step returns Top-3 probability distribution + confidence.
  • Cost guardrails: maxSteps default 10, 401 session latch, quota/network graceful degradation.

Benchmarks (real Jev API, T1~T5)

Task Wall-clock Jev calls Tokens (in/out) Engine
T1 navigate+search 2062ms 2 1737/52 jev
T2 form submit 586ms 2 1666/48 jev
T3 select option 249ms 1 781/24 jev
T4 modal handling 518ms 2 1654/50 jev
T5 multi-step 574ms 2 1654/51 jev

Single-step median ~260–350ms. End-to-end speedup >75%, token reduction >80% vs LLM loop baseline.

Verification

  • Native Server: 79/79 tests passed
  • Chrome Extension: 301/301 tests passed (42 suites)
  • E2E Compliance: 153/153 tests passed
  • Typecheck: 0 errors · Clean build

Assets

  • browserclaw-extension-v2.8.0.zip — Chrome MV3 extension (load unpacked / side-load)
  • browserclaw-skill-v2.8.0.zip — Agent skill pack (47-tool contract, dual-brain routing, escalation protocol)

Requirements

  • Set TYPESAFE_API_KEY environment variable to enable the Jev engine. Without it, the tool automatically falls back to the built-in heuristic engine — always functional, gracefully degraded.

BrowserClaw v2.7.1 - Production Hardening, Anti-Hijack Index Matching & Full-Page Screenshot Parity

Choose a tag to compare

@GoldenLoaf24h GoldenLoaf24h released this 18 Sep 15:08

BrowserClaw v2.7.1 - Production Hardening, Anti-Hijack Index Matching & Full-Page Screenshot Parity

2. 硬核审查发现的 8 大关键缺陷与根本修复 (Root Cause & Fix)

1. inPageLocateByText 传入选项对象时触发运行时 TypeError 崩溃

  • 触发场景form-pipeline.ts 在执行表单题目选项模糊匹配或推进查找时,调用 inPageLocateByText(text, { exact: false, visibleOnly: true })
  • 根本原因inPageLocateByText 的签名原为 (text: string, role?: string),内部盲目执行 role.toLowerCase()。当第二个参数为配置对象时,直接抛出 TypeError: role.toLowerCase is not a function,导致表单自治管线瞬间崩溃。
  • 修复方案:重构入参为 roleOrOptions?: string | LocateByTextOptions,自适应提取 roleexactthreshold 等,支持结构化选项与旧版字符串双轨运行,杜绝运行时异常。

2. WeakRef 索引逆向包含引发的“父容器劫持子元素”Bug

  • 触发场景:在包含 <form> 或包装 <div> 的页面中,调用 extractElementLocationDetailsinPageVerifyInputCommitment 解析子元素(如提交按钮)。
  • 根本原因:原代码在 isolatedMap 中遍历元素时,使用了宽松的 target?.contains(el) || el.contains(target)。当父容器先于子按钮被索引时,parent.contains(child) 命中,导致子按钮错误继承了父容器的 Index,引发按 Index 点击点偏或选错元素。
  • 修复方案:改用严格的三阶段匹配(Pass 1: target === el 严格相等 -> Pass 2: 交互式微代理包装匹配 -> Pass 3: 动态分配全新唯一 Index),彻底根除索引劫持。

3. extractElementLocationDetails 遗漏 isClickablerole 语义契约

  • 触发场景:非 <button> 的可点击元素(如 <a class="btn"><input type="submit">role="button")在表单流水线或高阶定位中被解析。
  • 根本原因:返回值遗漏了 isClickablerole 计算,导致上游 form-pipeline 仅将 <button> 视为推进候选按钮,无法识别 ASP.NET LinkButton 及现代 DIV/SPAN 封装的提交按钮。
  • 修复方案:在底层补齐 role 提取与 isClickable(综合 tag、tabIndex、onclick、cursor 判断)语义标注。

4. native-host.ts Storage 异步解析未捕获异常

  • 触发场景:全新 Chrome Profile 或单元测试环境下,chrome.storage.local.get 返回 undefined,或 chrome.runtime.connectNative 不存在。
  • 根本原因:未对异步解构结果做空值保护,直接 result.native_server_port 抛出 Cannot read properties of undefined
  • 修复方案:增加 const res = result || {} 保底防御,并在调用前检查 typeof chrome.runtime?.connectNative === 'function'

5. 内部微操作嵌套调用引发的“双重沉淀(Double-Settle)”延迟浪费

  • 触发场景:调用 chrome_fill_index({ submit: true })chrome_batch_actions 自动触发提交点击。
  • 根本原因:外部管线在所有操作完成后会统一执行 waitForPageSettle;然而其内部调用的 interactIndexTool.execute 默认也开启了 waitForSettle: true。这导致同一个网络往返内先后执行了两次 200~500ms 的网络空闲探测,单步耗时无端拉长了近 1 秒。
  • 修复方案:内部点击显式传入 waitForSettle: false,将沉淀控制权统一收归外层 Pipeline,单次点击耗时立减 350ms+。

6. 全页长截图切片样式剥离导致页面原有 id 属性被永久误删

  • 触发场景:在包含已有 id 属性的吸顶导航(如 <header id="main-nav">)的页面上执行 chrome_screenshot({ fullPage: true })
  • 根本原因:GoFullPage 机制在对吸顶元素应用切片样式时,若元素无 id 会生成临时 __gfp_fixed_N。但在截完执行 popAllFixed() 时,粗暴调用了 element.removeAttribute('id'),将网页原本拥有的 id 属性一并剔除,破坏了网页原有的 CSS/JS 选择器绑定。
  • 修复方案:在 StyleStack.addFixed 中显式记录 hadOriginalId 标记,仅对合成生成的临时 ID 执行 removeAttribute,100% 保护网页原始 DOM 属性。

7. 缺少规范消息类型定义与编译期约束

  • 触发场景:全页长截图切片回滚向 In-Page 引擎发送 SCREENSHOT_POP_SLICE_FIXED 消息。
  • 根本原因common/message-types.ts 中未定义该常量,导致代码中充斥散落的魔法字符串。
  • 修复方案:在 TOOL_MESSAGE_TYPES 中注册 SCREENSHOT_POP_SLICE_FIXED: 'popSliceFixed',建立静态类型保护。

8. UnifiedLocatorResult 缺失核心字段导致管道数据损耗

  • 触发场景:调用 resolveTargetLocation 进行混合定位(ref, selector, text, role)。
  • 根本原因packages/shared/src/types.ts 中的接口未包含 indexroleisClickable,使得底层解析出的丰富语义无法回传给上层工具。
  • 修复方案:完善共享包类型定义并重新编译构建 chrome-mcp-shared

3. 核心代码变更清单

文件路径 变更概述
packages/shared/src/types.ts 扩充 UnifiedLocatorResult 声明,新增 indexroleisClickable
app/chrome-extension/common/message-types.ts 补齐 `SCREENSHOT_POP_SLICE

BrowserClaw v2.7.0 - True Input Commitment, Frustum Anti-Ghosting, Perceptive Delta & Form Pipeline

Choose a tag to compare

@GoldenLoaf24h GoldenLoaf24h released this 18 Sep 09:01

BrowserClaw v2.7.0 - True Input Commitment, Frustum Anti-Ghosting, Perceptive Delta & Form Pipeline

Highlights (核心摘要)

BrowserClaw v2.7.0 是一次攻克单页应用深度交互死锁(如 Draft.js/Lexical 富文本、Typeform 15 步动态问卷)的系统级重大里程碑发布。全面解决了富文本状态假成功、水平跨屏幽灵节点干扰、无效全页 DOM 轮询以及多步表单通信往返时延四大真实痛点,新增第 46 个高可靠规范工具 chrome_form_pipeline

  1. 真·输入交付与跨平台深度清空 (True Input Commitment & Deep Reset)
    • 彻底解决富文本编辑器“工具返回 success: true 但页面实际未提交”与清空失败导致的文本拼接;
    • 验证 React / Draft.js / Lexical 等框架的响应式状态,对无效输入自动回退为逐字 CDP 原生按键流(cdp_key_by_key)并核实有效交付;
  2. 多屏活动视口视锥裁剪 (Frustum Clipping & Anti-Ghosting)
    • 引入 activeViewportOnly: true 水平边界过滤,彻底剔除屏幕外未滑入视图的幽灵问卷项;
    • 文本定位算法注入 +1000 活动视口绝对加权,根除匹配到前一屏同名按钮(如 "OK"、"Next")导致的死循环;
  3. 感知差量引擎 (Perceptive Delta Engine)
    • 交互操作前后自动比对标题、题目变更、步骤进度(如 3 of 15)与当前激活项,直接随响应回传 perceptiveDelta,消除 80% 的模型无效全页 DOM 探测;
  4. 多步表单流水线 (chrome_form_pipeline - 第 46 个规范工具)
    • 允许一次性下发整套问卷/表单答案,由扩展后台本地微循环驱动原生 CDP 事件快速流转,自带卡顿熔断与验证保护,将 20 步交互压缩至单次往返。

📦 包含资产 (Assets)

  • browserclaw-extension-v2.7.0.zip:v2.7.0 纯净版 Chrome 扩展安装包,解压后直接在 Chrome 开发者模式一键加载;
  • browserclaw-skill-v2.7.0.zip:包含 46 规范工具契约、最新 chrome_form_pipeline 规范与实操配方的 Agent Skill 资产包。

BrowserClaw v2.6.2 - 3-Phase Execution Hardening, Assertions & Action-Triggered Network Capture

Choose a tag to compare

@GoldenLoaf24h GoldenLoaf24h released this 18 Sep 02:09

BrowserClaw v2.6.2 - 3-Phase Execution Hardening, Assertions & Action-Triggered Network Capture

Highlights (核心摘要)

BrowserClaw v2.6.2 是一次全面强化批处理流水线断言能力、模态窗隔离感知以及动作触发级网络数据静默捕获的工业级系统加固发布。落地了三大核心能力:

  1. 多断言增强流水线 (batch-actions 丰富 Assert):新增 matches_regexstarts_withends_withattribute_equals 等高阶断言,支持 continue_on_failure 宽容容错与自愈提示;
  2. 模态窗/弹窗严格隔离感知 (read_dom Modal Isolation):在检测到全屏模态窗口时,自动隔离背景非模态干扰节点,输出 [Modal Active: ...] 焦点锁定标识,防止 Agent 在弹窗开启时盲目操作背景元素;
  3. 动作触发级网络抓包 (Action-Triggered Network Capture):在 batch-actionsinteract-index 中支持单次交互同步嗅探因点击触发的异步 JSON API 响应,直接回传结构化数据。

1. 🛡️ 批处理流水线多维断言与数据提取增强

  • 扩展断言条件:在 chrome_batch_actions 中新增 matches_regexstarts_withends_withattribute_equals 条件;
  • 智能失败容错:支持 continue_on_failure: true,断言失败记录详细上下文而不中断后续表单清理流程;
  • 自愈诊断提示:断言未通过时输出精确的 actualexpected 及上下文 Diff,为推理模型提供直接自愈依据。

2. 🎯 模态窗视口隔离与焦点锁定感知

  • 自动范围修剪:当页面存在活跃的 <dialog open>aria-modal="true"z-index 顶层遮罩时,chrome_read_dom 优先隔离输出模态树,屏蔽被遮蔽的非交互背景;
  • 语义标记提示:在 AX 树头部直观标注模态焦点归属,引导 Agent 优先闭环确认或关闭当前弹窗。

3. ⚡ 交互动作自驱网络拦截 (Action-Triggered Network Capture)

  • 零额外轮询捕获:在点击、表单提交时支持附带 captureNetwork 规则(如过滤 */api/*),单次 Tool Call 同时返回交互结果与后台异步返回的纯 JSON 数据;
  • 敏感字段脱敏:配合 output-sanitizer 自动脱敏响应体中的 Token、密钥与密码字段。

📦 包含资产 (Assets)

  • browserclaw-extension-v2.6.2.zip:v2.6.2 纯净版 Chrome 扩展安装包,解压后可直接在 Chrome 开发者模式一键加载;
  • browserclaw-skill-v2.6.2.zip:包含 45 规范工具契约与最新高阶断言范式的 Agent Skill 配置包。

BrowserClaw v2.6.1 - Modern SPA Edge-Case Hardening, Input Disambiguation & Mask Piercing

Choose a tag to compare

@GoldenLoaf24h GoldenLoaf24h released this 17 Sep 15:04

BrowserClaw v2.6.1 - Modern SPA Edge-Case Hardening, Input Disambiguation & Mask Piercing

Highlights (核心摘要)

BrowserClaw v2.6.1 是一次深度攻坚现代单页应用 (SPA,如 Twitter/X、Complex Dashboards、Rich Social Web) 极端交互场景的系统级稳固发布。彻底解决了现代 SPA 中多输入框歧义(富文本发帖框 vs 顶部搜索框)、透明遮罩阻断物理点击、动态滚动边界丢失以及会话标签页上下文串流等 4 大高频阻断痛点。


1. 🎯 输入框语义消歧与富文本状态同步 (Input Disambiguation)

  • 多输入框智能识别与标识
    • 自动通过 detectEditorSemantics 精确区分富文本发帖框(Draft.js、Lexical、ProseMirror、Quill、role="textbox")与普通搜索框(type="search"role="searchbox");
    • 在紧凑 DOM 树中为富文本编辑器追加 [composer][editor] 语义标记,引导 Agent 建立清晰上下文认知;
  • 状态无损同步与回填
    • inPageFillIndex 中为 contenteditable 输入域注入 document.execCommand('insertText') 与原生 InputEvent 派发双通道,确保 React / Virtual DOM 内部状态实时响应,彻底根治发帖按钮假灰死锁;
    • 交互工具支持 preferComposer: true 偏好,并在候选模糊时主动输出 disambiguationWarning

2. 🛡️ 透明遮罩穿透与自愈点击 (Mask Piercing)

  • 全屏透明层穿透解算
    • 升级 inPageCheckInterceptionelementsFromPoint:自动识别纯透明(opacity: 0 / rgba(0,0,0,0))或仅作过渡动画呈现的假遮罩层并递归穿透,准确定位底层真实交互目标;
    • 若物理 CDP 事件被未脱离的透明层阻断,自动降级为页内原生合成事件精准触发。

3. ⚡ 深度动态容器滚动与精准流静默 (Smart Scroll & Quiescence)

  • 局部多层溢出滚动优先
    • chrome_smart_scroll 深度重构滚动容器探查,沿 DOM 树向上自动探测带 overflow-y: auto/scroll 的实际可滚子容器,优先进行局部容器滚动,杜绝全页无效盲滚;
  • 自适应网络静默窗口
    • action-watchdog.ts 中优化 SPA 动态拉取请求监听,结合 100ms 滑动窗口与 React 水合状态判定,确保数据流拉取完毕后再交付下一轮操作。

4. 🔄 会话亲和度与严格标签页隔离 (Session Tab Affinity)

  • 强化 SessionTabAffinityManager,严格保证并行会话下的标签页操作上下文不发生串流,多任务多窗口操作安全隔离。

📦 包含资产 (Assets)

  • browserclaw-extension-v2.6.1.zip:v2.6.1 纯净版 Chrome 扩展安装包,解压后可直接在 Chrome 开发者模式一键加载;
  • browserclaw-skill-v2.6.1.zip:包含 45 规范工具契约与现代化 SPA 操控范式的最新 Agent Skill 配置包。