feat(extension): hover end to end capability support

[Ljy-0827]

问题描述

现有版本对 hover 触发展开型内容的端到端支持不完整，导致 agent 在处理需要先 hover 才能展开的内容时容易中断或卡住。

此前 VOM 的 probe 使得 observation 能够发现 hover 后出现的菜单内容，但这部分信息不一定会被正确标注到实际可操作的 handle 上。同时 CLI、daemon、extension 侧缺少稳定的 hover 动作链路，agent 即使知道菜单存在，也无法可靠地执行“先 hover、再观察、再点击具体内容”的流程。

另外，一些页面的展开逻辑并不只是依赖 CSS :hover，还可能通过 JS 事件、组件状态或 DOM 变化触发。如果只按静态 :hover 信号判断候选入口，容易漏掉实际可 hover 的控件。

这会导致 agent 在头像菜单、工具栏菜单、更多操作菜单等常见 UI 中，误以为触发器可以直接点击，从而点不到 hover 展开后的内部菜单项。

解决方案

本 PR 从执行链路、观察能力和映射逻辑三层补齐 hover 菜单支持：

• 补齐 hover 动作链路：新增并接通 hover tool，使 CLI 能通过 daemon 调用 extension，在目标页面上执行真实鼠标 hover，并等待 hover UI 稳定。
• 增强 VOM hover surface 探测：observe 会进行受控的 hover surface probe，对候选元素执行探测并收集 hover 后新增的菜单项，而不是只依赖静态 :hover 样式判断。
• 支持 hover 状态下继续观察：hover 后菜单会保持在可观察状态，后续 observe 能看到展开后的菜单结构和菜单项 refs。
• 修复 probe 结果到 handle 的映射：surface probe 结果使用 DOM/backend node 关系和几何信息映射到 VOM 中实际会渲染 ref 的节点，避免只按文本或 AX role 匹配导致标注丢失。
• 复用 VOM 交互节点判定：hover 引导信息只挂到 VOM 最终认为可操作、会生成 handle 的节点上，避免 observe 里出现“有菜单但不知道 hover 哪个 ref”的情况。
• 补充调试与测试覆盖：增加 hover surface debug 输出和相关测试，覆盖 hover 菜单发现、展开后观察、以及自定义 DOM 控件映射等场景。

完成后，agent 可以按以下流程稳定操作：
observe 识别可 hover 的入口和菜单摘要 → hover @ref 展开并保持菜单 → 再次 observe 获取展开菜单中的 refs → click 目标菜单项。

Problem Description

The current version does not provide complete end-to-end support for hover-triggered expandable content, which can cause the agent to get interrupted or stuck when handling content that must be revealed by hovering first.

Previously, VOM probes allowed observation to discover menu content that appears after hover, but this information was not always correctly attached to the actual actionable handle. At the same time, the CLI, daemon, and extension did not have a stable hover action path, so even when the agent knew the menu existed, it could not reliably execute the flow of “hover first, observe again, then click the target item”.

Additionally, some pages do not rely only on CSS :hover for expansion. They may use JavaScript events, component state, or DOM changes to reveal content. If candidate detection only depends on static :hover signals, it can miss controls that are actually hoverable.

As a result, in common UI patterns such as avatar menus, toolbar menus, and more-action menus, the agent may incorrectly assume the trigger can be clicked directly, and then fail to access the menu items that only appear after hover.

Solution

This PR completes hover menu support across three layers: the execution path, observation capability, and mapping logic.

• Complete the hover action path: add and wire up the hover tool so the CLI can call the extension through the daemon, perform a real mouse hover on the target page, and wait for the hover UI to settle.
• Improve VOM hover surface probing: observe now performs controlled hover surface probes on candidate elements and collects menu items that appear after hover, instead of relying only on static CSS :hover style checks.
• Support observing while hover state is active: after hovering, the menu remains observable, so a follow-up observe can see the expanded menu structure and menu item refs.
• Fix probe-to-handle mapping: surface probe results are mapped to the VOM node that will actually render a ref using DOM/backend node relationships and geometry, avoiding missing annotations caused by text-only or AX-role-only matching.
• Reuse VOM interactive-node detection: hover guidance is attached only to nodes that VOM ultimately treats as actionable and assigns handles to, avoiding cases where observe shows a menu exists but gives no clear ref to hover.
• Add debugging and test coverage: add hover surface debug output and tests covering hover menu discovery, observation after expansion, and mapping for custom DOM controls.

After this change, the agent can reliably follow this flow:

observe identifies a hoverable entry and menu summary → hover @ref expands and keeps the menu open → another observe returns refs for the expanded menu items → click the target menu item.

[Copilot]

Pull request overview

This PR adds end-to-end support for hover-triggered UI (menus/popovers) so agents can reliably discover hover-only content via VOM, perform a real hover action, and then re-observe/click newly revealed items.

Changes:

• Introduces a new hover tool wired through protocol → daemon/CLI → extension, including hover “latching” so follow-up observe/snapshot can keep hover state active.
• Enhances VOM conditional-surface probing and rendering to better detect hover-triggered menus and present guidance as [hover first: …], plus optional debug output.
• Extends recording/trace formats and adds tests for hover probing, mapping, dispatcher reassert/release behavior, and CLI parsing.

Reviewed changes

Copilot reviewed 39 out of 39 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
skill/SKILL.md	Updates user-facing skill docs to recommend observe first and introduces bsk hover usage guidance.
packages/vom/src/render.ts	Exposes reference/recovery helpers and updates surface rendering wording to “hover first”.
packages/vom/src/index.ts	Re-exports newly public VOM helper functions.
packages/vom/src/tests/render.test.ts	Updates renderer expectations for “hover first” surface text.
crates/bsk-protocol/src/tools/record.rs	Adds Hover as a recorded trace step variant.
crates/bsk-protocol/src/tools/observation.rs	Adds debug_surfaces param and optional debug payload for surface probes.
crates/bsk-protocol/src/tools/interaction.rs	Defines hover tool params/result types and serde tests.
crates/bsk-protocol/src/method.rs	Registers tool.hover and classifies it as transient input.
crates/bsk-protocol/src/bin/dump-schema.rs	Includes hover tool schemas in schema dump output.
crates/bsk-protocol/schema/trace.json	Updates trace JSON schema to include hover step shape.
crates/bsk-protocol/schema/trace_step.json	Updates trace step JSON schema to include hover step shape.
crates/bsk-protocol/schema/tool_record_stop_result.json	Extends record stop result schema to allow hover steps.
crates/bsk-protocol/schema/tool_record_await_result.json	Extends record await result schema to allow hover steps.
crates/bsk-protocol/schema/tool_observe_result.json	Adds debug shape for observe results.
crates/bsk-protocol/schema/tool_observe_params.json	Adds debug_surfaces to observe params schema.
crates/bsk-protocol/schema/tool_hover_result.json	Adds hover result JSON schema.
crates/bsk-protocol/schema/tool_hover_params.json	Adds hover params JSON schema.
crates/bsk-cli/tests/tools_ipc.rs	Updates observe IPC test fixtures for new debug field.
crates/bsk-cli/tests/cli_parse.rs	Adds CLI parsing test for bsk hover --settle.
crates/bsk-cli/src/main.rs	Wires Command::Hover into CLI dispatch.
crates/bsk-cli/src/daemon/ipc.rs	Allows daemon IPC handler to route tool.hover.
crates/bsk-cli/src/cli/observe.rs	Adds --debug-surfaces flag to bsk observe.
crates/bsk-cli/src/cli/mod.rs	Adds Hover command to CLI command enum/imports.
crates/bsk-cli/src/cli/interaction.rs	Implements bsk hover command and request/printing behavior.
crates/bsk-cli/skill/SKILL.md	Mirrors top-level skill docs update for CLI packaging.
apps/extension/src/transport/types.ts	Adds hover tool transport types + observe debug types.
apps/extension/src/tools/vom/capture.ts	Reworks hover probing to score candidates and diff menu items after hover.
apps/extension/src/tools/vom/tests/capture.test.ts	Adds/updates tests for hover probing behavior and dedupe logic.
apps/extension/src/tools/observation.ts	Maps hover probe results to actionable VOM refs using backend ids/geometry; adds debug serialization.
apps/extension/src/tools/interaction.ts	Implements tool.hover and adds overlay bypass/settle handling.
apps/extension/src/tools/dispatcher.ts	Adds hover latch/bypass lifecycle: remember, reassert, and release around follow-up tools.
apps/extension/src/tools/tests/observation.test.ts	Updates/extends hover probe mapping tests and observe debug behavior.
apps/extension/src/tools/tests/interaction.test.ts	Adds unit test for hover overlay bypass retention behavior.
apps/extension/src/tools/tests/dispatcher.test.ts	Adds tests for hover reassert/release behaviors and session isolation.
apps/extension/src/lib/trace-reducer.ts	Preserves hover steps in trace reduction.
apps/extension/src/lib/record-bridge.ts	Extends record bridge step union to include hover op.
apps/extension/src/lib/describe-target.ts	Adds resolveHoverElement() heuristic for recording hover targets.
apps/extension/src/lib/tests/trace-reducer.test.ts	Adds test ensuring hover stays before subsequent menu clicks.
apps/extension/src/content/record-capture.ts	Records hover-before-click sequences by tracking recent mouseover targets.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 44 out of 44 changed files in this pull request and generated no new comments.

Suppressed comments (3)

apps/extension/src/tools/observation.ts:931

• used.add(match.id) has no effect because the loop only checks used.has(probe.triggerBackendNodeId). This makes the used set partially unused and can allow multiple probes (with different backend ids) to attach to the same rendered trigger node. Either remove the match.id tracking or also guard against used.has(match.id) before pushing a surface.

    used.add(match.id);

apps/extension/src/content/record-capture.ts:427

• mouseover fires very frequently while recording; hoverTargetFromEvent() performs DOM queries plus getBoundingClientRect()/getComputedStyle() each time. You already keep a pendingHover latch, so you can skip the expensive recomputation when the pointer stays within the currently latched trigger subtree.

  const onMouseOver = (event: MouseEvent) => {
    const target = eventTarget(event);
    if (isOverlayTarget(target)) return;
    const hover = hoverTargetFromEvent(target);
    if (!hover) return;

apps/extension/src/tools/dispatcher.ts:546

• setHoverBypass() returns early when a tab is already in hoverBypassTabs, even if the existing entry belongs to a different session. That prevents ownership from transferring and can leave overlay bypass stuck enabled indefinitely (because later releaseHoverLatch() calls are session-scoped).

  private async setHoverBypass(sessionId: string, tabId: number, enabled: boolean): Promise<void> {
    if (enabled) {
      if (this.hoverBypassTabs.has(tabId)) return;
      await bypassOverlay(tabId, true);
      this.hoverBypassTabs.set(tabId, sessionId);