Skip to content

feat(browser): rewrite network for agent-native discovery#1100

Merged
jackwener merged 2 commits intomainfrom
feat/browser-network-refactor
Apr 20, 2026
Merged

feat(browser): rewrite network for agent-native discovery#1100
jackwener merged 2 commits intomainfrom
feat/browser-network-refactor

Conversation

@jackwener
Copy link
Copy Markdown
Owner

Summary

Replaces the index-based browser network listing + pretty-printed --detail with a structured JSON interface built around:

  1. Stable keys — GraphQL ops use operationName, REST uses METHOD host+pathname, collisions get #N. Agents no longer reference entries by array index that shifts on every capture.
  2. Shape-first previews — default output ships a compact path → type map (depth cap 6, 2KB/entry budget) instead of the full body. Agents see response structure without paying body tokens.
  3. Persistent capture cache — every browser network run snapshots to ~/.opencli/cache/browser-network/<workspace>.json (24h TTL), so --detail <key> still works after later commands clear the live buffer.

Structured error codes replace human-readable stderr messages so agents can branch on error.code without regex: cache_missing, cache_expired, cache_corrupt, key_not_found (the last returns available_keys inline).

Example

$ opencli browser network
{
  "workspace": "browser:default",
  "captured_at": "2026-04-21T01:10:00.000Z",
  "count": 2,
  "filtered_out": 14,
  "entries": [
    {
      "key": "UserTweets",
      "method": "GET",
      "status": 200,
      "url": "https://x.com/i/api/graphql/qid/UserTweets?...",
      "ct": "application/json",
      "size": 84721,
      "shape": {
        "$": "object",
        "$.data": "object",
        "$.data.user": "object",
        "$.data.user.result": "object",
        "$.data.user.result.timeline_v2": "object"
      }
    },
    { "key": "UserByScreenName", "...": "..." }
  ],
  "detail_hint": "Run \"browser network --detail <key>\" for full body."
}

$ opencli browser network --detail UserTweets
{ "key": "UserTweets", "body": { "data": { ... } }, "shape": { ... } }

Flags

  • --detail <key> — full body for the given key (from cache)
  • --raw — emit every full body inline (skip shape preview)
  • --all — include static resources / telemetry that the default filter drops
  • --ttl <ms> — override 24h cache TTL for --detail lookups

Breaking changes

Default output format changed from human-readable text to JSON, and --detail now takes a key instead of a numeric index. No backward-compat shim — per the design goal "把 agent 的不确定性变成确定的 CLI 返回值".

Supersedes #1051 (@freemandealer's cache prototype, credited as co-author).

Test plan

  • src/browser/shape.test.ts — primitives, nested objects, arrays, depth cap, long strings, byte budget, Twitter UserTweets envelope (9 tests)
  • src/browser/network-key.test.ts — graphql operationName extraction, REST fallback, collision #N suffixes (6 tests)
  • src/browser/network-cache.test.ts — round-trip save/load, missing/expired/corrupt states, filename sanitization (7 tests)
  • src/cli.test.ts — default JSON output, --all, --raw, --detail <key> happy path, key_not_found with available_keys, cache_missing (6 new tests)
  • npm test — 220 files, 1646 tests, 2 skipped, all green
  • npm run build — clean TypeScript build + manifest regeneration

Replace the index-based list + pretty-printed --detail flow with a
structured JSON interface built around stable keys, body-shape previews,
and a persistent capture cache. Agents can now reference captured
requests by operationName (GraphQL) or `METHOD host+pathname` (REST)
instead of array indexes that shift on every rerun.

- `browser network` now emits JSON: `{workspace, captured_at, count,
  filtered_out, entries: [{key, method, status, url, ct, size, shape}],
  detail_hint}` — no body payloads by default
- Shape inference (src/browser/shape.ts) walks response JSON into a
  flat path -> descriptor map with depth cap 6 and a 2KB budget per
  entry, so agents see structure without paying body tokens
- Stable key generator (src/browser/network-key.ts) derives
  `operationName` from graphql URLs and `METHOD host+pathname`
  elsewhere, disambiguating collisions with `#N` suffixes
- Persistent cache (src/browser/network-cache.ts) snapshots every
  capture to `~/.opencli/cache/browser-network/<workspace>.json` with
  a 24h TTL, so `--detail <key>` survives later commands
- `--detail <key>` returns `{key, url, method, status, ct, size, shape,
  body}` with structured error codes (cache_missing / cache_expired /
  cache_corrupt / key_not_found, the latter including available_keys)
- Add `--raw` for agents that want every full body inline, `--ttl` for
  cache lookups
- Update opencli-adapter-author + opencli-autofix skill docs to
  reference `--detail <key>` and the shape-first discovery flow

Supersedes the cache prototype in #1051.

Co-authored-by: freemandealer <freeman.zhang1992@gmail.com>
Self-review findings on the network refactor:

- captureNetworkItems throwing (browser crashed / CDP dropped) now emits
  `error.code: capture_failed` on stdout rather than leaking a bare
  stderr line from browserAction's generic handler — agents get a
  parseable JSON blob on every failure path, matching the design goal.
- saveNetworkCache throwing (disk full, read-only path) is a soft
  failure: the captured data is already in hand, so surface a
  `cache_warning` field in the envelope and keep going instead of
  aborting. `--detail` lookups on that run will miss the cache but the
  listing still reaches the agent.
- shape.ts: guard the sub-walk on `add()`'s return value so the
  "budget hits on the array/object descriptor itself" path can never
  emit a stray child without its parent marker.
- network-key.ts: document that `#N` suffixes start at `#2` — the first
  occurrence stays bare, there is no `#1`. Matches test + code.

Added regression tests: `capture_failed` on readNetworkCapture throw,
`cache_warning` on persistence failure, shape budget hit on array descriptor.

Co-authored-by: freemandealer <freeman.zhang1992@gmail.com>
@jackwener
Copy link
Copy Markdown
Owner Author

自 review 跑了一轮(code-reviewer subagent),改了三处:

[blocker] capture_failed 结构化错误 — 之前如果 page.evaluate / readNetworkCapture 抛(浏览器断了 / CDP 掉了),异常会被 browserAction 的通用 handler 吃掉,只写 stderr,stdout 是空的。agent 拿不到可解析的 JSON,只能读 exit code,违反 agent-native 契约。现在会 emit {"error":{"code":"capture_failed","message":"..."}}

[blocker 的软化版] cache_warningsaveNetworkCache 抛(磁盘满 / 路径只读)时不再整条命令报错。capture 已经在内存里了,直接在 envelope 里加一个 cache_warning 字段然后正常返回 entries。只是 --detail 那次会错过 cache,但列表仍然到 agent 手里。

[medium] shape.ts 数组深度 cap 的 fragility — 如果预算刚好在 array(N) 自己那条耗尽,之前 walk 还会进 node[0](靠顶部的 if (truncated) return 短路,但行间顺序敏感)。改成用 add() 的返回值 guard 递归,任何时候改代码顺序都不会漏 marker。

[high 的 doc-only 修正] network-key.ts 注释明确 #N#2 起步,没有 #1(第一次保持裸 key)。行为本身没问题,只是注释不清。

新加 3 个回归测试:capture_failedcache_warning、shape 预算在 array descriptor 上耗尽。全量 1649 tests 过。

@jackwener jackwener merged commit 7fd8bd6 into main Apr 20, 2026
13 checks passed
@jackwener jackwener deleted the feat/browser-network-refactor branch April 20, 2026 17:32
luxiaolei pushed a commit to luxiaolei/OpenCLI that referenced this pull request Apr 21, 2026
…1100)

* feat(browser): rewrite network command for agent-native discovery

Replace the index-based list + pretty-printed --detail flow with a
structured JSON interface built around stable keys, body-shape previews,
and a persistent capture cache. Agents can now reference captured
requests by operationName (GraphQL) or `METHOD host+pathname` (REST)
instead of array indexes that shift on every rerun.

- `browser network` now emits JSON: `{workspace, captured_at, count,
  filtered_out, entries: [{key, method, status, url, ct, size, shape}],
  detail_hint}` — no body payloads by default
- Shape inference (src/browser/shape.ts) walks response JSON into a
  flat path -> descriptor map with depth cap 6 and a 2KB budget per
  entry, so agents see structure without paying body tokens
- Stable key generator (src/browser/network-key.ts) derives
  `operationName` from graphql URLs and `METHOD host+pathname`
  elsewhere, disambiguating collisions with `#N` suffixes
- Persistent cache (src/browser/network-cache.ts) snapshots every
  capture to `~/.opencli/cache/browser-network/<workspace>.json` with
  a 24h TTL, so `--detail <key>` survives later commands
- `--detail <key>` returns `{key, url, method, status, ct, size, shape,
  body}` with structured error codes (cache_missing / cache_expired /
  cache_corrupt / key_not_found, the latter including available_keys)
- Add `--raw` for agents that want every full body inline, `--ttl` for
  cache lookups
- Update opencli-adapter-author + opencli-autofix skill docs to
  reference `--detail <key>` and the shape-first discovery flow

Supersedes the cache prototype in jackwener#1051.

Co-authored-by: freemandealer <freeman.zhang1992@gmail.com>

* fix(browser): structured errors for capture/save, shape budget guard

Self-review findings on the network refactor:

- captureNetworkItems throwing (browser crashed / CDP dropped) now emits
  `error.code: capture_failed` on stdout rather than leaking a bare
  stderr line from browserAction's generic handler — agents get a
  parseable JSON blob on every failure path, matching the design goal.
- saveNetworkCache throwing (disk full, read-only path) is a soft
  failure: the captured data is already in hand, so surface a
  `cache_warning` field in the envelope and keep going instead of
  aborting. `--detail` lookups on that run will miss the cache but the
  listing still reaches the agent.
- shape.ts: guard the sub-walk on `add()`'s return value so the
  "budget hits on the array/object descriptor itself" path can never
  emit a stray child without its parent marker.
- network-key.ts: document that `#N` suffixes start at `#2` — the first
  occurrence stays bare, there is no `#1`. Matches test + code.

Added regression tests: `capture_failed` on readNetworkCapture throw,
`cache_warning` on persistence failure, shape budget hit on array descriptor.

Co-authored-by: freemandealer <freeman.zhang1992@gmail.com>

---------

Co-authored-by: freemandealer <freeman.zhang1992@gmail.com>
(cherry picked from commit 7fd8bd6)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant