Skip to content

feat(traffic-recording): runtime control + logs integration#164

Merged
g1331 merged 12 commits into
masterfrom
traffic-recording-runtime-control
May 19, 2026
Merged

feat(traffic-recording): runtime control + logs integration#164
g1331 merged 12 commits into
masterfrom
traffic-recording-runtime-control

Conversation

@g1331
Copy link
Copy Markdown
Owner

@g1331 g1331 commented May 18, 2026

Summary

把请求录制从环境变量驱动的工作模式升级为可在管理端实时控制的运维功能,并打通录制页与日志页:日志展开行可以直接看到该请求的完整 fixture,录制行可以反向跳回原始日志。本分支同时落地两个 openspec change(traffic-recording-runtime-controltraffic-recording-logs-integration),共 9 个 commit。

Related Issue

Closes #160

Type of Change

  • New feature (non-breaking change that adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)
  • Refactoring (code improvement without changing functionality)
  • Tests (adding or updating tests)

Breaking 部分说明:RECORDER_ENABLED / RECORDER_MODE / RECORDER_REDACT_SENSITIVE 三个环境变量不再控制运行时录制行为,改由管理端配置接管;RECORDER_FIXTURES_DIR 仍作为 fixture 存储根目录保留。部署方需把这些值迁移到新的管理页。

Changes

traffic-recording-runtime-control(issue #160 主功能)

  • 新增 traffic_recording_settings 单例表,承载启用状态、录制模式(all/success/failure)、脱敏开关、保留天数;管理员可在不重启服务的情况下修改,新请求立即生效。
  • 新增 traffic_recordings 索引表,记录 fixture 文件路径、大小、请求日志 ID、模型、状态码、脱敏标记、创建时间,供列表查询、详情读取与统计使用。
  • 代理 /api/proxy/v1/* 在每个请求开始处读取一次 settings 快照,后续录制(成功、失败、流式)使用同一份配置;fixture 写入时同步落数据库索引。
  • 新增 /system/traffic-recording 管理页:状态面板、配置表单、按时间/状态码/模型/API key/上游筛选、单条详情查看与删除、手动清理过期记录。
  • 新增 traffic_recording_cleanup 后台任务(默认启用,24h 间隔),复用既有 background sync 框架。
  • 全部接口受 admin token 保护:GET /api/admin/traffic-recording/settingsPATCH /api/admin/traffic-recording/settingsGET /api/admin/traffic-recordingsGET /api/admin/traffic-recordings/[id]DELETE /api/admin/traffic-recordings/[id]POST /api/admin/traffic-recordings/cleanup
  • 数据库迁移:drizzle/0033_shocking_emma_frost.sql(PostgreSQL)、drizzle-sqlite/0011_lush_kitty_pryde.sql(SQLite)。

traffic-recording-logs-integration(评审延伸:打通两个页面)

  • 抽出 RecordingJsonBlock 及其 JSON 树辅助函数到 src/components/admin/recording-json-block.tsx,供录制页与日志页复用。
  • GET /api/admin/traffic-recordings 新增 request_log_id 查询参数;GET /api/admin/logs 新增 id 查询参数。
  • 日志展开行新增 LogRecordingSection:用 useTrafficRecordingByLogId(logId, enabled) 在行展开后按需探测录制并加载详情,状态机覆盖 idle / loading / absent / present / missing-file / error 六种 UI。
  • 录制管理页表格行追加「打开原始日志」入口(仅在 request_log_id 非空时显示),链接 /logs?focus=<id>
  • 日志页读取 focus query 参数后进入单条聚焦模式:用 id filter 单条查询、自动展开目标行、顶部展示聚焦提示条与「清除聚焦」按钮;命中失败时显示「找不到该日志」状态。
  • 无数据库 schema 变更。

顺手修复

  • tests/unit/scripts/migrate-sqlite.test.ts 中硬编码的「Applied 11 migration(s)」与 11 个 hash 数组改为「Applied 12」与 12 个 hash,配合本分支新增的 0011_lush_kitty_pryde 迁移。

Test Plan

  • Local tests pass (pnpm test:run) — 2343 passed, 1 skipped, 0 failed
  • Type check passes (pnpm exec tsc --noEmit)
  • Lint passes (pnpm lint)
  • Format check passes (pnpm format:check)
  • git diff --check 通过
  • Manual testing completed(待 reviewer 在浏览器中确认以下场景):
    • /system/traffic-recording 切换启用状态、模式、脱敏开关、保留天数并保存,下一次代理请求按新配置录制。
    • 展开已录制日志行 → 显示录制分区与 JSON 树。
    • 展开未录制日志行 → 显示「该请求未被录制」提示。
    • 录制行点击「打开原始日志」 → 跳到 /logs?focus=<id> 并自动展开目标行。
    • 在聚焦模式下点击「清除聚焦」 → 回到普通分页列表。
    • 在录制管理页点击「清理过期」 → 过期记录被删除,索引与文件同步清理。

Checklist

  • Code follows the project's coding standards
  • Tests have been added where necessary(新增 6 个测试文件、扩充 4 个既有测试文件)
  • Documentation has been updated(两个 openspec change 的 proposal/design/specs/tasks 完整)
  • Changes do not introduce security vulnerabilities(录制路径用 assertPathInsideRecordingRoot 防越权;所有管理端接口受 admin token 保护)
  • Commit messages follow conventions(feat: / refactor: / test: / chore:

Screenshots

待 reviewer 在浏览器中验证后附上录制管理页与日志展开行的截图。

Additional Notes

部署与迁移

  • 升级前请执行 pnpm db:migrate,依次应用 PostgreSQL 0033_shocking_emma_frost.sql 或 SQLite 0011_lush_kitty_pryde.sql
  • RECORDER_ENABLED / RECORDER_MODE / RECORDER_REDACT_SENSITIVE 三个环境变量不再生效,需要在 /system/traffic-recording 重新配置;如果之前设置为开启状态,部署后会回落到默认值(关闭、failure 模式、脱敏开启、保留 7 天),需手动开启。
  • RECORDER_FIXTURES_DIR 仍作为 fixture 存储根目录,可继续使用。

Capabilities 摘要

  • 新增 capability:traffic-recording-runtime-controlrequest-log-record-integration
  • 修改 capability:background-sync-tasks(新增 cleanup 任务)、traffic-recording-runtime-control(在 logs-integration change 内追加 request_log_id filter scenario)。

Commit 序列

6e51219 chore(openspec): mark phase 7 tasks.md fully complete
0308336 test(migrate): bump expected sqlite migration count to 12
34090c4 feat(logs): support focus deep-link from recording rows
63b4f4c feat(recordings): link recording rows back to their source log
429969c feat(logs): embed traffic recording section in log row details
8acc2c7 feat(api): allow filtering request logs by id
d21bb22 feat(api): allow listing traffic recordings by request_log_id
7835f65 refactor: extract recording-json-block as shared component
d7e5d6d feat: add traffic recording runtime controls

g1331 added 9 commits May 18, 2026 22:30
Move RecordingJsonBlock and its JSON tree helpers (JsonTreeNode,
JsonPrimitiveValue, collectExpandedJsonPaths, isJsonBranch,
getJsonBranchEntries, getJsonBranchSummary) from the traffic recording
admin page into src/components/admin/recording-json-block.tsx so they
can be reused by the logs view in upcoming phases.

Behavior is preserved: the traffic recording page now imports the
shared component and renders identically. Adds component tests
covering helpers, default depth-1 expansion, expand/collapse all,
clipboard copy success/error paths, and branch toggling.

Part of openspec change traffic-recording-logs-integration (phase 1).
Adds the `requestLogId` filter to `TrafficRecordingListFilters` and
wires the `request_log_id` query parameter through the admin
`/api/admin/traffic-recordings` route. Empty values are treated as
absent so the existing filter semantics for api_key_id / upstream_id
are preserved.

This unblocks the logs view from probing whether a given log entry has
a corresponding recording without scanning unrelated records. Service
and route tests cover hit, miss, and unauthenticated paths.

Part of openspec change traffic-recording-logs-integration (phase 2).
Adds the `id` filter to `ListRequestLogsFilter` and exposes it via the
admin `/api/admin/logs` route. Empty values are ignored so the
existing filter semantics are preserved.

This lets the logs view focus on a single record (used by the upcoming
`/logs?focus=<id>` deep link from the recording management page) without
introducing a separate detail endpoint. Service and route tests cover
hit, miss, empty-value, and unauthenticated paths.

Part of openspec change traffic-recording-logs-integration (phase 3).
Adds an inline recording panel inside the expanded log row so admins
can view the full fixture for a request without leaving the logs page.

- `useTrafficRecordingByLogId(logId, enabled)` is a new hook that
  probes `/admin/traffic-recordings?request_log_id=<id>&page_size=1`
  and, on hit, fetches the detail endpoint. Both queries stay disabled
  until the caller opts in, so collapsed rows make no requests.
- `LogRecordingSection` renders four UI states (idle, loading, absent,
  missing-file, error, present) and reuses `RecordingJsonBlock` for
  fixture rendering. Status badge, model, size, redaction marker, and
  creation time are surfaced in a compact metadata row.
- `logs-table.tsx` mounts the section at the end of
  `renderExpandedDetails`, gated on `expandedRows.has(log.id)`.
- Adds English / Simplified Chinese translations for all new copy.
- Tests cover the hook's state machine via the component (idle through
  present) and verify that `LogsTable` mounts the section with the
  expected props when a row is expanded.

Part of openspec change traffic-recording-logs-integration (phase 4).
Adds an "open source log" outline button to each recording table row
that has a non-null request_log_id. The button is a localized link
into /logs?focus=<id>, which will be wired up in the next phase so the
focused log expands automatically.

Component tests cover both branches: a row with a request_log_id
renders exactly one link with the expected href, while a row without
one renders none.

Part of openspec change traffic-recording-logs-integration (phase 5).
Adds /logs?focus=<id> deep-linking so admins clicking "open source log"
on a recording row land directly on the matching log with its details
already expanded.

- `useRequestLogs` now accepts an `id` filter passed through to the
  admin API.
- The logs page reads the `focus` search param, switches into
  single-log mode (page=1, pageSize=1, no polling) when present, and
  swaps the live-status header for a focus banner. Hits render the
  banner with the log id; misses switch to a "not found" copy. Both
  branches expose a localized "clear focus" link back to /logs.
- `LogsTable` gained an `initialExpandedIds` prop so the focused row
  is expanded on mount without lifting `expandedRows` to the page.
- Adds English / Simplified Chinese strings for the focus banner copy
  and clear-focus action.
- A new `tests/components/logs-page.test.tsx` covers no-focus,
  focus-hit, and focus-miss flows including filter forwarding and the
  clear link's href.

Part of openspec change traffic-recording-logs-integration (phase 6).
The traffic-recording-runtime-control change added migration 0011
(0011_lush_kitty_pryde) to drizzle-sqlite, which made the adoption
test fail because it still asserted an "Applied 11 migration(s)"
banner and an 11-hash list.

Updates the count and appends the new hash. Finalizes the
traffic-recording-logs-integration openspec change by marking phase 7
tasks complete (manual UI verification deferred to the user).
@codecov
Copy link
Copy Markdown

codecov Bot commented May 18, 2026

Codecov Report

❌ Patch coverage is 45.81006% with 194 lines in your changes missing coverage. Please review.
✅ Project coverage is 79.19%. Comparing base (6b44e5c) to head (7d5a784).
✅ All tests successful. No failed tests found.

❌ Your patch check has failed because the patch coverage (45.81%) is below the target coverage (50.00%). You can increase the patch coverage or adjust the target coverage.

Additional details and impacted files
@@            Coverage Diff             @@
##           master     #164      +/-   ##
==========================================
- Coverage   80.23%   79.19%   -1.04%     
==========================================
  Files         120      125       +5     
  Lines       10401    10739     +338     
  Branches     3642     3785     +143     
==========================================
+ Hits         8345     8505     +160     
- Misses       1312     1444     +132     
- Partials      744      790      +46     
Flag Coverage Δ *Carryforward flag
typescript 76.03% <ø> (-0.07%) ⬇️ Carriedforward from e331d48
verify 74.58% <45.81%> (-1.09%) ⬇️

*This pull request uses carry forward flags. Click here to find out more.

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

g1331 added 3 commits May 19, 2026 20:24
启用 vitest 的 default reporter,让 stdout 同时输出失败摘要,方便从
GitHub Actions 日志直接看到具体哪条测试或哪个阶段出错;并新增
actions/upload-artifact 步骤,把 test-report.junit.xml 与
coverage/lcov.info 作为 vitest-report artifact 保留 7 天,便于
事后下载排查 worker 崩溃、coverage 生成失败等 CI 特有问题。
之前断言 fixture 读取路径时硬编码了 Windows 反斜杠("data\\traffic-recordings\\…"),
在 Linux runner 上实际收到的是正斜杠路径而失配,导致 Quality CI 任务整体退出码非 0。
改用 node:path 的 path.join 构造期望片段,让分隔符随当前平台变化,兼容 Windows 与 Linux。
之前 listTrafficRecordings 的 count 与 items 查询都按 whereClause 过滤,
但 totalSizeBytes/latestCreatedAt 的聚合查询直接 from(trafficRecordings)
扫全表,导致按 api_key、upstream、request_log_id、status_code、model、
时间区间任一过滤时,管理页顶部的「磁盘占用」「最近记录」与列表中实际
看到的行无关,stats 卡片内部 total 与其它字段口径不一致。

补上 .where(whereClause) 让 stats 聚合与列表使用同一过滤集合,并把
service 测试中 stats mock 链改为 .from().where() 的形态,未来误删
.where() 时测试会因解构非 Promise 失败而立即报警。
@g1331 g1331 merged commit bab786f into master May 19, 2026
12 checks passed
@g1331 g1331 deleted the traffic-recording-runtime-control branch May 19, 2026 14:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

将请求录制改进为实时控制

1 participant