Skip to content

ACT-103: reaction latency bench + workspace bench unification#699

Merged
Rotorsoft merged 5 commits into
masterfrom
ACT-103
May 10, 2026
Merged

ACT-103: reaction latency bench + workspace bench unification#699
Rotorsoft merged 5 commits into
masterfrom
ACT-103

Conversation

@Rotorsoft
Copy link
Copy Markdown
Owner

Summary

Closes #674.

ACT-103 surface — new libs/act/bench/reaction-latency.bench.ts measures single-process commit→reaction latency on InMemoryStore at three steady-state rates (idle / 100 per sec / 1000 per sec). Reports p50/p95/p99 with a regression bound (idle p99 < 50 ms). Numbers populate a new "Reaction latency" section in libs/act/PERFORMANCE.md and the README links to it.

Scenario p50 p95 p99
idle ~7 ms ~8 ms ~8 ms
low (100/s) ~8 ms ~12 ms ~14 ms
high (1000/s) ~1.8 s ~3.0 s ~3.0 s

Workspace bench unification — three benchmark shapes coexisted on master with inconsistent locations and run commands. They now have explicit homes:

Shape Lives in Run Use when
A — microbench (bench()) libs/<pkg>/bench/ pnpm bench:micro Inner-loop comparisons (delta vs JSON-patch, snap intervals)
B — research script (tsx standalone) libs/<pkg>/scripts/ npx tsx libs/<pkg>/scripts/<file>.bench.ts Open-ended exploration
C — scenario bench (it() + assertions) libs/<pkg>/bench/ pnpm bench:scenarios Realistic scenarios with regression bounds; CI-friendly

13 bench files migrated from test/ to the new directories. tsconfig.build.json excludes bench/ so dist doesn't pull them in. Per-package bench:micro and bench:scenarios scripts plus workspace-level fan-out (pnpm bench:micro / pnpm bench:scenarios).

CI integrationbench:scenarios now runs on every PR. Built-in regression assertions in each Shape C bench catch order-of-magnitude perf regressions. Captured stdout appends to $GITHUB_STEP_SUMMARY so reviewers see the numbers at the top of the workflow page. Microbenches (too noisy on shared runners) and research scripts (no regression bounds) stay manual-only.

Coveralls best-effort — added continue-on-error: true to the coveralls upload step. The previous PR got stuck on a coveralls outage; we'd rather lose a coverage badge than block a release.

BENCH.md — new repo-root index. Every benchmark, what it measures, run command, current numbers, and a link to the deeper writeup.

Coverage

100% statements / branches / functions / lines workspace-wide. 1025 tests.

Test plan

  • Workspace tests pass (pnpm test) — 1025/1025
  • Lint clean (pnpm lint)
  • Workspace build clean (pnpm build)
  • pnpm bench:scenarios works for both libs (act + act-pg) under workspace fan-out
  • pnpm bench:micro works for all three (act + act-pg + act-patch)
  • Latency bench numbers stable across 3 runs
  • Three Shape B scripts moved to scripts/ still runnable via npx tsx
  • CI workflow change is additive — existing perf-baseline regression guard untouched

🤖 Generated with Claude Code

rotorsoft and others added 5 commits May 10, 2026 10:49
CLAUDE.md was 810 lines of mixed reference, procedural guidance, and
project meta. Auto-loading that into every session burns context for
material that's already in docs/docs/ (Docusaurus). Reframed CLAUDE.md
as a 147-line index that points into docs and skills, keeping only
project meta and safety-critical one-liners inline.

- concurrency-model.md: appended "Why no framework-level request
  deduplication" — the TOCTOU/semantic-overloading/no-TTL reasoning
  that previously only existed in CLAUDE.md.
- error-handling.md: appended a Debugging section covering LOG_LEVEL,
  lifecycle event subscriptions, and query_array/query_streams
  introspection.
- guides/contributing-new-package.md (new, sidebar-wired): captures the
  seed-tag gotcha and the file-update checklist for adding a new
  @rotorsoft/act-* package.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…103)

ACT-103 surface — `libs/act/bench/reaction-latency.bench.ts` measures
single-process commit→reaction latency at three steady-state rates
(idle / 100 per sec / 1000 per sec) on InMemoryStore. Reports
p50/p95/p99 with a regression bound (idle p99 < 50 ms). Numbers feed
the new "Reaction latency" section in `libs/act/PERFORMANCE.md`.

Workspace bench unification — three coexisting benchmark shapes now
have explicit homes and consistent run commands:

  Shape A (microbench, `bench()` blocks)        → libs/<pkg>/bench/
  Shape B (research script, `tsx` standalone)   → libs/<pkg>/scripts/
  Shape C (scenario bench, `it()` + assertions) → libs/<pkg>/bench/

Files migrated from `libs/<pkg>/test/` and `libs/act-pg/test/` to the
new directories. tsconfig.build.json excludes `bench` so it doesn't
land in dist. New per-package `bench:micro` (vitest bench) and
`bench:scenarios` (vitest run --config vitest.bench.config.ts)
scripts; workspace-level `pnpm bench:micro` / `pnpm bench:scenarios`
fan-out across packages.

CI: bench:scenarios runs on every PR (Shape C built-in regression
assertions catch order-of-magnitude perf regressions). Captured
stdout appends to $GITHUB_STEP_SUMMARY so reviewers see the numbers
without scrolling logs. Microbenches (Shape A, too noisy) and
research scripts (Shape B, no bounds) stay manual-invocation only.

Coveralls upload marked `continue-on-error` — outages have blocked
master merges before. We keep the coverage badge when coveralls is
up; we keep merging releases when it's down.

Plus: new `BENCH.md` at the repo root indexes every benchmark with
its shape, run command, what it measures, and links to the deeper
writeup.

Coverage: 100% across the board (1025 tests).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The InMemory bench shipped earlier in the PR but the ticket
explicitly asked for both adapters. This adds the same three
steady-state scenarios (idle / 100 per sec / 1000 per sec) on
PostgresStore so the docs cover both sides.

PG numbers (single run, docker PG, no other load):

  idle          p50  4 ms  p95  20 ms  p99 ~500 ms (single outlier)
  low (100/s)   p50 10 ms  p95  22 ms  p99   70 ms
  high (1000/s) p50 ~125ms p95 ~1.2 s  p99 ~1.5 s (saturating)

Per-source warm-up pre-pays connection setup + dynamic-resolver
subscribe so the timed window starts from steady state. Regression
bound asserts on p50 (stable) instead of p99 (single-outlier-prone
with the small idle sample size); tail variance lives in the
reported table.

PERFORMANCE.md gets the PG result table alongside InMemory and
drops the now-stale "out of scope" note. BENCH.md updated.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
# Conflicts:
#	CLAUDE.md
#	libs/act-pg/vitest.bench.config.ts
The merge-resolution earlier in this PR took master's 846-line
CLAUDE.md, undoing the docs split optimization that lived on ACT-103
(commit 0b8a0fe — "split CLAUDE.md into Docusaurus reference and
slim index"). The slim version (147 lines) is the right shape: deep
reference moved to docs/docs/, the file stays an index.

Re-applied the slim version and added one new pointer line for
priority-lanes (since ACT-102 landed `docs/docs/architecture/priority-lanes.md`
on master). All other ACT-102 doc additions (priority sections in
the act and act-pg READMEs, the scaffold-act-app skill update,
priority-lanes.md itself, sidebars entry) stay — those were
correctly merged.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@Rotorsoft Rotorsoft merged commit 00384c5 into master May 10, 2026
6 checks passed
@github-actions
Copy link
Copy Markdown

🎉 This PR is included in version @rotorsoft/act-patch-v1.2.0 🎉

The release is available on:

Your semantic-release bot 📦🚀

@github-actions
Copy link
Copy Markdown

🎉 This PR is included in version @rotorsoft/act-v0.35.0 🎉

The release is available on:

Your semantic-release bot 📦🚀

@github-actions
Copy link
Copy Markdown

🎉 This PR is included in version @rotorsoft/act-pg-v0.20.0 🎉

The release is available on:

Your semantic-release bot 📦🚀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ACT-103: add commit-to-reaction latency benchmark scenarios

1 participant