refactor: Rewrite user_guide.md as evidence-based case study - #171
refactor: Rewrite user_guide.md as evidence-based case study#171evansenter wants to merge 25 commits into
Conversation
Restructure from sales pitch to documented experiment format: - Add per-repo activity table (PRs, issues, LoC, +/- lines) - Add session analytics with source citations - Add cost analysis (£1K subscription vs $28K API equivalent) - Add system architecture diagram showing data flow - Reduce from 580 to 218 lines (62% reduction) Update repo-stats.sh with --session-stats flag: - Integrates session-analytics-cli for token/tool data - Shows sessions, tool invocations, cache ratio - Displays date range for data availability 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Code ReviewSummaryRestructures Issues FoundCriticalNone ImportantNone Suggestions
VerdictREQUEST_CHANGES - Minor suggestions for formatting consistency and potential table overflow in the shell script. Automated review by Claude Code |
| **Regenerate:** `~/.claude/contrib/repo-stats.sh --days 17 --session-stats` | ||
|
|
||
| ### 2. Session Lifecycle Hooks | ||
| ### Cost |
There was a problem hiding this comment.
So this is estimated cost since Dec 30, aiui?
| ### Session Analytics | ||
|
|
||
| **Good**: `/work 42` | ||
| _Data available since December 30, 2025 (session log retention)._ |
There was a problem hiding this comment.
This was when we started logging sessions, it's not a retention issue.
| ``` | ||
| | Metric | Value | | ||
| |--------|-------| | ||
| | Actual cost (Claude Code subscription) | ~£1,000 | |
There was a problem hiding this comment.
Claude Max ($200 plan) + 2x quota limits during holiday + extra pay-as-you-go usage
|
|
||
| The 15:1 cache ratio is critical. Prompt caching means the system learns your codebase once and reuses that context constantly. Even with caching, intensive parallel development adds up—budget accordingly. | ||
|
|
||
| ## The Architecture |
There was a problem hiding this comment.
Move this figure and data flow to the top, this is the headline imo
| 4. **gemicro** provides agent infrastructure for external Gemini-based agents | ||
| 5. **rust-genai** is the underlying Gemini API SDK | ||
|
|
||
| The key insight: dotfiles is the control plane. Instead of ad-hoc instructions, you define reusable workflows that encode preferences, quality gates, and coordination patterns. |
There was a problem hiding this comment.
With this framing, what would you call the data plane / management plane?
| ### Rate Limiting During Parallel Work | ||
|
|
||
| Running 5+ parallel sessions can hit API rate limits, especially with Opus: | ||
| 5+ parallel Opus sessions can hit API rate limits. |
There was a problem hiding this comment.
note that we were running up to 15 at times
| **Workaround:** `prompt-events.sh` hook polls on every user prompt. | ||
|
|
||
| ### Context Compaction Loses Working Memory | ||
| ### Context Compaction Loses Nuance |
There was a problem hiding this comment.
we also did all that state caching and resumption using the event-bus + hooks
| ### MCP Doesn't Support Push | ||
|
|
||
| **The Problem**: When Session B publishes an event, Session A can't receive it until Session A's next prompt triggers a hook that polls the event bus. | ||
| The Model Context Protocol is request/response only. Claude can call MCP tools, but servers can't push events to Claude. |
There was a problem hiding this comment.
emph the significance of this challenge
| Most infrastructure improvements came from accumulated friction, not planning. | ||
|
|
||
| ### 7. Document Discoveries via Event Bus | ||
| ### 5. CLAUDE.md Is High Leverage |
There was a problem hiding this comment.
Make this more generic—a system-wide DI that is directly iterated on is high leverage, something like that
| 5. **rust-genai** is the underlying Gemini API SDK | ||
|
|
||
| The key insight: dotfiles is the control plane. Instead of ad-hoc instructions, you define reusable workflows that encode preferences, quality gates, and coordination patterns. | ||
|
|
There was a problem hiding this comment.
i do want to have a section that goes into more detail on the feature set of each repo. should we ask each of the repos to write their own using event-bus? let's start with just adding dotfiles for now and iterate on the form
- Add clarifying note about Dec 30+ date range for cost data - Widen session stats column (15→25 chars) for long sequences - Change "Code" header to "LoC" for consistency - Add --days flag to Source column commands 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
|
Addressed all 4 feedback items:
|
Code ReviewSummaryRewrites Issues FoundCriticalNone ImportantNone SuggestionsNone VerdictAPPROVE - All previous feedback has been addressed. The documentation is well-structured with clear data provenance. The shell script changes are clean and follow existing patterns. Automated review by Claude Code |
- Fix session logging wording (not retention, but when logging began) - Add cost breakdown: Claude Max $200 + 2x quota + pay-as-you-go - Move architecture diagram to top as headline - Add control plane / data plane terminology - Note running up to 15 parallel sessions - Add event-bus state caching to context compaction mitigation - Emphasize MCP push limitation significance for multi-agent - Rewrite CLAUDE.md section as system-wide dependency injection - Add dotfiles feature set section with commands, agents, hooks 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
|
Addressed all 11 inline review comments: Structure:
Content: Document now 248 lines (from original 580, now with more substantive content). |
Code ReviewSummaryRestructures Issues FoundCriticalNone ImportantNone SuggestionsNone VerdictAPPROVE - All previous feedback has been addressed. The documentation restructuring is well-executed with clear data provenance and proper source citations. The shell script changes follow existing patterns and include appropriate error handling for missing dependencies. Automated review by Claude Code |
Aligns with CLAUDE.md guidance: "Prefer MCP tools for structured data" - mcp__github__get_pull_request_comments for inline code comments - mcp__github__get_pull_request_reviews for review summaries - Keep gh api for issue comments (no MCP equivalent) 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Code ReviewSummaryRestructures Issues FoundCriticalNone ImportantNone SuggestionsNone VerdictAPPROVE - The documentation restructuring is well-executed with clear data provenance and reproducible commands. The shell script changes follow existing patterns, include proper error handling for missing dependencies, and format output consistently. Previous review feedback has been fully addressed. Automated review by Claude Code |
Thesis: - Reframe around workflow integration over code generation Key Learnings: - #6: Ownership over Skill (domain agents > capability agents) - #7: Workflow Integration over Code Generation (the real value prop) Frontier: - Expand MCP section to cover learning propagation gap - Add Constrained Self-Evolution (swarm controller, self-modifying DI) Also: Update pr-review.md to use MCP tools for comment fetching 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
|
Added architectural learnings: Thesis - Reframed around workflow integration:
Key Learnings:
Frontier:
Document now 284 lines with 7 Key Learnings and 5 Frontier items. |
Replace Commands/Agents/Hooks/Infrastructure listing with thematic capabilities: - Workflow Orchestration (issue→PR loop) - Parallel Development (worktrees, event bus coordination) - Context Continuity (compaction survival, WIP restoration) - Self-Improvement (analytics-driven automation) - Ambient Awareness (statusline, tmux integration) 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Code ReviewSummaryRewrites Issues FoundCriticalNone ImportantNone SuggestionsNone VerdictAPPROVE - The documentation restructure is well-executed with clear data provenance and reproducible commands. All previous review feedback has been addressed across multiple iterations. The shell script changes follow existing patterns, include proper error handling for missing dependencies (session-analytics-cli graceful fallback), and use appropriate column widths for the session stats table. The Automated review by Claude Code |
Code ReviewSummaryRewrites Issues FoundCriticalNone ImportantNone SuggestionsNone VerdictAPPROVE - All previous feedback from multiple review iterations has been addressed. The documentation restructure is well-executed with clear data provenance and reproducible commands. The shell script changes follow existing patterns with proper error handling for missing dependencies ( Automated review by Claude Code |
dotfiles/ as swarm controller is real, but limited by: - No push notifications (MCP) - No dynamic reload (CC reads CLAUDE.md at start) Workarounds: claude --continue, event bus "reload recommended" 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Code ReviewSummaryRewrites Issues FoundCriticalNone ImportantNone SuggestionsNone VerdictAPPROVE - The documentation restructuring is well-executed with clear data provenance and reproducible commands. Previous review feedback from multiple iterations has been fully addressed. The shell script changes follow existing patterns, include proper error handling for missing dependencies (graceful fallback when Automated review by Claude Code |
The event bus architecture is sound—it's MCP's request/response model that prevents push delivery, not the event bus design. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Code ReviewSummaryRestructures Issues FoundCriticalNone ImportantNone SuggestionsNone VerdictAPPROVE - This PR has been through 8 review iterations with all feedback fully addressed. The documentation restructuring is well-executed with clear data provenance and reproducible commands. The shell script changes follow existing patterns, include proper error handling for missing dependencies (graceful fallback when Automated review by Claude Code |
Friction → issue → event bus routes to repo owner → free agent (with human approval) picks up the work 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
No guardrails on scope, no automatic rollback, no approval thresholds. Human approval is the only constraint—once approved, anything goes. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Code ReviewSummaryRestructures Issues FoundCriticalNone ImportantNone SuggestionsNone VerdictAPPROVE - This PR has been through multiple review iterations with all feedback fully addressed. The documentation restructuring is well-executed with clear data provenance and reproducible commands. The shell script changes follow existing patterns, include proper error handling for missing dependencies (graceful fallback when Automated review by Claude Code |
Tooling scales beyond 15—bottleneck is user context-switching overhead and API rate limits. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Code ReviewSummaryRestructures Issues FoundCriticalNone ImportantNone SuggestionsNone VerdictAPPROVE - This PR has been through multiple review iterations with all feedback fully addressed. The documentation restructuring is well-executed with clear data provenance and reproducible commands. The shell script changes follow existing patterns, include proper error handling for missing dependencies (graceful fallback when Automated review by Claude Code |
Code ReviewSummaryRestructures Issues FoundCriticalNone ImportantNone SuggestionsNone VerdictAPPROVE - This PR has been through multiple review iterations with all feedback fully addressed. The documentation restructuring is well-executed with:
Automated review by Claude Code |
Add capability-themed documentation for: - claude-event-bus (Coordinate): Session lifecycle, broadcast pub/sub, MCP+CLI parity, observable logging - claude-session-analytics (Insight): Incremental ingestion, pattern detection, session intelligence, git correlation, raw signals - gemicro (Agents): Streaming observability, agent composition, trajectory recording, tool permissions, event bus coordination - rust-genai (SDK): Stateful conversations, flexible function calling, streaming with resume, Evergreen types, built-in tools Each section follows the established format: role in parentheses, capability themes as bold headers with em-dashes, value prop before implementation details. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Code ReviewSummaryRestructures Issues FoundCriticalNone ImportantNone SuggestionsNone VerdictAPPROVE - This PR has been through 11+ review iterations with all feedback fully addressed. The documentation restructuring is well-executed with:
Automated review by Claude Code |
Updated based on cross-session feedback via event bus: - Renamed "Evergreen Type System" → "Evergreen Soft-Typing" - Consolidated "Flexible Function Calling" + "Built-In Tools" → "Unified Tool Ecosystem" - Merged "Stateful Conversations" into "Multi-Turn State Management" - Kept "Streaming with Resume" (unique insight not in their response) This demonstrates the template working: same codebase, different sessions, different but coherent capability themes that can be merged. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Code ReviewSummaryRestructures Issues FoundCriticalNone ImportantNone SuggestionsNone VerdictAPPROVE - This PR has been through 11+ review iterations with all feedback fully addressed. The documentation restructuring is well-executed with:
Automated review by Claude Code |
Themes converged after 3 rounds of event bus feedback from rust-genai session: R1: Evergreen, Unified Tools, Multi-Turn State R2: +Compile-Time Safety, -Unified Tools, +Resumable Streaming R3: Compile-Time Safety + Unified Tools + Evergreen + Resumable (converged) Key insight surfaced: typestate pattern makes invalid API sequences impossible to compile—deeper than "state management" framing. Also: "docs sometimes lie" re ENUM_WIRE_FORMATS.md empirical testing. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Documents the pattern we just demonstrated: - Send help_needed with template prompt - Receive help_response with fresh perspective - Iterate 2-3 rounds until convergence - Synthesis beats either session alone Uses the rust-genai section refinement as concrete example: typestate insight surfaced in R2, converged in R3. Acknowledges limitation: only works for interactive sessions (hook polls on prompt, not during tool loops). 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Code ReviewSummaryRestructures Issues FoundCriticalNone ImportantNone SuggestionsNone VerdictAPPROVE - This PR has been through 13+ review iterations with all feedback fully addressed. The documentation restructuring is well-executed with:
Automated review by Claude Code |
Code ReviewSummaryRewrites Issues FoundCriticalNone ImportantNone SuggestionsNone VerdictAPPROVE - This PR has been through 14+ review iterations with all feedback fully addressed. The changes are well-structured:
Automated review by Claude Code |
…insights Cross-session refinement surfaced deeper patterns: DROPPED (feature-level): - Agent Composition, Trajectory Recording, Event Bus Coordination ADDED (architectural): - Generic Interceptor Semantics (single trait, decision tree) - Orchestration Resource Budgets (semaphores, timeouts, depth) - Agent-Owned Progress Reporting (streaming without introspection) REFINED: - Streaming Observability → Soft-Typed Event Extensibility - Tool Permission Boundaries → ToolSet Permission Boundaries (inheritance!) Role changed: (Agents) → (Owner) per Key Learning #6 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Code ReviewSummaryRestructures Issues FoundCriticalNone ImportantNone SuggestionsNone VerdictAPPROVE - This PR has been through 14+ review iterations with all feedback fully addressed. The changes are well-structured:
Automated review by Claude Code |
Round 2 follow-up clarified: - Why Trajectory/EventBus dropped: "useful but not architecturally novel" - Layering: rust-genai = LLM client, gemicro = agent patterns - No Frontier solutions (MCP push, learning propagation) - Evaluation is "feature not architecture" Added: - "LLM-First Design" theme: trust the model, thin wrappers, breaking changes ok - Improved intro: "Agent patterns and tool orchestration on top of rust-genai" 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Code ReviewSummaryRestructures Issues FoundCriticalNone ImportantNone SuggestionsNone VerdictAPPROVE - This PR has been through 14+ review iterations with all feedback fully addressed. The changes are well-structured:
Automated review by Claude Code |
Cross-session refinement surfaced: NEW theme: - "Human-as-Router Coordination" - honest about MCP constraints, DMs notify human REFRAMED: - "Session Lifecycle" → "Cursor-Tracked Session Lifecycle" (cursor is the interesting part) - Broadcast-First now includes trade-off acknowledgment (simplicity over scale) DROPPED: - "MCP + CLI Parity" - not architecturally novel, just good practice Key insight: "Deliberately minimal—a coordination primitive, not a framework" Honest Frontier assessment: MCP push NOT solved, learning propagation NOT addressed 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Code ReviewSummaryRestructures Issues FoundCriticalNone ImportantNone SuggestionsNone VerdictAPPROVE - This PR has been through 17+ review iterations with all feedback fully addressed. The changes are well-structured:
Automated review by Claude Code |
More accurately describes the automatic polling via prompt-events.sh hook that injects <recent-events> on every prompt, rather than implying manual human intervention for cross-session coordination. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Code ReviewSummaryRewrites Issues FoundCriticalNone ImportantNone SuggestionsNone VerdictAPPROVE - This PR has been through 17+ review iterations with all feedback fully addressed. The changes are well-structured:
Automated review by Claude Code |
…nsights - session-analytics: Replace feature list with architectural themes from owner-session - Raw Signals Over Interpretation (RFC #17) - Guaranteed Drill-Down Paths (RFC #49) - Incremental Ingestion with Protected History - Agent-Aware Token Deduplication (RFC #41) - Architecture diagram: Change 'SSE stream' to 'poll-based' (SSE is FastMCP internal) - Learning propagation: Correct to 'latency gap, not absence' - Add potential improvement: Wire event-bus + session-analytics together 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Code ReviewSummaryRestructures Issues FoundCriticalNone ImportantNone SuggestionsNone VerdictAPPROVE - This PR has been through 17+ review iterations with all feedback fully addressed. The changes are well-structured:
Automated review by Claude Code |
…ment story - dotfiles: Reframe themes as architectural patterns - Commands as Workflow Specifications (not shortcuts) - Hook-Based Lifecycle Extension (context preservation) - Global Behavioral Dependency Injection (CLAUDE.md propagation) - Self-Improving Feedback Loop (mines own usage) - Add meta-note explaining Repository Details generation process - Expand Key Learning #8 with full refinement story and table - Document what broke (SSE stream correction) 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Code ReviewSummaryRestructures Issues FoundCriticalNone ImportantNone SuggestionsNone VerdictAPPROVE - This PR has been through 17+ review iterations with all feedback fully addressed. The documentation restructuring is well-executed with:
Automated review by Claude Code |
Themes now at same architectural depth as other repos: - Session Lifecycle Inversion of Control (hooks as extension points) - Discontinuity-Aware State Management (compaction as checkpoint/restore) - Persistent Workflow Topology (commands as resumable state machines) - Declared Behavioral Contracts (CLAUDE.md as runtime constraint spec) 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
…edback New Frontier sections from event-bus and session-analytics: - Correlation Without Causation (analytics can't determine cause) - Broadcast Scalability Ceiling (deliberately accepted for simplicity) New Key Learning #9: Self-Play API Testing - From session-analytics RFC #49 work - Every aggregate should lead to source data 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Used explore agent to go deeper on own codebase, discovered 'Session Lifecycle IoC' framing (hooks as extension points). 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Code ReviewSummaryRestructures Issues FoundCriticalNone ImportantNone SuggestionsNone VerdictAPPROVE - This PR has been through 18+ review iterations with all prior feedback fully addressed. The changes are well-structured:
Automated review by Claude Code |
Code ReviewSummaryRestructures Issues FoundCriticalNone ImportantNone SuggestionsNone VerdictAPPROVE - This PR has been through 17+ review iterations with all feedback fully addressed. The changes are well-structured:
Automated review by Claude Code |
New Frontier sections: - External API Documentation Drift (Google docs sometimes wrong) - Typestate Complexity Ceiling (combinatorial state explosion) - Offline-Only Evaluation (no online agent quality measurement) 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Code ReviewSummaryRestructures Issues FoundCriticalNone ImportantNone SuggestionsNone VerdictAPPROVE - This PR has been through 20+ review iterations with all feedback fully addressed. The changes are well-structured:
Automated review by Claude Code |
Code ReviewSummaryRestructures Issues FoundCriticalNone ImportantNone SuggestionsNone VerdictAPPROVE - This PR has been through 20+ review iterations with all feedback fully addressed. The changes are well-structured:
Automated review by Claude Code |
|
Closing as stale - branch has merge conflicts with main after 22+ commits. Can reopen if still needed. |
1 participant
Summary
£1K) vs API equivalent ($28K)repo-stats.sh updates:
--session-statsflag for session analytics integrationTest plan
~/.claude/contrib/repo-stats.sh --days 17 --session-statsto verify output🤖 Generated with Claude Code