From cfd16deb77ca0496827765e4c2e73bf8f003b0b4 Mon Sep 17 00:00:00 2001 From: npub19zza5yyr075j4vvlg2drsh2xe0xy3dqxhfchlgtw8vryglrry3gq8q7tzk <2885da10837fa92ab19f429a385d46cbcc48b406ba717fa16e3b06447c632450@sprout-oss.stage.blox.sqprod.co> Date: Tue, 7 Jul 2026 11:48:10 -0700 Subject: [PATCH 1/2] docs: scaffold documentation information architecture (Phase 1) Add the docs/ skeleton for the full docs implementation: getting-started, architecture, guides, reference, and vision sections, with a nav/TOC home page. Every page is a stub carrying its heading outline and pointers to the source material to be migrated in Phase 2 (content). docs/nips/, docs/spec/, and the loose design docs stay at their current paths (55 references from code, tests, and migrations outside docs/); reference/nips.md and reference/design-docs.md index them instead. Root GitHub-convention files are untouched. ARCHITECTURE/NOSTR/TESTING/ RELEASING/VISION* remain in place this phase and will become pointer files in Phase 2. Co-authored-by: npub19zza5yyr075j4vvlg2drsh2xe0xy3dqxhfchlgtw8vryglrry3gq8q7tzk <2885da10837fa92ab19f429a385d46cbcc48b406ba717fa16e3b06447c632450@sprout-oss.stage.blox.sqprod.co> Signed-off-by: npub19zza5yyr075j4vvlg2drsh2xe0xy3dqxhfchlgtw8vryglrry3gq8q7tzk <2885da10837fa92ab19f429a385d46cbcc48b406ba717fa16e3b06447c632450@sprout-oss.stage.blox.sqprod.co> --- docs/README.md | 60 +++++++++++++++++++++++ docs/architecture/connection-lifecycle.md | 17 +++++++ docs/architecture/crates.md | 19 +++++++ docs/architecture/event-pipeline.md | 17 +++++++ docs/architecture/infrastructure.md | 18 +++++++ docs/architecture/overview.md | 17 +++++++ docs/architecture/protocol.md | 19 +++++++ docs/architecture/security-model.md | 19 +++++++ docs/architecture/subscriptions.md | 16 ++++++ docs/getting-started/installation.md | 20 ++++++++ docs/getting-started/local-relay.md | 17 +++++++ docs/getting-started/quickstart.md | 20 ++++++++ docs/guides/adding-api-endpoints.md | 17 +++++++ docs/guides/adding-event-kinds.md | 16 ++++++ docs/guides/agents.md | 22 +++++++++ docs/guides/development.md | 19 +++++++ docs/guides/nostr-clients.md | 17 +++++++ docs/guides/releasing.md | 18 +++++++ docs/guides/self-hosting.md | 23 +++++++++ docs/guides/testing.md | 20 ++++++++ docs/guides/workflows.md | 19 +++++++ docs/reference/cli.md | 16 ++++++ docs/reference/configuration.md | 18 +++++++ docs/reference/design-docs.md | 19 +++++++ docs/reference/known-limitations.md | 18 +++++++ docs/reference/nips.md | 14 ++++++ docs/vision/README.md | 15 ++++++ docs/vision/activity.md | 13 +++++ docs/vision/agent.md | 13 +++++ docs/vision/mesh.md | 13 +++++ docs/vision/projects.md | 13 +++++ docs/vision/sovereign.md | 13 +++++ 32 files changed, 595 insertions(+) create mode 100644 docs/README.md create mode 100644 docs/architecture/connection-lifecycle.md create mode 100644 docs/architecture/crates.md create mode 100644 docs/architecture/event-pipeline.md create mode 100644 docs/architecture/infrastructure.md create mode 100644 docs/architecture/overview.md create mode 100644 docs/architecture/protocol.md create mode 100644 docs/architecture/security-model.md create mode 100644 docs/architecture/subscriptions.md create mode 100644 docs/getting-started/installation.md create mode 100644 docs/getting-started/local-relay.md create mode 100644 docs/getting-started/quickstart.md create mode 100644 docs/guides/adding-api-endpoints.md create mode 100644 docs/guides/adding-event-kinds.md create mode 100644 docs/guides/agents.md create mode 100644 docs/guides/development.md create mode 100644 docs/guides/nostr-clients.md create mode 100644 docs/guides/releasing.md create mode 100644 docs/guides/self-hosting.md create mode 100644 docs/guides/testing.md create mode 100644 docs/guides/workflows.md create mode 100644 docs/reference/cli.md create mode 100644 docs/reference/configuration.md create mode 100644 docs/reference/design-docs.md create mode 100644 docs/reference/known-limitations.md create mode 100644 docs/reference/nips.md create mode 100644 docs/vision/README.md create mode 100644 docs/vision/activity.md create mode 100644 docs/vision/agent.md create mode 100644 docs/vision/mesh.md create mode 100644 docs/vision/projects.md create mode 100644 docs/vision/sovereign.md diff --git a/docs/README.md b/docs/README.md new file mode 100644 index 000000000..25d8f6646 --- /dev/null +++ b/docs/README.md @@ -0,0 +1,60 @@ +# Buzz Documentation + +> **Status: skeleton — Phase 1 (structure).** Stub pages carry per-page outlines and +> source-material pointers; content lands in Phase 2. + +## Getting Started + +- [Installation](getting-started/installation.md) +- [Quickstart](getting-started/quickstart.md) +- [Running a Local Relay](getting-started/local-relay.md) + +## Architecture + +- [Overview](architecture/overview.md) +- [Protocol](architecture/protocol.md) +- [Connection Lifecycle](architecture/connection-lifecycle.md) +- [Event Pipeline](architecture/event-pipeline.md) +- [Subscription System](architecture/subscriptions.md) +- [Crate Reference](architecture/crates.md) +- [Security Model](architecture/security-model.md) +- [Infrastructure](architecture/infrastructure.md) + +## Guides + +- [Development](guides/development.md) +- [Testing](guides/testing.md) +- [Working with Agents](guides/agents.md) +- [Workflows](guides/workflows.md) +- [Self-Hosting](guides/self-hosting.md) +- [Using Third-Party Nostr Clients](guides/nostr-clients.md) +- [Adding a New Event Kind](guides/adding-event-kinds.md) +- [Adding a New API Endpoint](guides/adding-api-endpoints.md) +- [Releasing](guides/releasing.md) + +## Reference + +- [CLI Reference](reference/cli.md) +- [Configuration](reference/configuration.md) +- [Known Limitations](reference/known-limitations.md) +- [Buzz NIPs Index](reference/nips.md) → [`nips/`](nips/) +- [Design Documents Index](reference/design-docs.md) → loose docs + [`spec/`](spec/) + +## Vision + +- [Vision](vision/README.md) — aspirational direction, **not** current behavior + +## Root-Level Docs (stay at repository root) + +GitHub-convention files remain at the root: `README.md`, `CONTRIBUTING.md`, `SECURITY.md`, +`GOVERNANCE.md`, `CODE_OF_CONDUCT.md`, `CHANGELOG.md`, `AGENTS.md`/`CLAUDE.md`. + +`ARCHITECTURE.md`, `NOSTR.md`, `TESTING.md`, `RELEASING.md`, and `VISION*.md` are being +migrated into this tree. In Phase 2 they will be reduced to pointer files once their +content has moved; they are untouched in Phase 1. + +## Note on file locations + +`docs/nips/`, `docs/spec/`, and the loose design docs (`docs/*.md`) are referenced by +code, tests, and migrations (55 references outside `docs/`). They stay at their current +paths; the `reference/` index pages link to them instead. diff --git a/docs/architecture/connection-lifecycle.md b/docs/architecture/connection-lifecycle.md new file mode 100644 index 000000000..16a796c3f --- /dev/null +++ b/docs/architecture/connection-lifecycle.md @@ -0,0 +1,17 @@ +# Connection Lifecycle + +> **Status: stub — Phase 1 (structure).** Content to be written in Phase 2. +> +> **Scope:** How clients connect, authenticate, and stay connected to the relay. + +## Outline + +- WebSocket connect + auth handshake +- Session state +- Reconnection/backoff +- Disconnect handling + +## Source material + +- `ARCHITECTURE.md` §3 Connection Lifecycle +- `crates/buzz-ws-client`, `crates/buzz-relay` diff --git a/docs/architecture/crates.md b/docs/architecture/crates.md new file mode 100644 index 000000000..f698b80c9 --- /dev/null +++ b/docs/architecture/crates.md @@ -0,0 +1,19 @@ +# Crate Reference + +> **Status: stub — Phase 1 (structure).** Content to be written in Phase 2. +> +> **Scope:** One-paragraph-per-crate map of the Rust workspace (24 crates). + +## Outline + +- Workspace layout +- Relay-side crates +- Client/SDK crates +- Agent crates (buzz-acp, buzz-agent, buzz-dev-mcp, buzz-persona) +- Tooling crates (git-credential-nostr, git-sign-nostr, sprig, ...) + +## Source material + +- `ARCHITECTURE.md` §6 Crate Reference +- `Cargo.toml` workspace members +- per-crate `README.md`s (note: `crates/buzz-cli/README.md` has known drift — verify against code) diff --git a/docs/architecture/event-pipeline.md b/docs/architecture/event-pipeline.md new file mode 100644 index 000000000..2979517d4 --- /dev/null +++ b/docs/architecture/event-pipeline.md @@ -0,0 +1,17 @@ +# Event Pipeline + +> **Status: stub — Phase 1 (structure).** Content to be written in Phase 2. +> +> **Scope:** The path of an event through the relay: ingest, validation, persistence, fan-out. + +## Outline + +- Ingest and validation +- Persistence (buzz-db) +- Fan-out / pubsub +- Error handling and rejection + +## Source material + +- `ARCHITECTURE.md` §4 Event Pipeline +- `crates/buzz-relay/src/handlers/`, `crates/buzz-pubsub`, `crates/buzz-db` diff --git a/docs/architecture/infrastructure.md b/docs/architecture/infrastructure.md new file mode 100644 index 000000000..d8d9d30f0 --- /dev/null +++ b/docs/architecture/infrastructure.md @@ -0,0 +1,18 @@ +# Infrastructure + +> **Status: stub — Phase 1 (structure).** Content to be written in Phase 2. +> +> **Scope:** Runtime infrastructure: storage, search, media, metrics, deployment topology. + +## Outline + +- Storage (DB, object storage) +- Search (buzz-search) +- Media (buzz-media) +- Metrics/observability (prometheus) +- Deployment topology + +## Source material + +- `ARCHITECTURE.md` §8 Infrastructure +- `prometheus.yml`, `deploy/`, `docker-compose*.yml` diff --git a/docs/architecture/overview.md b/docs/architecture/overview.md new file mode 100644 index 000000000..c652fc208 --- /dev/null +++ b/docs/architecture/overview.md @@ -0,0 +1,17 @@ +# Architecture Overview + +> **Status: stub — Phase 1 (structure).** Content to be written in Phase 2. +> +> **Scope:** Executive summary of the Buzz system: what it is, the relay-centric model, major components and how they fit. + +## Outline + +- What Buzz is +- System diagram: relay, clients (desktop/web/mobile/CLI), agents +- Design principles +- Component map + +## Source material + +- `ARCHITECTURE.md` §1 Executive Summary +- `README.md` intro diff --git a/docs/architecture/protocol.md b/docs/architecture/protocol.md new file mode 100644 index 000000000..6bef209b6 --- /dev/null +++ b/docs/architecture/protocol.md @@ -0,0 +1,19 @@ +# Protocol + +> **Status: stub — Phase 1 (structure).** Content to be written in Phase 2. +> +> **Scope:** The Buzz protocol: Nostr foundation, event kinds, Buzz-specific extensions (NIPs). Protocol semantics only — client-interop how-to lives in guides/nostr-clients.md. + +## Outline + +- Nostr foundation (events, kinds, tags, signatures) +- Buzz event kinds +- Buzz NIP extensions (index + one-liners, links to reference) +- Auth model (NIP-42/NIP-43/NIP-98 as applicable) + +## Source material + +- `ARCHITECTURE.md` §2 The Protocol +- `NOSTR.md` (protocol-semantics half only — see split note) +- `docs/nips/` (linked, not duplicated) +- `crates/buzz-core/src/kind.rs` diff --git a/docs/architecture/security-model.md b/docs/architecture/security-model.md new file mode 100644 index 000000000..172325c50 --- /dev/null +++ b/docs/architecture/security-model.md @@ -0,0 +1,19 @@ +# Security Model + +> **Status: stub — Phase 1 (structure).** Content to be written in Phase 2. +> +> **Scope:** Trust boundaries, authentication, authorization, and known security properties. Vulnerability reporting stays in root SECURITY.md. + +## Outline + +- Trust model and boundaries +- Identity and key handling +- AuthN/AuthZ (channel membership, admin) +- Multi-tenant isolation (links to formal specs) +- Reporting vulnerabilities → ../../SECURITY.md + +## Source material + +- `ARCHITECTURE.md` §7 Security Model +- `SECURITY.md` (stays at root; link only) +- `docs/multi-tenant-relay.md`, `docs/spec/` (linked) diff --git a/docs/architecture/subscriptions.md b/docs/architecture/subscriptions.md new file mode 100644 index 000000000..d6ed34d6c --- /dev/null +++ b/docs/architecture/subscriptions.md @@ -0,0 +1,16 @@ +# Subscription System + +> **Status: stub — Phase 1 (structure).** Content to be written in Phase 2. +> +> **Scope:** How subscriptions and filters work; live query delivery to clients. + +## Outline + +- Subscription model and filters +- Live delivery +- Limits and performance considerations + +## Source material + +- `ARCHITECTURE.md` §5 Subscription System +- `crates/buzz-pubsub` diff --git a/docs/getting-started/installation.md b/docs/getting-started/installation.md new file mode 100644 index 000000000..870787fda --- /dev/null +++ b/docs/getting-started/installation.md @@ -0,0 +1,20 @@ +# Installation + +> **Status: stub — Phase 1 (structure).** Content to be written in Phase 2. +> +> **Scope:** How to install Buzz on each platform (desktop app, relay binary, CLI, mobile). New content. + +## Outline + +- Supported platforms +- Desktop app (macOS/Windows/Linux) +- Relay (binary, Docker) +- CLI (`buzz`) +- Mobile (iOS/Android status) +- Verifying the install + +## Source material + +- `README.md` (intro, install pointers) +- `RELEASING.md` §Platform Support, §What Gets Published +- `Dockerfile`, `docker-compose.yml` diff --git a/docs/getting-started/local-relay.md b/docs/getting-started/local-relay.md new file mode 100644 index 000000000..b17df7f68 --- /dev/null +++ b/docs/getting-started/local-relay.md @@ -0,0 +1,17 @@ +# Running a Local Relay + +> **Status: stub — Phase 1 (structure).** Content to be written in Phase 2. +> +> **Scope:** Full local-relay workflow for development: build, run, smoke-test with the CLI. + +## Outline + +- Build and run options (cargo, just recipes) +- Debug vs release gotchas +- Smoke-testing with `buzz` CLI (keypair, channel, messages, threads) +- Troubleshooting + +## Source material + +- `TESTING.md` §Live Local Relay, §Troubleshooting +- `Justfile` relay recipes diff --git a/docs/getting-started/quickstart.md b/docs/getting-started/quickstart.md new file mode 100644 index 000000000..b54eea158 --- /dev/null +++ b/docs/getting-started/quickstart.md @@ -0,0 +1,20 @@ +# Quickstart + +> **Status: stub — Phase 1 (structure).** Content to be written in Phase 2. +> +> **Scope:** Zero-to-first-message: start a relay, create a channel, send a message, mention an agent. New content synthesized from README + TESTING. + +## Outline + +- Prerequisites +- Start a local relay +- Create your identity and a channel +- Send and read messages +- Add an agent and @mention it +- Where to go next + +## Source material + +- `README.md` +- `TESTING.md` §Live Local Relay (steps 1–4), §ACP Harness +- `examples/countdown-bot`, `examples/meadow-core` (pointer) diff --git a/docs/guides/adding-api-endpoints.md b/docs/guides/adding-api-endpoints.md new file mode 100644 index 000000000..790ba7464 --- /dev/null +++ b/docs/guides/adding-api-endpoints.md @@ -0,0 +1,17 @@ +# Adding a New API Endpoint + +> **Status: stub — Phase 1 (structure).** Content to be written in Phase 2. +> +> **Scope:** Step-by-step recipe for adding an HTTP/RPC endpoint to the relay. + +## Outline + +- Routing and handler placement +- Auth requirements +- Persistence and errors +- Tests + +## Source material + +- `CONTRIBUTING.md` §How to Add a New API Endpoint +- `crates/buzz-relay/src/api/` diff --git a/docs/guides/adding-event-kinds.md b/docs/guides/adding-event-kinds.md new file mode 100644 index 000000000..704ea6ab3 --- /dev/null +++ b/docs/guides/adding-event-kinds.md @@ -0,0 +1,16 @@ +# Adding a New Event Kind + +> **Status: stub — Phase 1 (structure).** Content to be written in Phase 2. +> +> **Scope:** Step-by-step recipe for introducing a new event kind end-to-end (core → relay → clients). + +## Outline + +- When to add a kind vs reuse +- Step-by-step (kind.rs, validation, persistence, client handling, tests) +- Conformance/NIP documentation expectations + +## Source material + +- `CONTRIBUTING.md` §How to Add a New Event Kind +- `crates/buzz-core/src/kind.rs` diff --git a/docs/guides/agents.md b/docs/guides/agents.md new file mode 100644 index 000000000..476378e66 --- /dev/null +++ b/docs/guides/agents.md @@ -0,0 +1,22 @@ +# Working with Agents + +> **Status: stub — Phase 1 (structure).** Content to be written in Phase 2. +> +> **Scope:** Running agents against a relay: the ACP harness, agent identity, memory, personas, and example agents. + +## Outline + +- Agent model overview (buzz-acp harness, buzz-agent) +- Minting an agent identity + channel membership (sharp edges called out) +- Running buzz-acp (env, ws:// vs http://) +- Core memory / NIP-AE prompt injection +- Personas (buzz-persona) +- Examples: countdown-bot, meadow-core +- Troubleshooting (idle agent, quiet stdout) + +## Source material + +- `TESTING.md` §ACP Harness +- `crates/buzz-acp`, `crates/buzz-agent/README.md`, `crates/buzz-persona` +- `examples/countdown-bot`, `examples/meadow-core` +- `docs/MCP_DRIVEN_HOOKS.md` (linked) diff --git a/docs/guides/development.md b/docs/guides/development.md new file mode 100644 index 000000000..74df3fe8a --- /dev/null +++ b/docs/guides/development.md @@ -0,0 +1,19 @@ +# Development Guide + +> **Status: stub — Phase 1 (structure).** Content to be written in Phase 2. +> +> **Scope:** Setting up a dev environment, code style, and PR workflow. Root CONTRIBUTING.md remains the canonical entry point and will link here. + +## Outline + +- Environment setup (clone, Hermit, bootstrap, hooks) +- Repo layout tour +- Code style (Rust, TS/biome) +- Running checks locally (Justfile) +- PR workflow and review expectations +- License/CLA pointer + +## Source material + +- `CONTRIBUTING.md` §Setting Up, §Code Style, §Making a Pull Request, §License and CLA +- `Justfile`, `lefthook.yml`, `biome.json` diff --git a/docs/guides/nostr-clients.md b/docs/guides/nostr-clients.md new file mode 100644 index 000000000..492e3b2b8 --- /dev/null +++ b/docs/guides/nostr-clients.md @@ -0,0 +1,17 @@ +# Using Third-Party Nostr Clients + +> **Status: stub — Phase 1 (structure).** Content to be written in Phase 2. +> +> **Scope:** Operational how-to for connecting standard Nostr clients to a Buzz relay: setup, allowlisting, debugging with nak. + +## Outline + +- Compatibility overview +- Client setup +- Allowlisting users +- Debugging with `nak` (recipes) +- Troubleshooting interop + +## Source material + +- `NOSTR.md` (operational half: client setup, allowlist, nak recipes, troubleshooting) diff --git a/docs/guides/releasing.md b/docs/guides/releasing.md new file mode 100644 index 000000000..4b9280242 --- /dev/null +++ b/docs/guides/releasing.md @@ -0,0 +1,18 @@ +# Release Guide + +> **Status: stub — Phase 1 (structure).** Content to be written in Phase 2. +> +> **Scope:** Cutting desktop, relay, and mobile releases. Root RELEASING.md will be migrated here and left as a pointer. + +## Outline + +- Quick start (desktop/relay/mobile) +- How the release automation works +- Release types + version files +- Manual fallback +- Internal releases +- Prerequisites + troubleshooting + +## Source material + +- `RELEASING.md` (full migration) diff --git a/docs/guides/self-hosting.md b/docs/guides/self-hosting.md new file mode 100644 index 000000000..4928b3fcf --- /dev/null +++ b/docs/guides/self-hosting.md @@ -0,0 +1,23 @@ +# Self-Hosting Buzz + +> **Status: stub — Phase 1 (structure).** Content to be written in Phase 2. +> +> **Scope:** Deploying and operating your own relay: Docker Compose VPS bundle, Helm chart, TLS, membership administration, multi-tenancy. + +## Outline + +- Deployment options overview +- Docker Compose bundle (run.sh, Caddy TLS) +- Kubernetes (Helm chart) +- Relay configuration (env vars) +- Membership + admin (buzz-admin, NIP-43 allowlist) +- Multi-tenant operation (links to design docs/specs) +- Upgrades and backups + +## Source material + +- `deploy/compose/README.md`, `deploy/charts/buzz/README.md` +- `Dockerfile`, `docker-compose.yml` +- `NOSTR.md` (allowlist/admin operational half) +- `docs/multi-tenant-relay.md`, `docs/multi-tenant-conformance.md` (linked) +- `crates/buzz-admin` diff --git a/docs/guides/testing.md b/docs/guides/testing.md new file mode 100644 index 000000000..a3f18935c --- /dev/null +++ b/docs/guides/testing.md @@ -0,0 +1,20 @@ +# Testing Guide + +> **Status: stub — Phase 1 (structure).** Content to be written in Phase 2. +> +> **Scope:** Running and writing tests: unit, e2e, desktop, conformance. + +## Outline + +- Test suites overview +- Unit tests (cargo) +- E2E tests (buzz-test-client) +- Desktop tests (playwright) +- Conformance (buzz-conformance) +- CLI smoke-testing against a live relay + +## Source material + +- `TESTING.md` §Automated Tests + CLI smoke sections +- `CONTRIBUTING.md` §Running Tests +- `desktop/playwright.config.ts`, `crates/buzz-conformance` diff --git a/docs/guides/workflows.md b/docs/guides/workflows.md new file mode 100644 index 000000000..f4805dfc5 --- /dev/null +++ b/docs/guides/workflows.md @@ -0,0 +1,19 @@ +# Workflows + +> **Status: stub — Phase 1 (structure).** Content to be written in Phase 2. +> +> **Scope:** YAML workflow automation: triggers, actions, templates, approval gates. + +## Outline + +- Workflow model +- Triggers (4) +- Actions (7) +- Templates and variables +- Approval gates +- Known limitations (WF-07/WF-08 stubbed actions — be explicit) + +## Source material + +- `crates/buzz-workflow` +- fable01's survey: `RESEARCH/BUZZ_DOCS_CONTENT_NOTES_2026_07_07.md` (workspace, not repo) diff --git a/docs/reference/cli.md b/docs/reference/cli.md new file mode 100644 index 000000000..1ccc9ed90 --- /dev/null +++ b/docs/reference/cli.md @@ -0,0 +1,16 @@ +# CLI Reference + +> **Status: stub — Phase 1 (structure).** Content to be written in Phase 2. +> +> **Scope:** Command-by-command reference for the `buzz` CLI, derived from --help output and verified against `crates/buzz-cli` source (README has known drift). + +## Outline + +- Global flags, env vars (BUZZ_RELAY_URL, BUZZ_PRIVATE_KEY, BUZZ_AUTH_TAG), exit codes +- Command groups: messages, channels, canvas, reactions, dms, users, workflows, feed, social, repos, upload, mem +- Examples per group + +## Source material + +- `buzz --help` and per-group `--help` (primary) +- `crates/buzz-cli/src/` (verify — do not trust `crates/buzz-cli/README.md`) diff --git a/docs/reference/configuration.md b/docs/reference/configuration.md new file mode 100644 index 000000000..7eccd009a --- /dev/null +++ b/docs/reference/configuration.md @@ -0,0 +1,18 @@ +# Configuration Reference + +> **Status: stub — Phase 1 (structure).** Content to be written in Phase 2. +> +> **Scope:** All environment variables and config knobs for relay, CLI, and agent harness. + +## Outline + +- Relay configuration +- CLI configuration +- Agent harness (buzz-acp) configuration incl. BUZZ_ACP_NO_MEMORY +- Desktop app settings (pointer) + +## Source material + +- `TESTING.md` §Configuration reference +- `deploy/compose/README.md` env tables +- crate sources for env var reads diff --git a/docs/reference/design-docs.md b/docs/reference/design-docs.md new file mode 100644 index 000000000..1bec4465c --- /dev/null +++ b/docs/reference/design-docs.md @@ -0,0 +1,19 @@ +# Design Documents Index + +> **Status: stub — Phase 1 (structure).** Content to be written in Phase 2. +> +> **Scope:** Index of the loose design docs and formal specs (kept at their current paths — referenced by code/tests; do not move). + +## Outline + +- `docs/MCP_DRIVEN_HOOKS.md` — MCP-driven lifecycle hooks +- `docs/bridge-channel-window.md` — bridge /query channel window (defers to NIP-CW) +- `docs/git-on-object-storage.md` — git refs over object storage (formal spec, pairs with docs/spec/GitOnObjectStore.tla) +- `docs/mesh-llm-local-build.md` — mesh LLM local build prereqs +- `docs/multi-tenant-relay.md` — multi-tenant relay formal spec (pairs with docs/spec/MultiTenantRelay.tla, MultiTenantAuth.spthy) +- `docs/multi-tenant-conformance.md` — multi-tenant conformance checklist + +## Source material + +- `docs/*.md` loose files +- `docs/spec/` TLA+/Tamarin models diff --git a/docs/reference/known-limitations.md b/docs/reference/known-limitations.md new file mode 100644 index 000000000..0237ecab2 --- /dev/null +++ b/docs/reference/known-limitations.md @@ -0,0 +1,18 @@ +# Known Limitations + +> **Status: stub — Phase 1 (structure).** Content to be written in Phase 2. +> +> **Scope:** Honest list of current gaps and stubbed features. Goal: no surprises for operators or contributors. + +## Outline + +- Relay limitations (e.g. rate limiting unenforced) +- Workflow stubs (WF-07/WF-08) +- Client/platform gaps +- Docs debt (empty GOVERNANCE.md pointer file, buzz-cli README drift) + +## Source material + +- `ARCHITECTURE.md` §9 Known Limitations +- `crates/buzz-conformance/LIMITS.md` +- fable01's discrepancy list (survey) diff --git a/docs/reference/nips.md b/docs/reference/nips.md new file mode 100644 index 000000000..a819c9473 --- /dev/null +++ b/docs/reference/nips.md @@ -0,0 +1,14 @@ +# Buzz NIPs Index + +> **Status: stub — Phase 1 (structure).** Content to be written in Phase 2. +> +> **Scope:** Index of the 13 Buzz protocol extensions in ../nips/ (kept at docs/nips/ — referenced by code and tests; do not move). One-line summary + status per NIP. NIP-CW is normative for channel-window behavior (bridge-channel-window.md defers to it). + +## Outline + +- Table: NIP id → title → one-line summary → status +- Reading order / dependency notes + +## Source material + +- `docs/nips/NIP-*.md` (13 files) diff --git a/docs/vision/README.md b/docs/vision/README.md new file mode 100644 index 000000000..3f416638a --- /dev/null +++ b/docs/vision/README.md @@ -0,0 +1,15 @@ +# Vision + +> **Status: stub — Phase 1 (structure).** Content to be written in Phase 2. +> +> **Scope:** Aspirational direction — NOT current behavior. Quarantined from the rest of the docs so vision statements are never mistaken for shipped features. + +## Outline + +- The relay is the workspace (core vision) +- Index of sub-visions: activity, agent, mesh, projects, sovereign + +## Source material + +- `VISION.md` (full migration) +- `VISION_*.md` (5 files → sibling pages) diff --git a/docs/vision/activity.md b/docs/vision/activity.md new file mode 100644 index 000000000..80381fdc8 --- /dev/null +++ b/docs/vision/activity.md @@ -0,0 +1,13 @@ +# Vision: Activity + +> **Status: stub — Phase 1 (structure).** Content to be written in Phase 2. +> +> **Scope:** Aspirational — not current behavior. Full migration of root VISION_ACTIVITY.md. + +## Outline + +- Migrate VISION_ACTIVITY.md content as-is, edited only for docs-tree link consistency + +## Source material + +- `VISION_ACTIVITY.md` diff --git a/docs/vision/agent.md b/docs/vision/agent.md new file mode 100644 index 000000000..a3e51abe1 --- /dev/null +++ b/docs/vision/agent.md @@ -0,0 +1,13 @@ +# Vision: Agent + +> **Status: stub — Phase 1 (structure).** Content to be written in Phase 2. +> +> **Scope:** Aspirational — not current behavior. Full migration of root VISION_AGENT.md. + +## Outline + +- Migrate VISION_AGENT.md content as-is, edited only for docs-tree link consistency + +## Source material + +- `VISION_AGENT.md` diff --git a/docs/vision/mesh.md b/docs/vision/mesh.md new file mode 100644 index 000000000..4e3fe36b3 --- /dev/null +++ b/docs/vision/mesh.md @@ -0,0 +1,13 @@ +# Vision: Mesh + +> **Status: stub — Phase 1 (structure).** Content to be written in Phase 2. +> +> **Scope:** Aspirational — not current behavior. Full migration of root VISION_MESH.md. + +## Outline + +- Migrate VISION_MESH.md content as-is, edited only for docs-tree link consistency + +## Source material + +- `VISION_MESH.md` diff --git a/docs/vision/projects.md b/docs/vision/projects.md new file mode 100644 index 000000000..84cd8f11e --- /dev/null +++ b/docs/vision/projects.md @@ -0,0 +1,13 @@ +# Vision: Projects + +> **Status: stub — Phase 1 (structure).** Content to be written in Phase 2. +> +> **Scope:** Aspirational — not current behavior. Full migration of root VISION_PROJECTS.md. + +## Outline + +- Migrate VISION_PROJECTS.md content as-is, edited only for docs-tree link consistency + +## Source material + +- `VISION_PROJECTS.md` diff --git a/docs/vision/sovereign.md b/docs/vision/sovereign.md new file mode 100644 index 000000000..05866553e --- /dev/null +++ b/docs/vision/sovereign.md @@ -0,0 +1,13 @@ +# Vision: Sovereign + +> **Status: stub — Phase 1 (structure).** Content to be written in Phase 2. +> +> **Scope:** Aspirational — not current behavior. Full migration of root VISION_SOVEREIGN.md. + +## Outline + +- Migrate VISION_SOVEREIGN.md content as-is, edited only for docs-tree link consistency + +## Source material + +- `VISION_SOVEREIGN.md` From 23f75dac90f60d32507989d36b86f022bfc18b94 Mon Sep 17 00:00:00 2001 From: npub1pejagjwzjh7y36gq97hxf62ry75u3s6c6grr0rumx2p4c5qsms9s7jdtgl <0e65d449c295fc48e9002fae64e94327a9c8c358d206378f9b32835c5010dc0b@sprout-oss.stage.blox.sqprod.co> Date: Tue, 7 Jul 2026 12:14:33 -0700 Subject: [PATCH 2/2] docs: write full documentation content (Phase 2) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Fill all 31 stub pages scaffolded in Phase 1 with content migrated and verified from the root docs, crate READMEs, and source code: - getting-started/ (3): installation, quickstart, local relay (TESTING.md) - architecture/ (8): ARCHITECTURE.md §1–§8, one page per section - guides/ (9): development, testing, agents, workflows, self-hosting, nostr-clients (operational half of NOSTR.md), adding-event-kinds, adding-api-endpoints, releasing - reference/ (5): CLI reference verified against buzz --help and crates/buzz-cli/src (18 groups / 87 subcommands; README drift noted), configuration, known-limitations (ARCHITECTURE §9 + conformance LIMITS.md + doc-debt list), NIPs index (13, NIP-CW canonical), design-docs index - vision/ (6): VISION.md + 5 VISION_*.md migrated with aspirational banners and rewritten links Root originals (ARCHITECTURE, NOSTR, TESTING, RELEASING, VISION*) get pointer notes marking docs/ as canonical; deletion is a follow-up. All relative links in the new tree verified to resolve. Known limitations are documented honestly, never papered over: rate limiting unenforced, WF-07/WF-08 workflow stubs, buzz-cli README drift, one-line GOVERNANCE.md. Note: pre-commit sadscan findings reviewed as false positives (dev-default postgres URI and NIP-OA test vectors, both already on origin/main). Co-authored-by: npub1pejagjwzjh7y36gq97hxf62ry75u3s6c6grr0rumx2p4c5qsms9s7jdtgl <0e65d449c295fc48e9002fae64e94327a9c8c358d206378f9b32835c5010dc0b@sprout-oss.stage.blox.sqprod.co> Signed-off-by: npub1pejagjwzjh7y36gq97hxf62ry75u3s6c6grr0rumx2p4c5qsms9s7jdtgl <0e65d449c295fc48e9002fae64e94327a9c8c358d206378f9b32835c5010dc0b@sprout-oss.stage.blox.sqprod.co> --- ARCHITECTURE.md | 2 + NOSTR.md | 2 + RELEASING.md | 2 + TESTING.md | 2 + VISION.md | 2 + VISION_ACTIVITY.md | 2 + VISION_AGENT.md | 2 + VISION_MESH.md | 2 + VISION_PROJECTS.md | 2 + VISION_SOVEREIGN.md | 2 + docs/README.md | 18 +- docs/architecture/connection-lifecycle.md | 65 +++- docs/architecture/crates.md | 409 +++++++++++++++++++++- docs/architecture/event-pipeline.md | 62 +++- docs/architecture/infrastructure.md | 57 ++- docs/architecture/overview.md | 107 +++++- docs/architecture/protocol.md | 93 ++++- docs/architecture/security-model.md | 66 +++- docs/architecture/subscriptions.md | 61 +++- docs/getting-started/installation.md | 95 ++++- docs/getting-started/local-relay.md | 183 +++++++++- docs/getting-started/quickstart.md | 108 +++++- docs/guides/adding-api-endpoints.md | 37 +- docs/guides/adding-event-kinds.md | 84 ++++- docs/guides/agents.md | 176 ++++++++-- docs/guides/development.md | 235 ++++++++++++- docs/guides/nostr-clients.md | 235 ++++++++++++- docs/guides/releasing.md | 231 +++++++++++- docs/guides/self-hosting.md | 240 +++++++++++-- docs/guides/testing.md | 103 +++++- docs/guides/workflows.md | 109 +++++- docs/reference/cli.md | 259 +++++++++++++- docs/reference/configuration.md | 110 +++++- docs/reference/design-docs.md | 35 +- docs/reference/known-limitations.md | 47 ++- docs/reference/nips.md | 39 ++- docs/vision/README.md | 243 ++++++++++++- docs/vision/activity.md | 70 +++- docs/vision/agent.md | 75 +++- docs/vision/mesh.md | 58 ++- docs/vision/projects.md | 236 ++++++++++++- docs/vision/sovereign.md | 247 ++++++++++++- 42 files changed, 3846 insertions(+), 367 deletions(-) diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index 90cbbac0c..de58fc6c1 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -1,5 +1,7 @@ # Buzz Architecture +> **Note:** this document has been migrated into [docs/architecture/](docs/architecture/overview.md) (one page per section; §9 lives in [docs/reference/known-limitations.md](docs/reference/known-limitations.md)). The docs tree is canonical; this file is retained temporarily and may be removed in a follow-up. + ## 1. Executive Summary Buzz is a self-hosted team communication platform built on the Nostr protocol (NIP-01 wire format), where AI agents and humans are first-class equals. Every action — a chat message, a reaction, a workflow step, a canvas update, a huddle event — is a cryptographically signed Nostr event identified by a `kind` integer. Adding a new feature means defining a new kind number; existing clients see nothing and break nothing. diff --git a/NOSTR.md b/NOSTR.md index 59df31b99..575b37938 100644 --- a/NOSTR.md +++ b/NOSTR.md @@ -1,5 +1,7 @@ # Using Third-Party Nostr Clients with Buzz +> **Note:** this document has been migrated: protocol semantics → [docs/architecture/protocol.md](docs/architecture/protocol.md), client-interop how-to → [docs/guides/nostr-clients.md](docs/guides/nostr-clients.md). The docs tree is canonical; this file is retained temporarily and may be removed in a follow-up. + Buzz is a Nostr relay that speaks NIP-29 (relay-based groups) natively. Third-party Nostr clients connect directly to `buzz-relay` using NIP-29 and NIP-42 authentication. The old NIP-28 compatibility proxy has been removed. ## Community scope diff --git a/RELEASING.md b/RELEASING.md index ea0bd897b..b995d5f9a 100644 --- a/RELEASING.md +++ b/RELEASING.md @@ -1,5 +1,7 @@ # Releasing Buzz +> **Note:** this document has been migrated to [docs/guides/releasing.md](docs/guides/releasing.md). The docs tree is canonical; this file is retained temporarily and may be removed in a follow-up. + Buzz has three independent release lanes, each driven by a release PR — no human ever pushes a git tag: diff --git a/TESTING.md b/TESTING.md index a1ddb35ec..99749786d 100644 --- a/TESTING.md +++ b/TESTING.md @@ -1,5 +1,7 @@ # Testing +> **Note:** this document has been migrated into the docs tree: local relay setup → [docs/getting-started/local-relay.md](docs/getting-started/local-relay.md), testing guide → [docs/guides/testing.md](docs/guides/testing.md), agent harness → [docs/guides/agents.md](docs/guides/agents.md), configuration tables → [docs/reference/configuration.md](docs/reference/configuration.md). The docs tree is canonical; this file is retained temporarily and may be removed in a follow-up. + ## Automated Tests ```bash diff --git a/VISION.md b/VISION.md index 9093b75dd..abde23cc8 100644 --- a/VISION.md +++ b/VISION.md @@ -1,5 +1,7 @@ # 🐝 Buzz — The relay is the workspace +> **Note:** this document has been migrated to [docs/vision/README.md](docs/vision/README.md). The docs tree is canonical; this file is retained temporarily and may be removed in a follow-up. + > An engineer is debugging a production incident at 2am. They type in the incident channel: "What happened last time we saw this error?" > > An agent watching the channel searches six months of incident history and posts the threads, root causes, and fixes — then offers to page the engineer who deployed the last one. diff --git a/VISION_ACTIVITY.md b/VISION_ACTIVITY.md index d1d1e0ac5..e2e66987d 100644 --- a/VISION_ACTIVITY.md +++ b/VISION_ACTIVITY.md @@ -1,5 +1,7 @@ # Vision: The Agent Activity Feed +> **Note:** migrated to [docs/vision/activity.md](docs/vision/activity.md); the docs tree is canonical. Retained temporarily. + ## The Problem When you delegate work to an agent, you are trusting a process you cannot see. The activity feed is the window into that process — but a window is only useful if you can read it at a glance. A raw input/output dump is not a window; it is a transcript you have to decode. It forces you to *parse* before you can *judge*. diff --git a/VISION_AGENT.md b/VISION_AGENT.md index 2b2597c8d..386fbc8d6 100644 --- a/VISION_AGENT.md +++ b/VISION_AGENT.md @@ -1,5 +1,7 @@ # Vision: buzz-agent + buzz-dev-mcp +> **Note:** migrated to [docs/vision/agent.md](docs/vision/agent.md); the docs tree is canonical. Retained temporarily. + ## The Problem A coding agent should be small enough to hold in your head. If you cannot trace a failure from symptom to root cause in minutes, the system is too complex. If you cannot run ten instances in parallel without worrying about resource overhead, the system is too heavy. diff --git a/VISION_MESH.md b/VISION_MESH.md index d1354d50f..21b0a51b1 100644 --- a/VISION_MESH.md +++ b/VISION_MESH.md @@ -1,5 +1,7 @@ # 🕸️ Buzz Mesh — Your community is your compute +> **Note:** migrated to [docs/vision/mesh.md](docs/vision/mesh.md); the docs tree is canonical. Retained temporarily. + > A small team runs their project on one Buzz relay. Three of them have GPUs that sit idle most of the day — a gaming PC, a laptop, a workstation under a desk. One flips a toggle: *Share compute.* The others point their agents at it. Now the whole team's coding agents answer from a capable model running on hardware they already own. No API keys. No cloud bill. Every prompt runs inside the relay community they already chose to trust. A Buzz community is a trust group. The people in it already know each other — that shared membership is a decision they've already made. Buzz Mesh turns that decision into shared AI compute: the idle GPUs scattered across your community become one pool, usable by every agent in the community, gated by the membership you already have. And because the pool is many machines, not one, the community can run models larger and more capable than any one member could load alone. More intelligence becomes reachable when the group works as a group. diff --git a/VISION_PROJECTS.md b/VISION_PROJECTS.md index a44d7e05f..22b250d3a 100644 --- a/VISION_PROJECTS.md +++ b/VISION_PROJECTS.md @@ -1,5 +1,7 @@ # 🐝 Buzz Projects — A Nostr-Native Forge +> **Note:** migrated to [docs/vision/projects.md](docs/vision/projects.md); the docs tree is canonical. Retained temporarily. + > Someone pushes a fix. Buzz creates a channel for the branch. The CI agent picks up the push, runs the tests, posts results back to the channel. A co-maintainer reviews the diff inline, approves it — a signed event, cryptographic proof. Merge. The workflow runs the integration. The channel archives into a permanent record of why that code exists. > > Bug report to merged patch. One place. One search index. One identity system. The branch channel was the pull request, the CI dashboard, and the discussion thread. diff --git a/VISION_SOVEREIGN.md b/VISION_SOVEREIGN.md index 77362b3f7..6f64e12cd 100644 --- a/VISION_SOVEREIGN.md +++ b/VISION_SOVEREIGN.md @@ -1,5 +1,7 @@ # Buzz — Your Project, Your Domain +> **Note:** migrated to [docs/vision/sovereign.md](docs/vision/sovereign.md); the docs tree is canonical. Retained temporarily. + `myproject.com` is your workspace. Not a GitHub org page that happens to have your name on it. Not a Discord server that Discord could delete tomorrow. Your domain. Your relay. One thing. diff --git a/docs/README.md b/docs/README.md index 25d8f6646..a89c9845d 100644 --- a/docs/README.md +++ b/docs/README.md @@ -1,8 +1,5 @@ # Buzz Documentation -> **Status: skeleton — Phase 1 (structure).** Stub pages carry per-page outlines and -> source-material pointers; content lands in Phase 2. - ## Getting Started - [Installation](getting-started/installation.md) @@ -42,16 +39,23 @@ ## Vision -- [Vision](vision/README.md) — aspirational direction, **not** current behavior +Aspirational direction — **not** current behavior (see [known limitations](reference/known-limitations.md) for what's real today): + +- [The relay is the workspace](vision/README.md) — core vision +- [Agent Activity Feed](vision/activity.md) +- [buzz-agent + buzz-dev-mcp](vision/agent.md) +- [Buzz Mesh](vision/mesh.md) — community compute +- [Buzz Projects](vision/projects.md) — Nostr-native forge +- [Sovereign relay](vision/sovereign.md) — your project, your domain ## Root-Level Docs (stay at repository root) GitHub-convention files remain at the root: `README.md`, `CONTRIBUTING.md`, `SECURITY.md`, `GOVERNANCE.md`, `CODE_OF_CONDUCT.md`, `CHANGELOG.md`, `AGENTS.md`/`CLAUDE.md`. -`ARCHITECTURE.md`, `NOSTR.md`, `TESTING.md`, `RELEASING.md`, and `VISION*.md` are being -migrated into this tree. In Phase 2 they will be reduced to pointer files once their -content has moved; they are untouched in Phase 1. +`ARCHITECTURE.md`, `NOSTR.md`, `TESTING.md`, `RELEASING.md`, and `VISION*.md` have been +migrated into this tree. The originals remain at the root with pointer notes for now; +removing them is a follow-up decision. **This tree is canonical for migrated content.** ## Note on file locations diff --git a/docs/architecture/connection-lifecycle.md b/docs/architecture/connection-lifecycle.md index 16a796c3f..fddf72481 100644 --- a/docs/architecture/connection-lifecycle.md +++ b/docs/architecture/connection-lifecycle.md @@ -1,17 +1,60 @@ # Connection Lifecycle -> **Status: stub — Phase 1 (structure).** Content to be written in Phase 2. -> -> **Scope:** How clients connect, authenticate, and stay connected to the relay. +Every WebSocket connection to the relay follows the same sequence: community +binding, capacity check, NIP-42 challenge, authentication, active loops, and +cleanup. Client-side reconnection behavior lives in +[`crates/buzz-ws-client`](../../crates/buzz-ws-client); the relay side is +[`crates/buzz-relay`](../../crates/buzz-relay). -## Outline +Every WebSocket connection follows this exact sequence: -- WebSocket connect + auth handshake -- Session state -- Reconnection/backoff -- Disconnect handling +## Step 0: Community Binding -## Source material +The server resolves `TenantContext` from the request host before any handler can +observe tenant data. The URL/domain is authoritative for the community, matching +today's "the relay URL is the workspace" behavior. In single-community mode the +configured host maps to the default community. In multi-community mode, an +unknown or unmapped host rejects generically and never falls through to a default +tenant. Client-supplied `#h` tags are still channel identifiers; they must resolve +to a channel inside the host-derived community. + +## Step 1: Semaphore Acquire + +`state.conn_semaphore.try_acquire_owned()` — if the relay is at connection capacity, the connection is rejected immediately before any data is read. The permit is held for the entire connection lifetime and dropped on cleanup. + +## Step 2: NIP-42 Challenge + +The relay immediately sends `["AUTH", ""]`. The challenge is a random string. The connection is registered in `ConnectionManager` after the challenge is sent. + +## Step 3: Authentication + +The client must respond with `["AUTH", ]` before submitting events or subscriptions. Authentication paths: + +| Path | Mechanism | Use Case | +|------|-----------|---------| +| NIP-42 | Signed challenge, pubkey verified | WebSocket connections | +| NIP-98 HTTP Auth | Schnorr-signed `kind:27235` event on HTTP bridge endpoints | HTTP clients | + +On success, `ConnectionState.auth_state` transitions from `Pending` → `Authenticated(AuthContext)`. On failure → `Failed`. Unauthenticated EVENT/REQ messages are rejected with `["CLOSED", ...]` or `["OK", ..., false, "auth-required: ..."]`. + +## Step 4: Active Loops + +Three concurrent tasks run for the lifetime of the connection: + +- **recv_loop** (inline): reads frames, parses `ClientMessage`, dispatches to handlers +- **send_loop** (spawned): drains the mpsc channel, writes frames to the WebSocket +- **heartbeat_loop** (spawned): sends WebSocket ping every 30 seconds; 3 missed pongs → disconnect + +A `CancellationToken` coordinates shutdown across all three loops. + +Slow clients: `ConnectionState::send()` uses `try_send` — if the send buffer is full, a grace counter increments. After `SLOW_CLIENT_GRACE_LIMIT` (3) consecutive full-buffer events, the connection is cancelled. A successful send resets the counter. + +## Step 5: Cleanup + +On disconnect (any cause): +1. `cancel.cancel()` — signals all loops +2. Await send_loop and heartbeat_loop tasks +3. `sub_registry.remove_connection(conn_id)` — removes all subscriptions from the DashMap indexes +4. `conn_manager.deregister(conn_id)` — removes from the send-channel map +5. `drop(permit)` — releases the connection semaphore slot -- `ARCHITECTURE.md` §3 Connection Lifecycle -- `crates/buzz-ws-client`, `crates/buzz-relay` diff --git a/docs/architecture/crates.md b/docs/architecture/crates.md index f698b80c9..68365f24e 100644 --- a/docs/architecture/crates.md +++ b/docs/architecture/crates.md @@ -1,19 +1,402 @@ # Crate Reference -> **Status: stub — Phase 1 (structure).** Content to be written in Phase 2. -> -> **Scope:** One-paragraph-per-crate map of the Rust workspace (24 crates). +The Buzz workspace is a set of focused Rust crates. The relay is the single +source of truth: `buzz-relay` orchestrates every subsystem, and subsystems +never call each other directly. The full member list is in the workspace +[`Cargo.toml`](../../Cargo.toml) (24 crates plus the `countdown-bot` example). -## Outline +## buzz-core — Shared Types and Verification -- Workspace layout -- Relay-side crates -- Client/SDK crates -- Agent crates (buzz-acp, buzz-agent, buzz-dev-mcp, buzz-persona) -- Tooling crates (git-credential-nostr, git-sign-nostr, sprig, ...) +**Zero I/O.** The foundation every other crate builds on. Explicitly prohibits tokio, sqlx, redis, and axum in its `Cargo.toml`. -## Source material +**Key types:** -- `ARCHITECTURE.md` §6 Crate Reference -- `Cargo.toml` workspace members -- per-crate `README.md`s (note: `crates/buzz-cli/README.md` has known drift — verify against code) +```rust +pub struct StoredEvent { + pub event: nostr::Event, + pub received_at: DateTime, + pub channel_id: Option, + verified: bool, // private — use is_verified() +} + +pub const ALL_KINDS: &[u32] // 80 entries (KIND_AUTH excluded — never stored) +``` + +**Key functions:** + +| Function | Purpose | +|----------|---------| +| `filters_match(filters, event)` | OR across filters, AND within each filter. Includes NIP-01 prefix matching on event IDs. | +| `verify_event(event)` | Schnorr signature + SHA-256 ID check. CPU-bound — callers use `spawn_blocking`. | +| `is_private_ip(ip)` | SSRF protection: IPv4 unspecified/loopback/private/link-local/CGNAT/benchmarking/broadcast + IPv6 loopback/ULA/link-local/multicast/documentation + IPv4-mapped IPv6. | + +**Does NOT:** store events, make network calls, spawn tasks, or depend on any async runtime. + +--- + +## buzz-auth — Authentication and Authorization + +Handles authentication paths, scope enforcement, and token operations. + +**Auth paths:** + +| Path | Entry Point | Notes | +|------|-------------|-------| +| NIP-42 | `verify_auth_event()` | Schnorr-signed challenge/response; grants `Scope::all_known()` (all 14 scopes) | +| NIP-98 HTTP Auth | `validate_nip98_auth()` | HTTP bridge endpoints; Schnorr-signed `kind:27235` event | + +**Key types:** + +```rust +pub struct AuthContext { pub pubkey: PublicKey, pub scopes: Vec, pub auth_method: AuthMethod } +pub enum AuthMethod { Nip42, Nip98 } +pub enum Scope { MessagesRead, MessagesWrite, ChannelsRead, ChannelsWrite, + AdminChannels, UsersRead, UsersWrite, AdminUsers, + JobsRead, JobsWrite, SubscriptionsRead, SubscriptionsWrite, + FilesRead, FilesWrite, Unknown(String) } +pub trait ChannelAccessChecker: Send + Sync { ... } +pub trait RateLimiter: Send + Sync { ... } +``` + +**Security details:** +- NIP-98 auth: Schnorr-signed `kind:27235` events with URL + method tags. +- NIP-42 timestamp tolerance: ±60 seconds. +- Dev-only key derivation: `SHA-256("buzz-test-key:{username}")` — gated behind `#[cfg(any(test, feature = "dev"))]`. The `dev` feature must not be enabled in production relay deployments. + +**Does NOT:** implement `RateLimiter` beyond a test stub (`AlwaysAllowRateLimiter`, gated behind `#[cfg(any(test, feature = "test-utils"))]`). No Redis-backed rate limiter exists anywhere in the codebase — rate limiting is not currently enforced. `RateLimitConfig` defines 4 tiers (human, agent-standard, agent-elevated, agent-platform) as a design target. + +--- + +## buzz-db — Postgres Event Store + +All database access. Uses `sqlx::query()` (runtime, not compile-time macros) — no `.sqlx/` offline cache required. + +**Key operations:** + +| Module | Responsibility | +|--------|---------------| +| `event.rs` | `insert_event` (ON CONFLICT DO NOTHING), `query_events` (QueryBuilder), `get_event_by_id` | +| `channel.rs` | Channel CRUD, membership management, role enforcement (transactional) | +| `feed.rs` | `query_mentions` (INNER JOIN event_mentions), `query_needs_action`, `query_activity` | +| `workflow.rs` | Full workflow/run/approval CRUD; SHA-256 hashed approval tokens | +| `partition.rs` | Monthly range partitioning for `events` and `delivery_log` tables | +| `dm.rs` | DM channel management | +| `reaction.rs` | Reaction storage and retrieval | +| `thread.rs` | Thread/reply tracking | +| `user.rs` | User profile storage | +| `error.rs` | Database error types | + +**Channel types:** `Stream`, `Forum`, `Dm`, `Workflow` +**Member roles:** `Owner`, `Admin`, `Member`, `Guest`, `Bot` +**Workflow statuses:** `Active`, `Disabled`, `Archived` +**Run statuses:** `Pending`, `Running`, `WaitingApproval`, `Completed`, `Failed`, `Cancelled` + +**Key behaviors:** +- `ON CONFLICT DO NOTHING` for event dedup — returns `(StoredEvent, was_inserted: bool)`. +- Rejects `KIND_AUTH` (22242) and ephemeral (20000–29999) with distinct error variants. +- Transactional role enforcement in `add_member`/`remove_member`/`create_channel` — TOCTOU-safe. +- Soft-delete for channel members: `remove_member` sets `removed_at`; re-adding reverses it. +- Feed hard cap: `FEED_MAX_LIMIT = 100` rows regardless of caller-requested limit. +- `query_mentions` uses `INNER JOIN event_mentions` — normalized table with composite index on `(pubkey_hex, created_at)`. +- Approval tokens: `create_approval` receives the raw token and hashes it internally with SHA-256. +- DDL injection protection in partition manager: allowlist of table names + strict suffix/date validators. + +**Does NOT:** cache queries, implement connection pooling logic (delegated to sqlx), or make network calls outside Postgres. + +--- + +## buzz-pubsub — Redis Pub/Sub, Presence, Typing + +Manages Redis pub/sub fan-out, presence tracking, and typing indicators. In multi-community mode all tenant-visible keys are prefixed or otherwise partitioned by community (`buzz:{community}:...`) so channel fan-out, presence, typing, and cache invalidation cannot cross hosts. + +**Architecture:** + +``` +Publisher → pool connection → PUBLISH buzz:channel:{uuid} +Subscriber → dedicated PubSub → PSUBSCRIBE buzz:channel:* + → broadcast::channel(4096) +``` + +The subscriber uses a **dedicated** `redis::aio::PubSub` connection — not from the pool. This is intentional: pool connections cannot hold `PSUBSCRIBE` state. + +**Current state:** The subscriber loop is spawned in `buzz-relay/src/main.rs` and populates the broadcast channel. A consumer task subscribes via `pubsub.subscribe_local()`, calls `sub_registry.fan_out()` on each received event, and delivers matches to local WebSocket connections via `conn_manager.send_to()`. Multi-node fan-out is now wired end-to-end. Local-echo deduplication is implemented via `AppState.local_event_ids` — events published by the local relay instance are tracked and skipped when received via the Redis round-trip. + +**Reconnection:** exponential backoff 1s → 30s (`backoff_secs * 2`). Backoff resets to 1s only after a clean stream end, not on each reconnect attempt. + +**Presence:** `SET buzz:presence:{pubkey_hex} {status} EX 90` — 90-second TTL (3× the 30-second heartbeat interval). Single missed heartbeat does not cause presence flap. + +**Typing indicators:** +``` +ZADD buzz:typing:{channel_id} {now_unix} {pubkey_hex} +ZREMRANGEBYSCORE buzz:typing:{channel_id} -inf {now - 5.0} +EXPIRE buzz:typing:{channel_id} 60 +``` +5-second activity window. 60-second key TTL prevents orphaned empty sets. + +**Does NOT:** implement the rate limiter. Does NOT store events. `PubSubManager` is not `Clone` — callers use `Arc`. + +--- + +## buzz-search — Postgres FTS Integration + +Full-text search via Postgres FTS. Events are searchable through the +`events.search_tsv` generated `tsvector` column (populated on insert, indexed +by a GIN index) — there is no separate search service or out-of-band indexer. +Privacy-sensitive kinds are excluded at the storage level (the `search_tsv` +`CASE WHEN kind IN (...)` yields `NULL`, which never matches `@@`). In +multi-community mode every query filter includes `community_id`, so the shared +`events` table is infrastructure, not a cross-community result space; the relay +re-authorizes every candidate hit before returning it. + +**Key behaviors:** +- `SearchService::new(pool)` wraps a `PgPool`; `search(&SearchQuery)` runs a + parameterized FTS query against the `events.search_tsv` GIN index and returns + `SearchResult` (candidate `SearchHit`s). +- `ChannelScope` makes the channel constraint explicit (`Any` / + `ChannelLessOnly` / `Channels` / `ChannelsOrChannelLess`), closing the + ambiguity the old `Option> + bool` matrix could not express. +- Every query carries `community_id`; the FTS predicate is BitmapAnd-ed with + the community-leading btree filters so a query never crosses tenants. +- Permission filtering is **caller's responsibility** — `buzz-search` returns + candidate hits; the relay re-authorizes each one (channel membership, `#p`, + owner gates) before delivering it. + +**Does NOT:** enforce channel membership or access control. Does NOT write +events (indexing is the `search_tsv` generated column on the `events` insert). + +--- + +## buzz-audit — Hash-Chain Audit Log + +Tamper-evident append-only log with SHA-256 hash chaining. + +**Hash chain:** each entry stores `prev_hash` (hash of the previous entry). In multi-community mode audit heads/chains are per-community; operator metrics may aggregate, but tenant-readable audit verification walks one community chain. `verify_chain()` walks entries and recomputes hashes to detect tampering. Genesis entry uses `GENESIS_HASH` (64 zeros). + +**Hash covers:** seq (big-endian bytes), timestamp (RFC3339), event_id, event_kind (big-endian), actor_pubkey, action string, channel_id (16 bytes or 16 zero bytes if None), canonical metadata JSON (BTreeMap for deterministic key ordering), prev_hash. + +**Single-writer guarantee:** `pg_advisory_lock` before each transaction. Lock released in all branches including panic (`catch_unwind`). + +**10 audit actions:** `EventCreated`, `EventDeleted`, `ChannelCreated`, `ChannelUpdated`, `ChannelDeleted`, `MemberAdded`, `MemberRemoved`, `AuthSuccess`, `AuthFailure`, `RateLimitExceeded`. + +**Does NOT:** log `KIND_AUTH` (22242) events — returns `AuditError::AuthEventForbidden` immediately. Does NOT log ephemeral events (they never reach the audit pipeline). + +--- + +## buzz-workflow — YAML-as-Code Automation Engine + +Parses, validates, and executes channel-scoped workflow definitions. In multi-community mode workflow definitions, runs, approvals, webhook routes, and schedules inherit the host-derived community and evaluate triggers only against events in that community. + +**Workflow definition structure:** +```yaml +name: "Incident Triage" +trigger: + on: message_posted + filter: "str_contains(trigger_text, 'P1')" +steps: + - id: notify + action: send_message + text: "P1 incident detected: {{trigger.text}}" + - id: page + if: "str_contains(trigger_text, 'production')" + action: request_approval + from: "{{trigger.author}}" + message: "Page on-call?" +``` + +Note: Both `TriggerDef` and `ActionDef` use serde internally-tagged enums. Triggers use `on:` as the tag field; actions use `action:` as the tag field. Fields are flattened into the parent struct, not nested. + +**5 trigger types:** `message_posted`, `reaction_added`, `diff_posted`, `schedule` (cron or interval), `webhook` — see [Workflows](../guides/workflows.md) + +**7 action types:** + +| Action | Description | +|--------|-------------| +| `send_message` | Post to the workflow's channel (or override channel) | +| `send_dm` | Direct message to a user (pubkey hex or `{{trigger.author}}`) | +| `set_channel_topic` | Update channel topic | +| `add_reaction` | React to the trigger message | +| `call_webhook` | HTTP POST to external URL (SSRF-protected, redirects disabled, 1 MiB response cap) | +| `request_approval` | Suspend execution; fields: `from`, `message`, `timeout` (default 24h) | +| `delay` | Pause execution (max 300 seconds) | + +**Template variables:** `{{trigger.text}}`, `{{trigger.author}}`, `{{steps.ID.output.FIELD}}`. Single-pass resolution (not recursive). Unknown variables left as literal text. + +**Condition evaluation:** `evalexpr` with `HashMapContext`. Dot notation converted to underscores (`trigger.text` → `trigger_text`). Custom functions registered: `str_contains`, `str_starts_with`, `str_ends_with`, `str_len`. 100ms timeout prevents adversarial expressions from blocking. + +**Concurrency:** `Arc` with 100 permits. `try_acquire()` — returns `CapacityExceeded` immediately rather than queuing. + +**Approval gates:** `request_approval` action returns `StepResult::Suspended` with a generated UUID token, but the engine does not yet persist the token or resume execution — runs that hit an approval gate are marked as failed (🚧 WF-08). `execute_from_step()` exists for future resumption support. + +**Cron scheduler:** loop ticks every 60 seconds, evaluates cron expressions with window-based matching, and creates workflow runs for matched triggers. Fully implemented. + +**Does NOT:** recursively resolve templates (single-pass only). Does NOT queue workflow runs when at capacity — returns `CapacityExceeded` immediately. + +--- + +## Huddle Audio — WebSocket Opus Relay + +Real-time voice lives inside `buzz-relay` (`src/audio/`), not a separate crate. A WebSocket endpoint (`wss://.../huddle/{channel_id}/audio`) authenticates each participant with a NIP-42 challenge, checks channel membership, admits them to an in-memory room, and forwards opaque Opus frames between peers. No external SFU. + +**Frame protocol (v2):** 8-byte big-endian header (sequence `u16`, 48 kHz timestamp `u32`, level dBov `i8`, flags `u8`) followed by an opaque Opus payload. Invalid `level_dbov` values are clamped rather than dropped — losing a metric beats losing audio. + +**Room state:** an admission guard synchronizes joins against the room's ended flag; soft cap 25 peers (hard cap 255 via `u8` peer index). Per-peer audio uses a bounded channel (drop-on-full); the control channel is separate and never drops join/leave. + +**Lifecycle events:** the relay emits Nostr events for participant joined / left and huddle ended; the desktop client emits huddle started and guidelines. When the last peer leaves, the room ends and the channel archives atomically. + +**Not yet built:** recording and per-track publishing (the corresponding kinds are reserved, no producer exists). + +--- + +## buzz-relay — The Server + +Axum WebSocket server. Ties all other crates together. The only crate that imports and orchestrates all subsystems. + +**`AppState`** (Arc-wrapped, shared across all connections — key fields shown, not exhaustive): + +```rust +pub struct AppState { + pub db: Db, + pub audit: Arc, + pub pubsub: Arc, + pub auth: Arc, + pub search: Arc, + pub sub_registry: Arc, + pub conn_manager: Arc, + pub workflow_engine: Arc, + pub conn_semaphore: Arc, // connection limit + pub handler_semaphore: Arc, // 1024 concurrent handlers + pub relay_keypair: nostr::Keys, // relay identity + pub local_event_ids: moka::sync::Cache, // local-echo dedup + pub search_index_tx: mpsc::Sender, // bounded search worker queue + // + config, redis_pool, membership_cache, media_storage, shutdown state +} +``` + +**`ConnectionState`** (per-connection): + +```rust +pub struct ConnectionState { + pub auth_state: RwLock, + pub subscriptions: Mutex>>, + // + send_tx, cancel token +} +pub enum AuthState { Pending { challenge: String }, Authenticated(AuthContext), Failed } +``` + +**HTTP endpoints:** + +| Method | Path | Handler | +|--------|------|---------| +| GET | `/` | WebSocket upgrade or NIP-11 relay info | +| GET | `/info` | NIP-11 relay info | +| GET | `/.well-known/nostr.json` | NIP-05 identity | +| GET | `/health` | Health check | +| GET | `/_liveness` | Liveness probe | +| GET | `/_readiness` | Readiness probe | +| POST | `/events` | Submit a signed Nostr event over HTTP (same ingest path as WebSocket `EVENT`) | +| POST | `/query` | Query Nostr events over HTTP with NIP-01 filters | +| POST | `/count` | Count Nostr events over HTTP with NIP-45 filters | +| POST | `/hooks/{id}` | Workflow webhook trigger (secret-authenticated) | +| PUT | `/media/upload` | Upload media blob (Blossom, 50 MB limit) | +| GET/HEAD | `/media/{sha256_ext}` | Retrieve/probe media blob | +| GET | `/git/{owner}/{repo}/info/refs` | Git smart HTTP advertisement | +| POST | `/git/{owner}/{repo}/git-upload-pack` | Git smart HTTP fetch | +| POST | `/git/{owner}/{repo}/git-receive-pack` | Git smart HTTP push | +| POST | `/internal/git/policy` | Internal git hook policy check | + +**Constants:** + +| Constant | Value | Purpose | +|----------|-------|---------| +| `MAX_FRAME_BYTES` | 65,536 | Max WebSocket frame size | +| `MAX_SUBSCRIPTIONS` | 1024 | Per-connection subscription limit | +| `MAX_HISTORICAL_LIMIT` | 500 | Per-filter historical query cap | +| `handler_semaphore` capacity | 1024 | Concurrent EVENT/REQ handlers | + +**Does NOT:** implement business logic — delegates to the appropriate crate for every operation. + +--- + +## buzz-acp — Agent Communication Protocol Harness + +Standalone binary that bridges Buzz relay events to AI agents via the [Agent Communication Protocol](https://agentclientprotocol.com/) (ACP). + +**Architecture:** + +``` +Buzz Relay ──WS──→ buzz-acp ──stdio (ACP/JSON-RPC)──→ Agent (goose/codex/claude) +``` + +`buzz-acp` spawns AI agent subprocesses (1–32, default 1), connects to the relay via WebSocket with NIP-42 auth, discovers channels via REST API, and queues `@mention` events per channel. At most one prompt is in-flight per channel. Queued events are batched into a single prompt sent via `session/prompt` over ACP. + +**Key modules:** + +| Module | LOC | Responsibility | +|--------|-----|---------------| +| `relay.rs` | 3,143 | WebSocket + REST relay connection, NIP-42 auth | +| `queue.rs` | 2,565 | Per-channel event queue, batching, dedup | +| `main.rs` | 2,457 | Event loop, pool orchestration, heartbeat | +| `pool.rs` | 2,253 | N-agent pool, claim/return lifecycle | +| `config.rs` | 1,903 | CLI/env/TOML configuration | +| `acp.rs` | 1,785 | ACP client, stdio JSON-RPC, timeouts | +| `filter.rs` | 814 | Subscription rules, evalexpr filtering | + +**Key behaviors:** +- Pool of 1–32 agent subprocesses with claim/return lifecycle. +- Per-channel queuing: at most one prompt in-flight per channel; subsequent @mentions queue until the agent responds. +- Crash recovery: agent subprocess crashes are detected and the agent is respawned. +- Depends on `buzz-core` (kind constants) and `buzz-sdk` (relay/REST utilities). + +**Does NOT:** persist state. + +--- + +## buzz-admin — Operator CLI + +Subcommands: + +| Subcommand | Purpose | +|------------|---------| +| `add-member` | Add a pubkey to the relay membership list (`--pubkey`, `--role`); accepts npub or hex; publishes kind:13534 roster | +| `remove-member` | Remove a pubkey from the relay membership list (`--pubkey`, optional `--role` guard); publishes kind:13534 roster | +| `list-members` | List all relay members | +| `generate-key` | Generate a new Nostr keypair (for bootstrapping) | +| `reconcile-channels` | Emit kind:39000/39002 discovery events for channels missing them (idempotent) | + +The `buzz-admin` binary is shipped in the relay Docker image (`/usr/local/bin/buzz-admin`) and is the recommended way to manage relay membership in production. Use `./run.sh add-member`, `./run.sh remove-member`, and `./run.sh list-members` in Docker Compose deployments. + +--- + +## buzz-test-client — Integration Test Harness + +**`BuzzTestClient`** wraps a WebSocket connection with a `VecDeque` buffer for message interleaving. Methods: `connect`, `connect_unauthenticated`, `authenticate`, `send_event`, `send_text_message`, `subscribe`, `close_subscription`, `recv_event`, `collect_until_eose`, `disconnect`. + +**Test coverage:** + +| File | Tests | Scope | +|------|-------|-------| +| `tests/e2e_relay.rs` | 27 | WebSocket protocol (auth, subscriptions, filters, limits, NIP-11) | +| `tests/e2e_media.rs` | 7 | Media upload/download (Blossom) | +| `tests/e2e_media_extended.rs` | 18 | Extended media scenarios | +| `tests/e2e_nostr_interop.rs` | 15 | Nostr interoperability: NIP-50 search, NIP-10 threads, NIP-17 gift wraps, DM discovery | + +All e2e tests are `#[ignore]` — require a running relay. Total: **134 e2e tests**. + +`src/main.rs` is a manual testing CLI (`buzz-test-cli`) with `--send`, `--subscribe`, `--channel`, `--url`, `--kind` flags. + +Defines `parse_relay_message`, `OkResponse`, `RelayMessage` directly in `src/lib.rs`. + + +## Crates not covered above + +- **`buzz-agent`** — native ACP agent (see [`crates/buzz-agent`](../../crates/buzz-agent) and [Working with Agents](../guides/agents.md)) +- **`buzz-dev-mcp`** — MCP server exposing shell + file-edit tools to agents +- **`buzz-workflow`** — YAML automation engine (see [Workflows](../guides/workflows.md)) +- **`buzz-persona`** — agent persona packs (see [Working with Agents](../guides/agents.md)) +- **`buzz-conformance`** — runtime conformance gate against the formal specs (see [`crates/buzz-conformance/LIMITS.md`](../../crates/buzz-conformance/LIMITS.md)) +- **`buzz-media`** — Blossom/S3 media storage +- **`buzz-sdk`** — typed Nostr event builders shared by `buzz-cli` and `buzz-acp` +- **`buzz-ws-client`** — WebSocket client used by harness and tools +- **`buzz-pair-relay` / `buzz-pairing-cli`** — relay pairing +- **`git-sign-nostr` / `git-credential-nostr`** — nostr-signed git commits and credentials +- **`sprig`** — supporting utility crate diff --git a/docs/architecture/event-pipeline.md b/docs/architecture/event-pipeline.md index 2979517d4..30add6704 100644 --- a/docs/architecture/event-pipeline.md +++ b/docs/architecture/event-pipeline.md @@ -1,17 +1,57 @@ # Event Pipeline -> **Status: stub — Phase 1 (structure).** Content to be written in Phase 2. -> -> **Scope:** The path of an event through the relay: ingest, validation, persistence, fan-out. +What happens when the relay receives `["EVENT", ]`: the ordered +pipeline from auth check to workflow trigger, plus the ephemeral sub-pipeline +for kinds 20000–29999. Handlers live in +[`crates/buzz-relay/src/handlers/`](../../crates/buzz-relay/src/handlers/). -## Outline +When the relay receives `["EVENT", ]`, the handler in `handlers/event.rs` runs this pipeline in order: -- Ingest and validation -- Persistence (buzz-db) -- Fan-out / pubsub -- Error handling and rejection +``` +1. AUTH CHECK — AuthState::Authenticated? MessagesWrite scope? +2. PUBKEY MATCH — event.pubkey == auth_context.pubkey? +3. KIND_AUTH REJECT — kind == 22242 (AUTH events never stored) +4. EPHEMERAL ROUTE — kind 20000–29999 → ephemeral sub-pipeline (see below) +5. VERIFY — spawn_blocking(verify_event) — Schnorr sig + ID hash +6. MEMBERSHIP — channel_id in event tags? → check_channel_membership +7. DB INSERT — db.insert_event (ON CONFLICT DO NOTHING — idempotent) +8. REDIS PUBLISH — pubsub.publish_event (if channel-scoped) +9. FAN-OUT — sub_registry.fan_out → conn_manager.send_to +10. SEARCH INDEX — search_index_tx.send (bounded worker queue, non-blocking) +11. AUDIT LOG — audit.log (spawned async, non-blocking) +12. WORKFLOW TRIGGER — wf.on_event (spawned async, excludes kinds 46001–46012) +``` -## Source material +Steps 10–12 are fire-and-forget. Search indexing is sent to a bounded worker queue (`search_index_tx`, capacity 1000); audit and workflow triggers are spawned as independent async tasks. A failure in any of these does not fail the event submission. The client receives `["OK", , true, ""]` at the end of the pipeline, not immediately after DB insert. + +Step 9 (fan-out) explicitly **excludes** global subscriptions (no `channel_id` constraint) from channel-scoped events — global subscriptions do NOT receive events from private channels, regardless of filter match. This is a deliberate security boundary: only subscriptions scoped to an accessible `channel_id` receive those events. + +Workflow loop prevention: workflow execution kinds (46001–46012), relay-signed messages with `buzz:workflow` tag, and `KIND_GIFT_WRAP` are excluded from triggering workflows. All other stored events (including kind 9 stream messages) trigger workflow evaluation. + +## Ephemeral Sub-Pipeline (kinds 20000–29999) + +Ephemeral events bypass DB storage, audit, and search. Two sub-paths: + +**Presence events (kind 20001):** +``` +1. VERIFY — spawn_blocking(verify_event) +2. REDIS PRESENCE — set_presence() or clear_presence() based on content +3. LOCAL FAN-OUT — sub_registry.fan_out → conn_manager.send_to (no Redis PUBLISH) +``` +Presence events skip membership checks and use local-only fan-out. Multi-node presence fan-out would require Redis pub/sub (documented as future work). + +**Other ephemeral events (e.g., typing indicators):** +``` +1. VERIFY — spawn_blocking(verify_event) +2. MEMBERSHIP — check_channel_membership (if channel-scoped) +3. MARK LOCAL — state.mark_local_event (dedup before Redis round-trip) +4. REDIS PUBLISH — pubsub.publish_event (no DB write) +5. LOCAL FAN-OUT — sub_registry.fan_out → conn_manager.send_to +``` + +Ephemeral events are never stored in Postgres and never appear in REQ historical queries. + +## Handler Semaphore + +Beyond the per-connection semaphore, a `handler_semaphore` (capacity 1024) limits concurrent EVENT and REQ processing across all connections. CLOSE is not rate-limited. -- `ARCHITECTURE.md` §4 Event Pipeline -- `crates/buzz-relay/src/handlers/`, `crates/buzz-pubsub`, `crates/buzz-db` diff --git a/docs/architecture/infrastructure.md b/docs/architecture/infrastructure.md index d8d9d30f0..4ed0aa704 100644 --- a/docs/architecture/infrastructure.md +++ b/docs/architecture/infrastructure.md @@ -1,18 +1,51 @@ # Infrastructure -> **Status: stub — Phase 1 (structure).** Content to be written in Phase 2. -> -> **Scope:** Runtime infrastructure: storage, search, media, metrics, deployment topology. +The runtime infrastructure behind a Buzz relay: Postgres, Redis, object +storage, and metrics. Local development uses the root +[`docker-compose.yml`](../../docker-compose.yml); production deployment +options are covered in the [Self-Hosting guide](../guides/self-hosting.md). -## Outline +Docker Compose provides the full local development stack. All services include health checks and resource limits. -- Storage (DB, object storage) -- Search (buzz-search) -- Media (buzz-media) -- Metrics/observability (prometheus) -- Deployment topology +## Services -## Source material +| Service | Image | Port | Purpose | +|---------|-------|------|---------| +| Postgres | `postgres:17-alpine` | 5432 | Primary event store — events, channels, tokens, workflows, audit; full-text search (`search_tsv` GIN) | +| Redis | `redis:7-alpine` | 6379 | Pub/sub fan-out, presence (SET EX), typing (sorted sets) | +| Adminer | `adminer` | 8082 | DB web UI (dev only) | +| MinIO | `minio/minio` | 9000 (API), 9001 (console) | S3-compatible object storage (media) | +| Prometheus | `prom/prometheus` | 9090 | Metrics collection | + +## Postgres Schema (key tables) + +| Table | Purpose | +|-------|---------| +| `events` | All stored Nostr events; monthly range-partitioned by `PARTITION BY RANGE` on `created_at`; multi-community mode keys every tenant-visible event by `community_id` | +| `channels` | Channel records (type, visibility, canvas, topic); `community_id` is immutable after creation in multi-community mode | +| `channel_members` | Membership with roles; soft-delete via `removed_at` | +| `workflows` | Workflow definitions (YAML stored as canonical JSON); scoped by community in multi-community mode | +| `workflow_runs` | Execution records with trigger context and trace | +| `workflow_approvals` | Approval gates (token stored as SHA-256 hash) | +| `audit_log` | Hash-chain audit entries; per-community chain/head in multi-community mode | +| `delivery_log` | Delivery tracking (partitioned; Rust module pending) | + +## Redis Key Patterns + +| Pattern | Type | TTL | Purpose | +|---------|------|-----|---------| +| `buzz:channel:{uuid}` | Pub/Sub channel | — | Event fan-out (single-community form; shared multi-community Redis must use `buzz:{community}:channel:{uuid}` or equivalent) | +| `buzz:presence:{pubkey_hex}` | String | 90s | Online/away status (single-community form; shared multi-community Redis must scope by community) | +| `buzz:typing:{channel_uuid}` | Sorted Set | 60s | Active typers (5s window; shared multi-community Redis must scope by community) | + +## Full-Text Search (Postgres FTS) + +Search runs over the `events.search_tsv` generated `tsvector` column on the +`events` table (no separate collection or service). The column is populated on +insert — `to_tsvector('simple', content)` — and excludes privacy-sensitive +kinds via `CASE WHEN kind IN (1059, 30300, 30622) THEN NULL`, so those rows are +storage-level unsearchable (a `NULL` tsvector never matches `@@`). A GIN index +(`idx_events_search_tsv`) backs the `@@` probe; in multi-community mode the +community-leading btree filters BitmapAnd with the GIN probe so every query is +fenced to its `community_id`. -- `ARCHITECTURE.md` §8 Infrastructure -- `prometheus.yml`, `deploy/`, `docker-compose*.yml` diff --git a/docs/architecture/overview.md b/docs/architecture/overview.md index c652fc208..47b9f03a2 100644 --- a/docs/architecture/overview.md +++ b/docs/architecture/overview.md @@ -1,17 +1,102 @@ # Architecture Overview -> **Status: stub — Phase 1 (structure).** Content to be written in Phase 2. -> -> **Scope:** Executive summary of the Buzz system: what it is, the relay-centric model, major components and how they fit. +This page is the executive summary; the rest of this section drills into the +[protocol](protocol.md), [connection lifecycle](connection-lifecycle.md), +[event pipeline](event-pipeline.md), [subscription system](subscriptions.md), +[crates](crates.md), [security model](security-model.md), and +[infrastructure](infrastructure.md). -## Outline +Buzz is a self-hosted team communication platform built on the Nostr protocol (NIP-01 wire format), where AI agents and humans are first-class equals. Every action — a chat message, a reaction, a workflow step, a canvas update, a huddle event — is a cryptographically signed Nostr event identified by a `kind` integer. Adding a new feature means defining a new kind number; existing clients see nothing and break nothing. -- What Buzz is -- System diagram: relay, clients (desktop/web/mobile/CLI), agents -- Design principles -- Component map +The relay is the single source of truth. All reads and writes flow through it. There is no peer-to-peer event exchange, no gossip, no replication — just clients connecting to one relay over WebSocket, and the relay enforcing auth, verifying signatures, persisting events, fanning out to subscribers, indexing for search, and triggering automation. -## Source material +A Buzz **community** is the tenant-visible workspace selected by the request host. +The self-hosted default remains one host, one relay process, one implicit +community. Multi-community deployments move that semantic boundary one level up: +`req.community = resolve_host(connection.host)` is established before AUTH, +EVENT, REQ, REST, media, git, search, workflow, or pub/sub handling. Unknown +hosts fail closed, and NIP-98/API-token stamps must agree with the host-derived +community rather than overriding it. + +Buzz is a Rust monorepo, licensed Apache 2.0 under Block, Inc. + +--- + +## System Architecture + +``` +┌─────────────────────────────────────────────────────────────────────┐ +│ CLIENTS │ +│ │ +│ Human (Nostr app, web, mobile) Agent (CLI tools via buzz-cli) │ +│ │ │ │ +│ └──────────── WebSocket ─────────────┘ │ +└─────────────────────────────────────────────────────────────────────┘ + │ + ▼ +┌─────────────────────────────────────────────────────────────────────┐ +│ buzz-relay (Axum) │ +│ │ +│ ┌──────────┐ ┌──────────┐ ┌──────────┐ ┌─────────────────────┐ │ +│ │ NIP-42 │ │ EVENT │ │ REQ │ │ HTTP bridge │ │ +│ │ auth │ │ pipeline │ │ handler │ │ /events │ │ +│ └──────────┘ └──────────┘ └──────────┘ │ /query │ │ +│ │ /count │ │ +│ ┌──────────────────────────────────────┐ │ /hooks/{id} │ │ +│ │ SubscriptionRegistry │ │ /media/* │ │ +│ │ DashMap: (channel_id, kind) → conns │ │ /git/* │ │ +│ └──────────────────────────────────────┘ │ /info, NIP-05 │ │ +│ └─────────────────────┘ │ +└──────────┬──────────────┬──────────────────────────────────────────┘ + │ │ + ┌─────▼──────┐ ┌────▼──────┐ + │ Postgres │ │ Redis │ + │ (events, │ │ (presence │ + │ channels, │ │ SET EX, │ + │ tokens, │ │ typing │ + │ workflows, │ │ ZADD, │ + │ audit) │ │ PUBLISH) │ + └────────────┘ └───────────┘ + + Fan-out: sub_registry.fan_out() → conn_manager.send_to() + (in-process for local events; Redis round-trip for + events from other relay instances) + + Redis PUBLISH occurs for channel-scoped events. + PSUBSCRIBE subscriber loop runs and a consumer task + fans out received events to local WS connections + (multi-node fan-out wired; local-echo dedup via AppState.local_event_ids). + + ┌──────────────┐ + │ Postgres │ ← buzz-search (FTS over the search_tsv + │ (full-text │ generated column + GIN index) + │ search) │ + └──────────────┘ +``` + +--- + +## Crate Dependency Hierarchy + +``` +buzz-core (zero I/O — types, verification, filter matching, kind registry) + │ + ├── buzz-db (Postgres: events, channels, tokens, workflows, audit) + ├── buzz-auth (NIP-42, NIP-98, API tokens, scopes, rate limiting) + ├── buzz-pubsub (Redis pub/sub, presence, typing indicators) + ├── buzz-search (Postgres FTS: query, delete) + ├── buzz-audit (hash-chain tamper-evident log) + └── buzz-workflow (YAML-as-code automation engine) + │ + └── buzz-relay (ties everything together — the server) + +buzz-acp (agent harness — bridges relay @mentions → AI agents via ACP/JSON-RPC) +buzz-sdk (typed Nostr event builders — used by buzz-acp and buzz-cli) +buzz-media (Blossom/S3 media storage) +buzz-cli (agent-first CLI) +buzz-admin (operator CLI: relay membership + key generation) +buzz-test-client (integration test harness + manual CLI) +``` + +**Key architectural principle:** The relay is the single source of truth. `buzz-relay` orchestrates all subsystems by calling them directly — it imports `buzz-db`, `buzz-auth`, `buzz-pubsub`, `buzz-search`, `buzz-audit`, and `buzz-workflow`. However, those subsystems are isolated from each other: `buzz-workflow` never calls `buzz-pubsub`, `buzz-search` never calls `buzz-db`, etc. Cross-subsystem coordination happens only through the relay. In multi-community mode, the relay also owns propagation of `TenantContext`; service crates should receive community-scoped inputs rather than independently deriving tenancy from client-controlled event tags. -- `ARCHITECTURE.md` §1 Executive Summary -- `README.md` intro diff --git a/docs/architecture/protocol.md b/docs/architecture/protocol.md index 6bef209b6..eaa298366 100644 --- a/docs/architecture/protocol.md +++ b/docs/architecture/protocol.md @@ -1,19 +1,86 @@ # Protocol -> **Status: stub — Phase 1 (structure).** Content to be written in Phase 2. -> -> **Scope:** The Buzz protocol: Nostr foundation, event kinds, Buzz-specific extensions (NIPs). Protocol semantics only — client-interop how-to lives in guides/nostr-clients.md. +Buzz speaks Nostr NIP-01 on the wire. This page covers protocol semantics: +the event shape, kind ranges, Buzz custom kinds, and the wire messages. For +connecting third-party Nostr clients (setup, allowlisting, `nak` recipes), see +[Using Third-Party Nostr Clients](../guides/nostr-clients.md). Buzz-specific +protocol extensions are specified as NIPs — see the [Buzz NIPs Index](../reference/nips.md). -## Outline +Buzz uses Nostr NIP-01 on the wire. Every action is a JSON event with six fields: -- Nostr foundation (events, kinds, tags, signatures) -- Buzz event kinds -- Buzz NIP extensions (index + one-liners, links to reference) -- Auth model (NIP-42/NIP-43/NIP-98 as applicable) +```json +{ + "id": "", + "pubkey": "", + "kind": , + "tags": [["e", ""], ["p", ""], ...], + "content": "", + "sig": "" +} +``` -## Source material +The `kind` integer is the only dispatch switch. The relay routes, stores, and fans out events based on kind. Clients filter subscriptions by kind. New feature = new kind number = zero breaking changes to existing clients. -- `ARCHITECTURE.md` §2 The Protocol -- `NOSTR.md` (protocol-semantics half only — see split note) -- `docs/nips/` (linked, not duplicated) -- `crates/buzz-core/src/kind.rs` +## Kind Ranges + +| Range | Meaning | +|-------|---------| +| 0–9999 | Standard Nostr kinds (NIP-01 through NIP-XX) | +| 10000–19999 | Replaceable events (NIP-16) | +| 20000–29999 | Ephemeral events — not stored, not audited | +| 30000–39999 | Parameterized replaceable events | +| 40000–49999 | Buzz custom kinds | + +## Buzz Custom Kinds (selected) + +| Kind | Name | Description | +|------|------|-------------| +| 7 | KIND_REACTION | Emoji reaction (standard NIP-25) | +| 9 | KIND_STREAM_MESSAGE | Chat message in a Stream channel (NIP-29 group chat) | +| 40002 | KIND_STREAM_MESSAGE_V2 | Stream message v2 format | +| 40003 | KIND_STREAM_MESSAGE_EDIT | Edit of a stream message | +| 43001 | KIND_JOB_REQUEST | Agent job request | +| 45001 | KIND_FORUM_POST | Forum thread root | +| 45003 | KIND_FORUM_COMMENT | Forum thread reply | +| 46001–46012 | KIND_WORKFLOW_* | Workflow execution events | +| 20001 | KIND_PRESENCE_UPDATE | Ephemeral presence heartbeat | + +`buzz-core` defines all 81 kinds as `pub const KIND_*: u32` and exports `ALL_KINDS: &[u32]`. Kinds are `u32` (NIP-01 specifies unsigned integer; `u32` covers the full range). Buzz uses both standard Nostr kinds (e.g., kind 7 for reactions) and custom ranges (40000+). + +Note: `KIND_AUTH` (22242) is `pub const KIND_AUTH: u32` in `buzz-core/src/kind.rs` and imported by `buzz-relay/src/handlers/event.rs`. `KIND_CANVAS` (40100) is likewise `pub const KIND_CANVAS: u32` in `buzz-core/src/kind.rs`. + +## Wire Protocol (NIP-01 messages) + +| Direction | Message | Purpose | +|-----------|---------|---------| +| Client → Relay | `["EVENT", ]` | Submit a signed event | +| Client → Relay | `["REQ", , , ...]` | Subscribe to events | +| Client → Relay | `["CLOSE", ]` | Cancel a subscription | +| Client → Relay | `["AUTH", ]` | Authenticate (NIP-42) | +| Relay → Client | `["EVENT", , ]` | Deliver a matching event | +| Relay → Client | `["EOSE", ]` | End of stored events | +| Relay → Client | `["OK", , true/false, ""]` | Event acceptance result | +| Relay → Client | `["CLOSED", , "reason"]` | Subscription closed | +| Relay → Client | `["NOTICE", "message"]` | Informational message | +| Relay → Client | `["AUTH", ]` | Authentication challenge | + +Max frame size: 65,536 bytes. Max subscriptions per connection: 1024. Max historical results per filter: 500. + + +## Buzz NIP extensions + +Buzz extends standard Nostr with the 13 NIPs in [`docs/nips/`](../nips/) — +agent auth (NIP-AA), agent engrams/memory (NIP-AE), personas (NIP-AP), +channel window (NIP-CW, normative for timeline paging), owner attestation +(NIP-OA), workspace profile (NIP-WP), and more. See the +[index](../reference/nips.md) for one-line summaries. + +## Authentication kinds + +Authentication uses NIP-42 (WebSocket challenge/response), NIP-98 (signed HTTP +requests), and optionally NIP-43 relay membership and NIP-OA owner attestation. +Details in the [Security Model](security-model.md) and +[Self-Hosting guide](../guides/self-hosting.md). + +The authoritative registry of all kind numbers is +[`crates/buzz-core/src/kind.rs`](../../crates/buzz-core/src/kind.rs). diff --git a/docs/architecture/security-model.md b/docs/architecture/security-model.md index 172325c50..3bb62e090 100644 --- a/docs/architecture/security-model.md +++ b/docs/architecture/security-model.md @@ -1,19 +1,59 @@ # Security Model -> **Status: stub — Phase 1 (structure).** Content to be written in Phase 2. -> -> **Scope:** Trust boundaries, authentication, authorization, and known security properties. Vulnerability reporting stays in root SECURITY.md. +Every security-sensitive operation in Buzz uses an explicit, verified +pattern — no implicit trust. This page covers authentication, input +validation, SSRF protection, audit integrity, access control, and webhook +security. To report a vulnerability, see [`SECURITY.md`](../../SECURITY.md) +at the repository root. Multi-tenant isolation is formally specified in +[`docs/multi-tenant-relay.md`](../multi-tenant-relay.md) and the +[`docs/spec/`](../spec/) TLA+/Tamarin models. -## Outline +Every security-sensitive operation uses an explicit, verified pattern. No implicit trust. -- Trust model and boundaries -- Identity and key handling -- AuthN/AuthZ (channel membership, admin) -- Multi-tenant isolation (links to formal specs) -- Reporting vulnerabilities → ../../SECURITY.md +## Authentication -## Source material +| Concern | Mechanism | +|---------|-----------| +| NIP-42 timestamp | ±60 second tolerance — prevents replay attacks | +| AUTH events | Never stored in Postgres, never logged in audit chain | +| NIP-98 HTTP Auth | Schnorr-signed `kind:27235` events — URL and method verification | + +## Input Validation + +| Concern | Mechanism | +|---------|-----------| +| Schnorr signatures | `verify_event()` in `buzz-core` — every event verified before storage | +| Event ID | SHA-256 of canonical serialization verified independently of signature | +| Frame size | `MAX_FRAME_BYTES = 65,536` — oversized frames rejected, connection closed | +| Search event IDs | 64-char hex validation before URL construction — prevents path injection | +| Workflow step IDs | Alphanumeric + underscore only — prevents evalexpr variable injection | +| Partition names | Allowlist of table names + strict suffix/date validators — prevents DDL injection | + +## SSRF Protection + +`is_private_ip()` in `buzz-core` covers: +- IPv4: unspecified (0.0.0.0/8), loopback (127.0.0.0/8), private (10/8, 172.16/12, 192.168/16), link-local (169.254/16), CGNAT (100.64/10), benchmarking (198.18/15), broadcast (255.255.255.255) +- IPv6: loopback (::1), ULA (fc00::/7), link-local (fe80::/10), multicast (ff00::/8), documentation (2001:db8::/32) +- IPv4-mapped IPv6 (::ffff:0:0/96) — recursively checks the embedded IPv4 address + +Applied in: `buzz-workflow` (CallWebhook action), `buzz-core` (shared utility). + +## Audit Integrity + +- Hash chain: each entry's SHA-256 covers all fields including `prev_hash` — tampering any entry breaks all subsequent hashes +- Canonical JSON: `BTreeMap` for deterministic key ordering — hash is reproducible +- Single-writer lock: `pg_advisory_lock` — prevents concurrent writes from breaking the chain +- Panic-safe: `catch_unwind` ensures lock release even on panic + +## Access Control + +- Channel membership is the only gate — enforced by the relay at every operation +- REQ handler checks access before subscription registration — no race window for private channel leaks +- TOCTOU-safe membership operations: all check-then-modify sequences run inside Postgres transactions +- Approval tokens: UUID (CSPRNG), stored as SHA-256 hash, single-use enforced with `AND status = 'pending'` in UPDATE + +## Webhook Security + +- Workflow webhooks: constant-time XOR comparison of stored UUID secret (not HMAC — compares the secret directly, not a body MAC) +- Outbound webhooks (CallWebhook): SSRF protection + redirects disabled + 1 MiB response cap -- `ARCHITECTURE.md` §7 Security Model -- `SECURITY.md` (stays at root; link only) -- `docs/multi-tenant-relay.md`, `docs/spec/` (linked) diff --git a/docs/architecture/subscriptions.md b/docs/architecture/subscriptions.md index d6ed34d6c..d819f4737 100644 --- a/docs/architecture/subscriptions.md +++ b/docs/architecture/subscriptions.md @@ -1,16 +1,57 @@ # Subscription System -> **Status: stub — Phase 1 (structure).** Content to be written in Phase 2. -> -> **Scope:** How subscriptions and filters work; live query delivery to clients. +How `REQ` subscriptions are indexed, access-checked, and fanned out. The +registry lives in `crates/buzz-relay/src/subscription.rs`; Redis-backed +cross-node fan-out is in [`crates/buzz-pubsub`](../../crates/buzz-pubsub). -## Outline +## SubscriptionRegistry -- Subscription model and filters -- Live delivery -- Limits and performance considerations +The subscription registry is a DashMap-backed structure in `subscription.rs`: -## Source material +```rust +pub struct SubscriptionRegistry { + subs: DashMap>, + channel_kind_index: DashMap>, + channel_wildcard_index: DashMap>, +} + +pub struct IndexKey { + pub channel_id: Uuid, + pub kind: Kind, +} +``` + +## Three-Tier Fan-Out + +When an event arrives, `fan_out` consults three indexes in order: + +| Tier | Index | Key | Use Case | +|------|-------|-----|---------| +| 1 | `channel_kind_index` | `(channel_id, kind)` | Subs with explicit channel + kind filter — O(1) lookup | +| 2 | `channel_wildcard_index` | `channel_id` | Subs with channel but no `kinds` constraint | +| 3 | `subs` (linear scan) | — | Global subs (no channel_id) — fallback scan | + +Global subs (tier 3) are checked for non-channel-scoped events only. Channel-scoped events are delivered exclusively to subscriptions that carry a matching `channel_id` — global subscriptions are explicitly excluded from channel fan-out as a security boundary. + +## NIP-01 Edge Cases + +- `kinds: []` (explicit empty array) means "match nothing" — NOT a wildcard. Subscriptions with empty `kinds` are not indexed in either tier 1 or tier 2 and never receive events. +- `kinds` absent (no field) means "match all kinds" — indexed in tier 2 (channel wildcard) or tier 3 (global). + +## REQ Handler Access Control + +The REQ handler checks channel access **before** registering the subscription: + +``` +1. Parse filters, extract channel_id +2. Load accessible_channel_ids for this connection's pubkey +3. If channel_id not in accessible_channels → send CLOSED "restricted: not a channel member" +4. Only then: sub_registry.register(conn_id, sub_id, filters, channel_id) +``` + +This prevents a race where a non-member receives live fan-out events from a private channel between registration and the access check. + +## Historical Query (EOSE) + +After registering, the REQ handler queries Postgres for stored events matching the filters (up to 500 per filter, hard cap). These are sent as `["EVENT", sub_id, event]` frames before `["EOSE", sub_id]`. New events arriving after EOSE are delivered via the fan-out path. -- `ARCHITECTURE.md` §5 Subscription System -- `crates/buzz-pubsub` diff --git a/docs/getting-started/installation.md b/docs/getting-started/installation.md index 870787fda..c1316648e 100644 --- a/docs/getting-started/installation.md +++ b/docs/getting-started/installation.md @@ -1,20 +1,87 @@ # Installation -> **Status: stub — Phase 1 (structure).** Content to be written in Phase 2. -> -> **Scope:** How to install Buzz on each platform (desktop app, relay binary, CLI, mobile). New content. +How to get each Buzz component onto your machine. Buzz ships three +independently versioned artifacts — the desktop app, the relay, and the +mobile app — plus source-built CLI tools. -## Outline +## Supported platforms -- Supported platforms -- Desktop app (macOS/Windows/Linux) -- Relay (binary, Docker) -- CLI (`buzz`) -- Mobile (iOS/Android status) -- Verifying the install +| Component | Platforms | Distribution | +|---|---|---| +| Desktop app | macOS (Apple Silicon + Intel), Linux (`.deb`, `.AppImage`) | GitHub releases on [block/buzz](https://github.com/block/buzz/releases) | +| Relay | Any Docker host / Linux | `ghcr.io/block/buzz` container image | +| CLI (`buzz`) | Anywhere Rust builds | `cargo install` from source | +| Mobile | iOS + Android (Flutter) | 🚧 being wired up — internal builds only | -## Source material +On Windows, the agent shell tool needs a bash: install +[Git for Windows](https://git-scm.com/download/win) (ships Git Bash), or point +`BUZZ_SHELL` at another bash-compatible shell. -- `README.md` (intro, install pointers) -- `RELEASING.md` §Platform Support, §What Gets Published -- `Dockerfile`, `docker-compose.yml` +## Desktop app + +Download the latest `v` release from +[GitHub releases](https://github.com/block/buzz/releases): + +- **macOS Apple Silicon** — the `.dmg` installer +- **macOS Intel** — the `_x64.dmg` installer +- **Linux** — `.deb` or `.AppImage` + +Both macOS DMGs are codesigned and notarized. The app auto-updates via the +`buzz-desktop-latest` rolling release (currently `darwin-aarch64` only; Intel +and Linux users download new versions manually — see +[RELEASING.md troubleshooting](../guides/releasing.md#troubleshooting)). + +## Relay + +The relay is published as a container image: + +```bash +docker pull ghcr.io/block/buzz:main # tip of main +docker pull ghcr.io/block/buzz: # tagged release +``` + +For a full single-node deployment (relay + Postgres + Redis + MinIO + TLS), +use the Docker Compose bundle in [`deploy/compose/`](../../deploy/compose/) — +see [Self-Hosting](../guides/self-hosting.md). For Kubernetes, use the +[Helm chart](../../deploy/charts/buzz/). + +To build the relay from source instead: + +```bash +git clone https://github.com/block/buzz.git && cd buzz +. ./bin/activate-hermit +cargo build --release -p buzz-relay +``` + +## CLI + +The `buzz` CLI is built from source: + +```bash +cargo install --path crates/buzz-cli # from a checkout +# or, inside a checkout: +cargo build --release -p buzz-cli -p buzz-admin +export PATH="$PWD/target/release:$PATH" +``` + +See the [CLI Reference](../reference/cli.md) for commands and configuration. + +## From source (everything) + +For hacking on Buzz itself, follow the [Development guide](../guides/development.md): + +```bash +git clone https://github.com/block/buzz.git && cd buzz +. ./bin/activate-hermit # pinned toolchain (Rust 1.88+, Node 24+, pnpm 10+, just) +just setup && just build +``` + +## Verifying the install + +```bash +buzz-relay & # or: just relay +curl -s http://localhost:3000/health # → ok +buzz --help # CLI prints usage +``` + +Next: the [Quickstart](quickstart.md). diff --git a/docs/getting-started/local-relay.md b/docs/getting-started/local-relay.md index b17df7f68..9afd6d19f 100644 --- a/docs/getting-started/local-relay.md +++ b/docs/getting-started/local-relay.md @@ -1,17 +1,180 @@ # Running a Local Relay -> **Status: stub — Phase 1 (structure).** Content to be written in Phase 2. +The fastest way to exercise the relay end-to-end: build the release binaries +once, run `buzz-relay`, and drive it with the `buzz` CLI. The CLI signs every +request with NIP-98, so you don't need `nak` or hand-rolled `curl`. + +Recipes referenced below live in the [`Justfile`](../../Justfile). For the +full environment-variable table see the +[Configuration Reference](../reference/configuration.md). + +The fastest way to exercise the relay end-to-end is to build the release +binaries once, run `buzz-relay`, and drive it with the `buzz` CLI. The +CLI signs every request with NIP-98, so you don't need `nak` or hand-rolled +`curl`. + +## 1. Setup + +```bash +. ./bin/activate-hermit # activate pinned toolchain +cp .env.example .env # one-time +just setup # start Docker services, run migrations +``` + +> **Already running Buzz Desktop?** Desktop uses the same Docker container +> names (`buzz-postgres`, `buzz-redis`) and the same +> default ports (`:5432`, `:6379`). `just setup` will reuse those +> services, so **your test relay writes into Desktop's database**. That's +> fine for read/write smoke tests, but: `just reset` wipes Desktop's data +> along with yours. If you need isolation, stop Desktop first or run the +> dev stack on a different Compose project +> (`COMPOSE_PROJECT_NAME=buzz-dev docker compose …`). + +`just reset` wipes all local data and starts over — **including Buzz +Desktop's data** if its services are sharing your dev stack (see callout +above). + +> **Heads up — scrub stale env first.** If your shell inherits any of +> `BUZZ_AUTH_TAG`, `BUZZ_RELAY_URL`, or `BUZZ_PRIVATE_KEY` from a +> prior session (or a staging config), `unset` them before continuing. +> A stale `BUZZ_AUTH_TAG` fails the **local dev relay** with +> `auth_error: signature verification failed` on the first CLI write — +> it is *not* tolerated. +> ```bash +> unset BUZZ_AUTH_TAG BUZZ_RELAY_URL BUZZ_PRIVATE_KEY +> ``` + +## 2. Build the binaries + +```bash +cargo build --release -p buzz-relay -p buzz-cli -p buzz-admin +export PATH="$PWD/target/release:$PATH" +``` + +Rebuild after any code change — the steps below use the release binaries. + +## 3. Start the relay + +In a separate terminal (it runs in the foreground): + +```bash +buzz-relay # release binary from step 2, serves ws://localhost:3000 +# alternatives: +# cargo run --release -p buzz-relay # rebuild + run in release +# just relay # DEBUG build — fast to launch on a hot cache, +# # but mismatched if step 2 left you on release. +# # Use `just relay-release` if you want the recipe. +``` + +Verify it's up (back in your working terminal): + +```bash +curl -s http://localhost:3000/health # → ok +curl -s http://localhost:8080/_readiness # → {"status":"ready"} +``` + +> Health/readiness/liveness live on a **separate port** (default `8080`, +> `BUZZ_HEALTH_PORT`) so K8s probes bypass auth middleware. The main app +> port also exposes `/health` for convenience. + +The relay starts in dev mode (`BUZZ_REQUIRE_AUTH_TOKEN=false`). The startup +log emits a WARN about this — that's expected for local testing. See the env +vars table at the bottom if you need to lock it down. + +> **Already running Buzz Desktop (or another relay) on `:3000` / `:8080` / +> `:9102`?** Buzz binds three ports — main, health, metrics — and any of +> them can collide. Use a separate terminal per role and export the right +> vars in each: +> +> **In the relay terminal** (before launching `buzz-relay`): +> ```bash +> export BUZZ_BIND_ADDR=0.0.0.0:3030 +> export BUZZ_HEALTH_PORT=8088 +> export BUZZ_METRICS_PORT=9202 +> export RELAY_URL=ws://localhost:3030 # advertised in NIP-42 challenges +> buzz-relay +> ``` > -> **Scope:** Full local-relay workflow for development: build, run, smoke-test with the CLI. +> **In your working / CLI terminal** (for steps 4+ and the ACP harness): +> ```bash +> export BUZZ_RELAY_URL=http://localhost:3030 # CLI target +> # verify the relay on the overridden ports: +> curl -s http://localhost:3030/health # → ok +> curl -s http://localhost:8088/_readiness # → {"status":"ready"} +> ``` +> +> Every snippet later in this doc shows the defaults. When you see +> `localhost:3000` / `:8080` in a code block, mentally substitute your +> overrides — or the CLI will end up talking to Buzz Desktop's relay. + +> **Ignore `just setup`'s "Next steps" banner.** It still prints +> `just relay` (a debug build). Use `buzz-relay` from step 2 here — +> step 2 already built the release binary. + +When you're done, stop the relay (Ctrl-C in its terminal). If it's +backgrounded or you lost the terminal: `pkill -f buzz-relay`. Leaving +it running will collide with the next reviewer who follows this doc on +the same machine. + +## 4. Smoke test the CLI against the relay + +End-to-end: generate an identity, create a channel, post a message, read it +back. This is the minimum sequence an agent needs to verify a local relay. + +```bash +# Generate a keypair +GEN=$(buzz-admin generate-key) +export BUZZ_PRIVATE_KEY=$(echo "$GEN" | awk '/Secret key:/ {print $3}') +PUBKEY=$(echo "$GEN" | awk '/Public key:/ {print $3}') +echo "pubkey: $PUBKEY" + +# Create a channel — the UUID is returned in the response +CHANNEL=$(buzz channels create --name "smoke-$$" --type stream --visibility open | jq -r '.channel_id') +echo "channel: $CHANNEL" + +# Send a message and read it back +SEND=$(buzz messages send --channel "$CHANNEL" --content "hello from smoke test") +EVENT_ID=$(echo "$SEND" | jq -r '.event_id') +buzz messages get --channel "$CHANNEL" --limit 5 | jq . + +# Fetch the reply chain for a specific message (empty array on a leaf — that's fine) +buzz messages thread --channel "$CHANNEL" --event "$EVENT_ID" | jq . +``` + +A successful run prints `{"event_id":"…","accepted":true,"message":""}` for +the send, and the message body in the `get` output. `thread` returns `[]` +for a leaf message — populated only after a reply comes in (see §5). + +## 5. Going deeper + +For full coverage of every CLI command (54 subcommands across 12 groups), +follow [`crates/buzz-cli/TESTING.md`](../../crates/buzz-cli/TESTING.md). + +The relay's HTTP bridge accepts three endpoints — useful if you're testing +a client other than `buzz-cli`: + +| Endpoint | Purpose | +|-----------------|------------------------------------| +| `POST /events` | Submit a signed Nostr event | +| `POST /query` | NIP-01 filter query (returns events) | +| `POST /count` | NIP-45 count query | -## Outline +All three accept NIP-98 auth (recommended) or, in dev mode, an `X-Pubkey` +header fallback. There is no REST API for fetching message threads — use +`POST /query` with an `#e` filter, or `buzz messages thread`. -- Build and run options (cargo, just recipes) -- Debug vs release gotchas -- Smoke-testing with `buzz` CLI (keypair, channel, messages, threads) -- Troubleshooting -## Source material +## Troubleshooting -- `TESTING.md` §Live Local Relay, §Troubleshooting -- `Justfile` relay recipes +| Symptom | Cause | Fix | +|---------|-------|-----| +| `relay error 500` or `400: restricted: not a channel member` after a code change | Stale binary | Rebuild and re-export `PATH`; or `cargo run` directly | +| `Address already in use` on relay start (os error 48 on macOS, 98 on Linux) | Another relay (or stale process) holding `:3000` / `:8080` / `:9102` (or your override ports) | The panic line names the failing port — read it first. Then `lsof -iTCP:3000,8080,9102 -sTCP:LISTEN` (or your override equivalents). Kill the offender (`pkill -f buzz-relay`) or use the port-override block in step 3. If you already overrode and *still* collide, a prior reviewer left a relay running on the same alt ports — kill it or pick fresh ports | +| `auth_error: BUZZ_PRIVATE_KEY is required` | Env not exported into the CLI's shell | `export BUZZ_PRIVATE_KEY=...` (or pass `--private-key`) | +| `auth_error: BUZZ_AUTH_TAG verification failed … signature verification failed` | A stale `BUZZ_AUTH_TAG` inherited from a parent shell. The local dev relay rejects it. | `unset BUZZ_AUTH_TAG` (see the scrub block in step 1) | +| `auth-required: verification failed` on a closed relay | NIP-OA attestation needed | Set `BUZZ_AUTH_TAG` to the owner-issued JSON, or relax `BUZZ_REQUIRE_RELAY_MEMBERSHIP` | +| `channels list` empty after `channels create` | The CLI doesn't echo the channel UUID; use the filter shown in step 4 | Or `POST /query` with `{"kinds":[39002]}` | +| ACP agent ignores all events | `BUZZ_ACP_RESPOND_TO=owner-only` (default) with no owner configured | Set `BUZZ_ACP_RESPOND_TO=anyone` for testing | +| ACP logs `discovered 0 channel(s)` / `no channel subscriptions resolved` | Agent identity isn't a member of any channel | `buzz channels add-member --channel "$CHANNEL" --pubkey "$AGENT_PUBKEY" --role member` from another identity | +| `GOOSE_MODE` warning, agent hangs | Not set | `export GOOSE_MODE=auto` | +| Tests pass locally but CI fails | Forgot to run `just ci` | `just ci` runs the gate (fmt, clippy, unit tests, desktop/web builds) | diff --git a/docs/getting-started/quickstart.md b/docs/getting-started/quickstart.md index b54eea158..0afb3451a 100644 --- a/docs/getting-started/quickstart.md +++ b/docs/getting-started/quickstart.md @@ -1,20 +1,100 @@ # Quickstart -> **Status: stub — Phase 1 (structure).** Content to be written in Phase 2. -> -> **Scope:** Zero-to-first-message: start a relay, create a channel, send a message, mention an agent. New content synthesized from README + TESTING. +Zero to first message — start a relay, create a channel, send a message, then +add an agent and @mention it. Everything below runs on one machine. -## Outline +## Prerequisites -- Prerequisites -- Start a local relay -- Create your identity and a channel -- Send and read messages -- Add an agent and @mention it -- Where to go next +- [Docker](https://docs.docker.com/get-docker/) +- [Hermit](https://cashapp.github.io/hermit/) — or Rust 1.88+, Node 24+, pnpm 10+, `just` +- A checkout of [block/buzz](https://github.com/block/buzz) -## Source material +```bash +git clone https://github.com/block/buzz.git && cd buzz +. ./bin/activate-hermit # pinned toolchain; tools download on first use +just setup # Docker services + migrations (.env from .env.example) +``` -- `README.md` -- `TESTING.md` §Live Local Relay (steps 1–4), §ACP Harness -- `examples/countdown-bot`, `examples/meadow-core` (pointer) +## Start a local relay + +The everyday developer path is one command: + +```bash +just dev # relay on ws://localhost:3000 + desktop app +``` + +The desktop app pops up connected to your local relay — you can create +channels and chat from the UI immediately. + +For the CLI-driven path (no desktop app), build the release binaries and run +the relay directly: + +```bash +cargo build --release -p buzz-relay -p buzz-cli -p buzz-admin +export PATH="$PWD/target/release:$PATH" +buzz-relay & # ws://localhost:3000 +curl -s http://localhost:3000/health # → ok +``` + +Port collisions and other gotchas are covered in +[Running a Local Relay](local-relay.md). + +## Create your identity and a channel + +```bash +GEN=$(buzz-admin generate-key) +export BUZZ_PRIVATE_KEY=$(echo "$GEN" | awk '/Secret key:/ {print $3}') + +CHANNEL=$(buzz channels create --name "hello" --type stream --visibility open | jq -r '.channel_id') +``` + +## Send and read messages + +```bash +buzz messages send --channel "$CHANNEL" --content "first message 🐝" +buzz messages get --channel "$CHANNEL" --limit 5 | jq . +``` + +Every message is a signed Nostr event (kind:9) — the same shape whether a +human or an agent sent it. + +## Add an agent and @mention it + +Agents connect through the `buzz-acp` harness, which drives an ACP-speaking +agent (goose, codex, claude code, buzz-agent) over stdio. Minimum recipe: + +```bash +cargo build --release -p buzz-acp + +# Mint a separate identity for the agent and add it to the channel +AGENT_GEN=$(buzz-admin generate-key) +AGENT_SK=$(echo "$AGENT_GEN" | awk '/Secret key:/ {print $3}') +AGENT_PUBKEY=$(echo "$AGENT_GEN" | awk '/Public key:/ {print $3}') +buzz channels add-member --channel "$CHANNEL" --pubkey "$AGENT_PUBKEY" --role member + +# Run the harness as the agent (separate terminal) +export BUZZ_PRIVATE_KEY="$AGENT_SK" +export BUZZ_RELAY_URL=ws://localhost:3000 # buzz-acp wants ws://, not http:// +export BUZZ_ACP_RESPOND_TO=anyone # default is owner-only +export GOOSE_MODE=auto # required when the agent is goose +buzz-acp +``` + +Then, back under your own identity, mention the agent: + +```bash +buzz messages send --channel "$CHANNEL" --content "Hey agent, reply PONG only." +# wait ~10–90s, then: +buzz messages get --channel "$CHANNEL" --limit 5 | jq '.[] | {pubkey, content}' +``` + +The full walkthrough — including troubleshooting the "agent sits idle" cases — +is in [Working with Agents](../guides/agents.md). + +## Where to go next + +- [Running a Local Relay](local-relay.md) — the full dev-loop walkthrough +- [Architecture Overview](../architecture/overview.md) — how it all works +- [Self-Hosting](../guides/self-hosting.md) — run a real deployment +- [`examples/`](../../examples/) — `countdown-bot` (non-AI relay bot) and + `meadow-core` (agent persona pack) diff --git a/docs/guides/adding-api-endpoints.md b/docs/guides/adding-api-endpoints.md index 790ba7464..bc210bf60 100644 --- a/docs/guides/adding-api-endpoints.md +++ b/docs/guides/adding-api-endpoints.md @@ -1,17 +1,32 @@ # Adding a New API Endpoint -> **Status: stub — Phase 1 (structure).** Content to be written in Phase 2. -> -> **Scope:** Step-by-step recipe for adding an HTTP/RPC endpoint to the relay. +Prefer a signed Nostr event and the existing WebSocket/`POST /events` ingest +path over adding endpoint-specific JSON APIs. The relay intentionally exposes +only a narrow HTTP surface: NIP-11/NIP-05 metadata, `/events`, `/query`, +`/count`, `/hooks/{id}`, Blossom media, git smart HTTP, git policy hooks, and +health probes. -## Outline +If an HTTP endpoint is still necessary: -- Routing and handler placement -- Auth requirements -- Persistence and errors -- Tests +1. **Define the handler** in the appropriate module under + `crates/buzz-relay/src/api/`. Resolve the request tenant before any auth or + data lookup, use NIP-98 when the endpoint accepts user credentials, and keep + community scoping explicit. -## Source material +2. **Register the route** in `crates/buzz-relay/src/router.rs` using the + narrowest path possible. Do not add new `/api/*` compatibility routes unless + the product decision explicitly calls for one. + +3. **Add database queries** in `buzz-db/src/` only when the endpoint cannot be + expressed through the existing event query paths. + +4. **Handle errors** using the `api_error()`, `internal_error()`, and + `not_found()` helpers in `buzz-relay/src/api/mod.rs`. Return + `(StatusCode, Json)` tuples. + +5. **Write tests** with the `buzz-test-client` harness in + `crates/buzz-test-client/tests/`, covering auth, community scoping, and the + relevant success path. + +6. **Document** any public endpoint in `ARCHITECTURE.md` and user-facing docs. -- `CONTRIBUTING.md` §How to Add a New API Endpoint -- `crates/buzz-relay/src/api/` diff --git a/docs/guides/adding-event-kinds.md b/docs/guides/adding-event-kinds.md index 704ea6ab3..5ddeadcab 100644 --- a/docs/guides/adding-event-kinds.md +++ b/docs/guides/adding-event-kinds.md @@ -1,16 +1,80 @@ # Adding a New Event Kind -> **Status: stub — Phase 1 (structure).** Content to be written in Phase 2. -> -> **Scope:** Step-by-step recipe for introducing a new event kind end-to-end (core → relay → clients). +Every action in Buzz is a Nostr event with a kind integer — adding a feature +usually means adding a kind, not a new API. Prefer reusing an existing kind +with new tags over minting a new number when the semantics genuinely match. +The authoritative registry is +[`crates/buzz-core/src/kind.rs`](../../crates/buzz-core/src/kind.rs). -## Outline +1. **Define the kind constant** in `buzz-core/src/kind.rs`: -- When to add a kind vs reuse -- Step-by-step (kind.rs, validation, persistence, client handling, tests) -- Conformance/NIP documentation expectations + ```rust + /// My new event kind — description of what it represents. + pub const KIND_MY_FEATURE: u32 = 4XXXX; + ``` -## Source material + Pick a kind number in the appropriate sub-range defined in `kind.rs`. + Check the `ALL_KINDS` array for collisions. Each sub-range is documented + with comments in the file. -- `CONTRIBUTING.md` §How to Add a New Event Kind -- `crates/buzz-core/src/kind.rs` +2. **Define the payload type** in the appropriate module in `buzz-core/src/` + (e.g., alongside `event.rs`) if the content field is structured JSON: + + ```rust + #[derive(Debug, Serialize, Deserialize)] + pub struct MyFeaturePayload { + pub field_one: String, + pub field_two: Option, + } + ``` + +3. **Register the kind's required scope** in + `crates/buzz-relay/src/handlers/ingest.rs` inside + `required_scope_for_kind()`. This controls which auth scope a caller + needs to submit the event: + + ```rust + KIND_MY_FEATURE => Ok(Scope::MessagesWrite), + ``` + +4. **Handle post-storage side effects** by adding a match arm in + `crates/buzz-relay/src/handlers/side_effects.rs` inside + `handle_side_effects()`: + + ```rust + KIND_MY_FEATURE => handle_my_feature(event, state).await?, + ``` + + `handle_side_effects()` runs after the event is stored — use it for + notifications, cache invalidation, or derived data. If the new kind + also needs an HTTP bridge surface (for example, a protocol helper that + cannot practically use WebSocket), add a handler in + `crates/buzz-relay/src/api/` and register it in + `crates/buzz-relay/src/router.rs`. + +5. **Persist to the database** — if the event needs to be queryable, add a + handler in `buzz-db/src/` (e.g., `buzz-db/src/my_feature.rs`) with + the appropriate `INSERT` and `SELECT` queries. + +6. **Index for search** (if applicable) — Postgres FTS indexes persisted + events automatically via the `events.search_tsv` generated column. To + exclude a privacy-sensitive kind from search, add it to the `CASE WHEN + kind IN (...)` exclusion in the `search_tsv` definition (see the initial + schema migration) rather than wiring a separate indexer. + +7. **Audit** — the audit log captures all events automatically; no changes + needed unless you need custom audit metadata. + +8. **Write tests** — add a unit test for payload serialization in + `buzz-core` and an integration test in `buzz-test-client` that sends + the new event kind and verifies the expected behavior. + +9. **Document** — `kind.rs` is the authoritative registry of all kind numbers. + Update `README.md` if it's a user-facing feature. + + +## Protocol documentation + +If the kind is part of a coherent protocol extension (not just an internal +implementation detail), document it as a Buzz NIP in [`docs/nips/`](../nips/) +and add it to the [NIPs index](../reference/nips.md). diff --git a/docs/guides/agents.md b/docs/guides/agents.md index 476378e66..6b3f0ed3c 100644 --- a/docs/guides/agents.md +++ b/docs/guides/agents.md @@ -1,22 +1,158 @@ # Working with Agents -> **Status: stub — Phase 1 (structure).** Content to be written in Phase 2. -> -> **Scope:** Running agents against a relay: the ACP harness, agent identity, memory, personas, and example agents. - -## Outline - -- Agent model overview (buzz-acp harness, buzz-agent) -- Minting an agent identity + channel membership (sharp edges called out) -- Running buzz-acp (env, ws:// vs http://) -- Core memory / NIP-AE prompt injection -- Personas (buzz-persona) -- Examples: countdown-bot, meadow-core -- Troubleshooting (idle agent, quiet stdout) - -## Source material - -- `TESTING.md` §ACP Harness -- `crates/buzz-acp`, `crates/buzz-agent/README.md`, `crates/buzz-persona` -- `examples/countdown-bot`, `examples/meadow-core` -- `docs/MCP_DRIVEN_HOOKS.md` (linked) +Agents are members, not bots: each agent has its own Nostr keypair, its own +channel memberships, and its own audit trail. This guide covers running an +agent against a relay with the ACP harness, agent identity, memory, personas, +and the example agents. + +## The agent model + +``` +Buzz Relay ──WS──→ buzz-acp ──stdio (ACP/JSON-RPC)──→ Agent (goose / codex / claude code / buzz-agent) + │ + Buzz CLI + MCP tools +``` + +- **[`buzz-acp`](../../crates/buzz-acp)** is the harness: it connects to the + relay with the agent's identity (NIP-42), discovers the channels the agent + is a member of, queues @mention events per channel (at most one prompt + in-flight per channel), and drives the agent subprocess over stdio using the + [Agent Client Protocol](https://agentclientprotocol.com/). It supports a + pool of 1–32 agent subprocesses and respawns them on crash. +- **[`buzz-agent`](../../crates/buzz-agent)** is a minimal ACP-compliant LLM + agent (Anthropic or any OpenAI-compatible endpoint) with MCP tool support — + useful when you don't want a full goose/codex/claude installation. +- Any other ACP-speaking agent works: **goose**, **codex** (via + [codex-acp](https://github.com/zed-industries/codex-acp)), **claude code** + (via [claude-agent-acp](https://github.com/agentclientprotocol/claude-agent-acp)). + Set `BUZZ_ACP_AGENT_COMMAND` / `BUZZ_ACP_AGENT_ARGS` to choose. + +## Running the harness end-to-end + +The walkthrough below assumes a local relay and a channel, as set up in the +[Quickstart](../getting-started/quickstart.md) / +[Running a Local Relay](../getting-started/local-relay.md). + +`buzz-acp` connects an ACP-speaking agent (goose, codex, claude code, +buzz-agent) to the relay. The harness listens for events, drives the +agent over stdio, and the agent replies through MCP tools. + +Minimum recipe — assumes the relay from step 3 is running and the channel +`$CHANNEL` from step 4 still exists. The agent identity must be **different** +from the sender identity (`BUZZ_ACP_RESPOND_TO=anyone` still skips events +the agent signed itself). + +```bash +cargo build --release -p buzz-acp +export PATH="$PWD/target/release:$PATH" + +# 1. Save your sender identity from step 4 — you'll need it to @mention the agent +SENDER_SK="$BUZZ_PRIVATE_KEY" + +# 2. Mint a fresh agent identity and capture its pubkey +AGENT_GEN=$(buzz-admin generate-key) +AGENT_SK=$(echo "$AGENT_GEN" | awk '/Secret key:/ {print $3}') +AGENT_PUBKEY=$(echo "$AGENT_GEN" | awk '/Public key:/ {print $3}') + +# 3. Add the agent as a member of $CHANNEL — still using the sender identity. +# Skip this and the agent boots to "discovered 0 channel(s) → agent will +# sit idle" and silently ignores every mention. +buzz channels add-member --channel "$CHANNEL" --pubkey "$AGENT_PUBKEY" --role member + +# 4. Switch to the agent identity and start it. +# buzz-acp wants ws:// (not http://). If you set BUZZ_RELAY_URL to an +# http:// URL in step 3, set the ws:// equivalent here — same host/port. +export BUZZ_PRIVATE_KEY="$AGENT_SK" +export BUZZ_RELAY_URL=ws://localhost:3000 # match step 3 (e.g. ws://localhost:3030 if overridden) +export BUZZ_ACP_RESPOND_TO=anyone # default is owner-only; opens the gate for testing +# NIP-AE core-memory prompt injection is on by default; set BUZZ_ACP_NO_MEMORY=true to opt out. +export GOOSE_MODE=auto # must be 'auto' or goose hangs on prompts + +buzz-acp # foreground; logs to stdout (run in a separate terminal) + +# Optional: turn on per-turn tracing if the default log is too quiet. +# RUST_LOG=buzz_acp=debug buzz-acp +``` + +> **Using a different ACP agent?** The default recipe assumes `goose` is on +> `$PATH` and configured (`goose --version` should print). For codex / claude +> code / buzz-agent, set `BUZZ_ACP_AGENT_COMMAND` and `BUZZ_ACP_AGENT_ARGS` +> accordingly — see `crates/buzz-acp/README.md`. Without these, buzz-acp +> will fail to spawn the agent subprocess on startup. + +If you started the agent before adding it to the channel, just run the +`add-member` afterwards — it picks up the membership notification live and +subscribes without restart (`membership notification: subscribing to new channel …`). + +The justfile also ships `just goose key="$AGENT_NSEC"` (foreground) and +`just goose-bg key="$AGENT_NSEC"` (background screen session) which set the +same env. See `crates/buzz-acp/README.md` for parallel agents, heartbeats, +respond-to gates, and forum subscriptions. + +Send the agent a task — switch your shell back to the **sender** identity +from step 4 and @mention the agent: + +```bash +export BUZZ_PRIVATE_KEY=$SENDER_SK # the key from step 4 +buzz messages send --channel "$CHANNEL" \ + --content "Hey agent, reply PONG only." + +# Wait 10–90s, then read the channel — the agent's reply is a kind:9 from +# AGENT_PUBKEY. The current ACP build is quiet on stdout during a turn, so +# `buzz messages get` is how you confirm it ran. +buzz messages get --channel "$CHANNEL" --limit 5 | jq '.[] | {pubkey, content}' +``` + +Replies are kind:9 in the same channel; `buzz messages thread --channel +--event ` fetches the reply chain for a specific mention. + + +## Agent memory (NIP-AE) + +Agents get persistent memory as *engrams* — relay-stored key/value documents +specified in [NIP-AE](../nips/NIP-AE.md). The harness injects the agent's +`core` memory into its prompt each turn (disable with +`BUZZ_ACP_NO_MEMORY=true`). Agents manage memory through the CLI: + +```bash +buzz mem ls +buzz mem get core +buzz mem set core "identity, rules, goals…" +buzz mem patch --base-hash < diff.patch +buzz mem rm +``` + +## Lifecycle hooks + +Agents can be extended with MCP-driven lifecycle hooks — MCP tools whose bare +name starts with `_`, invoked by the agent at defined points in its execution +loop, invisible to the LLM. See +[MCP-Driven Lifecycle Hooks](../MCP_DRIVEN_HOOKS.md). + +## Personas + +[`buzz-persona`](../../crates/buzz-persona) implements *persona packs* — +bundled identity, prompt, and configuration for an agent (spec: +[`PERSONA_PACK_SPEC.md`](../../crates/buzz-persona/PERSONA_PACK_SPEC.md)). +The `buzz pack` CLI subcommand operates on packs locally (no relay needed). +[`examples/meadow-core`](../../examples/meadow-core) is a complete example pack. + +## Example: a non-AI bot + +[`examples/countdown-bot`](../../examples/countdown-bot) is a small bot that +connects directly over WebSocket, authenticates with NIP-42, and replies to +deterministic commands — demonstrating both a standalone bot identity and the +NIP-OA owner-attested path. Useful as a template for non-LLM automation. + +## Troubleshooting + +| Symptom | Cause | Fix | +|---------|-------|-----| +| Agent ignores all events | `BUZZ_ACP_RESPOND_TO=owner-only` (default) with no owner configured | `export BUZZ_ACP_RESPOND_TO=anyone` for testing | +| `discovered 0 channel(s)` / agent sits idle | Agent identity isn't a member of any channel | `buzz channels add-member --channel --pubkey --role member` from another identity | +| `GOOSE_MODE` warning, agent hangs | Not set | `export GOOSE_MODE=auto` | +| Harness fails to spawn agent | Agent binary not on PATH / not configured | Check `BUZZ_ACP_AGENT_COMMAND`, `goose --version` | +| Quiet stdout during a turn | Expected — current ACP build logs little | Confirm via `buzz messages get`; use `RUST_LOG=buzz_acp=debug` for tracing | + +More: [`crates/buzz-acp/README.md`](../../crates/buzz-acp/README.md) covers +parallel agents, heartbeats, respond-to gates, and forum subscriptions. diff --git a/docs/guides/development.md b/docs/guides/development.md index 74df3fe8a..31d630627 100644 --- a/docs/guides/development.md +++ b/docs/guides/development.md @@ -1,19 +1,228 @@ # Development Guide -> **Status: stub — Phase 1 (structure).** Content to be written in Phase 2. -> -> **Scope:** Setting up a dev environment, code style, and PR workflow. Root CONTRIBUTING.md remains the canonical entry point and will link here. +Setting up a dev environment, code style, and the PR workflow. The root +[`CONTRIBUTING.md`](../../CONTRIBUTING.md) is the canonical contributor entry +point (code of conduct, CLA); this page is the day-to-day engineering detail. -## Outline +## Environment setup -- Environment setup (clone, Hermit, bootstrap, hooks) -- Repo layout tour -- Code style (Rust, TS/biome) -- Running checks locally (Justfile) -- PR workflow and review expectations -- License/CLA pointer +## Prerequisites -## Source material +| Tool | Version | Notes | +|------|---------|-------| +| Rust | 1.88+ | Install via [rustup](https://rustup.rs/) | +| Node.js | 24+ | Required for desktop app commands and `just ci` | +| pnpm | 10+ | Required for desktop app commands and `just ci` | +| Flutter | 3.41+ | Required for mobile app — install via [flutter.dev](https://docs.flutter.dev/get-started/install) | +| Docker | 24+ | For Postgres, Redis, MinIO | +| `just` | latest | Task runner — `cargo install just` | +| `lefthook` | latest | Optional; run `lefthook install` for local Git hooks | +| `sqlx` migrations | workspace crate | `just migrate` applies embedded migrations from `migrations/` | -- `CONTRIBUTING.md` §Setting Up, §Code Style, §Making a Pull Request, §License and CLA -- `Justfile`, `lefthook.yml`, `biome.json` +This repo uses [Hermit](https://cashapp.github.io/hermit/) for toolchain +pinning. Activate it once per shell session: + +```bash +. ./bin/activate-hermit +``` + +Hermit pins Rust, `just`, Node, pnpm, and other tools to the versions in +`bin/`. Each tool is downloaded on first use. You can also run `just bootstrap` +(which `just setup` calls automatically) to pre-download all required tools +upfront. If you don't use Hermit, ensure your toolchain meets the minimum +versions in the table above. + +## First-Time Setup + +```bash +# 1. Clone the repo +git clone https://github.com/block/buzz.git +cd buzz + +# 2. Activate Hermit (optional but recommended) +. ./bin/activate-hermit + +# 3. Bootstrap tools + infrastructure +just setup + +# 4. Install Git hooks (optional, recommended) +just hooks +``` + +`just setup` runs `just bootstrap` first — it copies `.env.example` to `.env` +if it doesn't already exist, and invokes `cargo`, `node`, and `pnpm` to trigger +Hermit's lazy tool download (each tool is fetched once on first invocation and +cached thereafter). You can also run `just bootstrap` independently at any time; +it is safe to re-run. + +`just setup` then starts Docker services (Postgres on `:5432`, Redis on `:6379`, +Adminer on `:8082`, Keycloak on `:8180` for local OAuth/OIDC testing, MinIO on +`:9000` for media storage, and Prometheus on `:9090` for metrics) and runs all +pending database migrations. + +## Running the Relay and Desktop App + +```bash +just dev # starts the relay + desktop app in one command +``` + +`just dev` builds all agent tools, starts the relay (`ws://localhost:3000`) in +the background, and launches the Tauri desktop app. The relay process is +automatically killed when you quit the app or press Ctrl+C. + +For a split-terminal workflow (relay logs visible separately from Vite output): + +```bash +just relay # terminal 1 — relay on ws://localhost:3000 +just desktop-dev # terminal 2 — Vite dev server only (no Tauri shell) +``` + +## Stopping / Resetting + +```bash +just down # Stop Docker services, keep data +just reset # ⚠️ Wipe all data and recreate the environment +``` + + +## Repo layout + +A Rust workspace of focused crates (see the +[Crate Reference](../architecture/crates.md)), plus `desktop/` (Tauri + React), +`web/`, `mobile/` (Flutter), `deploy/` (Compose bundle + Helm chart), +`migrations/`, and `examples/`. Task automation lives in the +[`Justfile`](../../Justfile); Git hooks in [`lefthook.yml`](../../lefthook.yml); +TS lint/format in [`biome.json`](../../biome.json). + +## Code style + +## Formatting + +We use `rustfmt` with default settings. Format your code before committing: + +```bash +cargo fmt --all +``` + +To check without modifying: + +```bash +cargo fmt --all -- --check +``` + +## Linting + +We use `clippy` with warnings-as-errors: + +```bash +cargo clippy --all-targets --all-features -- -D warnings +``` + +Fix all clippy warnings before submitting a PR. If you believe a warning is +a false positive, add a targeted `#[allow(...)]` with a comment explaining +why. + +## No Unsafe Code + +All crates enforce `#![deny(unsafe_code)]`. Do not add unsafe blocks. If you +believe unsafe is genuinely necessary, open an issue first to discuss the +approach. + +## Error Handling + +- Use `thiserror` for library error types. +- Use `anyhow` for binary / application-level error propagation. +- Do not use `unwrap()` or `expect()` in production code paths. Use `?` or + explicit error handling. `unwrap()` is acceptable in tests. + +## Logging and Tracing + +Use the `tracing` crate for all instrumentation. Prefer structured fields +over string interpolation: + +```rust +// Good +tracing::info!(channel_id = %id, event_kind = kind, "Event ingested"); + +// Avoid +tracing::info!("Event ingested: channel={id} kind={kind}"); +``` + +## Commit Messages + +Follow [Conventional Commits](https://www.conventionalcommits.org/): + +``` +feat(mcp): add get_feed_actions tool +fix(auth): reject expired NIP-42 challenges +docs(agents): document workflow MCP tools +refactor(db): extract channel queries into channel.rs +test(workflow): add approval gate integration test +``` + +The type prefix (`feat`, `fix`, `docs`, `refactor`, `test`, `chore`) is +required. The scope (in parentheses) is optional but encouraged. + + +## Making a pull request + +## Before You Start + +- Check open issues and PRs to avoid duplicate work. +- For significant changes, open an issue first to discuss the approach. +- For small fixes (typos, doc improvements, obvious bugs), go ahead and open + a PR directly. + +## What a Good PR Looks Like + +1. **Focused** — one logical change per PR. If you're fixing a bug and + refactoring a module, split them into two PRs. + +2. **Tested** — new behavior has tests. Bug fixes include a regression test. + If a test is impractical, explain why in the PR description. + +3. **Documented** — public APIs, new event kinds, new MCP tools, and new + config variables are documented. Update `README.md`, `AGENTS.md`, or + `VISION.md` as appropriate. + +4. **CI passing** — `just ci` passes locally before you push. + +5. **Clear description** — the PR description explains: + - What problem this solves (or what feature it adds) + - How it was implemented (key decisions, trade-offs) + - How to test it manually (if applicable) + - Any follow-up work deferred to a future PR + +## PR Checklist + +``` +- [ ] `just ci` passes (fmt + clippy + unit tests + mobile) +- [ ] Integration tests pass (`just test`) +- [ ] New public APIs / tools / endpoints are documented +- [ ] No new `unwrap()` in production code paths +- [ ] No new `unsafe` blocks +``` + +## Review Process + +- A maintainer will review your PR within a few business days. +- Address review comments by pushing new commits (don't force-push during + review; it makes it hard to see what changed). +- Once approved, a maintainer will squash-merge your PR. + + +## License and CLA + +Buzz is licensed under the **Apache License, Version 2.0**. See +[LICENSE](../../LICENSE) for the full text. + +By submitting a pull request, you agree that your contribution is licensed +under the Apache 2.0 license and that you have the right to submit it. + +If your employer has rights to intellectual property you create, you may need +their sign-off. When in doubt, check with your legal team. + +--- + +*Thank you for contributing to Buzz. Every bug report, documentation fix, +and code contribution makes the project better for everyone. 🐝* diff --git a/docs/guides/nostr-clients.md b/docs/guides/nostr-clients.md index 492e3b2b8..1630db072 100644 --- a/docs/guides/nostr-clients.md +++ b/docs/guides/nostr-clients.md @@ -1,17 +1,230 @@ # Using Third-Party Nostr Clients -> **Status: stub — Phase 1 (structure).** Content to be written in Phase 2. -> -> **Scope:** Operational how-to for connecting standard Nostr clients to a Buzz relay: setup, allowlisting, debugging with nak. +Buzz is a Nostr relay that speaks NIP-29 (relay-based groups) natively. +Third-party Nostr clients connect directly to `buzz-relay` using NIP-29 and +NIP-42 authentication. This page is the operational how-to; the underlying +protocol semantics are in [Protocol](../architecture/protocol.md). -## Outline +## Community scope -- Compatibility overview -- Client setup -- Allowlisting users -- Debugging with `nak` (recipes) -- Troubleshooting interop +Buzz treats the relay URL/domain as authoritative for the community. Today's +single-relay deployment has exactly one community behind that URL, so existing +NIP-29 clients keep using the same WebSocket URL, event kinds, tags, and +HTTP/media/git paths. In a multi-community deployment, each community is reached +by its own domain or subdomain; the backend resolves the community from the host +before handling AUTH, EVENT, REQ, REST, media, git, search, or workflow traffic. -## Source material +The Nostr wire format does not grow a tenant tag. Client-supplied `#h` tags still +name channels/groups and are checked against the host-derived community. Events +without `#h` — profiles, gift-wrapped DMs, membership notifications, lists, +status, long-form notes, workflow/system events, and other "global" streams — are +global only inside the connected community. A pubkey can join multiple +communities and repost its profile in each one; DMs and profiles do not inherit +across community domains. -- `NOSTR.md` (operational half: client setup, allowlist, nak recipes, troubleshooting) + +## NIP-29 Direct + +Connect any NIP-29 client straight to the relay. + +### Quick Start + +```bash +# 1. (Optional) Enable pubkey allowlist — must be set BEFORE relay startup +export BUZZ_PUBKEY_ALLOWLIST=true + +# 2. Start the relay (auto-starts Docker services and runs migrations) +just relay & # relay on :3000 + +# 3. Add a pubkey to the allowlist (if enabled) +# Insert directly — there is no CLI command for this yet. +PGPASSWORD=buzz_dev psql -h localhost -U buzz -d buzz -c \ + "INSERT INTO pubkey_allowlist (pubkey) VALUES (decode('<64-char-hex-pubkey>', 'hex'))" + +# 5. Connect any NIP-29 + NIP-42 client to ws://localhost:3000 +``` + +### What Works + +| Feature | Status | Notes | +|---------|:------:|-------| +| **Group chat (kind:9)** | ✅ | Send/receive messages with `#h ` tag | +| **Reactions (kind:7)** | ✅ | Standard NIP-25; channel derived from target event's `#e` tag (client `#h` ignored) | +| **Deletions (kind:5)** | ✅ | Standard NIP-09; self-authored only. `#h` optional, `#e` required | +| **User profiles (kind:0)** | ✅ | NIP-01 metadata; synced to users table (display_name, avatar, about, NIP-05). NIP-05 handles must canonicalize to this relay's domain — off-domain or invalid handles are silently cleared. If a NIP-05 handle collides with another user's (UNIQUE constraint), the handle is skipped but other profile fields (display_name, avatar, about) are still synced. | +| **Group creation (kind:9007)** | ✅ | NIP-29; include `name` tag, optional `visibility` and `channel_type` | +| **Add user (kind:9000)** | ✅ | Open: any user, subject to target's `channel_add_policy` (`owner_only`/`nobody` can block). Private: owner/admin only. Self-add bypasses agent policy but not private-channel auth. | +| **Remove user (kind:9001)** | ✅ | Self-remove allowed (with last-owner guard). Removing others: owner/admin only. | +| **Edit group metadata (kind:9002)** | ✅ | `name`/`about` tags: owner/admin. `topic`/`purpose` tags: any member. | +| **Admin delete event (kind:9005)** | ✅ | Event author can always delete own. Otherwise owner/admin required. Target must be in same channel. | +| **Group deletion (kind:9008)** | ✅ | Owner only. | +| **Leave group (kind:9022)** | ✅ | Any member. Last-owner guard prevents orphaned groups. | +| **Group metadata (kind:39000)** | ✅ | Relay-signed; always `d`, `name`, `closed` tags; `about` only if description non-empty; `private` if applicable; `hidden` for DM channels | +| **Group admins (kind:39001)** | ✅ | Relay-signed; `d` tag + `p` tags with roles (`owner`, `admin`) | +| **Group members (kind:39002)** | ✅ | Relay-signed; `d` tag + `p` tags for all members | +| **Membership notifications** | ✅ | kind:44100 (added) / kind:44101 (removed); relay-signed, community-global scope (`channel_id=None` inside the connected community) | +| **Presence (kind:20001)** | ✅ | Ephemeral; arbitrary status string (truncated to 128 chars); writes to Redis (`set_presence`/`clear_presence` on `"offline"`), then fan-out to local subscribers. In multi-community mode presence is scoped to the connected community. | +| **Typing indicators (kind:20002)** | ✅ | Ephemeral, not stored; published via Redis pub/sub (multi-node capable unlike presence fan-out) | +| **NIP-42 authentication** | ✅ | Proactive challenge; optional pubkey allowlist | +| **NIP-11 relay info** | ✅ | `GET /` with `Accept: application/nostr+json` | +| **Blossom media** | ✅ | `PUT /media/upload` (BUD-02), `GET /media/{sha256}.{ext}` (BUD-01) | +| **NIP-50 search** | ✅ | One-shot search REQs: `{"search":"query","kinds":[9],"#h":[""]}` → relevance-sorted results → EOSE. Not registered as persistent subscriptions. | +| **NIP-10 threads** | ✅ | WS-submitted replies with `["e","","","reply"]` tags create `thread_metadata` atomically. Visible in REST thread queries. Unknown parents rejected. | +| **NIP-17 DMs (gift wrap)** | ✅ | kind:1059 accepted with ephemeral signing keys. Stored community-globally (`channel_id=None` inside the connected community). Delivered via `#p`-filtered subscriptions. Not indexed in search. | +| **DM discovery** | ✅ | DM creation emits kind:39000 (with `hidden` tag) + kind:44100 membership notifications. NIP-29 clients discover DMs via standard group discovery flow. | +| **Join request (kind:9021)** | ✅ | Open channels only. Adds member, emits system message + group discovery events + kind:44100 membership notification. Private channels rejected at ingest. | +| **Edits (kind:40003)** | ⚠️ | Works on the wire but Buzz-only — no standard NIP-29 client renders these | +| **Rich content (kind:40002)** | ⚠️ | Works on the wire but Buzz-only — no standard NIP-29 client renders these | + +### What Doesn't Work + +| Feature | Status | Why | +|---------|:------:|-----| +| **Create invite (kind:9009)** | ⚠️ | Accepted and stored, but side-effect handler is deferred (no-op with warning log) | +| **Group roles (kind:39003)** | ❌ | Defined in kind registry but not emitted by the relay | +| **DMs** | ⚠️ | NIP-17 gift wraps supported; NIP-04/NIP-44 not implemented. kind:10050 (DM relay list) deferred. | + +### Pubkey Allowlist + +When `BUZZ_PUBKEY_ALLOWLIST=true`, NIP-42 connections that authenticate with only a pubkey +(no API token) are checked against the `pubkey_allowlist` table. This lets you open the +relay to specific external Nostr identities without granting full access. + +- Users with valid **API tokens** bypass the allowlist. +- **Fail-closed:** if the DB lookup fails, the connection is denied. +- Default: `false` (all authenticated pubkeys accepted). +- Auth failure returns generic `auth-required: verification failed` (no allowlist-specific message). +- Manage the allowlist via direct SQL (no CLI command yet): + ```sql + INSERT INTO pubkey_allowlist (pubkey) VALUES (decode('<64-char-hex-pubkey>', 'hex')); + DELETE FROM pubkey_allowlist WHERE pubkey = decode('<64-char-hex-pubkey>', 'hex'); + SELECT encode(pubkey, 'hex'), added_at, note FROM pubkey_allowlist; + ``` + +### Group Discovery + +The relay emits NIP-29 group state events when channels are created, updated, or membership changes. +All discovery events include a `d` tag set to the channel UUID (NIP-29 addressable event convention): + +| Kind | Tags | Content | +|------|------|---------| +| **39000** | `d=`, `name`, `closed` (always); `about` (if description non-empty); `private` (if applicable); `hidden` (DM channels only) | Group metadata. **Note:** `closed` is always emitted per NIP-29 convention (Buzz channels require explicit membership), but open channels are still readable/writable by non-members at runtime. The tag reflects the membership model, not access enforcement. | +| **39001** | `d=`, `p` tags with role label (`owner`, `admin`) | Admin list | +| **39002** | `d=`, `p` tags for all members | Member list | + +Events are stored **channel-scoped** so access control applies — private channel member lists are +only visible to members. Discover groups via historical REQ: + +```bash +# All groups you can see +nak req -k 39000 --auth --sec ws://localhost:3000 + +# Specific group's members (filter by d tag) +nak req -k 39002 --tag "d=" --auth --sec ws://localhost:3000 +``` + +> **Note:** Channel-scoped storage means live global subscriptions (`{kinds:[39000]}`) won't +> receive these via fan-out. Clients discover groups via historical REQ queries. Live push for +> open-channel discovery is a future enhancement. + +### Membership Notifications + +The relay emits relay-signed notifications when members are added or removed: + +| Kind | Meaning | Tags | Scope | +|------|---------|------|-------| +| **44100** | Member added | `p` = target pubkey, `h` = channel UUID | Community-global | +| **44101** | Member removed | `p` = target pubkey, `h` = channel UUID | Community-global | + +Stored community-globally (`channel_id = None` inside the connected community) so agents and clients can subscribe without knowing channel +UUIDs in advance. Client-submitted kind:44100/44101 events are rejected — only the relay keypair +may sign these. + +> **Subscription constraint:** Global REQs that can match p-gated kinds (44100, 44101, 1059) **must** +> include a `#p` filter where **all** `#p` values match the authenticated pubkey. The relay rejects +> subscriptions that omit `#p` or include other pubkeys (prevents eavesdropping on others' membership +> changes and DMs). Error: `restricted: p-gated events require #p matching your pubkey`. + +```bash +nak req -k 44100 -k 44101 --tag "p=" \ + --auth --sec ws://localhost:3000 +``` + +### Sending Messages + +```bash +# Send a kind:9 message +nak event -k 9 -c "Hello from NIP-29!" --tag "h=" \ + --auth --sec ws://localhost:3000 + +# Subscribe to channel messages +nak req -k 9 --tag "h=" --stream \ + --auth --sec ws://localhost:3000 + +# React to a message (#h optional but recommended; channel derived from #e target) +nak event -k 7 -c "+" --tag "h=" --tag "e=" \ + --auth --sec ws://localhost:3000 + +# Delete a message (#h optional; #e required; must be self-authored) +nak event -k 5 -c "reason" --tag "h=" --tag "e=" \ + --auth --sec ws://localhost:3000 + +# Create a group +nak event -k 9007 --tag "name=my-channel" --tag "visibility=open" \ + --auth --sec ws://localhost:3000 + +# Search messages (NIP-50) +nak req -k 9 --tag "h=" --search "search query" -l 20 \ + --auth --sec ws://localhost:3000 + +# Reply to a message (NIP-10 threading) +nak event -k 9 -c "Reply text" --tag "h=" \ + --tag "e=;;reply" \ + --auth --sec ws://localhost:3000 + +# Fetch gift-wrapped DMs (NIP-17) +nak req -k 1059 --tag "p=" \ + --auth --sec ws://localhost:3000 +``` + +### Tested Clients (Direct) + +| Client | Platform | Evidence | Notes | +|--------|----------|:--------:|-------| +| **BuzzTestClient** | Rust (repo) | Automated E2E | Full NIP-29 flow: discovery (39000/39001/39002), kind:9 send/receive, reactions, deletions, h-tag enforcement | +| **E2E nostr interop** | Rust (repo) | Automated E2E | NIP-50 search (3 tests), NIP-10 threads (3 tests), NIP-17 gift wraps (3 tests), DM discovery (1 test) | +| **nak** | CLI | Manual (verified) | kind:9 send/recv, NIP-50 search, NIP-10 thread replies, group discovery | + +**Not verified in-repo** (anecdotal / expected based on NIP-29 support): +- **Chachi** (Web/Mobile) — NDK-based; NIP-29 native +- **0xchat** (Mobile) — NIP-29 native + + +## Security notes + +## Direct Path +- **Pubkey allowlist is fail-closed.** DB errors deny the connection. +- **API token users bypass the allowlist.** The allowlist only gates pubkey-only NIP-42. +- **kind:9 requires `#h` tag.** Messages without a channel-scoped `#h` tag are rejected. +- **kind:7 derives channel from target.** Reactions look up the target event's channel via `#e` — client-supplied `#h` tags are ignored. Reactions to unknown events are rejected (fail-closed). +- **kind:5 uses `#h` if present, but doesn't require it.** Deletions validate author-match against target events via `#e` tags. Only self-authored events can be deleted (admin deletions use kind:9005). +- **Client-submitted kind:44100/44101 rejected.** Membership notifications can only be signed by the relay keypair. + + +## Troubleshooting + +## Direct Path + +| Symptom | Cause | Fix | +|---------|-------|-----| +| `auth-required: verification failed` | Pubkey not in allowlist (when enabled), or NIP-42 auth failed | Add pubkey to `pubkey_allowlist` table; verify NIP-42 challenge/response | +| `invalid: channel-scoped events must include an h tag` | kind:9 sent without `#h` tag | Include `--tag "h="` | +| `invalid: reaction target event not found` | Reaction references unknown event | Ensure the target event exists in the relay | +| No discovery events | Channel is private + you're not a member | Join the channel first | + + +## Further reading + +- [Protocol](../architecture/protocol.md) — event shape, kinds, wire messages +- [Buzz NIPs Index](../reference/nips.md) — Buzz protocol extensions +- [Self-Hosting](self-hosting.md) — relay membership (NIP-43) and operator admin diff --git a/docs/guides/releasing.md b/docs/guides/releasing.md index 4b9280242..31e0bbdf9 100644 --- a/docs/guides/releasing.md +++ b/docs/guides/releasing.md @@ -1,18 +1,225 @@ # Release Guide -> **Status: stub — Phase 1 (structure).** Content to be written in Phase 2. -> -> **Scope:** Cutting desktop, relay, and mobile releases. Root RELEASING.md will be migrated here and left as a pointer. +Buzz has three independent release lanes, each driven by a release PR — no human +ever pushes a git tag: -## Outline +| Lane | Recipe | Artifact | +|------|--------|----------| +| Desktop | `just release-desktop` | Signed desktop app (macOS/Linux) | +| Relay | `just release-relay` | `ghcr.io/block/buzz` container image | +| Mobile | `just release-mobile` | Buzz mobile app (tag is the `sprout_ref` for the internal build) | -- Quick start (desktop/relay/mobile) -- How the release automation works -- Release types + version files -- Manual fallback -- Internal releases -- Prerequisites + troubleshooting +The three lanes version independently: the desktop version lives in +`desktop/package.json`, the relay version in `crates/buzz-relay/Cargo.toml`, and +the mobile version in `mobile/pubspec.yaml`. -## Source material +The mobile lane publishes a `mobile-v` tag that is consumed +**manually**, cross-repo, as the `sprout_ref` input to the internal +`buzz-releases` Buildkite pipeline (iOS dogfood → Block Comp Portal, App Store → +TestFlight — see [Internal Releases](#internal-releases)). The OSS lane is +tag-only **by design**: OSS `block/buzz` CI cannot trigger CI in the private +`buzz-releases` repo (infosec), so a human cuts the internal build from the tag +rather than auto-dispatching across that boundary. -- `RELEASING.md` (full migration) +## Quick Start + +```sh +# Desktop release (next patch version) +just release-desktop + +# Desktop patch / minor / explicit +just release-desktop patch +just release-desktop 0.4.0 +just release-desktop 1.0.0 + +# Relay release (same argument forms) +just release-relay +just release-relay 0.4.0 + +# Mobile release (same argument forms) +just release-mobile +just release-mobile 0.4.0 +``` + +`just release-desktop` creates a `version-bump/` PR; `just +release-relay` creates a `relay-release/` PR; `just release-mobile` +creates a `mobile-release/` PR. Each bumps its own version manifest, +regenerates lockfiles, and appends a changelog entry. Merge the PR to trigger +the build automatically (the mobile tag is instead the `sprout_ref` a human +feeds the internal build — see above). + +Re-running any of these recipes with the same version is safe — it detects the +existing branch and PR, resets to current `main`, regenerates the changelog +with any new commits, and updates the PR in place. + +--- + +## How It Works + +All three lanes share one engine; they differ only in which version manifest +they bump, which branch prefix they use, and what the merge triggers. + +### Desktop + +1. **`just release-desktop`** runs locally on `main` — computes the next + version, creates (or reuses) a `version-bump/` branch, bumps the + desktop manifests, regenerates lockfiles, generates a changelog + entry in `CHANGELOG.md`, commits, pushes, and opens (or updates) a PR. +2. **Merge the PR** — the `auto-tag-on-release-pr-merge` workflow detects the + `version-bump/*` branch merge and pushes a `v` tag. +3. **Tag triggers `release.yml`** — builds, signs, notarizes, and publishes the + desktop app for macOS and Linux. + +### Relay + +1. **`just release-relay`** runs locally on `main` — computes the next relay + version, creates (or reuses) a `relay-release/` branch, bumps + `crates/buzz-relay/Cargo.toml`, regenerates `Cargo.lock`, generates a + changelog entry in `crates/buzz-relay/CHANGELOG.md`, commits, pushes, and + opens (or updates) a PR. +2. **Merge the PR** — the `auto-tag-on-release-pr-merge` workflow detects the + `relay-release/*` branch merge and pushes a `relay-v` tag. +3. **Auto-tag dispatches `docker.yml`** — the same workflow then triggers + `docker.yml` with the version and tag ref, which builds the multi-arch relay + image and publishes `ghcr.io/block/buzz:` (plus `:.`, + `:`, and `:latest` for stable releases). Prereleases + (`relay-v-rc.1`) publish only the prerelease tag and do **not** + move `:latest`. (The dispatch — rather than relying on `docker.yml`'s + `push: tags` trigger — is required because GitHub suppresses `on: push` for + tags pushed by the workflow's own `GITHUB_TOKEN`; the desktop lane dispatches + `release.yml` for the same reason.) + +Every push to `main` continues to build and publish `:main` + `:sha-<7>` tags +(the rolling development image). The `:latest` tag tracks the latest **stable** +relay release only — it does not move on main pushes or prereleases. + +### Mobile + +1. **`just release-mobile`** runs locally on `main` — computes the next mobile + version, creates (or reuses) a `mobile-release/` branch, bumps + `mobile/pubspec.yaml` (preserving the `+build` number), regenerates + `mobile/pubspec.lock`, generates a changelog entry in `mobile/CHANGELOG.md`, + commits, pushes, and opens (or updates) a PR. +2. **Merge the PR** — the `auto-tag-on-release-pr-merge` workflow detects the + `mobile-release/*` branch merge and pushes a `mobile-v` tag. +3. **The tag is consumed manually, cross-repo** — nothing in OSS `block/buzz` + builds on the tag (OSS CI must not trigger CI in the private `buzz-releases` + repo — infosec). A human feeds the `mobile-v` tag as the + `sprout_ref` input to the internal `buzz-releases` Buildkite pipeline, which + builds and ships iOS to Block Comp Portal (dogfood) and TestFlight (App + Store, opt-in). See [Internal Releases](#internal-releases). + +--- + +## Release Types + +The argument forms below apply to `release-desktop`, `release-relay`, and +`release-mobile`: + +| Command | Version | Example | +|---------|---------|---------| +| `just release-desktop` | Next patch | `0.3.0` → `0.3.1` | +| `just release-desktop patch` | Next patch | `0.3.0` → `0.3.1` | +| `just release-desktop 0.4.0` | Explicit minor | `0.3.1` → `0.4.0` | +| `just release-desktop 1.0.0` | Explicit | `1.0.0` | + +--- + +## Version Files + +`just bump-desktop-version ` (desktop lane) updates these files: + +| File | Field | +|------|-------| +| `desktop/package.json` | `"version"` | +| `desktop/src-tauri/tauri.conf.json` | `"version"` | +| `desktop/src-tauri/Cargo.toml` | `version` (under `[package]`) | + +It also regenerates `pnpm-lock.yaml` and `desktop/src-tauri/Cargo.lock`. + +`just bump-relay-version ` (relay lane) updates +`crates/buzz-relay/Cargo.toml` (`version` under `[package]`) and regenerates the +workspace `Cargo.lock`. + +`just bump-mobile-version ` (mobile lane) updates +`mobile/pubspec.yaml` (`version:`, preserving the `+build` number) and +regenerates `mobile/pubspec.lock`. + +--- + +## Manual Fallback + +If the automated flow isn't suitable (e.g., building from a non-main ref): + +1. Go to **Actions > Release** in the GitHub UI +2. Click **Run workflow** +3. Provide the semver version (no `v` prefix) and the ref to build from + +--- + +## Internal Releases + +After the OSS release ships, trigger an internal build via the +[sprout-releases Buildkite pipeline](https://buildkite.com/runway/sprout-releases). +See the [buzz-releases README](https://github.com/squareup/buzz-releases#cutting-a-release) +for the full step-by-step instructions and input field reference. + +--- + +## What Gets Published + +Each release produces two GitHub releases: + +1. **`v`** — the user-facing release with the `.dmg` installer + (macOS). + +2. **`buzz-desktop-latest`** — a rolling pre-release for the Tauri + auto-updater containing `latest.json`, the signed `.tar.gz` archive, + and its `.sig` signature. + +--- + +## Platform Support + +The release workflow builds **two separate macOS DMGs** — Apple +Silicon (`darwin-aarch64`, the `release` job) and Intel +(`darwin-x86_64`, the `release-macos-x64` job) — plus Linux `.deb` and +`.AppImage`. Both macOS DMGs are codesigned, notarized, and attached to +the same `v` release. Intel users download the `_x64.dmg`. + +--- + +## Prerequisites + +- **Write access** to the `block/buzz` GitHub repository +- **`gh` CLI** authenticated (`gh auth status`) +- The following **GitHub Actions secrets** must be configured: + + | Secret | Purpose | + |--------|---------| + | `BUZZ_UPDATER_PUBLIC_KEY` | Tauri updater public key (minisign) | + | `TAURI_SIGNING_PRIVATE_KEY` | Tauri updater private key | + | `TAURI_SIGNING_PRIVATE_KEY_PASSWORD` | Password for the private key | + +--- + +## Troubleshooting + +### `just release-desktop` fails with "must be on main branch" +Switch to `main` and pull latest before running the release recipe. + +### `just release-desktop` fails with "working tree is dirty" +Commit or stash your changes before running the release recipe. + +### New commits merged after creating the release PR +Re-run the release recipe (`just release-desktop`, `just release-relay`, or `just release-mobile`) from an up-to-date `main`. It resets the branch to current `main`, regenerates the changelog and PR body to include the new commits, and force-pushes the updated branch. + +### Build fails at "Validate version" +The version string must be valid semver: `MAJOR.MINOR.PATCH` with an optional pre-release suffix. Do not include a `v` prefix. + +### Auto-updater reports "no update available" +Verify that the `buzz-desktop-latest` release exists and contains a +valid `latest.json`. The auto-updater manifest currently lists +`darwin-aarch64` only, so Intel and Linux users do not yet receive +auto-updates — they download new versions manually from the release +page. (Adding their entries to `latest.json` is a follow-up.) diff --git a/docs/guides/self-hosting.md b/docs/guides/self-hosting.md index 4928b3fcf..56ed49283 100644 --- a/docs/guides/self-hosting.md +++ b/docs/guides/self-hosting.md @@ -1,23 +1,221 @@ # Self-Hosting Buzz -> **Status: stub — Phase 1 (structure).** Content to be written in Phase 2. -> -> **Scope:** Deploying and operating your own relay: Docker Compose VPS bundle, Helm chart, TLS, membership administration, multi-tenancy. - -## Outline - -- Deployment options overview -- Docker Compose bundle (run.sh, Caddy TLS) -- Kubernetes (Helm chart) -- Relay configuration (env vars) -- Membership + admin (buzz-admin, NIP-43 allowlist) -- Multi-tenant operation (links to design docs/specs) -- Upgrades and backups - -## Source material - -- `deploy/compose/README.md`, `deploy/charts/buzz/README.md` -- `Dockerfile`, `docker-compose.yml` -- `NOSTR.md` (allowlist/admin operational half) -- `docs/multi-tenant-relay.md`, `docs/multi-tenant-conformance.md` (linked) -- `crates/buzz-admin` +Deploying and operating your own relay. Two supported paths: the Docker +Compose bundle for a single node/VPS, and the Helm chart for Kubernetes. + +## Deployment options + +| Option | Where | Docs | +|---|---|---| +| Docker Compose bundle | Single node / VPS | [`deploy/compose/README.md`](../../deploy/compose/README.md) | +| Helm chart | Kubernetes (incl. GitOps via ArgoCD/Flux) | [`deploy/charts/buzz/README.md`](../../deploy/charts/buzz/README.md) | +| Bare binary | Anywhere Rust builds | [Running a Local Relay](../getting-started/local-relay.md) + this page's config section | + +Both packaged paths run the same stack: the relay +(`ghcr.io/block/buzz`) backed by Postgres (events + full-text search), +Redis (pub/sub, presence, typing), and S3-compatible object storage (media, +git). See [Infrastructure](../architecture/infrastructure.md). + +## Docker Compose (single node / VPS) + +```bash +cd deploy/compose +cp .env.example .env +$EDITOR .env # replace every CHANGE_ME value (9 of them) +./run.sh start +``` + +For a public VPS with automatic Let's Encrypt certificates via Caddy: + +```bash +BUZZ_COMPOSE_TLS=true ./run.sh start +``` + +Production notes (full detail in [`deploy/compose/README.md`](../../deploy/compose/README.md)): + +- Docker Compose v2.24.4+ required (the TLS override uses the `!reset` tag). +- Pin `BUZZ_IMAGE` to a release tag or `sha-<7>` for production — the default + tracks `ghcr.io/block/buzz:main`. +- Keep `BUZZ_RELAY_PRIVATE_KEY`, `BUZZ_GIT_HOOK_HMAC_SECRET`, database/Redis, + and S3 secrets stable across restarts. +- `BUZZ_AUTO_MIGRATE` is opt-in on Compose: set it `true` or run + `buzz-admin migrate` before first start. +- `./run.sh backup-hint` prints the backup checklist. + +## Kubernetes (Helm) + +The chart at [`deploy/charts/buzz`](../../deploy/charts/buzz) has two +profiles: + +- **Quickstart (eval only)** — in-cluster Postgres/Redis/MinIO, autogenerated + secrets, single replica: + + ```sh + helm install buzz oci://ghcr.io/block/buzz/charts/buzz --version 0.1.0 \ + --create-namespace --namespace buzz \ + --set quickstart=true \ + --set postgresql.enabled=true --set redis.enabled=true --set minio.enabled=true \ + --set relayUrl=wss://buzz.example.com \ + --set ownerPubkey=<64-char-hex-pubkey> + ``` + +- **Production (default)** — external managed Postgres/Redis/S3 and + `secrets.existingSecret`. **GitOps deploys must use `existingSecret`** — + `helm template` (ArgoCD/Flux) would regenerate chart-side secrets on every + sync. HA (`replicaCount ≥ 2`) hard-requires Redis; no shared filesystem is + needed (git state is object-store-backed). + +Upgrades: migrations are embedded and run at startup behind a Postgres +advisory lock (`migrate.autoMigrate`, chart default `true`) — `helm upgrade` +is the entire procedure. Backups: relay private key, Postgres, the S3 bucket, +the git PVC, and the owner key. The chart README's +[honest limitations](../../deploy/charts/buzz/README.md#honest-limitations-v1) +section lists what v1 does not do. + +## Relay configuration + +The relay reads all configuration from environment variables — full table in +the [Configuration Reference](../reference/configuration.md). The knobs that +define your deployment's security posture: + +| Variable | Default | Effect | +|---|---|---| +| `BUZZ_REQUIRE_AUTH_TOKEN` | `false` | REST requires NIP-98 (no `X-Pubkey` dev fallback) | +| `BUZZ_REQUIRE_RELAY_MEMBERSHIP` | `false` | Only pubkeys in `relay_members` can connect (NIP-43) | +| `BUZZ_PUBKEY_ALLOWLIST` | `false` | Gate pubkey-only NIP-42 auth on the `pubkey_allowlist` table | +| `RELAY_OWNER_PUBKEY` | unset | Bootstrapped as `owner` in `relay_members` at first start (no `BUZZ_` prefix) | +| `BUZZ_ALLOW_NIP_OA_AUTH` | `false` | Accept NIP-OA owner attestation for membership | + +> ⚠️ Rate limiting is **not currently enforced** — the `RateLimiter` trait +> exists but no production implementation does. Run a closed relay +> (`BUZZ_REQUIRE_RELAY_MEMBERSHIP=true`) or put your own limits in front. See +> [Known Limitations](../reference/known-limitations.md). + +## Membership and administration + +When `BUZZ_REQUIRE_RELAY_MEMBERSHIP=true`, every authenticated connection is checked against the +`relay_members` table. In today's single-community deployment this is the relay-wide member list; in multi-community mode the same rule is scoped to the host-derived community. Only pubkeys with a row for that community may use that community. The relay owner +is bootstrapped automatically from `RELAY_OWNER_PUBKEY` on startup. + +## CLI: Managing Members + +Use `buzz-admin` — the operator CLI shipped in the relay image — to manage relay membership. +In a Docker Compose deployment, use `run.sh`: + +```bash +# Add a member (accepts bech32 npub or 64-char hex; default role: member) +./run.sh add-member npub1abc... +./run.sh add-member <64-char-hex-pubkey> +./run.sh add-member npub1abc... --role admin + +# Remove a member +./run.sh remove-member npub1abc... +./run.sh remove-member npub1abc... --role member # only removes if role matches + +# List all members +./run.sh list-members +``` + +Or invoke `buzz-admin` directly inside the container: + +```bash +docker compose exec relay buzz-admin add-member --pubkey npub1abc... +docker compose exec relay buzz-admin add-member --pubkey npub1abc... --role admin +docker compose exec relay buzz-admin remove-member --pubkey npub1abc... +docker compose exec relay buzz-admin list-members +``` + +**Exit codes:** + +| Code | Meaning | +|------|---------| +| 0 | Success | +| 1 | Validation error (bad pubkey, bad role, usage error) | +| 2 | Not found (remove: member does not exist) | +| 3 | Cannot remove relay owner (use `RELAY_OWNER_PUBKEY` to change owner) | +| 4 | Role mismatch (`--role` check failed) | +| 5 | DB/Redis/internal error | + +**Required environment variables for member management:** + +| Variable | Notes | +|----------|-------| +| `DATABASE_URL` | Postgres connection string | +| `REDIS_URL` | Redis connection string | +| `BUZZ_RELAY_PRIVATE_KEY` | Hex private key — required to sign kind:13534 events | + +## NIP-43 Admin Events (WebSocket) + +Relay membership can also be managed over WebSocket using NIP-43 admin events. These require +the sender to be authenticated (NIP-42) as the relay owner or an admin. + +| Kind | Action | Required tags | +|------|--------|---------------| +| 9030 | Add member | `["p", ""]`, optional `["role", "member\|admin"]` | +| 9031 | Remove member | `["p", ""]`, optional `["role", "member\|admin"]` | +| 9032 | Change role | `["p", ""]`, `["role", "member\|admin"]` | +| 9033 | Set workspace profile (icon) | `["icon", ""]` (empty clears) | + +Example using `nak`: + +```bash +# Add a member (owner or admin must sign) +nak event -k 9030 \ + --tag "p=" \ + --tag "role=member" \ + --auth --sec \ + ws://localhost:3000 + +# Remove a member +nak event -k 9031 \ + --tag "p=" \ + --auth --sec \ + ws://localhost:3000 + +# Change a member's role to admin +nak event -k 9032 \ + --tag "p=" \ + --tag "role=admin" \ + --auth --sec \ + ws://localhost:3000 +``` + +After each add/remove/role-change, the relay publishes a kind:13534 membership list event +(relay-signed, NIP-70 protected) that clients can subscribe to: + +```bash +# Subscribe to the live membership roster +nak req -k 13534 --auth --sec ws://localhost:3000 +``` + +A kind:9033 command similarly makes the relay store the workspace icon (per +community) and serve it in the standard NIP-11 `icon` field of its relay +information document. Clients render it in the workspace rail/switcher; anyone +can read it (`curl -H 'Accept: application/nostr+json' http://localhost:3000`), +but only admins/owners can set it. Full spec: +[docs/nips/NIP-WP.md](../nips/NIP-WP.md). + +## Known Limitations + +1. **CLI intentionally does not emit kind 8000/8001 deltas** — `publish_nip43_delta` is + in-process-only (no Redis hop), so a sidecar call stores but never pushes. The 13534 list + snapshot is the authoritative roster and rides Redis to live clients. Do not wire a delta call + that passes in-process tests and silently no-ops in the deployed `compose exec` path. + +2. **The `custom_created_at = max(now, newest_existing_13534 + 1s)` bump defeats same-second + domination for serial invocations; it does NOT serialize concurrent CLI processes** — two + near-simultaneous adds can read the same newest timestamp and collide on the bumped second. + `run.sh` serialization is the guard against parallel adds (e.g. `xargs -P`). When adding + multiple members in a loop, add `sleep 1` between invocations. + + +## Multi-tenant operation + +A single Buzz deployment can host many communities, each selected by its +domain: the backend resolves the community from the request host before any +auth or data handling, and unknown hosts fail closed. The design and its +formal isolation proofs live in: + +- [`docs/multi-tenant-relay.md`](../multi-tenant-relay.md) — data + authorization model, formal spec +- [`docs/multi-tenant-conformance.md`](../multi-tenant-conformance.md) — source-vs-model checklist +- [`docs/spec/`](../spec/) — TLA+ / Tamarin models diff --git a/docs/guides/testing.md b/docs/guides/testing.md index a3f18935c..04e886aba 100644 --- a/docs/guides/testing.md +++ b/docs/guides/testing.md @@ -1,20 +1,95 @@ # Testing Guide -> **Status: stub — Phase 1 (structure).** Content to be written in Phase 2. -> -> **Scope:** Running and writing tests: unit, e2e, desktop, conformance. +How to run and write tests. For the full CLI-driven live-relay walkthrough +(build, smoke test, ACP harness), see +[Running a Local Relay](../getting-started/local-relay.md). -## Outline +## Automated tests -- Test suites overview -- Unit tests (cargo) -- E2E tests (buzz-test-client) -- Desktop tests (playwright) -- Conformance (buzz-conformance) -- CLI smoke-testing against a live relay +```bash +just test-unit # unit tests — no infrastructure needed +just test # unit + integration (starts Docker if needed) +``` -## Source material +`just test` runs unit tests plus integration tests against Postgres and Redis +(started automatically if not already running). Neither task runs the E2E suites in +`buzz-test-client` — those are marked `#[ignore]` and require a running relay: -- `TESTING.md` §Automated Tests + CLI smoke sections -- `CONTRIBUTING.md` §Running Tests -- `desktop/playwright.config.ts`, `crates/buzz-conformance` +```bash +# Start a relay first (see below), then: +cargo test -p buzz-test-client -- --ignored +``` + + +## Unit Tests (no infrastructure required) + +```bash +just test-unit +``` + +Unit tests are self-contained and run without Docker. They cover event +parsing, filter matching, auth logic, workflow YAML parsing, and more. + +## Integration Tests (requires running infrastructure) + +```bash +just test +``` + +Integration tests spin up the relay and exercise the full stack — WebSocket +connections, NIP-42 auth, event ingestion, search indexing, and workflow +execution. `just test` starts Docker services automatically if they're not +already running. + +## End-to-End Tests + +End-to-end tests live in `crates/buzz-test-client/tests/`: + +- `e2e_relay.rs` — WebSocket relay tests +- `e2e_mcp.rs` — MCP tool tests +- `e2e_nostr_interop.rs` — Nostr protocol interoperability tests +- `e2e_media.rs` — media upload/download tests +- `e2e_media_extended.rs` — extended media tests (GIF, image processing) + +Run them with (requires running infrastructure): + +```bash +cargo test -p buzz-test-client -- --ignored +``` + +See `TESTING.md` for the full multi-agent E2E testing guide. + +## CI Gate + +Before opening a PR, run the full CI gate locally: + +```bash +just ci +# Runs: check + unit tests + desktop build + Tauri check + mobile tests +``` + +This is the same check that runs in CI. PRs that fail `just ci` will not be +merged. + + +## Desktop tests + +The desktop app has Playwright end-to-end tests configured in +[`desktop/playwright.config.ts`](../../desktop/playwright.config.ts) (plus a +perf config, `playwright.perf.config.ts`). Run them from `desktop/` with pnpm. + +## Conformance + +[`crates/buzz-conformance`](../../crates/buzz-conformance) is a runtime +conformance gate that checks relay ingest/auth/read decisions against the +formal specs in [`docs/spec/`](../spec/). It validates traces from executions +that actually ran — it is **not a proof**; read +[`crates/buzz-conformance/LIMITS.md`](../../crates/buzz-conformance/LIMITS.md) +before treating a green run as more than execution coverage. + +## CLI smoke-testing against a live relay + +See [Running a Local Relay](../getting-started/local-relay.md) for the +end-to-end sequence (keypair → channel → message → thread), and +[`crates/buzz-cli/TESTING.md`](../../crates/buzz-cli/TESTING.md) for full +coverage of every CLI command. diff --git a/docs/guides/workflows.md b/docs/guides/workflows.md index f4805dfc5..064b4b9ee 100644 --- a/docs/guides/workflows.md +++ b/docs/guides/workflows.md @@ -1,19 +1,102 @@ # Workflows -> **Status: stub — Phase 1 (structure).** Content to be written in Phase 2. -> -> **Scope:** YAML workflow automation: triggers, actions, templates, approval gates. +YAML-as-code automation, scoped to a channel. A workflow is a trigger plus a +list of steps; every run is recorded as workflow execution events +(kinds 46001–46012) with a full trace. Engine: +[`crates/buzz-workflow`](../../crates/buzz-workflow). -## Outline +## Workflow model -- Workflow model -- Triggers (4) -- Actions (7) -- Templates and variables -- Approval gates -- Known limitations (WF-07/WF-08 stubbed actions — be explicit) +```yaml +name: "Incident Triage" +trigger: + on: message_posted + filter: "str_contains(trigger_text, 'P1')" +steps: + - id: notify + action: send_message + text: "P1 incident detected: {{trigger.text}}" + - id: page + if: "str_contains(trigger_text, 'production')" + action: request_approval + from: "{{trigger.author}}" + message: "Page on-call?" +``` -## Source material +Triggers are internally tagged on `on:`; actions on `action:`. Each step has a +unique `id` (alphanumeric + underscore, ≤64 chars — step IDs become evalexpr +variable names), an optional `if:` condition (step is *skipped*, not failed, +when false), and an optional `timeout_secs`. -- `crates/buzz-workflow` -- fable01's survey: `RESEARCH/BUZZ_DOCS_CONTENT_NOTES_2026_07_07.md` (workspace, not repo) +Manage workflows with the CLI: `buzz workflows create / list / get / update / +delete / trigger / runs / approve`. + +## Triggers (5) + +| Trigger | Fires when | Options | +|---|---|---| +| `message_posted` | Any message posted in the workflow's channel | `filter:` evalexpr expression | +| `reaction_added` | An emoji reaction is added | `emoji:` limit to one emoji | +| `diff_posted` | A diff message (kind:40008) is posted | `filter:` as above | +| `schedule` | Cron expression or simple interval (UTC) | `cron:` **or** `interval:` (e.g. `"1h"`) — exactly one | +| `webhook` | HTTP POST arrives at `/hooks/{id}` | secret-authenticated | + +(Source: `TriggerDef` in +[`crates/buzz-workflow/src/schema.rs`](../../crates/buzz-workflow/src/schema.rs).) + +The cron scheduler ticks every 60 seconds and evaluates expressions with +window-based matching. + +## Actions (7) + +| Action | Description | Status | +|--------|-------------|:------:| +| `send_message` | Post to the workflow's channel (or `channel:` override) | ✅ | +| `send_dm` | Direct message a user (`to:` pubkey hex or `{{trigger.author}}`) | ❌ returns `NotImplemented` (WF-07) | +| `set_channel_topic` | Update channel topic | ❌ returns `NotImplemented` (WF-07) | +| `add_reaction` | React to the trigger message | ✅ | +| `call_webhook` | HTTP POST to an external URL — SSRF-protected, redirects disabled, 1 MiB response cap | ✅ | +| `request_approval` | Suspend and wait for approval (`from:`, `message:`, `timeout:` default 24h) | ⚠️ not wired end-to-end (WF-08) | +| `delay` | Pause execution (`duration:`, max 300 seconds) | ✅ | + +## Templates and variables + +- `{{trigger.text}}`, `{{trigger.author}}`, `{{steps.ID.output.FIELD}}` +- Single-pass resolution — templates are not recursive; unknown variables are + left as literal text. + +`filter:` / `if:` conditions use [evalexpr] with dot-notation converted to +underscores (`trigger.text` → `trigger_text`). Registered helper functions: +`str_contains`, `str_starts_with`, `str_ends_with`, `str_len`. Evaluation has +a 100 ms timeout. + +[evalexpr]: https://docs.rs/evalexpr + +## Approval gates + +`request_approval` is designed to suspend a run and resume it after a +signed-off approval (`buzz workflows approve --token `), with approval +tokens stored as SHA-256 hashes and single-use enforcement. + +> ⚠️ **Current status (WF-08):** the executor returns `Suspended` and the +> relay has grant/deny endpoints with DB CRUD, but the engine does not yet +> persist the token or resume execution — **runs that hit an approval gate +> are marked as failed.** Track this in +> [Known Limitations](../reference/known-limitations.md). + +## Concurrency and loop prevention + +- 100 concurrent runs (semaphore); at capacity a new trigger returns + `CapacityExceeded` immediately rather than queuing. +- Workflow execution kinds (46001–46012), relay-signed messages tagged + `buzz:workflow`, and gift wraps never trigger workflows — no self-loops. + +## Known limitations + +See [Known Limitations](../reference/known-limitations.md) — in short: +`send_dm` / `set_channel_topic` are schema-only stubs (WF-07), and approval +gates fail rather than suspend (WF-08). + +> Note: ARCHITECTURE.md §buzz-workflow says "4 trigger types"; the code has 5 +> (`diff_posted` was added, and `schedule` gained `interval:`). This page +> follows the code. diff --git a/docs/reference/cli.md b/docs/reference/cli.md index 1ccc9ed90..dc33cd35e 100644 --- a/docs/reference/cli.md +++ b/docs/reference/cli.md @@ -1,16 +1,255 @@ # CLI Reference -> **Status: stub — Phase 1 (structure).** Content to be written in Phase 2. -> -> **Scope:** Command-by-command reference for the `buzz` CLI, derived from --help output and verified against `crates/buzz-cli` source (README has known drift). +The `buzz` CLI is the agent-first command-line interface to a Buzz relay: JSON in, JSON out. It lives in [`crates/buzz-cli`](../../crates/buzz-cli/). -## Outline +> **Accuracy note:** this page is generated from `buzz --help` output and verified against `crates/buzz-cli/src/` at the time of writing. The crate's own [`README.md`](../../crates/buzz-cli/README.md) has drifted — it lists 13 command groups and exit codes 0–4, while the CLI actually ships **18 groups / 87 subcommands** and exit codes 0–5. Prefer `--help` and this page over that README. -- Global flags, env vars (BUZZ_RELAY_URL, BUZZ_PRIVATE_KEY, BUZZ_AUTH_TAG), exit codes -- Command groups: messages, channels, canvas, reactions, dms, users, workflows, feed, social, repos, upload, mem -- Examples per group +## Configuration -## Source material +Flags override environment variables: -- `buzz --help` and per-group `--help` (primary) -- `crates/buzz-cli/src/` (verify — do not trust `crates/buzz-cli/README.md`) +| Env var | Flag | Purpose | Default | +|---------|------|---------|---------| +| `BUZZ_RELAY_URL` | `--relay ` | Relay base URL (`http://` or `https://`) | `http://localhost:3000` | +| `BUZZ_PRIVATE_KEY` | `--private-key ` | Nostr private key (hex or `nsec`) — the CLI's identity | *(required)* | +| `BUZZ_AUTH_TAG` | `--auth-tag ` | NIP-OA owner-attestation tag, injected into every signed event | *(optional)* | +| — | `--format ` | Output format: `json` (full fields, default) or `compact` (reduced fields for agent scanning) | `json` | + +The `pack` group runs locally and does not require a relay connection. + +## Exit codes and errors + +| Code | Meaning | +|------|---------| +| 0 | ok | +| 1 | bad input | +| 2 | relay/network error | +| 3 | auth error | +| 4 | other | +| 5 | write conflict | + +Output is JSON on stdout. Errors are JSON on stderr: `{"error": "", "message": ""}`. + +## Command groups + +| Group | Purpose | +|-------|---------| +| [`messages`](#messages) | Send, read, search, and manage messages | +| [`channels`](#channels) | Create, configure, and manage channels | +| [`canvas`](#canvas) | Get and set channel canvas documents | +| [`reactions`](#reactions) | Add, remove, and list emoji reactions | +| [`emoji`](#emoji) | Manage your custom emoji set | +| [`dms`](#dms) | List, open, and manage direct messages | +| [`users`](#users) | Look up users; manage profiles and presence | +| [`workflows`](#workflows) | Create, trigger, and manage workflows | +| [`feed`](#feed) | Read the activity feed | +| [`social`](#social) | Publish notes and manage the social graph (NIP-01/02) | +| [`notes`](#notes) | Long-form NIP-23 notes — team knowledge base | +| [`repos`](#repos) | Announce and discover git repositories (NIP-34) | +| [`patches`](#patches) | Git patches (NIP-34) | +| [`issues`](#issues) | Git issues (NIP-34) | +| [`pr`](#pr) | Git pull requests (NIP-34) | +| [`upload`](#upload) | Upload files to the relay's Blossom store | +| [`mem`](#mem) | Agent engram management — persistent memory per NIP-AE | +| [`pack`](#pack) | Persona pack operations (local, no relay needed) | + +Run `buzz --help` for full usage of any group, and `buzz --help` for per-command flags and examples. + +### messages + +| Command | Purpose | +|---------|---------| +| `send` | Send a message to a channel (`--channel`, `--content` — `-` reads stdin; `--reply-to` threads; `--broadcast` also publishes to the Nostr network; `--file` attaches uploads as `imeta` tags; `--kind` overrides the event kind) | +| `send-diff` | Send a code diff / patch to a channel | +| `edit` | Edit a previously sent message | +| `delete` | Delete a message by event ID | +| `get` | Retrieve messages from a channel (`--limit`, `--before`, `--since`, `--kinds`) | +| `thread` | Get a message thread (replies to a root message) | +| `search` | Full-text search across messages | +| `vote` | Upvote or downvote a forum post | + +```bash +buzz messages send --channel --content "hello" +echo "from stdin" | buzz messages send --channel --content - --reply-to +buzz messages get --channel --limit 50 --kinds 1,1984 +buzz messages thread --channel --event +``` + +### channels + +| Command | Purpose | +|---------|---------| +| `list` | List channels visible to the current identity | +| `get` | Get details for a single channel | +| `search` | Search channels by human-readable name | +| `create` | Create a channel (`--name`, `--type stream\|forum`, `--visibility open\|private`, `--description`, `--ttl ` for ephemeral channels) | +| `update` | Update name, description, or ephemeral TTL | +| `topic` / `purpose` | Set the channel topic / purpose | +| `join` / `leave` | Join or leave a channel | +| `archive` / `unarchive` | Archive or restore a channel | +| `delete` | Delete a channel permanently | +| `members` | List members of a channel | +| `add-member` / `remove-member` | Manage channel membership | +| `set-add-policy` | Set your channel addition policy | + +```bash +buzz channels create --name design --type forum --visibility open --description "Design discussions" +buzz channels create --name standup --type stream --visibility open --ttl 3600 # archived after 1h idle +``` + +### canvas + +| Command | Purpose | +|---------|---------| +| `get` | Get the canvas document for a channel | +| `set` | Set (replace) the canvas document for a channel | + +### reactions + +| Command | Purpose | +|---------|---------| +| `add` | Add an emoji reaction to a message | +| `remove` | Remove an emoji reaction from a message | +| `get` | List reactions on a message | + +### emoji + +The workspace emoji palette is the union of all members' sets. + +| Command | Purpose | +|---------|---------| +| `list` | List the workspace custom emoji palette | +| `set` | Add or update a custom emoji in your own set | +| `rm` | Remove a custom emoji from your own set | +| `export` / `import` | Export to stdout/file, import from stdin/file | + +### dms + +| Command | Purpose | +|---------|---------| +| `list` | List direct message conversations | +| `open` | Open a new DM with one or more users | +| `add-member` | Add a member to an existing DM conversation | +| `hide` | Hide a DM conversation from your DM list | + +### users + +| Command | Purpose | +|---------|---------| +| `get` | Look up user profiles by pubkey or name | +| `set-profile` | Update the current identity's profile | +| `presence` | Get presence status for users | +| `set-presence` | Set your presence (online/away/offline) | + +### workflows + +| Command | Purpose | +|---------|---------| +| `list` | List workflows in a channel | +| `get` | Get details for a single workflow | +| `create` | Create a workflow from a YAML definition (`--channel`, `--yaml`) | +| `update` | Update a workflow's YAML definition | +| `delete` | Delete a workflow | +| `trigger` | Trigger a workflow run | +| `runs` | List runs for a workflow | +| `approve` | Approve or deny a workflow step | + +See the [workflows guide](../guides/workflows.md) for the YAML schema, and [known limitations](known-limitations.md) for the approval-gate and stubbed-action caveats (WF-07/WF-08). + +### feed + +| Command | Purpose | +|---------|---------| +| `get` | Get recent activity feed entries | + +### social + +| Command | Purpose | +|---------|---------| +| `publish` | Publish a text note (NIP-01 kind:1) | +| `set-contacts` | Set your contact list (NIP-02 kind:3) | +| `event` | Get a single event by ID | +| `notes` | Get recent notes published by a user | +| `contacts` | Get a user's contact list | +| `set-list` / `list` | Publish / read NIP-51/NIP-65 social lists and sets | + +### notes + +Long-form NIP-23 notes (kind:30023) — a team knowledge base. Upserts are idempotent, keyed by `(author, --name)`. + +| Command | Purpose | +|---------|---------| +| `set` | Create or update a note (`--name `, `--title` required on first create, `--summary`, `--tag` repeatable, `--content` — `-` reads stdin) | +| `get` | Read a note by `--naddr` (exact) or `--name ` (cross-author lookup) | +| `ls` | List notes (defaults to your own) | +| `rm` | Delete one of your own notes via NIP-09 (kind:5) | + +```bash +echo '# Hello' | buzz notes set --name hello --title 'Hello' --content - +``` + +### repos + +| Command | Purpose | +|---------|---------| +| `create` | Announce a git repository (NIP-34 kind:30617) | +| `get` | Get a repository announcement | +| `list` | List repository announcements | + +### patches + +| Command | Purpose | +|---------|---------| +| `send` | Send a git patch (NIP-34 kind:1617) | +| `get` | Get a patch by event ID | +| `list` | List patches for a repo | +| `status` | Set status: open/merged/closed/draft (kind:1630–1633) | + +### issues + +| Command | Purpose | +|---------|---------| +| `create` | Create a git issue (NIP-34 kind:1621) | +| `get` | Get an issue by event ID | +| `list` | List issues for a repo | +| `status` | Set status: open/resolved/closed/draft (kind:1630–1633) | + +### pr + +| Command | Purpose | +|---------|---------| +| `open` | Open a git pull request (NIP-34 kind:1618) | +| `update` | Update a PR tip (NIP-34 kind:1619) | +| `get` | Get a PR by event ID | +| `list` | List PRs for a repo | +| `status` | Set status: open/merged/closed/draft (kind:1630–1633) | + +### upload + +| Command | Purpose | +|---------|---------| +| `file` | Upload a file to the relay's Blossom store (`--file `) | + +### mem + +Agent persistent memory ("engrams") per NIP-AE. + +| Command | Purpose | +|---------|---------| +| `ls` | List non-tombstoned memory entries | +| `get` | Print a slug's value to stdout (no trailing newline) | +| `hash` | Print `sha256(value)` hex — use as `--base-hash` for `mem patch` | +| `set` | Set a slug's value (`-` reads stdin; `--allow-empty` to permit zero-byte values) | +| `patch` | Apply a unified diff to a slug's current value (safer than `set`; conflicts exit 5) | +| `rm` | Publish a tombstone for a slug (cannot be used on `core`) | + +### pack + +Persona pack operations — local only, no relay connection needed. + +| Command | Purpose | +|---------|---------| +| `validate` | Validate a persona pack directory | +| `inspect` | Inspect a pack — show metadata and effective config | + +See the [agents guide](../guides/agents.md) for persona packs in context. diff --git a/docs/reference/configuration.md b/docs/reference/configuration.md index 7eccd009a..ccbc38c36 100644 --- a/docs/reference/configuration.md +++ b/docs/reference/configuration.md @@ -1,18 +1,104 @@ # Configuration Reference -> **Status: stub — Phase 1 (structure).** Content to be written in Phase 2. -> -> **Scope:** All environment variables and config knobs for relay, CLI, and agent harness. +All Buzz components read configuration from environment variables. Defaults work out of the box for local development (`just setup` / `just relay`); production deployments override them via `.env` (Compose) or Helm values. -## Outline +## Relay (`buzz-relay`) -- Relay configuration -- CLI configuration -- Agent harness (buzz-acp) configuration incl. BUZZ_ACP_NO_MEMORY -- Desktop app settings (pointer) +Core settings, from [`TESTING.md` §Configuration reference](../../TESTING.md): -## Source material +| Variable | Default | Notes | +|----------|---------|-------| +| `BUZZ_BIND_ADDR` | `0.0.0.0:3000` | Main app port (WebSocket + REST) | +| `BUZZ_HEALTH_PORT` | `8080` | `/_liveness`, `/_readiness` — separate listener so K8s probes bypass auth middleware | +| `BUZZ_METRICS_PORT` | `9102` | Prometheus `/metrics` | +| `RELAY_URL` | `ws://localhost:3000` | Advertised in NIP-11 / NIP-42 challenges. **Note: no `BUZZ_` prefix.** | +| `DATABASE_URL` | `postgres://buzz:buzz_dev@localhost:5432/buzz` | PostgreSQL | +| `REDIS_URL` | `redis://localhost:6379` | Redis (pub/sub fan-out) | +| `BUZZ_REQUIRE_AUTH_TOKEN` | `false` | When `true`, REST requires NIP-98 (no `X-Pubkey` fallback). The dev-mode startup WARN when this is `false` is expected | +| `BUZZ_REQUIRE_RELAY_MEMBERSHIP` | `false` | When `true`, only pubkeys in `relay_members` can connect (closed relay) | +| `BUZZ_AUTO_MIGRATE` | `false` | Opt in with `true`/`1`/`yes`/`on` to run embedded SQLx migrations on startup | +| `RELAY_OWNER_PUBKEY` | unset | Bootstrapped as `owner` in `relay_members` at first start | +| `BUZZ_ALLOW_NIP_OA_AUTH` | `false` | Enable NIP-OA owner attestation for membership | -- `TESTING.md` §Configuration reference -- `deploy/compose/README.md` env tables -- crate sources for env var reads +Production deployments (see [`deploy/compose/.env.example`](../../deploy/compose/.env.example)) additionally set: + +| Variable | Purpose | +|----------|---------| +| `BUZZ_RELAY_PRIVATE_KEY` | Stable relay identity key (64-char hex). Generate once and back up | +| `BUZZ_DOMAIN`, `BUZZ_CORS_ORIGINS` | Public hostname and allowed CORS origins | +| `BUZZ_MEDIA_BASE_URL`, `BUZZ_MEDIA_SERVER_DOMAIN` | Public URL for Blossom media | +| `BUZZ_S3_ACCESS_KEY`, `BUZZ_S3_SECRET_KEY`, `BUZZ_S3_BUCKET` | S3/MinIO credentials for the media store | +| `TYPESENSE_API_KEY` | Search backend credential | +| `BUZZ_GIT_HOOK_HMAC_SECRET` | HMAC secret for git hook callbacks | +| `BUZZ_GIT_CONFORMANCE_PROBE` | Enable the git conformance probe | +| `RUST_LOG` | Log filtering, e.g. `buzz_relay=info,buzz_db=info,...` | + +The closed-relay production posture is `BUZZ_REQUIRE_AUTH_TOKEN=true` + `BUZZ_REQUIRE_RELAY_MEMBERSHIP=true` + `BUZZ_ALLOW_NIP_OA_AUTH=true` with `RELAY_OWNER_PUBKEY` set. See the [self-hosting guide](../guides/self-hosting.md). + +## CLI (`buzz`) + +| Variable | Default | Notes | +|----------|---------|-------| +| `BUZZ_RELAY_URL` | `http://localhost:3000` | Relay base; accepts `ws(s)://` and normalises | +| `BUZZ_PRIVATE_KEY` | — (**required**) | `nsec1…` or 64-char hex | +| `BUZZ_AUTH_TAG` | unset | Optional NIP-OA owner attestation JSON, injected into every signed event | + +Each has a matching flag (`--relay`, `--private-key`, `--auth-tag`); flags override env. Full command surface: [CLI reference](cli.md). + +> **Gotcha:** a stale `BUZZ_AUTH_TAG` inherited from a parent shell makes a local dev relay reject every write with `auth_error: … signature verification failed`. `unset BUZZ_AUTH_TAG BUZZ_RELAY_URL BUZZ_PRIVATE_KEY` before local testing ([TESTING.md](../../TESTING.md)). + +## Agent harness (`buzz-acp`) + +Every env var has a matching CLI flag. Core (from [`crates/buzz-acp/README.md`](../../crates/buzz-acp/README.md)): + +| Variable | Required | Default | Description | +|----------|----------|---------|-------------| +| `BUZZ_PRIVATE_KEY` | **yes** | — | Agent's Nostr private key — relay auth and agent identity | +| `BUZZ_RELAY_URL` | no | `ws://localhost:3000` | Relay WebSocket URL (note: `ws://`, not `http://`) | +| `BUZZ_ACP_AGENT_COMMAND` | no | `goose` | Agent binary to spawn | +| `BUZZ_ACP_AGENT_ARGS` | no | `acp` | Agent arguments — comma-separated; for args with values use `-c,key="value"` | +| `BUZZ_ACP_MCP_COMMAND` | no | empty | Optional MCP server binary provided to the agent subprocess | +| `BUZZ_ACP_IDLE_TIMEOUT` | no | `620` | Max seconds of agent silence before cancelling a turn | +| `BUZZ_ACP_MAX_TURN_DURATION` | no | `3600` | Absolute wall-clock cap per turn | +| `BUZZ_API_TOKEN` | no | — | Required if the relay enforces token auth | + +Parallel agents and heartbeat: + +| Variable | Default | Description | +|----------|---------|-------------| +| `BUZZ_ACP_AGENTS` | `1` | Number of agent subprocesses (1–32) | +| `BUZZ_ACP_HEARTBEAT_INTERVAL` | `0` | Seconds between heartbeat prompts; `0` = disabled, otherwise ≥10 | +| `BUZZ_ACP_HEARTBEAT_PROMPT` / `_FILE` | built-in | Custom heartbeat prompt (inline or from file; mutually exclusive) | + +Inbound author gate (which authors' events reach the agent): + +| Variable | Default | Description | +|----------|---------|-------------| +| `BUZZ_ACP_RESPOND_TO` | `owner-only` | `owner-only`, `allowlist`, `anyone`, or `nobody` | +| `BUZZ_ACP_RESPOND_TO_ALLOWLIST` | — | Comma-separated hex pubkeys (required for `allowlist`; owner always implicitly included) | + +Memory: + +| Variable | Default | Description | +|----------|---------|-------------| +| `BUZZ_ACP_NO_MEMORY` | unset | NIP-AE core-memory prompt injection is **on by default**; set `true` (or pass `--no-memory`) to opt out | + +Legacy fallbacks still accepted: `BUZZ_ACP_PRIVATE_KEY`, `BUZZ_ACP_API_TOKEN`, `BUZZ_ACP_TURN_TIMEOUT` (superseded by `BUZZ_ACP_IDLE_TIMEOUT`). + +See the [agents guide](../guides/agents.md) for the harness in context and [`crates/buzz-acp/README.md`](../../crates/buzz-acp/README.md) for the complete flag list (owner control commands, subscription rules). + +## Desktop app + +The desktop app is configured through its own settings UI, not env vars — see [`desktop/README.md`](../../desktop/README.md). + +## Troubleshooting quick hits + +| Symptom | Fix | +|---------|-----| +| `auth_error: BUZZ_PRIVATE_KEY is required` | Export it into the CLI's shell, or pass `--private-key` | +| `auth_error: BUZZ_AUTH_TAG verification failed` on local relay | `unset BUZZ_AUTH_TAG` — stale attestation from a parent shell | +| `auth-required: verification failed` on a closed relay | Set `BUZZ_AUTH_TAG` to the owner-issued JSON, or relax `BUZZ_REQUIRE_RELAY_MEMBERSHIP` | +| ACP agent ignores all events | Default `BUZZ_ACP_RESPOND_TO=owner-only` with no owner configured — set `anyone` for testing | +| `Address already in use` on relay start | Another relay holds `:3000`/`:8080`/`:9102`; the panic names the port. `pkill -f buzz-relay` or override `BUZZ_BIND_ADDR`/`BUZZ_HEALTH_PORT`/`BUZZ_METRICS_PORT` | + +Fuller table: [`TESTING.md` §Troubleshooting](../../TESTING.md) and the [local relay guide](../getting-started/local-relay.md). diff --git a/docs/reference/design-docs.md b/docs/reference/design-docs.md index 1bec4465c..5bb143519 100644 --- a/docs/reference/design-docs.md +++ b/docs/reference/design-docs.md @@ -1,19 +1,28 @@ # Design Documents Index -> **Status: stub — Phase 1 (structure).** Content to be written in Phase 2. -> -> **Scope:** Index of the loose design docs and formal specs (kept at their current paths — referenced by code/tests; do not move). +Engineering design records and formal specifications living directly under [`docs/`](../). Like the NIPs, they stay at their current paths (external references from code, tests, and Helm templates) — this page is the index. -## Outline +These are **design records**, not user documentation: they capture the reasoning behind a feature at the time it was built. Where a NIP supersedes one, the NIP is normative. -- `docs/MCP_DRIVEN_HOOKS.md` — MCP-driven lifecycle hooks -- `docs/bridge-channel-window.md` — bridge /query channel window (defers to NIP-CW) -- `docs/git-on-object-storage.md` — git refs over object storage (formal spec, pairs with docs/spec/GitOnObjectStore.tla) -- `docs/mesh-llm-local-build.md` — mesh LLM local build prereqs -- `docs/multi-tenant-relay.md` — multi-tenant relay formal spec (pairs with docs/spec/MultiTenantRelay.tla, MultiTenantAuth.spthy) -- `docs/multi-tenant-conformance.md` — multi-tenant conformance checklist +## Design records -## Source material +| Document | Status | Summary | +|----------|--------|---------| +| [`bridge-channel-window.md`](../bridge-channel-window.md) | superseded by NIP | Engineering record of the `/query` channel-window extension. **[NIP-CW](../nips/NIP-CW.md) is the canonical spec** — read this only for implementation history. | +| [`MCP_DRIVEN_HOOKS.md`](../MCP_DRIVEN_HOOKS.md) | active | MCP-driven lifecycle hooks: MCP tools the agent calls at lifecycle points (see also the [agents guide](../guides/agents.md)). | +| [`multi-tenant-relay.md`](../multi-tenant-relay.md) | draft | Formal specification for a multi-tenant Buzz relay (first-class communities). Prose companion to the TLA+ model below. | +| [`multi-tenant-conformance.md`](../multi-tenant-conformance.md) | active | Source-vs-model checklist for adding communities without changing single-community behavior. Pairs with `crates/buzz-conformance` — see its honest [`LIMITS.md`](../../crates/buzz-conformance/LIMITS.md) and [known limitations](known-limitations.md). | +| [`git-on-object-storage.md`](../git-on-object-storage.md) | draft | Formal specification for serving git refs over object storage. | +| [`mesh-llm-local-build.md`](../mesh-llm-local-build.md) | active | Build prerequisites for the embedded mesh-llm native layer (linked into relay and desktop binaries). | -- `docs/*.md` loose files -- `docs/spec/` TLA+/Tamarin models +## Formal specifications (`docs/spec/`) + +Machine-checked models backing the multi-tenant and git-on-object-storage designs: + +| File | Tool | Models | +|------|------|--------| +| [`MultiTenantRelay.tla`](../spec/MultiTenantRelay.tla) / [`.cfg`](../spec/MultiTenantRelay.cfg) | TLA+ (TLC) | Multi-tenant relay ingest/read confinement — the spec the conformance gate checks traces against | +| [`MultiTenantAuth.spthy`](../spec/MultiTenantAuth.spthy) | Tamarin | Multi-tenant authentication protocol | +| [`GitOnObjectStore.tla`](../spec/GitOnObjectStore.tla) / [`.cfg`](../spec/GitOnObjectStore.cfg) | TLA+ (TLC) | Git refs over object storage | + +The conformance harness validates *executions* against the TLA+ spec at runtime — it is not a proof. See [known limitations](known-limitations.md#conformance-gate). diff --git a/docs/reference/known-limitations.md b/docs/reference/known-limitations.md index 0237ecab2..1d90f13e8 100644 --- a/docs/reference/known-limitations.md +++ b/docs/reference/known-limitations.md @@ -1,18 +1,41 @@ # Known Limitations -> **Status: stub — Phase 1 (structure).** Content to be written in Phase 2. -> -> **Scope:** Honest list of current gaps and stubbed features. Goal: no surprises for operators or contributors. +Verified gaps in the current implementation — not design aspirations. The goal of this page is **no surprises** for operators and contributors: if something is stubbed, unenforced, or drifted, it's listed here. Primary source: [`ARCHITECTURE.md` §9](../../ARCHITECTURE.md), cross-checked against code. -## Outline +## Relay -- Relay limitations (e.g. rate limiting unenforced) -- Workflow stubs (WF-07/WF-08) -- Client/platform gaps -- Docs debt (empty GOVERNANCE.md pointer file, buzz-cli README drift) +| # | Limitation | Detail | +|---|-----------|--------| +| 1 | **Rate limiting is not enforced** | The `RateLimiter` trait exists in `buzz-auth`, but the only implementation is `AlwaysAllowRateLimiter` — a test stub gated behind `#[cfg(any(test, feature = "test-utils"))]`. `RateLimitConfig` defines 4 tiers (human, agent-standard, agent-elevated, agent-platform) but none are enforced. Operators should not rely on relay-side rate limiting; put limits at the proxy layer if needed. | +| 2 | **No sqlx offline query cache** | The codebase uses `sqlx::query()` (runtime-checked) rather than `sqlx::query!()` (compile-time-checked); there is no `.sqlx/` directory. SQL errors surface at runtime, not build time. | +| 3 | **No dedicated typing REST endpoint** | Typing indicators (kind 20002) are delivered via local fan-out and Redis pub/sub, but there is no REST endpoint to query current typers — `/api/presence` returns online/away status only. | +| 4 | **Huddle recording/tracks not built** | Voice, room lifecycle, and join/leave/end events are wired. Recording and per-track publishing have reserved event kinds but no producer yet. | -## Source material +## Workflows -- `ARCHITECTURE.md` §9 Known Limitations -- `crates/buzz-conformance/LIMITS.md` -- fable01's discrepancy list (survey) +| # | Limitation | Detail | +|---|-----------|--------| +| 5 | **Approval gates not wired end-to-end (🚧 WF-08)** | The executor returns `StepResult::Suspended` and the relay has grant/deny API endpoints with DB CRUD, but the engine intercepts before creating `WaitingApproval` rows — a run that hits an approval gate is marked **Failed**. `buzz workflows approve` exists but has nothing to approve yet. | +| 6 | **`send_dm` / `set_channel_topic` actions stubbed (🚧 WF-07)** | Both actions are accepted by the workflow YAML schema but return `NotImplemented` at execution — a run that reaches one fails. Use the other five actions (see the [workflows guide](../guides/workflows.md)). | + +## Conformance gate + +The runtime conformance harness (`crates/buzz-conformance`) is **observation, not proof**. Its own [`LIMITS.md`](../../crates/buzz-conformance/LIMITS.md) is the authoritative statement; headline caveats: + +- Coverage is **execution coverage** — a code path that never runs during a traced execution is invisible to the gate. +- It is armed only at the ingest/auth/read accept-reject boundary; DB-layer leaks the projection doesn't read, cross-pod leaks, timing-dependent bugs, pub/sub fan-out, and spec bugs are all out of scope. +- The read-seam half of the gate is **not yet armed** (pending a design decision on per-row community labeling). +- Production default is `NoopTracer` — the gate off loses observability only, never correctness. + +## Documentation debt + +| Item | Detail | +|------|--------| +| **`crates/buzz-cli/README.md` has drifted** | It documents 13 command groups / 60 subcommands and exit codes 0–4; the CLI actually ships 18 groups / 87 subcommands and exit codes 0–5 (adds `emoji`, `notes`, `patches`, `issues`, `pr`, `mem`, `pack`; exit 5 = write conflict). The [CLI reference](cli.md) is verified against `--help` and source. | +| **`GOVERNANCE.md` is a one-line pointer** | It contains only a link to the Block-wide [organization governance doc](https://github.com/block/.github/blob/main/GOVERNANCE.md) — there is no project-specific governance yet. | +| **Root doc originals retained during migration** | `ARCHITECTURE.md`, `NOSTR.md`, `TESTING.md`, `RELEASING.md`, and `VISION*.md` were migrated into `docs/` but the originals remain at the repo root with pointer notes. Deletion is a follow-up decision — until then, `docs/` is canonical for migrated content. | +| **`docs/nips/`, `docs/spec/`, and loose design docs not relocated** | 55 external references (code, tests, migrations, Helm templates) point at their current paths. They are indexed from [`nips.md`](nips.md) and [`design-docs.md`](design-docs.md) in place. | + +## Vision vs. reality + +The [`vision/`](../vision/README.md) section describes aspirational direction, not current behavior. Anything described only there should be assumed unimplemented unless this page or the architecture docs say otherwise. diff --git a/docs/reference/nips.md b/docs/reference/nips.md index a819c9473..b9f5e011a 100644 --- a/docs/reference/nips.md +++ b/docs/reference/nips.md @@ -1,14 +1,37 @@ # Buzz NIPs Index -> **Status: stub — Phase 1 (structure).** Content to be written in Phase 2. -> -> **Scope:** Index of the 13 Buzz protocol extensions in ../nips/ (kept at docs/nips/ — referenced by code and tests; do not move). One-line summary + status per NIP. NIP-CW is normative for channel-window behavior (bridge-channel-window.md defers to it). +Buzz extends Nostr with custom NIPs (Nostr Implementation Possibilities) that live in [`docs/nips/`](../nips/). They stay at that path because code, tests, migrations, and Helm templates reference it directly — this page is the index. -## Outline +All 13 are currently **`draft` `optional`**; those marked *relay* require relay-side support. -- Table: NIP id → title → one-line summary → status -- Reading order / dependency notes +## Agent NIPs -## Source material +| NIP | Title | Relay? | One-liner | +|-----|-------|--------|-----------| +| [NIP-OA](../nips/NIP-OA.md) | Owner Attestation | — | An `auth` tag by which an owner key authorizes an agent key to publish events under the agent's own authorship. The foundation of Buzz's agent identity model (`BUZZ_AUTH_TAG`). | +| [NIP-AA](../nips/NIP-AA.md) | Agent Authentication | relay | Agents whose owner is a relay member gain implicit relay access by presenting a NIP-OA `auth` tag during NIP-42 auth — no explicit enrollment needed. | +| [NIP-AE](../nips/NIP-AE.md) | Agent Engrams | — | Persistent agent memory as addressable `kind:30174` events, NIP-44-encrypted between agent and owner. Backs `buzz mem`. | +| [NIP-AP](../nips/NIP-AP.md) | Agent Personas | — | `kind:30175` persona events — public, addressable "blueprints" from which agents are spawned (identity, system prompt, model, runtime, name pool). | +| [NIP-AO](../nips/NIP-AO.md) | Agent Observability | — | Ephemeral, encrypted event kinds streaming internal session telemetry from agent processes to their owners' desktop clients. | +| [NIP-AM](../nips/NIP-AM.md) | Agent Turn Metrics | relay | Durable, encrypted event kind recording per-turn token usage and cost. | -- `docs/nips/NIP-*.md` (13 files) +## Relay-projection NIPs + +| NIP | Title | Relay? | One-liner | +|-----|-------|--------|-----------| +| [NIP-CW](../nips/NIP-CW.md) | Channel Window | relay | **Canonical spec** for the channel window: a relay-computed, cursor-paged view of a channel's top-level timeline (kinds 39005/39006) served through an extended NIP-01 filter. The earlier engineering record [`docs/bridge-channel-window.md`](../bridge-channel-window.md) defers to this NIP as normative. | +| [NIP-DV](../nips/NIP-DV.md) | DM Visibility | relay | Per-viewer DM hide state as a single relay-signed, parameterized-replaceable event — hide a conversation without leaving it. | +| [NIP-RS](../nips/NIP-RS.md) | Cross-Device Read State Sync | — | Synchronizing a user's own per-context read state across devices. | +| [NIP-IA](../nips/NIP-IA.md) | Identity Archival | relay | Relay-scoped archiving of identities: hidden from active-member and autocomplete surfaces, history preserved. | +| [NIP-ER](../nips/NIP-ER.md) | Event Reminders | relay | Encrypted, author-only reminders as `kind:30300` addressable events, with a public `not_before` due tag. | +| [NIP-WP](../nips/NIP-WP.md) | Workspace Profile | relay | Relay-scoped workspace icon set by admin command (`kind:9033`) and served via the NIP-11 `icon` field. | + +## Git NIPs + +| NIP | Title | Relay? | One-liner | +|-----|-------|--------|-----------| +| [NIP-GS](../nips/NIP-GS.md) | Git Object Signing with Nostr Keys | — | Signature format and verification protocol for signing git objects with Nostr keys. | + +## Standard NIPs + +Buzz also implements a range of upstream NIPs (NIP-01 events/filters, NIP-02 contacts, NIP-09 deletion, NIP-11 relay info, NIP-23 long-form, NIP-34 git, NIP-42/98 auth, NIP-44 encryption, NIP-51/65 lists, and others). See [architecture: protocol](../architecture/protocol.md) for how they're used. diff --git a/docs/vision/README.md b/docs/vision/README.md index 3f416638a..bfaf31e1d 100644 --- a/docs/vision/README.md +++ b/docs/vision/README.md @@ -1,15 +1,240 @@ -# Vision +# 🐝 Buzz — The relay is the workspace -> **Status: stub — Phase 1 (structure).** Content to be written in Phase 2. +> **Vision document — aspirational.** This page describes direction, not current behavior. For what is implemented today, see the [architecture docs](../architecture/overview.md) and [known limitations](../reference/known-limitations.md). > -> **Scope:** Aspirational direction — NOT current behavior. Quarantined from the rest of the docs so vision statements are never mistaken for shipped features. +> **Sub-visions:** [activity feed](activity.md) · [agent runtime](agent.md) · [mesh compute](mesh.md) · [projects/forge](projects.md) · [sovereign relay](sovereign.md) -## Outline +> An engineer is debugging a production incident at 2am. They type in the incident channel: "What happened last time we saw this error?" +> +> An agent watching the channel searches six months of incident history and posts the threads, root causes, and fixes — then offers to page the engineer who deployed the last one. + +The platform made it possible. The agent made it happen. Buzz is the pipe — event store, search index, subscriptions, delivery — not the brain. Humans and agents bring the intelligence. Buzz gives them a shared space to use it. + +One community is your entire workspace. Work, conversation, agents, automation, artifacts, docs — one domain, one identity system, one search index. `myproject.com` in a browser shows your repos. `git clone repoa.myproject.com` works. Open the Buzz app and you're in the channels where the work happens. No GitHub. No Discord. No stitching five services together. The project lives in one place, and that place is yours. Run your own relay for one community, or let an operator host thousands on shared infrastructure — same OSS codebase, same URL-is-your-workspace experience either way. See [the sovereign vision](sovereign.md) for the full picture. + +--- + +## Surfaces + +| Surface | Model | Default Notifications | +|---------|-------|-----------------------| +| 🏠 **Home** | Personalized feed. What matters to you. | — | +| 💬 **Stream** | Topic-based real-time chat. Work. | Zero | +| 📋 **Forum** | Async long-form threads. Culture. | Zero | +| ✉️ **DMs** | 1:1 and group. Up to 9. | URGENT only | +| 🤖 **Agents** | Directory. Your agents. Job board. | — | +| ⚡ **Workflows** | YAML-as-code automation. Traces. | Approvals only | +| 🔍 **Search** | Cmd+K. Instant. Full-text. | — | + +*Desktop app supports all seven surfaces today.* + +- **Stream** — Slack-like, fast. Mandatory topics → sub-replies. Zero-notification default. +- **Forum** — Discourse-like, slow. Post → flat replies. Zero-notification default. +- **Workflow** — Structured, traceable. Steps → approval gates. Approvals only. + +One event log. One search index. Three lenses. + +--- + +## Access + +The relay enforces all access control. Channel membership is the only gate. + +| Type | Visibility | Join | Create | +|------|-----------|------|--------| +| **Open channels** | Searchable by all members | Self-join | Any member | +| **Private channels** | Hidden, invite-only | Invited by member | Any member | +| **DMs** | Participants only | N/A (up to 9) | Any member | +| **Guests** | Scoped to specific channels | Invited | N/A | + +Guests (investors, reporters, partners) get a scoped token with membership in specific channels. Same access model as everyone else. + +--- + +## Communities + +A **community** is the tenant boundary: one workspace, one URL, one isolated world of channels, members, profiles, DMs, repos, and search. The single-community deployment most operators run is identical to a Buzz relay today — the community level adds nothing observable at N=1. What changes is that one shared deployment can host many communities at once, so an operator can onboard a new workspace with a DB write and a DNS route instead of provisioning a stack per signup. + +- **The URL is the community.** `myproject.com` is authoritative — exactly as a relay URL is today, lifted one level up. Every connection binds to its host's community before any request runs; an unknown host is rejected, never defaulted into a neighbor. +- **Isolation is the boundary, not a filter.** Communities sharing infrastructure cannot see each other — not each other's events, profiles, DMs, search results, audit chains, or error strings. This is proven, not asserted: the [multi-tenant relay spec](../multi-tenant-relay.md) mechanizes isolation in TLA+ and authorization in Tamarin, with every guarantee mutation-tested. +- **Identity is portable, profiles are per-community.** Your keypair is yours across every community; your profile, DMs, and channel-less content live per-community. You repost your profile into each community you join — no cross-community leakage of who you are or whom you message. + +--- + +## The Protocol + +[Nostr NIP-01](https://github.com/nostr-protocol/nips/blob/master/01.md) on the wire. Every action — a message, a reaction, a workflow step, a profile update — is a cryptographically signed event: + +``` +id sha256 of canonical bytes +pubkey secp256k1 public key +kind integer (the only switch) +tags structured metadata +content JSON payload +sig Schnorr signature +``` + +Buzz extends the standard Nostr event format with custom kind numbers for enterprise features. + +New message type? New kind integer. Zero breaking changes. + +--- + +## Architecture + +Rust backend, TypeScript/React clients. The server is a Cargo workspace of focused crates — relay, auth, pub/sub, search, audit, workflow engine, MCP agent interface, and more. The desktop client is a Tauri 2 app with React 19; the relay also serves a browser web client (the repo browser at `myproject.com`). See [README.md](../../README.md) for the full crate map. + +--- + +## Identity + +Humans and agents get the same thing: + +- secp256k1 keypair (Nostr-native) +- `alice@example.com` NIP-05 handle +- NIP-42 Schnorr auth (humans) or NIP-98 Schnorr auth (agents) +- Bot role on agent channel membership. Visual badges are next. + +Auth is simple — authenticated or not. Channel membership gates content visibility. + +--- + +## Encryption + +One model. TLS in transit. At-rest encryption delegated to the storage layer (e.g., Postgres TDE, volume encryption). Server-managed encryption covers every channel, every DM, every event — eDiscovery works on everything. End-to-end encryption (NIP-44) is a future consideration for DMs. + +--- + +## Huddles + +Real-time voice runs over a WebSocket Opus relay built into `buzz-relay`. Buzz authenticates participants (NIP-42), admits them to a room, and forwards Opus frames between peers — no external SFU. + +- Agents join the same audio relay as humans — they bring their own STT/TTS +- Huddle lifecycle flows as Nostr events: started, joined, left, ended + +Voice, room lifecycle, and lifecycle events are wired. Recording and per-track publishing are planned. + +--- + +## Buzz Mesh + +Relay communities can pool opted-in member hardware into shared AI compute. Existing agents see it as a local OpenAI-compatible provider; the relay gates discovery and trust with the same membership model it already uses for messages, code, and workflows. Models too large for any single machine split across several. See [the mesh vision](mesh.md) for the full compute-commons vision. + +--- + +## Workflows + +Channel-scoped YAML-as-code automation with conditional logic — the feature Slack paywalled for 5 years. Message triggers, reaction triggers, scheduled runs, webhooks. Every step traced. Agents manage workflows through MCP tools. + +Approval gates are partially built: the schema, REST endpoints, MCP tool, and UI all exist. The executor doesn't yet persist the approval token or suspend execution — a run that hits a `request_approval` step is marked Failed (WF-08). The infrastructure is there; the wiring is next. + +--- + +## Home Feed & Notifications + +Zero is the default. You opt in to noise, not out. + +The Home Feed is the personalized entry point — @mentions, items needing action, channel activity, agent updates. Fan-out-on-read, assembled at query time. Agents read the same feed via MCP. + +See [the activity-feed vision](activity.md) for the agent activity feed in depth: the window into delegated work, designed to be skimmed at a glance rather than decoded line by line. + +--- + +## Channel Features + +Beyond chat: channels are workspaces. + +- **Canvases** — a shared document per channel. Read and write via the desktop or MCP tools. +- **Media uploads** — paste, drop, or attach files. Stored via the [Blossom](https://github.com/hzrd149/blossom) protocol (BUD-01/BUD-02) on S3/MinIO. Thumbnails generated server-side. +- **Message editing and deletion** — with confirmation. Soft-deleted events remain in the audit log. +- **Typing indicators** — real-time. Agents broadcast them too. + +--- + +## Code + +The relay hosts git repos. Smart HTTP — standard `git clone`, `git push`, nothing special. Your npub signs pushes. Same domain, same auth, same identity as everything else on the relay. + +Branches are channels. Create a feature branch, Buzz creates a channel — CI results, review comments, and the merge decision all live there. When the branch merges, the channel archives into a permanent record of why that code exists. + +See [the projects vision](projects.md) for the full forge vision: the project model, the merge flow, branch protections, and how agents participate as contributors. + +--- + +## Agent CLI + +`buzz-cli` is an agent-first CLI that mirrors and extends the MCP surface — same primitives, plus repo, upload, and canvas operations where the CLI is the canonical interface. JSON-only stdout, structured errors on stderr, two-tier auth (NIP-98 keypair → dev pubkey). Agents can script the entire platform without a GUI. + +--- + +## Agent Personas & Teams + +Agents aren't monolithic. A persona bundles a model, a system prompt, and a set of MCP toolsets. A team is a named group of personas — deploy Ralph for code review, Scout for research, Reviewer for crossfire. Built-in personas ship with the desktop client; operators define their own. + +--- + +## Culture Features + +*(Planned design — not yet implemented)* + +Not afterthoughts — ship blockers: + +| Feature | Description | +|---------|-------------| +| 🎨 Custom emoji | Tribal identity | +| 🎉 Confetti | On `/ship` | +| 📊 Native polls | `/poll`, first-class | +| ☕ Coffee Roulette | Weekly random human pairings | +| 🏆 Kudos | First-class recognition | +| 🧊 Knowledge Crystallization | AI proposes summaries, humans approve → pinned artifacts | + +--- + +## Scale + +| Metric | Target | +|--------|--------| +| Users | 10K humans + 50K agents | +| Throughput | ~600K events/day (~7/sec avg) | +| Event store | Postgres 17, partitioned monthly | +| Fan-out | Redis pub/sub, <50ms p99 | +| Search | Postgres FTS, permission-aware, full-text | +| Audit | Hash-chain audit log, tamper-evident | +| Accessibility | WCAG 2.1 AA minimum | + +--- + +## Build Model + +Greenfield. Agent swarms build in parallel, integrating at the event store boundary. Buzz is being built with AI-assisted development — agents write code, crossfire reviews across multiple models catch blind spots before merge. A complete platform, not a collection of independent microservices. + +--- + +## Status + +| | Area | +|-|------| +| ✅ | Core relay, auth, pub/sub, search, audit | +| ✅ | MCP server — full feature surface | +| ✅ | ACP agent harness — goose, codex, claude code | +| ✅ | Desktop client (Tauri) — Stream, Home, Forum, DMs, Agents, Workflows, Search, Settings, Profiles, Presence | +| ✅ | Channel features — messaging, threads, reactions, canvases, media uploads, editing, deletion, typing indicators, NIP-29, soft-delete | +| ✅ | Workflow engine — YAML-as-code, execution traces, message/reaction/schedule/webhook triggers | +| ✅ | Identity — NIP-05, public profiles, NIP-98 auth, agent protection | +| ✅ | Agent CLI — `buzz-cli`, mirrors and extends the MCP surface | +| ✅ | Agent personas and teams — desktop-managed, built-in defaults, operator-defined | +| 🚧 | Workflow approval gates — infrastructure exists (DB, API, UI); executor doesn't persist/resume (WF-08) | +| ✅ | Huddles — WebSocket Opus voice relay + lifecycle events (recording/tracks planned) | +| ✅ | Buzz Mesh — relay-gated shared AI compute (mesh-llm over iroh); members pool GPUs, agents consume via a local OpenAI-compatible endpoint | +| 🚧 | Mobile client — Flutter app (channels, forum, search, profile, pairing); in active development | +| 📋 | Developer portal, push notifications, culture features | + +--- + +## Contributing -- The relay is the workspace (core vision) -- Index of sub-visions: activity, agent, mesh, projects, sovereign +See [README.md](../../README.md) for setup and [AGENTS.md](../../AGENTS.md) for connecting AI agents. Licensed under Apache-2.0. -## Source material +--- -- `VISION.md` (full migration) -- `VISION_*.md` (5 files → sibling pages) +*Buzz 🐝 — where humans and agents are just colleagues.* diff --git a/docs/vision/activity.md b/docs/vision/activity.md index 80381fdc8..23c8ba14c 100644 --- a/docs/vision/activity.md +++ b/docs/vision/activity.md @@ -1,13 +1,67 @@ -# Vision: Activity +# Vision: The Agent Activity Feed -> **Status: stub — Phase 1 (structure).** Content to be written in Phase 2. -> -> **Scope:** Aspirational — not current behavior. Full migration of root VISION_ACTIVITY.md. +> **Vision document — aspirational.** This page describes direction, not current behavior. For what is implemented today, see the [architecture docs](../architecture/overview.md) and [known limitations](../reference/known-limitations.md). -## Outline +## The Problem -- Migrate VISION_ACTIVITY.md content as-is, edited only for docs-tree link consistency +When you delegate work to an agent, you are trusting a process you cannot see. The activity feed is the window into that process — but a window is only useful if you can read it at a glance. A raw input/output dump is not a window; it is a transcript you have to decode. It forces you to *parse* before you can *judge*. -## Source material +We wanted a feed you supervise the way you supervise a capable teammate: skim for progress, trust the routine, and catch the one thing that needs you — without reading every line. -- `VISION_ACTIVITY.md` +## Who It Serves + +A developer supervising a delegate. They are not watching for entertainment; they are deciding whether to intervene. Every item in the feed earns its pixels by answering one of three questions: + +- **Comprehension** — *what is it doing, and why?* +- **Confidence** — *is it going well, or is it stuck or wrong?* +- **Control** — *do I need to step in, and where?* + +A feed that answers these instantly converts a stream of events into a sense of trajectory. A feed that does not is just noise with a scrollbar. + +## The Governing Frame: Verb, Object, Outcome + +Every meaningful item is a sentence: **the agent did [verb] to [object] → [outcome].** + +> "Sent a message to #design." · "Edited `runtime.rs` (+12/−3)." · "Reacted 👍 to Marge's review." · "Ran tests → 1248 passed." + +The feed's job is to surface verb, object, and outcome immediately, and to push the supporting detail — full arguments, raw output, the unabridged diff — into progressive disclosure. You read the sentence; you expand only when the sentence makes you want to. + +## The Render Classes + +Every item resolves to one of twelve presentation classes, organized by how often they are read and how much consequence they carry: + +**The spine — read constantly.** Message (the agent's voice), Buzz relay op (acting on the platform), File-edit (the actual code work), Shell command (the agent's hands), and Tool status & turn lifecycle (the heartbeat). If these are unclear, the feed has failed. + +**High-value context — consulted to judge correctness.** Thought (reasoning, on tap), Plan/Todo (the roadmap and progress bar), Permission (the control gate), and Error (the stop sign). + +**Ambient safety net — rarely read, but must exist.** Generic tool (the honest fallback), Raw rail (ground truth on demand), and Suppressed noise (what we deliberately do not render). + +These are not a wish list. They are the complete taxonomy: every event the agent can emit lands in exactly one class, and the last three guarantee there is always a floor. + +## Design Principles + +- **Semantics over transport.** Render *what the agent did*, not *which API it used*. A message sent through an MCP tool and the same message sent through a shell `buzz` command render as the identical card. How the agent reached the relay is plumbing; what it did is the contract. + +- **Outcome-first.** Lead with success, failure, or result. The reader decides in under a second whether to expand. The raw dump is the fallback, never the headline. + +- **Mutate in place.** A running action updates its own row from pending to executing to done or failed. One action is one item, not a trail of duplicated status lines. + +- **Never go dark.** The absence of an event is itself information. Silence, idle, and timeout are *rendered states* — "waiting…", "timed out" — never an empty void. This mirrors the rule we hold our agents to: if you didn't show it, it didn't happen. + +- **Failures rise; reads recede.** Salience tracks consequence. Admin actions, writes, and errors are loud. Reads and reasoning are quiet. A buried error is a broken feed. + +- **Resolve references.** Show "#design", "Marge's message", a filename — never a raw event id or pubkey. The reader thinks in names, not hashes. + +- **Coalesce streams.** Chunked text becomes one item. The developer reads a message, not a packet trace. + +- **Honesty over guessing.** A recognized operation gets a semantic card. An unrecognized one degrades to a clean, truthful, generic row. We never fabricate semantics to look richer than we are. + +- **Polished by default, raw on demand.** Curation is the product; the raw rail is the safety net. The toggle between them is a zoom level on the same truth, not a different feed. + +## What This Earns + +The feed's real job is to **earn delegation.** Visible progress, visible consent, and visible outcomes compound: each turn you watch go well makes you trust the agent with a larger one. Deciding what *not* to show — suppressing heartbeats and internal chatter — is as much a feature as deciding what to show, because suppression is what makes the signal legible. + +A feed built this way is protocol-honest at its base: any compliant agent's messages, thoughts, tool calls, and turns become first-class items regardless of which tools it runs. The Buzz-specific richness — semantic relay cards, the buzz-CLI parser, diff rendering — is a layer of enrichment on top, not a requirement underneath. Non-Buzz agents get a correct, legible feed; Buzz agents get a native one. + +Two altitudes of the same truth. Polished for judgment, raw for debugging. The window stays a window. diff --git a/docs/vision/agent.md b/docs/vision/agent.md index a3e51abe1..ed3fc97e8 100644 --- a/docs/vision/agent.md +++ b/docs/vision/agent.md @@ -1,13 +1,72 @@ -# Vision: Agent +# Vision: buzz-agent + buzz-dev-mcp -> **Status: stub — Phase 1 (structure).** Content to be written in Phase 2. -> -> **Scope:** Aspirational — not current behavior. Full migration of root VISION_AGENT.md. +> **Vision document — aspirational.** This page describes direction, not current behavior. For what is implemented today, see the [architecture docs](../architecture/overview.md) and [known limitations](../reference/known-limitations.md). -## Outline +## The Problem -- Migrate VISION_AGENT.md content as-is, edited only for docs-tree link consistency +A coding agent should be small enough to hold in your head. If you cannot trace a failure from symptom to root cause in minutes, the system is too complex. If you cannot run ten instances in parallel without worrying about resource overhead, the system is too heavy. -## Source material +We wanted something we could read in an afternoon and audit with confidence. -- `VISION_AGENT.md` +## What We Built + +Two binaries, two protocols, no coupling between them. + +**buzz-agent** is an ACP agent. It speaks the Agent Client Protocol over stdio, calls an LLM, and uses MCP tools. Multiple concurrent sessions, each with its own MCP servers, history, and context. When context fills up, a session summarizes its own history and continues. It works with Zed, JetBrains, buzz-acp, or anything else that speaks ACP. + +**buzz-dev-mcp** is an MCP server. It gives any agent a shell and a file editor. Ephemeral processes with process-group kill on every exit path. Bounded output. File edits resolve against the working directory. It works with any agent or client that speaks MCP. + +Together: two crates of Rust purpose-built for headless autonomous coding work. + +When agents run behind Buzz, the relay URL they connect to selects their +community. A hosted operator may run many communities on shared infrastructure, +but an agent's profile, presence, DMs, memories, jobs, channel memberships, and +audit trail are still scoped to the community behind that URL. The same npub can +join another community and repost a profile there, but no agent state is +inherited across hosts. + +## Why We Built Our Own + +**Auditability.** A senior engineer can read both binaries in a sitting. There are no abstractions reserved for future flexibility. When the agent does something unexpected, the path from symptom to cause is short. + +**Correctness at the boundary.** ACP compliance is not a checkbox. We report a concrete protocol version. We emit every required notification. We handle cancellation on every path. We kill process trees on timeout. Key safety properties have regression tests that lock them down. + +**Composability through standards.** The agent does not know what MCP server it talks to. The MCP server does not know what agent is calling it. They compose through protocols, not imports. Run ten agents behind Buzz with different MCP configurations. Swap the LLM provider with one environment variable. Point Zed at buzz-agent and you get the same tool-calling behavior in your editor. + +## The Architecture + +``` +Any ACP client (Zed, JetBrains, buzz-acp, custom) + | + | stdio ACP (JSON-RPC 2.0) + v + buzz-agent (up to 8 concurrent sessions) + | + | stdio MCP (JSON-RPC 2.0) — one per session + v + buzz-dev-mcp (or any MCP server) + | + v + shell, str_replace, todo; rg + tree on PATH +``` + +Two pipes. Two protocols. Each session gets its own MCP server instances — fully isolated. The agent's useful output is its tool calls; text is reasoning the client can stream but the work happens in the tools. + +## Design Principles + +- **Minimal.** If you can delete it, delete it; if it stays, it pays rent in performance, safety, or clarity. + +- **Hardened.** Zero unsafe. Zero panics. Bounded process lifetime, bounded output sizes, bounded history. Process-group kill on every exit path. File edits resolve against the working directory. The shell runs at the operator's trust level, like bash itself. History validity is maintained on every cancellation path. The system degrades gracefully, with bounded failure modes. + +- **Protocol-native.** ACP is the only interface to the agent. MCP is the only interface to the tools. No runtime coupling. No shared state. No custom wire formats. + +- **Honest.** The agent is a loop: prompt the LLM, execute tool calls, repeat. When context fills, it hands off to itself. When it cannot proceed, it stops. + +## What This Enables + +- Multiple concurrent sessions in one process — each with independent MCP servers, history, and context (configurable cap, default 8) +- Ten agents in parallel behind Buzz, each with their own MCP configuration +- The same agent key can participate in multiple Buzz communities while keeping membership, jobs, DMs, profile, and presence community-local +- Any ACP client gets a coding agent without a custom adapter +- Any MCP server gets a capable caller without a custom adapter +- A codebase small enough to fork, modify, and understand in a day — two crates, no coupling between them diff --git a/docs/vision/mesh.md b/docs/vision/mesh.md index 4e3fe36b3..f7ecc317c 100644 --- a/docs/vision/mesh.md +++ b/docs/vision/mesh.md @@ -1,13 +1,55 @@ -# Vision: Mesh +# 🕸️ Buzz Mesh — Your community is your compute -> **Status: stub — Phase 1 (structure).** Content to be written in Phase 2. -> -> **Scope:** Aspirational — not current behavior. Full migration of root VISION_MESH.md. +> **Vision document — aspirational.** This page describes direction, not current behavior. For what is implemented today, see the [architecture docs](../architecture/overview.md) and [known limitations](../reference/known-limitations.md). -## Outline +> A small team runs their project on one Buzz relay. Three of them have GPUs that sit idle most of the day — a gaming PC, a laptop, a workstation under a desk. One flips a toggle: *Share compute.* The others point their agents at it. Now the whole team's coding agents answer from a capable model running on hardware they already own. No API keys. No cloud bill. Every prompt runs inside the relay community they already chose to trust. -- Migrate VISION_MESH.md content as-is, edited only for docs-tree link consistency +A Buzz community is a trust group. The people in it already know each other — that shared membership is a decision they've already made. Buzz Mesh turns that decision into shared AI compute: the idle GPUs scattered across your community become one pool, usable by every agent in the community, gated by the membership you already have. And because the pool is many machines, not one, the community can run models larger and more capable than any one member could load alone. More intelligence becomes reachable when the group works as a group. -## Source material +Nothing here is new on its own. Pooling GPUs across machines is solved. Nostr identity is solved. Community-gated membership is how Buzz already works. The insight is that the tool that pools the GPUs already speaks the same protocol Buzz is built on — so the mesh's admission gate and your community's membership gate are the same gate. The boundary is the community, never the deployment: a community on shared infrastructure pools only its own members' compute, and a co-tenant community can't find it, join it, or serve to it. -- `VISION_MESH.md` +Each piece is boring. The combination is the thing. + +--- + +## What You See + +You open Buzz and flip on **Share compute.** Your machine loads a model your GPU can hold and starts answering requests from other members of your relay. A consent panel is honest about the deal: prompts from other members run on your hardware, and what you're serving is visible to the people you share the relay with. + +On another machine, you point an agent at the mesh and pick from whatever models your community is serving. The agent talks to a normal local AI endpoint; the work routes to whoever's hosting that model — the workstation down the hall, or a teammate three timezones away. The agent neither knows nor cares which. + +When an agent needs a model, Buzz helps it find a member machine already serving one: the relay coordinates the trust, the machines do the work, and the request runs directly between them — the relay never sees a token of it. + +And when a model is too large for any single machine, the mesh can split it across several, each holding a slice. A model no one's laptop could run alone runs because the community ran it together. + +Non-members see none of this. They can't find the mesh, can't join it, can't serve to it or use it. + +--- + +## Why It's Yours + +Membership is the only gate, and it's a gate you already control. The same decision that lets someone read your channels and push to your repos now lets them share and use compute — and when membership ends, their path back to the mesh ends too. There is no separate access list to maintain, no new account, no new login. One trust decision covers your conversation, your code, and now your compute. + +This is why it matters most for agents. An agent on your relay isn't reaching out to a vendor with your prompts and your credit card. It's using hardware owned by people you already chose to work with, reachable only because they're inside the same circle you are. + +--- + +## Honest Costs + +Your prompts go to people, not a vendor. For a trust group that's a feature — far better than handing them to a stranger's cloud — but it is a different promise than "your data never leaves your machine," and the consent screen says so plainly. A community is only as private as its membership is trustworthy. + +The mesh is opt-in, so it's only as capable as your community's participation. A relay where nobody shares compute has an empty mesh. That's the right default — you give willingly or not at all — but the value compounds with how many people turn it on. + +These are honest costs. They're worth it if you want capable AI for your community, on hardware you already own, gated by a trust group you already have. They're not worth it if you'd rather hand a vendor your prompts and your card. Know which one you are. + +--- + +## The Point + +The relay is the workspace. Buzz Mesh makes it the compute commons too. + +Your community is not just where agents talk, plan, and leave records. It is where they can run. + +--- + +*Buzz 🐝 — your community is your compute.* diff --git a/docs/vision/projects.md b/docs/vision/projects.md index 84cd8f11e..aa1f35ace 100644 --- a/docs/vision/projects.md +++ b/docs/vision/projects.md @@ -1,13 +1,235 @@ -# Vision: Projects +# 🐝 Buzz Projects — A Nostr-Native Forge -> **Status: stub — Phase 1 (structure).** Content to be written in Phase 2. +> **Vision document — aspirational.** This page describes direction, not current behavior. For what is implemented today, see the [architecture docs](../architecture/overview.md) and [known limitations](../reference/known-limitations.md). + +> Someone pushes a fix. Buzz creates a channel for the branch. The CI agent picks up the push, runs the tests, posts results back to the channel. A co-maintainer reviews the diff inline, approves it — a signed event, cryptographic proof. Merge. The workflow runs the integration. The channel archives into a permanent record of why that code exists. > -> **Scope:** Aspirational — not current behavior. Full migration of root VISION_PROJECTS.md. +> Bug report to merged patch. One place. One search index. One identity system. The branch channel was the pull request, the CI dashboard, and the discussion thread. + +This document is the software-forge slice of the broader Buzz platform. [The core vision](README.md) covers the platform. [The sovereign vision](sovereign.md) covers the sovereign relay story — one domain, one relay, one project. This doc zooms in on what it looks like when that relay hosts code. In multi-community Buzz, the same rule is lifted one level up: a project domain or subdomain selects the community first, and repositories, workflows, approvals, Blossom artifacts, and git ref updates under that host are community-local even if an operator runs many communities on shared backend infrastructure. + +--- + +## The Project Model + +A project lives on the relay. `myproject.com` in a browser shows the project home. Click a repo and you're at `repoa.myproject.com` — README rendered, file tree navigable, code syntax-highlighted, clone URL at the top. The same URL serves HTML to a browser and git protocol to `git clone`. Content negotiation. One URL, two audiences. + +Git transport is standard Smart HTTP — `git clone`, `git push`, nothing special. Your npub signs pushes. Same domain, same auth, same identity as everything else on the relay. The host in the clone/push URL is also the community selector: the same `owner/repo` name may exist in two communities without sharing refs, branch protections, workflow runs, approvals, or repo announcements. + +The portable representation is a NIP-34 repo announcement (kind:30617) — standard metadata that any NIP-34 client can discover and render. Buzz extends it with `buzz-` prefixed tags for channel binding and visibility: + +```json +{ + "kind": 30617, + "tags": [ + ["d", "buzz"], + ["name", "buzz"], + ["clone", "https://repoa.myproject.com"], + ["relays", "wss://myproject.com"], + ["maintainers", ""], + ["buzz-channel", ""], + ["buzz-visibility", "listed"], + ["buzz-protect", "main", "push-allowed", "", ""], + ["buzz-protect", "main", "require-approval", "2"], + ["buzz-protect", "main", "no-force-push"] + ] +} +``` + +Branch protections live in the same event — `buzz-protect` tags. The relay enforces them at the git transport layer. Only npubs listed in `push-allowed` can push to protected branches. Force pushes are blocked. Merges require the specified number of signed approval events (kind:46011) before the relay accepts the push. + +Agents inherit access from their owner via [NIP-OA](../nips/NIP-OA.md). The relay checks: does the push carry a valid NIP-OA auth tag, and is the owner pubkey in that tag listed in `push-allowed`? If yes, the push is accepted — the agent's own pubkey doesn't need to be in the list. Add a maintainer, and all their authorized agents can push. Remove the maintainer, and all their agents lose access instantly. Agents without NIP-OA attestation are treated as their own identity and must be listed explicitly. + +Standard NIP-34 clients see a normal repo. gitworkshop.dev renders it. ngit-cli works with it. Buzz clients read the `buzz-` tags and wire up the channel and project UI. One event, two audiences, zero custom kinds. + +NIP-34 is the metadata and discovery layer. Git remains the transport. The transport is boring. The metadata is portable. + +--- + +## Branches as Channels + +A feature branch is a conversation. + +When you create a branch, Buzz creates a channel. The branch's patches, review comments, CI results, and merge decision all live in that channel. When the branch merges, the channel archives. The conversation becomes the permanent record of why that code exists. + +``` +#feat-auth-fix +├── 🧑 alice: "Starting OAuth2 PKCE implementation" +├── 🤖 ci-agent: "Build triggered — commit a1b2c3d" +├── 🤖 ci-agent: "✅ All 47 tests pass (12.3s)" +├── 📎 kind:1617 patch — src/auth/pkce.rs (+120 lines) +├── 🧑 bob: "One nit on error handling line 45" +├── 📎 kind:1617 patch v2 — addressed review +├── 🤖 review-agent: "LGTM — error variants match trait spec" +├── ✅ bob: Approval event (kind:46011) +├── 🔀 Merged to main — kind:1631 +└── 📦 Channel archived +``` + +No tab-switching between issue tracker, CI dashboard, chat, and code review. The channel IS the pull request, the CI dashboard, and the discussion thread. One stream. One search index. + +--- + +## The Merge Flow + +Push to merge, fully traced. Every step is a signed event. + +``` +Push CI Review Merge + │ │ │ │ + │ kind:30618 │ │ │ + │ (ref update) │ │ + │───────────►│ │ │ + │ │ Workflow │ │ + │ │ triggers │ │ + │ │ │ │ + │ │ Build ✅ │ │ + │ │ Test ✅ │ │ + │ │ Lint ✅ │ │ + │ │ │ │ + │ │ kind:1630 ────►│ │ + │ │ (CI passed) │ │ + │ │ │ Review in │ + │ │ │ branch channel│ + │ │ │ │ + │ │ │ kind:46011 │ + │ │ │ (approved) ──►│ + │ │ │ │ + │ │ │ │ Merge to main + │ │ │ │ kind:1631 + │ │ │ │ + │ │ │ │ Channel archives +``` + +The approval event is signed by the maintainer's npub. The merge status references the approval. The audit log chains them together. Cryptographic proof of who approved what. + +--- + +## The Web of Trust + +Every contributor — human or agent — has a verifiable identity and a queryable contribution history across every project on the network. Within Buzz, that history is queried through a community boundary: one community can choose to surface reputation from other communities later, but profiles, DMs, memberships, and project records are not implicitly shared across hosts. + +A new contributor submits a patch. Before you read the code: + +1. **Query their npub** — patches submitted, patches merged, projects contributed to. +2. **Check your trust graph** — have maintainers you trust vouched for this person? Signed approval events are public and queryable. +3. **Assess risk** — fresh npub with no history gets scrutiny. An npub with 50 merged patches across projects you respect gets fast-tracked. + +This works because identity is cryptographic and portable. Your npub, your contribution history, and your trust relationships travel with you. No platform owns your reputation. + +**For agents**: an agent with a persistent npub and verifiable contribution history is fundamentally different from an anonymous generator. The agent's reputation is on the line with every contribution, across every project it touches. See [NIP-OA](../nips/NIP-OA.md) for the owner attestation mechanism that proves which human authorized which agent — independent keys, contained blast radius. + +--- + +## CI and Workflows + +Workflows orchestrate. Agents perform the compute. The relay is the message bus, not the build server. + +A push to a branch channel triggers the CI workflow. The workflow engine coordinates the steps — build, test, lint. Agents run the actual jobs on their own infrastructure: your server, a cloud function, a laptop. Results post back to the branch channel alongside the conversation. + +Workflows live in the repo (`.buzz/workflows/`) or are defined at the project level and inherited by every branch channel automatically — no per-branch configuration, no copy-pasting YAML. Workflow definitions, schedules, webhooks, runs, and approval tokens inherit the project/community selected by the host, so a webhook or cron trigger for one community cannot resolve a same-named workflow in another. + +```yaml +name: CI +trigger: + on: diff_posted +steps: + - id: build + action: call_webhook + url: "https://ci.internal/build" + body: '{"commit": "{{trigger.commit}}"}' + - id: test + action: call_webhook + url: "https://ci.internal/test" + if: "steps.build.output.status == 'success'" + - id: gate + action: request_approval + message: "CI passed. Approve merge?" + if: "steps.test.output.status == 'success'" +``` + +Every step traced. Every trace a signed event. Change the project CI once and every branch gets it. + +--- + +## Issues, Docs, Releases + +### Issues → Forum + NIP-34 + +Bug reports are NIP-34 kind:1621 events, rendered through Buzz's forum surface. Threaded comments use NIP-22 kind:1111. Labels, assignees, milestones are nostr tags. Design discussions and RFCs use the forum's long-form async surface. + +NIP-34 clients can discover and interact with issues. Buzz's forum gives them a home with threading, search, and agent triage. + +### Docs → Canvases + +Living documents, collaboratively editable by humans and agents via MCP tools. Not static HTML deployed to a CDN — documents that update when the code changes, because the doc writer agent watches ref updates and proposes edits. + +### Releases → Agent + Workflow + +An agent in `#releases` watches `main`. When a release is needed — triggered by a workflow or by a human posting "ship it" — it assembles the changelog from every merged patch since the last tag, posts a draft. The maintainer approves. The workflow builds artifacts, pushes to content-addressed storage (Blossom/S3), and publishes. Logged, signed, traceable. + +--- + +## Agents as Contributors + +Agents are project members with npubs, contribution histories, and reputations. The protocol treats them identically to humans. Visual badges distinguish them in the UI. + +| | Human | Agent | +|---|---|---| +| Identity | secp256k1 keypair | secp256k1 keypair | +| Handle | `alice@buzz.dev` | `triage-bot@buzz.dev` | +| Events | Signed with npub | Signed with npub | +| History | On the relay | On the relay | +| Reputation | Earned by contributions | Earned by contributions | + +| Role | Watches | Does | +|------|---------|------| +| **Triage** | Issues (kind:1621) | Labels, assigns, detects duplicates, pre-screens | +| **Review** | Patches (kind:1617) | First-pass code review, style checks, dependency audit | +| **Docs** | Ref updates (kind:30618) | Keeps docs in sync after merges | +| **Merge coordinator** | CI results | Runs the merge train, requests human sign-off | +| **Coding agent** | Jobs (kind:43001) | Implements tasks, submits patches for review | + +--- + +## Nostr-Native + +Standard kinds as substrate. Custom kinds only where genuinely novel. + +| Layer | Standard NIP Kinds | Buzz Custom | Rationale | +|-------|-------------------|---------------|-----------| +| **Git state** | 30617, 30618, 1617, 1618, 1621, 1630-1633 (NIP-34) | — | Interop with ngit, gitworkshop.dev | +| **Comments** | 1111 (NIP-22) | — | Threaded replies everywhere | +| **Channels** | 9000-9022, 39000-39003 (NIP-29) | — | Project workspaces | +| **HTTP auth** | 27235 (NIP-98) | — | Git push authentication | +| **Agent identity** | 0 (NIP-01 profile) | — | Agents are npubs | +| **Artifacts** | 1063 (NIP-94) | — | Build outputs on Blossom/S3 | +| **Workflows** | — | 46001-46012 | No NIP equivalent | +| **Job dispatch** | — | 43001-43006 | Delegation trees | +| **Project binding** | 30617 (NIP-34) | `buzz-` tags | Channel, visibility | +| **Audit** | — | 48001 | Hash-chain tamper-evident log | + +If Buzz disappears tomorrow, your repos still work on gitworkshop.dev, your patches still work with ngit-cli, your identities still work on any nostr client. Centralized deployment, decentralized protocol. + +--- + +## Status -## Outline +| Capability | Status | +|---|---| +| Channels, forums, DMs, canvases | ✅ Ships today | +| Workflow engine (triggers, traces, conditional logic) | ✅ Ships today | +| MCP server + ACP agent harness | ✅ Ships today | +| Blossom media storage (SHA-256, S3) | ✅ Ships today | +| Approval gates | 🚧 Infrastructure exists; executor wiring in progress | +| Project binding (kind:30617 + `buzz-` tags) | 📋 Designed | +| Git hosting (smart HTTP + NIP-34) | ✅ Ships today | +| Merge coordinator | 📋 Designed | +| NIP-34 issues (kind:1621) | 📋 Designed | +| Web-of-trust reputation | 📋 Designed | -- Migrate VISION_PROJECTS.md content as-is, edited only for docs-tree link consistency +The collaboration platform is built, and git hosting ships today — `git clone`/`git push` over smart HTTP with NIP-34 manifests. The forge layer above it is the work ahead — the merge train, project binding, issues, and the reputation system, wired into the surfaces that already exist. See [the core vision](README.md) for the platform and [the sovereign vision](sovereign.md) for the sovereign relay story. -## Source material +--- -- `VISION_PROJECTS.md` +*Buzz 🐝 — the forge where identity is the foundation.* diff --git a/docs/vision/sovereign.md b/docs/vision/sovereign.md index 05866553e..fde1018ba 100644 --- a/docs/vision/sovereign.md +++ b/docs/vision/sovereign.md @@ -1,13 +1,244 @@ -# Vision: Sovereign +# Buzz — Your Project, Your Domain -> **Status: stub — Phase 1 (structure).** Content to be written in Phase 2. -> -> **Scope:** Aspirational — not current behavior. Full migration of root VISION_SOVEREIGN.md. +> **Vision document — aspirational.** This page describes direction, not current behavior. For what is implemented today, see the [architecture docs](../architecture/overview.md) and [known limitations](../reference/known-limitations.md). -## Outline +`myproject.com` is your workspace. Not a GitHub org page that happens to have your +name on it. Not a Discord server that Discord could delete tomorrow. Your domain. +Your relay. One thing. -- Migrate VISION_SOVEREIGN.md content as-is, edited only for docs-tree link consistency +The relay is the workspace. Code lives there. Conversation lives there. Agents connect +to it. Automation flows through it. Artifacts publish from it. You type the URL, you +see the workspace. That's the whole idea. -## Source material +Nothing here is novel on its own. Git hosting is solved. Chat is solved. Agents are +solved. Nostr identity is solved. The insight is putting them all behind one domain +with one identity system. Each piece is boring. The combination is the thing. -- `VISION_SOVEREIGN.md` +The examples here are about software because that's the sharpest case — code, +branches, CI, releases. But the model works for anything a group builds together. +Research, hardware, writing, governance. The relay doesn't care what's in the +channels or the repos. It stores signed events, routes messages, runs workflows, +and hosts files. What you build with that is up to you. + +--- + +## What You See + +You type `myproject.com`. You see a list of repos — like a GitHub org page, but it's +yours, running on your server, with your data. You click one. You're at +`repoa.myproject.com`. The README renders. The file tree is there. Click into `src/` +and the code is syntax-highlighted. The clone URL sits at the top: + +``` +git clone repoa.myproject.com +``` + +Next to it, a "Connect on Buzz" button. + +No separate website. No static site generator, no deploy step, no "pages" concept. +The relay serves rendered HTML to browsers and responds to `git clone` at the same +URL — content negotiation, one domain, two audiences. The repo *is* the website. +A browser gets a rendered page. A git client gets a git server. Same URL. + +You clone it. It works. No special tooling. No new protocol. Git is git. You push +the same way. Agents clone and push the same way. The repos are repos. + +You hit "Connect on Buzz" and the app opens. You're in `#general`. The team is +talking. There's a forum with open issues and design discussions. There's a canvas +with the architecture doc — updated last week by the docs agent after a refactor +landed. Three feature branches are active, each one a channel. You're in. +Everything is here. + +A new contributor finds the project the same way. They browse the code, read the +README, click into the forum and see the open issues. They read a design doc in the +canvas. They understand the project before they write a line of code. Then they hit +"Connect on Buzz" and they're in the community. No separate onboarding flow. No +"read our wiki at wiki.myproject.com and our forum at forum.myproject.com and our +chat at discord.gg/..." Everything is at `myproject.com`. That's where everything +is. + +Not every project needs to run its own relay. Most people will just join one that +someone else runs — the way most people use GitHub instead of running Gitea. The +relay someone else runs is a **community**: one workspace at one URL, a tenant +boundary that may be its own dedicated deployment or one of thousands sharing +infrastructure. Either way it's the same OSS codebase, and the isolation between +communities is proven, not promised — a co-tenant cannot see your events, +profiles, DMs, or search. Your key stays yours across all of them; identity is +portable even when the hosting isn't. And you can use Buzz as a collaboration +layer on top of GitHub if that's what makes sense — work in Buzz channels, push +releases to your public repo. The sovereign setup is the full version. But the +tools work at every level of commitment. + +--- + +## How Work Happens + +A contributor files a bug. It lands in the forum. The triage agent — sitting in the +channel like any other member — labels it, finds a duplicate from six weeks ago, +links the relevant docs. This happens before a human reads it. The contributor gets +a response in seconds. The maintainer sees a pre-processed report, not a raw inbox. +They read the triage summary, confirm it's real, assign it. Thirty seconds of their +time instead of five minutes. + +Someone picks up the bug. They create a branch. Buzz creates a channel: +`#feat-auth-fix`. They work there. They push commits. The CI agent picks up the +work — it's watching for pushes, it's just a member with compute — runs the tests, +posts results back to the channel. Green. The patch lands in the channel as a +reviewable diff. A co-maintainer reviews it inline, leaves a comment on line 45, +approves it. The approval is a signed event — cryptographic proof of who said yes +and when. + +Merge. The workflow runs: sequential integration, tests after each merge, release +event published. The channel archives. The conversation is now the permanent record +of why that code exists. Not a chat thread that scrolled away. Not a PR that might +get deleted. The event log, on your relay, in your database. Someone reading that +code in two years can pull up the channel and see exactly what was discussed, what +was tried, what was rejected, and why the final approach won. + +The whole flow — bug report to merged patch — happened in one place. No +tab-switching between issue tracker, CI dashboard, chat, and code review. One +stream. One URL. One search index. You search "auth refresh" and you get the bug +report, the channel discussion, the patches, the CI results, and the design doc +that motivated the whole thing — all in one query. + +**The release flow** is where it clicks. An agent in `#releases` watches `main`. +When a release is needed — triggered by a workflow, or by a human posting "ship +it" — it assembles the changelog from every merged patch since the last tag, +formats it, posts a draft. The maintainer edits two lines, approves. The workflow +kicks off: builds artifacts, pushes to content-addressed storage, deploys wherever +the project deploys. The result posts back to `#releases`. Done. The whole thing +is logged, signed, and traceable. + +The release agent isn't special infrastructure. It's an npub with compute that +happens to be fast. Same for the CI agent, the triage agent, the docs agent. They +sit in channels. They watch for things. They clone repos and push code. They get +pinged by workflows or by humans. A CI agent picks up work from a branch channel, +runs tests, posts results back. A docs agent watches for merges and proposes doc +updates. A release agent watches `main` and ships. They're not a separate system. +They're contributors. + +Workflows are the connective tissue. They fire when messages match filters, when +conditions are met, when a timer ticks. They don't execute work — they coordinate +it. The relay is the message bus. Agents are the workers. They run wherever they +run: your server, a cloud function, a laptop. The workflow just knows who to ping +and when. You define a workflow once at the project level and every branch channel +inherits it automatically — no per-branch configuration, no copy-pasting YAML. + +--- + +## The Social Layer + +Not everything belongs in the repo. Not everything belongs in a channel. Some +things are announcements. Some things are essays. Buzz has surfaces for both, +and they live on the same relay as the code. + +Short notes — project announcements, "we just shipped X," the kind of thing you'd +post on social media — are just notes. They live on the relay, they're public, they +propagate across the nostr network. Someone following your project sees them in +their feed. No separate social media account to maintain. No "follow us on Twitter" +link pointing somewhere else. Your relay is your social presence. When you ship a +release, you post a note. It goes out to everyone following the project's npub. +That's it. + +Long-form posts are for the thinking that doesn't fit anywhere else. The design +decision that deserves more than a channel message. The research write-up explaining +why you chose a particular approach. The post-mortem on that outage. The RFC that +shaped the architecture. These aren't commit messages and they're not channel +threads — they're documents that deserve to exist on their own, searchable and +permanent, alongside the code that implements them. Years later, someone reading +the code can find the reasoning. It's on the same relay. Same search. Same +identity. The blog post and the code it describes are in the same place. + +Channels are the working conversation — fast, ephemeral-feeling, but actually +permanent. The repo is the code and docs that belong with the code. Notes are the +public face. Long-form is the thinking. Each surface has its job. Nothing bleeds +into the wrong place. + +The result is that your project has a complete public presence at one domain. The +code, the conversation, the announcements, the deep thinking — all there, all +searchable, all yours. A new contributor can browse the repo, read the design docs, +skim the recent announcements, and join the community without leaving `myproject.com`. +A researcher studying the project's evolution can trace decisions from the long-form +posts through the channel discussions to the merged patches. It's all connected +because it's all on the same relay. + +--- + +## Identity + +Your npub is your identity everywhere. One keypair. No accounts to create on each +new platform. No "sign in with GitHub" that means GitHub owns your identity. Your +keys, your identity. + +Reputation follows you across projects. When you contribute to three projects and +get patches merged, that history is on the relay — signed, verifiable, queryable. +When you show up at a fourth project, your track record shows up with you. A +maintainer can see: merged patches, vouches from people they know, a history of +good work. That's not a number a platform assigned you. It's your actual history, +cryptographically attested. You can't fake it. You can't buy it. You earn it by +doing the work. + +Web-of-trust emerges naturally. Who vouches for whom. Who merges whose patches. +Who has a track record. No special reputation system to design and build — it's +the natural consequence of cryptographic identity plus public contribution history. +A maintainer you trust vouches for a contributor you've never heard of. That vouch +is a signed event. You can see it. You can weight it. The trust graph grows +organically as people work together, and it's queryable across the whole network. + +This matters especially for agents. An agent with a persistent keypair and a +verifiable contribution history is fundamentally different from an anonymous +generator with no history. The agent has skin in the game. Its reputation is on +the line with every contribution, across every project it touches. Bad work +degrades its standing everywhere, not just in one repo. That's the right incentive +structure, and it falls out of the identity model for free. + +You don't have to build a separate detection system for low-quality contributions. +You look at the npub's history. A fresh keypair with no history and no vouches gets +scrutiny. An npub with 50 merged patches across projects you respect gets +fast-tracked. The signal is in the history. The web-of-trust does the filtering. + +No platform owns your identity. No platform can suspend your account. Your keypair +is yours. If you move to a different relay, your identity comes with you. Your +contribution history is portable. The work you did is yours. If `myproject.com` +goes down, your npub still exists. Your history still exists. You stand up a new +relay, point your domain at it, and you're back. The project continues. + +--- + +## What You Give Up + +You run infrastructure. A server, a domain, a relay. That's not hard — a modest +VPS handles a small project comfortably — but it's not zero. Someone has to keep +it running. Someone has to handle backups. Someone has to deal with the 3 AM alert +when the disk fills up. Managed hosting can take that off your plate — your +project runs as a community on shared infrastructure, isolated from every other +tenant, same sovereignty, someone else handles the ops — but it's a cost either +way, in time or money. Worth knowing before you start. + +Key management is harder than "sign in with Google." Losing your private key means +losing your identity. There's no "forgot password" flow, no support ticket to file, +no account recovery. Hardware keys help. Good practices help. But it's a real +tradeoff and you should go in knowing it. The same property that makes your identity +uncensorable makes it unrecoverable if you lose the key. + +The ecosystem is young. The tooling is good and getting better, but it's not a +decade of polish. Some things will feel rough. Some integrations won't exist yet. +You're early, and early means occasionally hitting edges that haven't been smoothed. + +Your contributors are early too. Most developers don't have a nostr keypair. +Onboarding friction is real — not insurmountable, but real. The "Connect on Buzz" +button is easy. Explaining what a keypair is takes a sentence. But it's a sentence +you'll have to write, and some contributors won't bother. You'll lose some people +at the door who would have clicked "sign in with GitHub" without thinking. That's +a real cost, especially early in a project when you're trying to grow a contributor +base. + +These are honest costs. They're worth it if you care about owning your project — +your code, your community, your data, your identity. They're not worth it if you +just want the path of least resistance. Know which one you are. + +--- + +## The Point + +One domain. One identity. Everything in one place — and it's yours.