Skip to content

v1.7.0

Choose a tag to compare

View all tags
@marmutapp marmutapp released this 27 May 12:34
· 16 commits to main since this release
v1.7.0
925cfa4

Headline: the Teams & Org Visibility feature (M0–M5) lands in full — an
optional, self-hosted observer-org server for org-wide cost/activity rollups
(SAML SSO, SCIM 2.0, Ed25519-signed content-free pushes) plus a default-off
OpenTelemetry exporter. It is purely additive: anyone who never enrols runs a
byte-identical solo-local tool (make test-invariant is the gate). This release
also carries a conversation-compression cache-accuracy fix and clarified
compression docs.

fix(compression): compress the cache_control marker block uniformly

cache_aware mode previously protected the tool_result block carrying the
SDK's rolling cache_control marker from compression. Because the SDK rolls
that marker forward every turn, the protected block went on the wire with two
different byte representations across consecutive turns — breaking Anthropic's
byte-hash prefix cache from that block onward, so cache_creation surged
(+65–245%) and ON-proxy cost ran +16–33% vs OFF. The fix compresses every
tool_result block uniformly (each compressor is a pure function of the block's
bytes, so a block serialises identically whether or not it currently carries the
marker), restoring cross-turn byte-stability and the prefix-cache hits. The
preserveMsg/preserveBlock plumbing and the dormant
BudgetOptions.PreserveBlockIdx are removed; findLastCacheBreakpoint stays
(still unit-tested). Regression-pinned by
TestPipelineCacheAware_MarkerRollPreservesPriorBytes and
TestPipelineCacheAware_CrossTurnInvariance.

docs(compression): mode × provider matrix + recipes

New docs/compression-modes.md (canonical reference), plus a condensed matrix +
per-provider recipes in the README and npm README, clarify what each
[compression.conversation].mode does on Anthropic (Claude Code) vs
OpenAI/Codex traffic: cache/cache_aware target Anthropic's content-hash
prefix cache (cache_control is an Anthropic Messages API concept), while the
proxy's OpenAI path is mode-agnostic (OpenAI prompt caching is automatic and
server-side). cache_aware is the shipped default and the recommended Claude
Code mode; token is the clearest choice for Codex. Also removes three
never-wired config knobs the npm README had documented (prefix_bytes,
min_bytes_to_compress, [compression.conversation.weights]).

feat(teams): M5 — hardening, release, documentation

Final milestone of the Teams & Org Visibility feature: takes it from "works on
the maintainer's machine" to "ships to customers". All additive; the solo-local
UX stays byte-identical and make test-invariant / go test -race ./... stay
green.

  • Linting tightened. .golangci.yml adds errorlint, gosec, gocritic,
    and gocyclo; gofumpt replaces gofmt as the canonical formatter (make fmt runs go tool gofumpt, pinned via the tool directive) and
    golangci-lint now runs in CI. Every finding the new linters surfaced is fixed
    or justified per-line — dead code deleted, errors.Is/%w wrapping, the
    excelize/Rollback errcheck gaps, slices.Concat, two internal renames
    (cost.Breakdown, antigravity.ConversationsDirs); 35 SQL findings carry
    per-line //nolint (SQL structure is built from constants, values bound via
    ?). Two documented, scoped deviations: gocyclo is enforced at
    complexity 30 (not the spec's 15 — the pre-Teams tree has ~250 functions over
    15) with the 20 monsters path-excluded and 15 as the post-Teams ratchet
    target; gosec G115 (noisy, memory-safe in Go) and gosec-on-tests are
    excluded with rationale.
  • Supply chain. scripts/release.sh gains a dry-run subcommand and
    CycloneDX SBOM generation (syft) for both binaries. The release workflow
    builds + pushes the observer-org Docker image to
    ghcr.io/marmutapp/observer-org, cosign-signs it keyless (OIDC) by
    digest, and generates SLSA Level 3 provenance for both binaries via the
    official slsa-github-generator. SBOMs + provenance + observer-org binaries
    attach to the release; the release notes carry the cosign verify command.
  • Kubernetes. New Helm chart charts/observer-org/ (singleton Deployment,
    PV for the DB, referenced Secret for the five server keys, Service, optional
    Ingress, hardened nonroot security context, configurable resources). A CI
    helm job lints + templates it and helm installs into a kind cluster.
  • Customer docs. docs/teams-getting-started.md,
    docs/teams-architecture.md, docs/teams-operations.md, and a README
    "Teams & Org Visibility" section.

After this milestone the Teams & Org Visibility feature (M0–M5) is complete.

feat(teams): M4 — OTel exporter with GenAI semconv v1.41.0

Fifth milestone of the Teams & Org Visibility feature: the "second rail". An
agent can emit one gen_ai.client span per api_turns row to any OTLP/HTTP
endpoint (Datadog, Grafana/Tempo, Honeycomb, Dynatrace, SigNoz, or a plain
OpenTelemetry Collector), so customers with an existing observability backend
ingest SuperBased Observer's data without running the org server. It requires
M0 only, never couples to the org server, and is OFF by default — a
solo-local install builds no OTLP client and makes zero exporter network calls.

  • internal/exporter/otel — Exporter{New,Start,Stop} tailing api_turns
    and emitting one span per turn to a batching OTLP/HTTP exporter. New never
    dials (the SDK connects lazily), so a bad/unreachable endpoint is a logged
    export failure, not a startup error. Span start = the turn timestamp,
    duration = total_response_ms; errored turns carry codes.Error.
  • Attribute mapping — GenAI v1.41.0 standard attrs (gen_ai.provider.name,
    operation.name, request/response.model, response.id,
    response.finish_reasons, usage.input/output_tokens, error.type) plus
    the SuperBased sbo.* namespace (project.root, session.id, cost.usd,
    cache.read_tokens, web_search.requests, tool.adapter,
    action.normalized_type, freshness.classification,
    redundancy.is_stale_reread). sbo.org.id is emitted only when enrolled;
    sbo.user.email only when enrolled and emit_user_email = true.
  • Row tail — store.SubscribeAPITurns drains api_turns in id-order pages
    and persists the published-id high-water mark in a schema_meta otel_cursor
    row, so a restart resumes exactly past the last delivered turn (at-least-once).
  • Opt-in prompt content — emit_prompt_content = true (default false)
    attaches a gen_ai.client.inference.operation.details event. The observer
    does not persist raw prompt/completion bodies (no-content-in-DB invariant), so
    the event carries the available content metadata (prompt/message hashes, error
    message); the data-volume implications are documented in doc.go.
  • Semconv transition — emits the v1.41.0 names; advertises
    OTEL_SEMCONV_STABILITY_OPT_IN=gen_ai_latest_experimental by default, env
    overridable. The contrib genai/semconv Go package does not exist at any
    version, so the gen_ai.* keys are defined as constants against the v1.41.0
    spec; resource service.name uses the released semconv v1.40.0 package.
  • Config + wiring — new [exporter.otel] section (all fields default to the
    safe option, OTEL_* env vars override at construction); wired into
    observer start as a P1-isolated goroutine that flushes on shutdown and never
    cancels the proxy/watcher/dashboard/push-loop.
  • Reference dashboards — docs/exporters/otel/ Grafana 11 + Datadog
    dashboards and Prometheus recording rules, with a README documenting the
    Collector spanmetrics pipeline they assume.
  • Tests — testing/synctest for the row-tail cadence + restart-from-cursor;
    tracetest in-memory unit tests for span emission, error status, and
    prompt-content gating; an OTLP/HTTP integration test against a fake collector
    asserting the on-the-wire protobuf attribute set field-by-field.

Deps (pinned): go.opentelemetry.io/otel + /sdk +
/exporters/otlp/otlptrace/otlptracehttp v1.43.0, semconv/v1.40.0.

feat(teams): M3 — org dashboard, drill-down audit, budgets

Fourth milestone of the Teams & Org Visibility feature: the data the server has
been ingesting becomes useful. An org admin logs in via SAML and sees
overview / team / project rollups, drills down into per-developer data (which
is audited), configures budgets with webhook alerts, and manages bearers and
team-lead roles. Still purely additive — the agent never imports a server
package and the tests/invariant/ gate stays green.

  • internal/orgserver/rollup — the six aggregation queries (overview,
    teams, team detail, developers, projects + detail, budgets, audit) over the
    pushed + identity tables, behind a read-through TTL cache (60 s overview /
    30 s team-project-developer / 5 s audit). Spend is a proxy-deduplicated UNION
    of api_turns.cost_usd and token_usage.estimated_cost_usd (the matching
    rows dropped), mirroring the local dashboard so org spend reconciles within
    the spec's ±2 % drift bound. Team spend is developer-attributed; the project
    list carries a cross-team overlap indicator. A golden test pins the JSON wire
    shapes.
  • /api/org/* endpoints — the generated dashboard ServerInterface
    (a compile-time routing-conformance gate) implemented by hand-written
    handlers, mounted behind a samlSecurity middleware. Role scoping is
    enforced per request in the handler     (defence in depth): an org admin (email
    in [dashboard] admin_emails) sees everything; a team lead
    (org_team_members.role = 'lead') sees only their teams — team detail /
    developers off-team is 403, an out-of-scope project id is 404.
  • Drill-down audit — GET …/developers writes a view_team_developers
    audit_log row (with the X-Forwarded-For source IP) before the data is
    fetched and refuses the disclosure if the audit write fails;
    POST /api/org/audit/log-drill-down records the explicit "show breakdown"
    click. GET /api/org/audit is scoped + paginated (admin sees all, lead sees
    their teams).
  • Budget engine — a budgets table (per-team/project monthly cap, JSON
    alert thresholds, last-fired high-water mark) with CRUD; an evaluator
    goroutine in the server polls every budget_poll_seconds (default 60 s),
    computes rolling-30-day spend per scope, and fires the OpenAPI BudgetAlert
    webhook once per newly-crossed threshold (async, exponential backoff to 1 h,
    then dropped). Wired into Server.Run, drained on shutdown.
  • Admin actions — list a developer's issued bearers and Revoke a jti
    (writes revoked_bearers, audited); promote/demote a team member's role
    (audited). Enrolment-at-time bearer jtis are now indexed (issued_bearers)
    so Revoke has a concrete target.
  • web2/ org dashboard SPA — a self-contained Vite + React 18 + Tailwind 3
    • react-router + recharts app (mirrors web/'s tooling), embedded into the
      observer-org binary and served at root behind the SAML session. Pages:
      Overview, Teams (+ detail), Projects (+ detail, overlap), Audit, Settings
      (budgets, enrolment-token mint, SCIM/SAML info). Every view opens on
      aggregates; the per-developer breakdown is behind a "Show developer
      breakdown" button that records the drill-down before fetching the data.
      Built + embedded via make web-build-org.
  • Migrations — server migration 002 adds budgets, audit_log, and
    issued_bearers.
  • Tests — rollup query + golden tests; dashboard handler tests (scoping,
    audit-before-data, budget CRUD, revoke, role); budget evaluator tests
    (crossing ladder, fire-once-per-crossing, live webhook); a full-stack
    org-dashboard E2E (SAML admin sees aggregates, lead gets 403 off-team) with a
    route-conformance phase.

feat(teams): M2 — agent enrolment, push loop, ingest

Third milestone of the Teams & Org Visibility feature: the agent↔server data
path. An enrolled agent ships content-free activity rollups on an interval; the
server verifies, deduplicates, and stores them; the local dashboard shows the
enrolment status and exactly what was last shared; the developer can unenrol at
will. Still purely additive — nothing runs and no keychain is touched unless
[org_client] enabled = true and the agent is enrolled (the
tests/invariant/ gate, including a new privacy test, stays green).

  • internal/orgclient — the push client: Enroll (generates an Ed25519
    keypair, exchanges the compound one-time token for a bearer, seeds the push
    cursor from the current high-water ids so only post-enrolment activity is
    ever shared), PushOnce (reads content-free rows above the cursor → gzip →
    per-push Ed25519 signature → POST; advances the cursor on 200), PushLoop
    (interval-paced, re-reads enrolment each cycle, 250 ms→30 s ±25 % jittered
    backoff on retryable failures, stops cleanly on an auth failure), plus
    Status / Unenroll / LastPayload. testing/synctest covers the timer.
  • Server ingest — POST /api/agent/push now enforces the per-push
    signature (X-SBO-Timestamp + X-SBO-Agent-Signature, verified over the
    exact gzip bytes against the enrol-bound key; missing/invalid/out-of-skew →
    401), then ingests via the new internal/orgserver/ingest package: one
    transaction, INSERT OR IGNORE keyed by the schema's composite keys, so
    re-pushes deduplicate server-side. Returns 200
    {accepted_rows, deduped_rows, next_cursor}; every row is tagged
    pushed_by_user_id = the authenticated pusher.
  • Compound enrolment token — EnrollResponse now echoes the resolved
    user_id so the agent records its own identity without decoding the bearer.
  • Agent CLI — observer enroll <org-url> <token>, observer unenroll,
    observer org status, observer org push-now.
  • Dashboard — additive, independently-degrading endpoints
    GET /api/enrolment/status, GET /api/enrolment/last-payload (the exact
    content-free JSON last shared, byte-for-byte), POST /api/enrolment/unenroll;
    a header "Enrolled in <Org>" badge (absent when not enrolled) and a
    Settings → Enrolment page (org/user/server, last push, "View raw push
    payload" drawer, "Unenrol").
  • Daemon — observer start runs the push loop as a goroutine when org mode
    is enabled; it never propagates an error, so a failing push cannot bring down
    the proxy, watcher, or dashboard (P1).
  • Privacy — the single SQL seam (store.SelectUnpushedSince) and the
    orgcontract row types carry no content columns; a new invariant test pushes
    a corpus whose every content column holds a distinctive secret and asserts
    none of those bytes ever cross the wire.

feat(teams): M1 — org server skeleton, SAML, SCIM, enrolment

Second milestone of the Teams & Org Visibility feature. Stands up
observer-org as a runnable server: an admin can run it, point a SAML IdP
at it, SCIM-provision users, log into a placeholder dashboard, mint an
enrolment token, and have an agent exchange it for a bearer. No data flows
yet — POST /api/agent/push authenticates and ACKs 202 but does not
ingest (M2). Still purely additive: the agent and the solo-local UX are
unchanged (the tests/invariant/ gate stays green).

  • cmd/observer-org — new binary with serve / migrate /
    new-enrolment-token / dump-config / doctor subcommands (cobra +
    slog, mirroring cmd/observer). doctor runs an 8-point health check.
    make build-orgserver → bin/observer-org.
  • internal/orgserver/db — server SQLite (WAL, pure-Go driver) +
    migration 001_initial.sql: identity tables (org singleton, members,
    teams, memberships, project→team), auth tables (argon2id-hashed
    enrolment tokens, jti revocation list), and content-free data tables
    (sessions/actions/api_turns/token_usage with pushed_at +
    pushed_by_user_id, empty until M2).
  • internal/orgserver/auth — Ed25519 bearer minting/validation (no
    JWT lib), HMAC-SHA256 session cookies (no session lib), SAML SP via
    crewjam/saml with a custom session provider, and per-prefix
    middlewares (RequireBearer / RequireSAMLSession / RequireSCIMToken).
  • internal/orgserver/scim — SCIM 2.0 Users + Groups via
    elimity-com/scim over SQLite; the 12 common operations have
    happy + sad path tests.
  • internal/orgserver/api — the generated ServerInterface
    (EnrollAgent runs token-burn → pubkey-bind → 90-day bearer;
    PushBatch → 202), the SAML-protected enrolment-token mint endpoint,
    and request-id / logging / rate-limit middleware.
  • OpenAPI — docs/openapi/orgserver.yaml extended with the SCIM,
    SAML, dashboard, and admin surfaces; codegen scoped to the agent tag
    so only enroll + push generate stubs.
  • Deploy — Dockerfile.observer-org (distroless static) +
    deploy/observer-org/docker-compose.yaml (keygen + dev SAML IdP + org
    server) + deploy/observer-org/README.md (run guide, nonroot ownership
    notes, rootless-Docker/WSL2 dev-box setup). New deps: crewjam/saml,
    elimity-com/scim, golang.org/x/crypto/argon2 (already pinned). The
    image runs as distroless nonroot (uid 65532): the Dockerfile seeds
    /var/lib/observer-org with nonroot ownership (before VOLUME) so the
    server can create server.db, and keygen.sh chowns the seeded secrets
    to 65532 so the read-only config mount is readable — verified by bringing
    the full compose stack up (metadata 200 / dashboard 302 / SCIM 401↔200).
  • E2E — tests/e2e/orgserver drives the full flow (SCIM-provision →
    real signed SAML login → mint → exchange → bearer-authorised push)
    through the assembled handler.

feat(teams): M0 — schema, identity, and wire-contract foundations

First milestone of the Teams & Org Visibility feature (spec:
docs/teams-features-spec-2026-05-25.md). Purely additive plumbing — no
behavioural change for any user who never enrols. A solo-local install's
dashboard responses are byte-identical, proven by a new invariant suite.

  • Migrations 028/029: six org-mode tables (org_enrolment,
    org_members, org_teams, org_team_members, org_project_team,
    org_push_log) and nullable org_id/user_email columns on
    actions/sessions/api_turns/token_usage with partial indexes
    (WHERE org_id IS NOT NULL) so solo-local installs pay no index cost.
  • internal/identity: a Stamper that attributes ingested rows to
    the enrolled org/user. No-op (and NULL columns) when not enrolled;
    wired into the watcher and proxy store via a chainable WithStamper.
  • internal/orgcontract: the single source of truth for the
    agent↔server wire types (push envelope, the four content-free pushed
    row types, enrolment + bearer claims), golden-tested.
  • OpenAPI 3.1 spec (docs/openapi/orgserver.yaml) for the agent
    protocol, with oapi-codegen client/server stubs reusing the
    orgcontract types via x-go-type. New make targets: test-invariant,
    gen-openapi, verify-openapi.

Downloads

Pre-built binaries for each supported platform are attached below. Linux variants bundle antigravity-bridge.exe next to the observer binary for WSL2 users of the Antigravity adapter.

Platform Asset
Linux x86_64 observer-v1.7.0-linux-x64.tar.gz
Linux arm64 observer-v1.7.0-linux-arm64.tar.gz
macOS x86_64 (Intel) observer-v1.7.0-darwin-x64.tar.gz
macOS arm64 (Apple Silicon) observer-v1.7.0-darwin-arm64.tar.gz
Windows x86_64 observer-v1.7.0-win32-x64.zip

Verify with sha256sum -c SHA256SUMS (or shasum -a 256 -c SHA256SUMS on macOS) from the directory containing the downloads.

Also available via npm: npm install -g @superbased/observer@1.7.0

Org server (Docker)

The self-hosted org server ships as a Docker image and as per-platform observer-org-v1.7.0-* archives (attached below).

docker pull ghcr.io/marmutapp/observer-org:v1.7.0

The image is keyless-signed with cosign. Verify it:

cosign verify ghcr.io/marmutapp/observer-org:v1.7.0 \
  --certificate-identity-regexp 'https://github.com/marmutapp/superbased-observer-private/.*' \
  --certificate-oidc-issuer https://token.actions.githubusercontent.com

Supply chain

CycloneDX SBOMs are attached: observer.cdx.json and observer-org.cdx.json.

SLSA Level 3 build provenance for the binaries is attached as a *.intoto.jsonl attestation on this release. Verify with slsa-verifier.

Assets 15
Loading