Skip to content

Contributor Metrics 1/3: closed local schema, consent, preview, and feedback #345

Description

@NotASithLord

Parent: #344

Outcome

Build the local-only half of Contributor Metrics:

  • A closed, versioned contribution schema.
  • An opt-in local accumulator over existing typed runtime outcomes.
  • Human-only consent state with versioned re-consent.
  • An exact pending-payload preview.
  • Optional binary task feedback ("worked" / "didn't work").
  • No network path and no collector origin in any artifact.

This issue must be independently mergeable while peerd still truthfully sends no telemetry.

Design constraints

Collection lifecycle

  • Defaults are OFF for store, preview, dev, and web-derived settings.
  • Before consent, telemetry-specific recorders are inert and create no queue/bucket state.
  • Consent carries a schema/disclosure version. A future schema expansion cannot silently inherit old consent; it must require a deliberate version bump and re-consent.
  • Only an explicitly trusted human Settings route can enable/disable participation.
  • Actors, models, tools, pages, hooks, imported state, dweb messages, and extension-controlled sandboxes have no route or descriptor that can change consent.
  • Disable/revoke clears all pending contribution buckets and feedback not yet contributed.
  • Reset/uninstall behavior follows existing local-data controls.

Closed schema, not an event bus

Do not add a generic analytics API.

The runtime-facing API should expose explicit, narrow operations such as typed outcome increments. It must not accept:

  • Arbitrary event names.
  • Arbitrary keys or property bags.
  • URLs, origins, hosts, selectors, prompts, text snippets, raw error strings, IDs, timestamps, or opaque caller strings.
  • Values derived directly from page/tool content.

Every outbound-capable field must be one of:

  • A versioned enum owned by the schema.
  • A bounded non-negative counter.
  • A named histogram bucket.
  • A generated extension version/channel/browser value.
  • A provider/model value normalized against a shipped registry; unknown/custom strings become custom.

Unknown fields or enum values fail closed during serialization; they are not stringified or preserved for debugging.

v0 contribution shape

The pure serializer should produce bounded cohort rows keyed only by product-decision dimensions:

  • Feature: initially web_actor_surface, with a deliberately small enum that can grow only through reviewed schema changes.
  • Variant: requested and resolved code / tools.
  • Enumerated fallback reason: none, unavailable worker, incomplete capability grant, or other reviewed values.
  • Browser family, extension version, and channel.
  • Known provider and catalog-known model family; unknown/custom names collapse to custom.
  • Counts/histograms for operational completion, cancellation, classified error, actor turns, action calls, duration, and token usage.
  • Binary user feedback count, not a per-session record.

Reuse the existing failure taxonomy and eval score vocabulary where their semantics match. Do not label a model response or clean turn as task success: only explicit user feedback may contribute to a user-reported-success metric.

No session/message/tool-use/actor/install identifier appears in the serialized shape. A locally generated one-batch idempotency token may be added only in the uploader issue, must rotate per sealed envelope, and must not be usable across batches.

Local aggregation

  • Aggregate before persistence; do not persist a raw telemetry event stream.
  • Keep the accumulator in a bounded, versioned local record separate from sessions, audit, memory, and the vault.
  • Concurrency-safe updates: simultaneous actor/tool settlements cannot lose counts.
  • A service-worker restart can resume the aggregate without double-counting a settled operation.
  • Unknown/newer local schema versions fail read-only with a deterministic diagnostic; no silent downgrade or destructive rewrite.
  • Retention/cap values live as named constants in code and are covered by tests, not duplicated as drifting prose.

Payload preview and consent UI

Add a dedicated Settings surface named Contributor Metrics (placement to follow the existing Options information architecture).

Before enablement it must explain, in plain language:

  • Participation is optional and peerd works fully without it.
  • Exactly which technical/interaction categories may be contributed.
  • The complete forbidden-content list.
  • That the first rollout is preview/dev only.
  • That disabling clears unsent local metrics.
  • That accepted aggregate server rows cannot be linked back or individually deleted because no identity is collected.

The UI must show:

  • Current consent state and disclosure/schema version.
  • The exact locally serialized pending payload after collection begins.
  • Clear Enable and Disable/Clear actions.
  • No pre-checked checkbox, bundled consent, urgency, reward, or degraded-product language.

The preview component must receive the exact frozen serialized bytes that a later uploader will use—not reconstruct a friendly approximation from separate state.

Binary task feedback

Add a small optional final-answer affordance:

  • "worked" / "didn't work"; no text box.
  • Available whether contribution is enabled or not, but only enters the contribution aggregate when valid consent is active.
  • The selection must not alter the model transcript or become model context.
  • Feedback is aggregated into the active fixed cohort; no session/message id is serialized.
  • Repeated clicks or state replay must not double-count; changing the choice adjusts the local aggregate consistently.
  • Do not add colorful accents beyond the existing brand/semantic rules.

If feedback cannot be made idempotent without retaining a local session/message association, retain that association only locally in a bounded dedupe record and prove it never enters serialization.

Suggested architecture

Keep the functional core pure and inject storage/time/runtime facts from the background shell:

  • Pure schema validation, normalization, aggregation, and serialization under the existing runtime/observability boundary or another justified peerd-runtime sub-area.
  • Browser storage and trusted-route wiring in the background chassis.
  • UI in Options and the existing final-answer rendering surface.
  • Public exports through module index.js; no cross-module deep imports.
  • No sixth top-level peerd-* module.

The exact file split is an implementation choice; the architectural boundaries above are not.

Security and privacy tests

Bun/pure

  • Serializer accepts every valid v0 combination and is deterministic.
  • Unknown keys/enums, negative/non-finite counters, over-cap rows, and malformed versions fail closed.
  • Property/fuzz tests prove arbitrary strings cannot survive into serialized output.
  • URL/origin/hostname/search-term/raw-error lookalikes cannot be smuggled through known fields.
  • Provider/model normalizer emits only registry-known enums or custom.
  • Aggregate math, histogram boundaries, concurrency reducer behavior, and feedback adjustment are correct.
  • Preview bytes are canonical and stable.
  • A newer stored version is retained read-only with a diagnostic.

In-browser

  • Fresh install/default settings create no contribution state.
  • Only the trusted human settings route can enable.
  • Enabling/disabling/re-consenting works across service-worker restart.
  • Disabling clears pending buckets and dedupe state.
  • Actor/tool/model/inbound-message routes cannot enable or record before consent.
  • Feedback is accessible, monochrome, idempotent, transcript-free, and does not render as a model-visible message.
  • Exact payload preview is readable and copyable.

Packaging/invariants

  • Store, preview, dev, Chrome, and Firefox defaults remain OFF.
  • No collector URL, upload route, alarm, or network client is introduced by this issue.
  • Generated settings/channel files remain generator-owned and drift-free.
  • Add a source invariant against a generic telemetry property-bag API.

Acceptance criteria

  • The closed schema and accumulator satisfy every invariant in Epic: opt-in Contributor Metrics without content telemetry #344.
  • No telemetry-specific data is recorded before consent.
  • Consent is human-only, versioned, revocable, and clearing.
  • Exact pending payload bytes are visible locally.
  • The v0 web-actor surface cohort can answer requested/resolved/fallback/efficiency/failure questions without content.
  • Binary feedback is optional, content-free, idempotent, and never model-visible.
  • All listed pure, browser, packaging, and UI verification lanes pass.
  • README/store/privacy claims remain unchanged because this issue adds no egress.
  • Adversarial privacy, schema-smuggling, and consent-lifecycle review has no unresolved material finding.

Non-goals

  • Any upload, endpoint, DNS name, remote configuration, or server component.
  • Automatic debug bundle, audit log, OTel, transcript, crash, or raw error sharing.
  • DAU/retention/funnel analytics.
  • Stable pseudonymous identifiers or per-session contribution rows.
  • Free-text feedback.
  • Store telemetry enablement.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    Status
    Done

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions