Skip to content

Add privacy-safe dogfood case inbox and regression promotion loop #36

Description

@maxjustships

Goal

Add a privacy-preserving dogfood case loop on top of the current accuracy-profile/eval system:

observed failure or correction
→ private local case inbox
→ dedupe / triage
→ sanitized export
→ explicit synthetic-regression promotion
→ targeted + frozen release gates

This is the intake/promotion layer missing from #34. It must help accumulate reusable failure classes without turning a private diary into a public test corpus or silently changing release expectations.

v0 CLI scope

Add a public nomnom feedback command group backed by a dedicated private SQLite database, separate from the nutrition diary:

  • feedback capture — store one allowlisted structured case receipt
  • feedback list / feedback show — inspect inbox cases
  • feedback export — produce a sanitized promotion receipt; raw input is redacted by default and requires an explicit opt-in flag to include
  • feedback mark — move a case through new, triaged, promoted, fixed, or duplicate, with optional issue/regression references
  • feedback remove --confirm — explicit deletion
  • feedback stats — counts by category/status/severity

The default feedback DB should live under the normal private XDG data root but must support an explicit test/operator path override. All reads/writes go through CLI commands; no direct diary SQLite access.

Case schema

Use a strict versioned allowlist. At minimum:

  • generated case ID and timestamp
  • category: semantic/integrity, atomicity, UX/reliability, actor protocol, or infrastructure
  • derived severity P0–P3
  • accuracy profile
  • raw input (private inbox only)
  • actual outcome and expected behavior
  • structured error code when applicable
  • whether a diary write occurred
  • source (dogfood or eval)
  • deterministic dedupe fingerprint
  • lifecycle status and optional duplicate/issue/regression references

Reject unknown fields, nutrition/macros/calories, credentials, arbitrary trace blobs, absolute private paths, non-finite values, and malformed JSON.

Privacy and safety boundaries

  • no automatic capture from normal diary commands in v0
  • no user diary DB access, copying, querying, or migration
  • no nutrition facts or provider payloads in feedback receipts
  • no secrets or environment dumps
  • no automatic GitHub issue creation or repository write
  • sanitized export must omit raw input by default
  • literal raw input may be exported only with an explicit flag
  • feedback database permissions must be private where supported
  • tests use only disposable paths and synthetic phrases
  • wheel/sdist still exclude all eval corpora and feedback fixtures

Promotion workflow

Add an eval-only helper that consumes a sanitized export and creates or prints a strict draft synthetic regression skeleton. It must not modify the canonical corpus automatically, invent provider facts, or weaken/fix expected outcomes. Human/agent review supplies synthetic provider fixtures and explicitly promotes the case.

Document the loop briefly in README and the agent skill. Keep #34 as the umbrella gate; create a separate issue only for a genuinely new reusable failure class.

Acceptance

  • strict TDD with RED evidence for each behavioral slice
  • focused tests for privacy, permissions, dedupe, lifecycle, redaction, forbidden fields, malformed/non-finite JSON, and no diary-DB access
  • full canonical suite, Ruff, and diff check pass
  • disposable installed-CLI smoke: capture → list/show → sanitized export → mark → stats → remove
  • package archives contain no eval corpus/feedback fixtures
  • no private diary database is read or written during development or verification

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions