Skip to content

feat(bandit): Thompson Sampling consumer closing the brand_rules feedback loop#21

Open
ninan-versioning wants to merge 1 commit into
mainfrom
jared-shoploop/bandit-consumer
Open

feat(bandit): Thompson Sampling consumer closing the brand_rules feedback loop#21
ninan-versioning wants to merge 1 commit into
mainfrom
jared-shoploop/bandit-consumer

Conversation

@ninan-versioning

Copy link
Copy Markdown
Owner

Summary

New package @shoploop/bandit (pure TypeScript, no Python runtime) that closes the architecture loop: Performance Ingest → Thompson Sampling → brand_rules.json. It is a consumer — it learns, writes posteriors, and emits DRAFT-only recommendations. It never calls a paid ad API.

It tails the shared BANDIT_QUEUE_PATH (written by Google Ads PR #18 and animated-shorts PR #19), maintains Beta-Bernoulli arms keyed by (twin_handle, preset) in LibSQL, merges arm posteriors into a bandit: namespace inside brand_rules.json, and writes BudgetRecommendations to a new typed BANDIT_RECOMMENDATIONS_PATH.

The 7-step consumer loop (consumer/loop.ts)

            BANDIT_QUEUE_PATH (JSONL, single-writer, shared with producers)
                       │
   ┌───────────────────┴────────────────────────────────────────────────┐
   │ 1. READ      tail new bytes since the LibSQL checkpoint              │
   │ 2. BUCKET    split by `schema`: performance_observation │ provenance │
   │ 3. JOIN      observation × provenance → per-arm reward               │
   │              key precedence: external_id ▶ product_gid ▶ campaign_id │
   │ 4. UPDATE    decay all arms ×BANDIT_DECAY, then α/β update each arm  │
   │ 5. RECOMMEND per twin w/ ≥2 presets: Thompson-sample; if leader      │
   │              beats runner-up by ≥BANDIT_PROMOTE_MARGIN → emit DRAFT  │
   │ 6. WRITE     merge bandit: namespace into brand_rules.json (locked)  │
   │ 7. CHECKPOINT advance the byte offset                                │
   └───────────────────┬─────────────────────────┬──────────────────────┘
                        │                         │
        brand_rules.json (bandit: ns)   BANDIT_RECOMMENDATIONS_PATH (DRAFT)
        ← Scene Composer reads this     ← distribution PR reads this (later)

Math (bandit/beta_arm.ts)

  • BetaArm { alpha, beta, reward_sum, pulls }, prior Beta(1,1).
  • Fractional conjugate update: α += r; β += (1−r) for reward r ∈ [0,1].
  • sample(rng) draws Beta via two Marsaglia-Tsang Gamma draws — seedable, so convergence is deterministic under test.
  • Per-loop multiplicative decay toward the prior (BANDIT_DECAY=0.995).

Numerical test results

Test Assertion Result
Beta posterior 8 successes / 2 failures → Beta(9,3), E[θ]=0.75 ✅ exact to 1e-12
Beta variance seeded sample() empirical var of Beta(9,3)0.0144 ✅ within 1e-2
Convergence 3 arms, true means [0.2, 0.5, 0.8], 2000 updates, seeded RNG ✅ best arm pulled >70% of last 500
probabilityBest Beta(2,8) vs Beta(8,2) ✅ strong arm >0.95
End-to-end fixture queue → once ✅ 6 joined, 1 skipped (noise), 1 recommendation, namespace written
Rules isolation non-bandit keys after write ✅ byte-identical

17/17 tests pass; typecheck clean.

bandit: namespace (only the bandit key is written; all others preserved)

{
  "bandit": {
    "version": 1,
    "updated_at": "2026-06-25T10:06:47.244Z",
    "arms": {
      "varitea_jasmine_pearl": {
        "hero":      { "alpha": 4,   "beta": 1,   "pulls": 3, "last_reward": 1 },
        "lifestyle": { "alpha": 1.1, "beta": 3.9, "pulls": 3, "last_reward": 0.02 }
      }
    },
    "last_recommendation": {
      "twin_handle": "varitea_jasmine_pearl",
      "winning_preset": "hero",
      "confidence": 0.983
    }
  }
}

Coordination with the distribution PR (the join seam)

PerformanceObservation knows the channel ad_id; RenderProvenance knows the arm. The bridge is provenance's external_id — the dispatched ad_id, stamped by the paid adapter. That adapter (request_budget_shift, list_recent_dispatches) is OUT OF SCOPE here and stays DRAFT. Until external_id is emitted, the joiner falls back product_gid ▶ campaign_id (flagged via JoinedSample.join_kind). Documented in packages/bandit/DESIGN.md with a TODO(distribution-PR) at the seam.

Cost

Pure CPU, co-locates on the existing t4g.small: $0 incremental. LibSQL state <$1. Total ≈ $0/mo.

Acceptance

shoploop-bandit once against the fixture queues updates the brand_rules.json bandit namespace and emits exactly one recommendation — verified by test/consumer.test.ts and by running the built CLI.

Test plan

  • pnpm typecheck — clean
  • pnpm test — 17/17 pass (beta math, convergence, joiner, rules isolation, e2e)
  • Built CLI: shoploop-bandit once / inspect against fixtures
  • Wire external_id on the animated-shorts provenance emitter (follow-up, additive)

🤖 Generated with Claude Code

…back loop

New package @shoploop/bandit. Pure TypeScript consumer of the shared
BANDIT_QUEUE_PATH: joins PerformanceObservation (Google Ads, PR #18) with
RenderProvenance (animated-shorts, PR #19) by (twin_handle, preset), maintains
Beta-Bernoulli arms in LibSQL, writes posteriors into the bandit: namespace of
brand_rules.json (file-locked, isolation-safe), and emits DRAFT-only
BudgetRecommendations to BANDIT_RECOMMENDATIONS_PATH. Distribution paid adapters
stay DRAFT — recommendations are visible, never auto-applied.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant