Skip to content

fix(taste): rejected preferences must carry rejection-strength confidence#1777

Open
jbetala7 wants to merge 1 commit into
garrytan:mainfrom
jbetala7:oss/fix-taste-rejected-confidence
Open

fix(taste): rejected preferences must carry rejection-strength confidence#1777
jbetala7 wants to merge 1 commit into
garrytan:mainfrom
jbetala7:oss/fix-taste-rejected-confidence

Conversation

@jbetala7
Copy link
Copy Markdown
Contributor

Repro / observed problem

In bin/gstack-taste-update, every preference in a rejected bucket is permanently stored with confidence: 0, regardless of how many times the value is rejected. On upstream main (19770ea):

$ gstack-taste-update rejected v1 --reason "colors: crimson"   # x3
$ gstack-taste-update rejected vx --reason "colors: beige"     # x1
$ gstack-taste-update show
[colors]
  rejected:
    crimson — conf 0.00 (+0/-3)
    beige — conf 0.00 (+0/-1)

Root cause

bumpPref() computed confidence as approval rate for both buckets:

const total = entry.approved_count + entry.rejected_count;
entry.confidence = entry.approved_count / (total + 1);

The approved and rejected lists are disjoint, so a rejected-bucket entry only ever gets rejected_count incremented — approved_count stays 0 and confidence is always 0. That broke three things:

  1. show prints conf 0.00 for every rejection.
  2. show ranks rejections by confidence * rejected_count = 0 for all, so the top-3 is insertion order, not the strongest rejections.
  3. The taste-drift warning (opp.confidence >= 0.6) is unreachable through real CLI use. The existing test only passed it by hand-seeding confidence: 0.8 via writeProfile, with a comment admitting the natural value is 0/5.

Fix

Compute confidence from this bucket's own count:

const total = entry.approved_count + entry.rejected_count;
const ownCount = action === 'approved' ? entry.approved_count : entry.rejected_count;
entry.confidence = ownCount / (total + 1);

Approved-bucket confidence is unchanged (there total === approved_count, so approved_count/(total+1) is identical). Rejected entries now reflect rejection strength, so crimson (3 rejections) → 0.75 and beige (1) → 0.50, the ranking orders by strength, and the drift warning fires after enough real rejections.

Testing

bun test test/taste-engine.test.ts — 22 pass (19 existing + 3 new). Added:

  • repeated rejections raise rejected confidence toward 1 (5 rejections → 5/6)
  • show ranks rejections by strength, not insertion order
  • drift warning fires from real CLI rejections (no seeding)

All three fail on main and pass with this change. Verified git diff --check clean and git merge-tree --write-tree HEAD origin/main merges cleanly.

Fixes #1776

…ence

bumpPref() computed confidence as approved_count/(total+1) for both
buckets. Rejected-bucket entries never gain approvals, so their
confidence was pinned to 0 — making `show` print "conf 0.00" for every
rejection, the rejection ranking a no-op, and the taste-drift warning
(opp.confidence >= 0.6) unreachable through real CLI use.

Compute confidence from this bucket's own count. Approved-bucket
behavior is unchanged (total == approved_count there); rejected entries
now reflect rejection strength.

Fixes garrytan#1776

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

taste profile: rejected preferences always have confidence 0 (broken show ranking + dead drift warning)

1 participant