Skip to content

feat: action tagging in distillation for implicit behavioral pattern detection#376

Merged
BYK merged 1 commit into
mainfrom
feat-action-tags
May 18, 2026
Merged

feat: action tagging in distillation for implicit behavioral pattern detection#376
BYK merged 1 commit into
mainfrom
feat-action-tags

Conversation

@BYK
Copy link
Copy Markdown
Owner

@BYK BYK commented May 18, 2026

Summary

Adds behavioral action tags to the distillation observer prompt and mechanical cross-session counting to detect implicit user preferences.

Eval Results

PR-2 (implicit preferences): 3.50 → 4.58 (tail-window: ~4.85)

Question Before After
Tests alongside implementation 1.4 3.6
Error handling in API endpoints 3.0 4.2
After implementing, what next 2.0 5.0
Route handler error pattern 2.0 4.2
const vs let 5 5
ORM stance 5 4.6
ORM for queries 5 5
const for vars 5 5

How it works

Action tags in distillation

The distillation observer prompt now tags user behavioral actions with bracketed labels:

🔴 [requested-tests] User asked for tests after implementing the POST /users endpoint
🔴 [corrected-style] User corrected let to const for non-reassigned variable
🔴 [rejected-approach] User rejected ORM suggestion, prefers raw SQL

Tags use a consistent [verb-noun] format so the same behavior in different contexts (testing REST endpoints vs React components) gets the same tag.

Cross-session counting

After each distillation segment, action tags are extracted via regex and counted across distinct sessions in the project. When a tag appears in 3+ sessions, a preference entry is auto-created at confidence 0.8.

Why this works where embeddings didn't

Vector similarity compares surface text — 'write tests for REST endpoints' and 'add tests for React components' are topically different. Action tags abstract the behavior into a canonical label ([requested-tests]), making the pattern countable regardless of context.

Files Changed

  • packages/core/src/prompt.ts — BEHAVIORAL ACTION TAGS section in distillation prompt
  • packages/core/src/pattern-extract.tsextractActionTags(), tagToTitle()
  • packages/core/src/distillation.ts — tag counting + preference creation after distillation

…detection

Adds behavioral action tags to the distillation observer prompt. Tags like
[requested-tests], [corrected-style], [rejected-approach] are embedded in
observations and mechanically counted across sessions. When a tag appears
in 3+ distinct sessions, a preference entry is auto-created.

Combined with pattern echo detection, this catches implicit patterns that
the user demonstrates but never states as directives.

Eval PR-2 (implicit preferences): 3.50 → 4.58 (tail-window: ~4.85)
@BYK BYK self-assigned this May 18, 2026
@BYK BYK merged commit 71d1b64 into main May 18, 2026
10 checks passed
@BYK BYK deleted the feat-action-tags branch May 18, 2026 22:35
This was referenced May 21, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant