VahterBanBot: honest LLM prompt for empty-text media, sender bio, LLM-kill attribution

[Szer]

Incident

2026-08-18: VahterBanBot auto-deleted an innocent static cat sticker in @AvaloniaRU (msg 217142, user @zefirixx). The sticker had no caption, so Azure OCR found no text and msg.Text stayed null. ML scored the null text 0.31478 — inside the warning band — so it escalated to LLM triage. The LLM prompt rendered Message: with an empty body (null interpolates to ""), so gpt-4o-mini judged a blank message on username/display-name alone and said SPAM. The verdict was recorded as BotAutoDeleted { reason = MlSpam, score = 0.31478 } — mislabeled, since the LLM decided, not the ML threshold.

Per the owner's constraint, the empty-text escalation itself is not disabled or threshold-gated — real spammers post content-less stickers/photos with the spam in their name or bio. The fix gives the LLM honest context instead.

Changes

1. Explicit media placeholder + prompt instruction (LlmTriage.fs) — when the LLM-triage message has no readable text but carries media, the prompt's Message: body renders a descriptive placeholder ([sticker "🐈‍⬛️" from set "catssenseoflife", no readable text], [photo, no readable text], degrading gracefully when sticker emoji/set_name are absent — the actual prod spam sticker had neither). A new system-prompt line tells the model a media-only, textless message is not itself a spam signal — judge sender signals (username/display name/bio) instead. The placeholder is rendered only in the LLM prompt string; msg.Text is never mutated.

2. Sender bio in the LLM prompt (LlmTriage.fs) — IUserProfileFetcher.Fetch (already used by reaction triage, 7-day DB cache, never throws) is now also called at message-LLM-triage escalation time, adding a Bio: ... line ((none) when empty/fetch-failed) alongside username/display name.

3. Deletion-reason attribution (Types.fs, Bot.fs) — AutoDeleteReason gains a distinct LlmSpam of {| score; modelName |} case, used when the kill came from LlmVerdict.Kill (Actor.LLM); MlSpam is kept for genuine ML-threshold deletions. Old stored events with reason.Case = "MlSpam" keep deserializing unchanged (compat test added).

The cache-key trap avoided

msg.Text feeds four things: the ML scorer, the spam-text cache, the LLM verdict-cache key, and the deleted-spam channel post text. SPAM/SKIP verdicts are cached globally by text hash — if the placeholder string had ever leaked into msg.Text (e.g. via AppendText), every sticker/photo with no OCR text would collapse onto the same cache key ("[photo, no readable text]"), and one SPAM verdict on a single photo would globally condemn every future photo. The placeholder therefore only exists inside the prompt-building closure; empty-text messages still take the pre-existing NoCache branch (now driven by a pulled-out hasStableTextCacheKey predicate, unit-tested to confirm placeholder-rendered messages still hit it).

Testing

• dotnet test tests/VahterBanBot.Unit.Tests — 83 passed (new: LlmMediaPlaceholderTests.fs covering placeholder rendering per media type, no-mutation-of-msg.Text, hasStableTextCacheKey/NoCache, formatBioLine; SpamDeleteReasonTests.fs covering LLM-kill vs ML-threshold attribution).
• dotnet test tests/VahterBanBot.Tests -c Release — 253 passed (local podman Testcontainers; includes new LlmSpam serialization round-trip + backward-compat + fold tests in EventSerializationTests.fs).
• dotnet test tests/SerializationCompat.Tests — 51 passed (unaffected, run as a regression check per project scope).
• dotnet build src/VahterBanBot/VahterBanBot.fsproj -c Release — clean, 0 warnings/errors.

🤖 Generated with Claude Code

https://claude.ai/code/session_01KVLgQPBYJPa1H9cemwQwHf