Skip to content

feat(auth): rotate api_token_pepper when blocking a user#4314

Merged
markijbema merged 7 commits into
mainfrom
mark/block-rotates-pepper
Jun 30, 2026
Merged

feat(auth): rotate api_token_pepper when blocking a user#4314
markijbema merged 7 commits into
mainfrom
mark/block-rotates-pepper

Conversation

@markijbema

Copy link
Copy Markdown
Contributor

Summary

Blocking a user previously set blocked_reason/blocked_at but left api_token_pepper unchanged. Since the pepper is the only per-user revocation lever that edge services check against the database, a blocked user's already-issued API tokens kept a valid pepper and continued to be accepted by pepper-validating services (e.g. KiloClaw) until the token's natural expiry (up to ~5 years for device-auth tokens).

This change makes blocking a user revoke their tokens everywhere by rotating the pepper as part of the block:

  • Adds a central blockUser helper (apps/web/src/lib/user/block.ts) that sets blocked_reason/blocked_at/blocked_by_kilo_user_id and rotates api_token_pepper, guarded by isNull(blocked_reason) so an existing block is never overwritten. Returns whether this call actually transitioned the user to blocked, and accepts an optional dbOrTx so it composes inside existing transactions.
  • Routes the scattered single-user block sites through it: Stripe dispute enforcement, kilo-pass duplicate-card block, kilo-pass cancel-and-refund, and Stytch SMART_RATE_LIMIT_BANNED autoban.
  • Updates bulkBlockUsers to rotate the pepper per-row in SQL (gen_random_uuid()::text) so each blocked user gets a distinct new pepper rather than a shared one.

This is Workstream B of a larger device-auth / token-revocation hardening effort. It is independent and safe to ship on its own; its benefit is fully realized on services that validate the pepper against the DB.

No schema change (the api_token_pepper, blocked_* columns already exist), so no migration. No new PII, so no GDPR soft-delete changes.

Verification

  • Added apps/web/src/lib/user/block.test.ts covering: blocks + rotates pepper; defaults blocked_by to null; does not overwrite an existing block and leaves the pepper untouched; returns false for a non-existent user; runs inside a provided transaction; and rolls back (block + rotation) when the surrounding transaction throws.
  • Extended call-site tests to assert pepper rotation: Stytch autoban (rotates; already-blocked left untouched), Stripe disputes (rotates), and bulkBlockUsers (each of 4 users gets a distinct, freshly-rotated pepper).
  • Ran the affected suites locally against the test DB: user/block, abuse/bulkBlock, stytch, stripe/disputes, kilo-pass/cancel-and-refund, kilo-pass/stripe-handlers-invoice-paid — all green.

Visual Changes

N/A

Reviewer Notes

  • Behavioral change: any user blocked after this ships has all existing API tokens invalidated. Unblocking does not restore the old pepper; the user simply re-authenticates. The existing unblock paths only clear blocked_* and do not depend on the pre-block pepper.
  • The single-user sites that ran inside a db.transaction now delegate to blockUser({ ..., dbOrTx: tx }); the dispute path keeps its SELECT ... FOR UPDATE for locking and admin-note logic and now derives didBlock from the helper's return value.
  • bulkBlock rotates per-row in SQL on purpose — a single JS randomUUID() in a bulk UPDATE would assign the same pepper to every blocked user.

@markijbema

Copy link
Copy Markdown
Contributor Author

Addressed [PR4314-1] in 6497112.

admin.users.updateBlockStatus was the primary admin block path and still wrote the blocked fields directly without rotating api_token_pepper. The unblocked→blocked transition now goes through blockUser({ kiloUserId, reason, blockedByKiloUserId: ctx.user.id, dbOrTx: tx }), so the pepper is rotated and existing tokens are revoked on every pepper-checking service.

Preserved:

  • Unblock still clears blocked_* and does not rotate the pepper (not a revocation event).
  • Already-blocked reason edits still update the reason directly without churning the pepper (it was already rotated at first block).
  • Event emission stays gated on the actual transition (didTransition).

Added apps/web/src/routers/admin-router.test.ts covering: block rotates pepper; unblock clears fields and leaves pepper intact; reason-change on an already-blocked user does not rotate.

@markijbema

Copy link
Copy Markdown
Contributor Author

Two notes:

[PR4314-1] is already addressed — that re-review looks stale. It cites admin-router.ts:505/522 with the old direct-write blockMetadata structure, but admin.users.updateBlockStatus was changed in 6497112 to route the block transition through blockUser, and further simplified in 241f745.

Re: using blockUser for the already-blocked reason-edit branch — done in 241f745. It couldn't be a drop-in before because blockUser's WHERE ... AND blocked_reason IS NULL guard makes it a no-op for an already-blocked user (which would silently drop the admin's reason edit). That guard is load-bearing for the automated callers (e.g. Stytch autoban must not clobber a manual admin block reason).

So I added an opt-in overwriteExisting flag that drops the guard, and collapsed both block branches of the admin mutation into a single blockUser({ ..., overwriteExisting: true }) call. Editing an already-blocked user's reason now also rotates the pepper — harmless as you noted, and it additionally covers users blocked before pepper rotation existed. Default callers (disputes, kilo-pass, stytch, bulkBlock) are unchanged; the guard is still in effect for them.

Tests updated: block.test.ts gains an overwriteExisting case; the admin already-blocked test now asserts the reason is overwritten and the pepper rotates.

Blocking a user previously set blocked_reason/blocked_at but left
api_token_pepper unchanged, so already-issued API tokens kept a valid
pepper and continued to be accepted by services that validate the pepper
against the DB (e.g. KiloClaw) until natural expiry (~5 years for
device-auth tokens).

Add a central blockUser helper that sets the blocked fields AND rotates
api_token_pepper, guarded by isNull(blocked_reason) so an existing block
is never overwritten and callers can detect the unblocked->blocked
transition. Route the scattered block sites through it: Stripe disputes,
kilo-pass duplicate-card, kilo-pass cancel-and-refund, Stytch autoban,
and the admin updateBlockStatus mutation. Rotate the pepper per-row in
bulkBlock via gen_random_uuid() so each blocked user gets a distinct
pepper.

Blocked users' tokens are now revoked on every pepper-checking service.
@markijbema markijbema force-pushed the mark/block-rotates-pepper branch from 14b4568 to 71ae194 Compare June 30, 2026 09:42

@markijbema markijbema left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LLM-generated review comment.

Findings:

  • PR4314-F1 High — apps/web/src/app/admin/api/backfills/block-blacklisted-domains/route.ts:99 still blocks users by directly setting blocked_reason, blocked_at, and blocked_by_kilo_user_id without rotating api_token_pepper. It only revokes gateway grants at apps/web/src/app/admin/api/backfills/block-blacklisted-domains/route.ts:119, so users blocked through this active admin backfill keep any existing pepper-valid API/device tokens. This should route through the same rotation behavior or add api_token_pepper: gen_random_uuid()::text in the guarded update.

  • PR4314-F2 High — apps/web/src/lib/user/block.ts:37 intentionally skips already-blocked users, and apps/web/src/routers/admin-router.ts:524 ignores the helper’s false result. That preserves old peppers for users who were already blocked before this PR, leaving the exact long-lived-token exposure described in the PR for the existing blocked cohort. A one-shot/backfill rotation for currently blocked users, or an explicit preserve-reason-but-rotate path, is needed if this PR is meant to close the revocation gap for existing blocks.

  • PR4314-F3 Medium — apps/web/src/lib/abuse/bulkBlock.ts:54 validates “not already blocked” in a separate read, but the update at apps/web/src/lib/abuse/bulkBlock.ts:64 is not guarded with isNull(kilocode_users.blocked_reason). If another block lands between the read and update, bulk block overwrites the original reason/actor/time and rotates the pepper again; later unblockBulkBlockedUsers can then clear the wrong block group. Add the blocked_reason IS NULL predicate to the update and treat a short update count as a conflict/retryable failure.

No automated tests were run; this was a code review only.

…4-F1)

This admin backfill blocked users without rotating api_token_pepper, so
users blocked through it kept pepper-valid API/device tokens. Add per-row
gen_random_uuid()::text rotation to the guarded update. Extract the batch
into backfillBlockBlacklistedDomainsBatch so it is testable without auth
mocking, mirroring the blocked-at backfill.
The 'already blocked' filter runs in a separate read, so a concurrent
block landing between the read and the update could overwrite the
original reason/actor/time (and re-rotate the pepper) — which
unblockBulkBlockedUsers could later clear as the wrong group. Add
isNull(blocked_reason) to the update WHERE and surface a short update
count as a retryable conflict listing the skipped users.
…-F2)

Block rotation only fires on the unblocked->blocked transition, so users
blocked before this change keep their original (usually null) pepper and
their existing tokens stay valid. Add a batched admin backfill, mirroring
the blocked-at backfill, that assigns a fresh gen_random_uuid()::text to
users with blocked_reason set and a null api_token_pepper. Non-null
peppers are excluded since they already reflect a prior rotation (block,
admin reset, or soft delete).
@markijbema

Copy link
Copy Markdown
Contributor Author

Addressed all three review findings as atomic commits:

  • PR4314-F1 (1da9190281) — block-blacklisted-domains backfill now rotates api_token_pepper per-row (gen_random_uuid()::text) in its guarded update, so users blocked through it have their tokens revoked. Extracted the batch into backfillBlockBlacklistedDomainsBatch so it's testable without auth mocking (mirrors blocked-at).
  • PR4314-F3 (8366a9800a) — bulkBlock update is now guarded with isNull(blocked_reason), and a short update count is surfaced as a retryable conflict listing the skipped users. Prevents a concurrent block (between the read and the update) from overwriting the original reason/actor/time, which unblockBulkBlockedUsers could otherwise clear as the wrong group.
  • PR4314-F2 (54f9b5bd2e) — new batched admin backfill blocked-user-pepper that assigns a fresh pepper to users with blocked_reason set and a null api_token_pepper. This closes the gap for the pre-existing blocked cohort (block rotation otherwise only fires on the unblocked→blocked transition). Non-null peppers are intentionally excluded — a non-null value already reflects a prior rotation (block, admin reset, or soft delete), so soft-deleted users fall out automatically.

Note on scope of F2: the predicate is blocked_reason IS NOT NULL AND api_token_pepper IS NULL. Since new users are created with a null pepper and it's only ever set by a rotation, "null pepper" is the dormant cohort whose tokens are currently un-revoked; filling those closes the gap without churning deliberately-set peppers.

Verification: 17 tests across the three areas pass; oxlint clean; full pnpm --filter web typecheck passes.

…314-F2)

Wire the blocked-user-pepper backfill into /admin/backfills with a
count badge + batched run button, mirroring the existing BlockedAt
backfill card so the new route is actually runnable from the admin UI.
@markijbema markijbema marked this pull request as ready for review June 30, 2026 11:59
Comment thread apps/web/src/lib/kilo-pass/cancel-and-refund.ts Outdated
@kilo-code-bot

kilo-code-bot Bot commented Jun 30, 2026

Copy link
Copy Markdown
Contributor

Code Review Summary

Status: No Issues Found | Recommendation: Merge

Files Reviewed (1 files)
  • apps/web/src/lib/kilo-pass/cancel-and-refund.ts
Previous Review Summary (commit 7c6238f)

Current summary above is authoritative. Previous snapshots are kept for context only.

Previous review (commit 7c6238f)

Status: 2 Issues Found | Recommendation: Address before merge

Overview

Severity Count
CRITICAL 0
WARNING 2
SUGGESTION 0
Issue Details (click to expand)

WARNING

File Line Issue
apps/web/src/lib/kilo-pass/cancel-and-refund.ts 194 blockUser() can now return false under a concurrent block, but the function still records Account blocked., emits user.blocked, and returns alreadyBlocked: false from the stale pre-transaction snapshot.
apps/web/src/routers/admin-router.ts 525 admin.users.updateBlockStatus now returns success without applying a new blocked_reason for already-blocked users, so admin reason updates are silently dropped at current HEAD.
Files Reviewed (18 files)
  • apps/web/src/app/admin/api/backfills/block-blacklisted-domains/route.test.ts - 0 issues
  • apps/web/src/app/admin/api/backfills/block-blacklisted-domains/route.ts - 0 issues
  • apps/web/src/app/admin/api/backfills/blocked-user-pepper/route.test.ts - 0 issues
  • apps/web/src/app/admin/api/backfills/blocked-user-pepper/route.ts - 0 issues
  • apps/web/src/app/admin/backfills/page.tsx - 0 issues
  • apps/web/src/app/admin/components/BlockedUserPepperBackfill.tsx - 0 issues
  • apps/web/src/lib/abuse/bulkBlock.test.ts - 0 issues
  • apps/web/src/lib/abuse/bulkBlock.ts - 0 issues
  • apps/web/src/lib/kilo-pass/cancel-and-refund.ts - 1 issue
  • apps/web/src/lib/kilo-pass/stripe-handlers-invoice-paid.ts - 0 issues
  • apps/web/src/lib/stripe/disputes.test.ts - 0 issues
  • apps/web/src/lib/stripe/disputes.ts - 0 issues
  • apps/web/src/lib/stytch.test.ts - 0 issues
  • apps/web/src/lib/stytch.ts - 0 issues
  • apps/web/src/lib/user/block.test.ts - 0 issues
  • apps/web/src/lib/user/block.ts - 0 issues
  • apps/web/src/routers/admin-router.test.ts - 0 issues
  • apps/web/src/routers/admin-router.ts - 1 issue

Fix these issues in Kilo Cloud


Reviewed by gpt-5.4-20260305 · Input: 38.6K · Output: 5K · Cached: 182.8K

Review guidance: REVIEW.md from base branch main

…lt (PR4314-W1)

cancel-and-refund read the user outside the transaction and then keyed
the admin note, user.blocked event, and alreadyBlocked result off that
stale blocked_reason snapshot while ignoring blockUser's return. Under a
concurrent block landing between the read and the guarded update, it would
falsely record 'Account blocked.', emit user.blocked, and return
alreadyBlocked: false. Capture didBlock from blockUser and drive the
note/event/result from it (matching disputes.ts).
@markijbema

Copy link
Copy Markdown
Contributor Author

Thanks for the review. Status on the two warnings:

  • WARNING cancel-and-refund.ts:194 — fixed in c51520f (now drives the note/event/alreadyBlocked off blockUser's didBlock result). Thread resolved.
  • WARNING admin-router.ts:525 — working as intended, not changing this. The non-overwrite behavior is deliberate: blockUser's isNull(blocked_reason) guard preserves an existing block's original reason and is load-bearing for the automated callers (e.g. a Stytch autoban must not clobber a manual admin block reason). The admin UI (UserAdminNotes) only ever offers Block for an unblocked user xor Unblock for a blocked one — they're mutually exclusive — so it never submits a new reason for an already-blocked user; the "dropped reason" path is unreachable from the product. The block→unblock event there is also already race-safe (it derives didTransition from the FOR UPDATE read, not a stale snapshot). To change a reason, an admin unblocks then re-blocks; to rotate a pepper directly there's the existing resetAPIKey admin action.

@markijbema markijbema merged commit da6a637 into main Jun 30, 2026
19 checks passed
@markijbema markijbema deleted the mark/block-rotates-pepper branch June 30, 2026 13:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants