Skip to content

fix(security): keep-alive Agent pinned to the Authentik origin — last documented mitigation for the sign-in timeout - #494

Merged
izzywdev merged 5 commits into
masterfrom
claude/signin-timeout-error-2avs7q
Aug 2, 2026
Merged

fix(security): keep-alive Agent pinned to the Authentik origin — last documented mitigation for the sign-in timeout#494
izzywdev merged 5 commits into
masterfrom
claude/signin-timeout-error-2avs7q

Conversation

@izzywdev

@izzywdev izzywdev commented Aug 2, 2026

Copy link
Copy Markdown
Owner

📋 Description

Follow-up to #455 and #459 on the same sign-in-timeout thread. A fresh production report on this thread confirmed the stall is still live post-#459: a Google sign-in redirect took ~10-15s (inside the documented 16-30s slow-path range), and a password sign-in for one account timed out outright with no session established.

Both #455 and #459 already fixed the symptoms — the client/server timeout budgets, the misleading 401, the redundant userinfo hop, and instrumented the cause (recordHop WARN logging) without confirming it. #455's own "Future Work" section named the one mitigation it did not apply:

If connect/DNS time dominates: an explicit keep-alive dispatcher pinned to the Authentik origin.

This PR applies exactly that, unconfirmed-hypothesis caveat intact — I don't have production log access to verify elapsedMs from recordHop, so this is the documented next step, not a proven cure.

Root cause (documented in authentikPassword.ts, unconfirmed)

Every hop of one login/signup attempt (2-3 flow-executor stages, the authorize→code chain, admin-API set_password) lands on the same in-cluster Authentik origin, several times per request. This pod's own dnsConfig documents CoreDNS "intermittently stalls lookups in 5s/10s retry multiples." DNS resolves per new connection. #455 already fixed one multiplier (leaked response bodies pinning sockets, forcing a fresh connection per hop) via drainBody. What's left: Node's default fetch dispatcher still uses a keep-alive window sized for general-purpose traffic, which can lapse between hops when a DNS stall spaces them out further than expected — forcing a reconnect (and a fresh lookup) anyway.

Fix

A dedicated undici.Agent (keepAliveTimeout: 30s, keepAliveMaxTimeout: 60s, connections: 32), created once at module scope in authentikPassword.ts and passed as fetch's dispatcher option from the single fetchWithTimeout helper all four Authentik-origin call sites already funnel through. Holds one socket open across an entire login chain, so only the first hop of a request should pay for DNS + connect.

undici was already resolving transitively (hoisted from another dependency); it's now declared directly in backend/security/package.json rather than relying on hoisting.

🔄 Type of Change

  • 🐛 Bug fix (non-breaking change which fixes an issue)
  • ⚡ Performance improvement

🧪 Testing

Test Configuration:

  • Node.js version: 22.22.2 (repo engines targets ≥24; CI covers 24.x)
  • npm version: 10.9.7
  • OS: Linux

Test Instructions

npm install
npm run type-check -w backend/security
npm test -w backend/security

Results:

  • type-check -w backend/security: clean
  • backend/security full suite: identical pass/fail set before and after (14 failed suites / 3 failed tests / 312 passed — all pre-existing, verified via git stash A/B on the unmodified tree; none touch authentikPassword.ts)
  • The tests that do exercise this file (authentik-password-login.test.ts, authentik-signup.test.ts) fully pass — they stub global.fetch directly, so the added dispatcher option is inert in tests

🔧 Implementation Details

Changes Made

  • Backend Changes:
    • backend/security/src/services/authentikPassword.ts — module-scoped undici.Agent, wired into fetchWithTimeout's single fetch call
    • backend/security/package.json — declares undici as a direct dependency

Code Quality

  • Code follows the project's coding standards
  • Self-review of code completed
  • Code is commented, particularly in hard-to-understand areas
  • No console.log or debugging statements left in code

Documentation

  • Documentation has been updated — reasoning captured in a doc-comment beside the Agent

🚨 Breaking Changes

None. Purely additive connection-pooling change to an internal HTTP client; no API surface, request shape, or response shape changes.

📋 Checklist

Pre-submission

  • I have performed a self-review of my code
  • I have commented my code, particularly in hard-to-understand areas
  • My changes generate no new warnings or errors
  • New and existing unit tests pass locally with my changes

Code Quality

Security Checklist

  • No sensitive data exposed
  • Authentication/authorization properly handled — unchanged; this only affects connection reuse, not the auth flow logic

📝 Additional Notes

Deployment Notes

master is deploy-on-push — merge in a deploy window per the repo's hardening convention.

Future Work

  • Read the authentikPassword: SLOW hop WARN lines after the next slow sign-in. If elapsedMs for early hops is still dominated by connect/DNS after this change, the dispatcher didn't fully address it and CoreDNS replica capacity is the remaining lever — that's FuzeInfra's, delegated via @claude per repo policy, not this repo.
  • If this does collapse the stalls, fix(login): un-break sign-in — right-size the timeouts, 503 for outages, and make the slow hop visible #455's 45s/40s timeout pair is sized to the bug, not the target — tighten it back down once the real p99 is confirmed.

Questions for Reviewers

🌐 Browser Compatibility

N/A — backend-only change.

🔄 Backwards Compatibility

  • Fully backwards compatible

Generated by Claude Code

…ver-side login

Every hop in the server-brokered password-login chain (flow-executor stages,
authorize hops, admin-API set_password) lands on the same in-cluster Authentik
origin, several times per request. The default fetch dispatcher's keep-alive
window is tuned for general traffic and can lapse between hops when CoreDNS's
documented multi-second stalls (see the recordHop comments in
authentikPassword.ts) space them out, forcing a fresh connection — and a fresh
DNS lookup — per hop. A dedicated undici Agent with a longer keep-alive holds
one socket open across the whole chain, so only the first hop pays for
DNS+connect. This was the last documented, unapplied mitigation in that file's
own incident notes (#362, #371) for the intermittent 16-30s+ sign-in stalls
that show up as an outright client-side timeout for some accounts.
@izzywdev izzywdev added the auto-merge Enable squash auto-merge once CI passes label Aug 2, 2026 — with Claude
@github-actions
github-actions Bot enabled auto-merge (squash) August 2, 2026 11:12
@github-actions

github-actions Bot commented Aug 2, 2026

Copy link
Copy Markdown
Contributor

Automated code review (gate-code-review)

Credit balance is too low

Report-only — this check never blocks merge.

@github-actions

github-actions Bot commented Aug 2, 2026

Copy link
Copy Markdown
Contributor

Automated code review (gate-code-review)

Credit balance is too low

Report-only — this check never blocks merge.

@github-actions

github-actions Bot commented Aug 2, 2026

Copy link
Copy Markdown
Contributor

Automated code review (gate-code-review)

Credit balance is too low

Report-only — this check never blocks merge.

@github-actions

github-actions Bot commented Aug 2, 2026

Copy link
Copy Markdown
Contributor

Automated code review (gate-code-review)

Credit balance is too low

Report-only — this check never blocks merge.

@github-actions

github-actions Bot commented Aug 2, 2026

Copy link
Copy Markdown
Contributor

Automated code review (gate-code-review)

Credit balance is too low

Report-only — this check never blocks merge.

@izzywdev
izzywdev merged commit 9410376 into master Aug 2, 2026
55 checks passed
@izzywdev
izzywdev deleted the claude/signin-timeout-error-2avs7q branch August 2, 2026 15:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

auto-merge Enable squash auto-merge once CI passes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants