Skip to content

fix: harden token pool timeout recovery - #159

Merged
FreeOnePlus merged 1 commit into
apache:masterfrom
FreeOnePlus:fix/issue-91-token-pool-timeout
Jul 30, 2026
Merged

fix: harden token pool timeout recovery#159
FreeOnePlus merged 1 commit into
apache:masterfrom
FreeOnePlus:fix/issue-91-token-pool-timeout

Conversation

@FreeOnePlus

Copy link
Copy Markdown
Member

Fixes #91

Root cause

Two independent paths amplified token-pool exhaustion in long-running HTTP deployments:

  1. Query timeout cancellation could only be trusted if the physical MySQL connection was discarded before pool release. Current bounded execution already performs that cancellation-safe close, but there was no repeated token-bound end-to-end regression.
  2. Static-token authentication validated Doris connectivity on every MCP request, including ping and discovery, by mutating active_db_config and rebuilding the shared global pool. This added connection pressure and allowed concurrent tenant authentication to interfere with shared pool state.

Change

  • Validate static tokens through the same dedicated token route used by query execution.
  • Never mutate the global active database configuration during token authentication.
  • Cache successful route validation for a configurable short TTL (TOKEN_DB_VALIDATION_TTL_SECONDS, default 30 seconds); failures are never cached and credential rotation changes the cache fingerprint.
  • Include exception type in token-pool acquire diagnostics.
  • Reserve the new TOKEN_... setting so it cannot be misread as a bearer token.
  • Add unit coverage and a real HTTP/token/Doris regression that uses a one-connection pool, forces three consecutive query timeouts, and proves recovery through the token-bound read-only account after every timeout.

Verification

  • Full local suite: 1104 passed, 69 skipped; coverage 58.46%.
  • Real Doris transport suite: 10 passed, 2 optional skipped.
  • Real regression: three consecutive SELECT SLEEP(3) timeouts with DORIS_MAX_CONNECTIONS=1; every follow-up SELECT CURRENT_USER() recovered in under 3 seconds and returned the token-bound read-only user.
  • Ruff: passed.
  • Mypy: passed (66 source files).
  • Bandit: passed.
  • Source distribution and wheel build: passed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Connection pool becomes exhausted after query timeouts with token-bound DB config

1 participant