fix(pool): pool performance #3565
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
ndyakov
force-pushed
the
ndyakov/pool-performance
branch
3 times, most recently
from
fce8679 to
3478f7a
Compare
…redefined state slices - Replace hookManager RWMutex with atomic.Pointer for lock-free reads in hot paths - Add predefined state slices to avoid allocations (validFromInUse, validFromCreatedOrIdle, etc.) - Add Clone() method to PoolHookManager for atomic updates - Update AddPoolHook/RemovePoolHook to use copy-on-write pattern - Update all hookManager access points to use atomic Load() Performance improvements: - Eliminates RWMutex contention in Get/Put/Remove hot paths - Reduces allocations by reusing predefined state slices - Lock-free reads allow better CPU cache utilization
ndyakov
force-pushed
the
ndyakov/pool-performance
branch
from
c914088 to
1b0168d
Compare
The state machine was calling notifyWaiters() on EVERY Get/Put operation, which acquired a mutex even when no waiters were present (the common case). Fix: Use atomic waiterCount to check for waiters BEFORE acquiring mutex. This eliminates mutex contention in the hot path (Get/Put operations). Implementation: - Added atomic.Int32 waiterCount field to ConnStateMachine - Increment when adding waiter, decrement when removing - Check waiterCount atomically before acquiring mutex in notifyWaiters() Performance impact: - Before: mutex lock/unlock on every Get/Put (even with no waiters) - After: lock-free atomic check, only acquire mutex if waiters exist - Expected improvement: ~30-50% for Get/Put operations
ndyakov
force-pushed
the
ndyakov/pool-performance
branch
from
079c217 to
78bcfbb
Compare
…ot path
The pool was creating new slice literals on EVERY Get/Put operation:
- popIdle(): []ConnState{StateCreated, StateIdle}
- putConn(): []ConnState{StateInUse}
- CompareAndSwapUsed(): []ConnState{StateIdle} and []ConnState{StateInUse}
- MarkUnusableForHandoff(): []ConnState{StateInUse, StateIdle, StateCreated}
These allocations were happening millions of times per second in the hot path.
Fix: Use predefined global slices defined in conn_state.go:
- validFromInUse
- validFromCreatedOrIdle
- validFromCreatedInUseOrIdle
Performance impact:
- Before: 4 slice allocations per Get/Put cycle
- After: 0 allocations (use predefined slices)
- Expected improvement: ~30-40% reduction in allocations and GC pressure
Further optimize the hot path by: 1. Remove redundant GetState() call in the loop 2. Only check waiterCount after successful CAS (not before loop) 3. Inline the waiterCount check to avoid notifyWaiters() call overhead This reduces atomic operations from 4-5 per Get/Put to 2-3: - Before: GetState() + CAS + waiterCount.Load() + notifyWaiters mutex check - After: CAS + waiterCount.Load() (only if CAS succeeds) Performance impact: - Eliminates 1-2 atomic operations per Get/Put - Expected improvement: ~10-15% for Get/Put operations
Introduced TryTransitionFast() for the hot path (Get/Put operations): - Single CAS operation (same as master's atomic bool) - No waiter notification overhead - No loop through valid states - No error allocation Hot path flow: 1. popIdle(): Try IDLE → IN_USE (fast), fallback to CREATED → IN_USE 2. putConn(): Try IN_USE → IDLE (fast) This matches master's performance while preserving state machine for: - Background operations (handoff/reauth use UNUSABLE state) - State validation (TryTransition still available) - Waiter notification (AwaitAndTransition for blocking) Performance comparison per Get/Put cycle: - Master: 2 atomic CAS operations - State machine (before): 5 atomic operations (2.5x slower) - State machine (after): 2 atomic CAS operations (same as master!) Expected improvement: Restore to baseline ~11,373 ops/sec
There was a problem hiding this comment.
Pull Request Overview
This PR aims to improve pool performance by replacing synchronization primitives with more efficient atomic-based implementations and optimizing hot path operations through inlining and allocation reduction.
Key Changes:
- Introduced
FastSemaphoreusing atomic operations instead of channel-based semaphores for connection limiting - Replaced mutex-protected hook manager with lock-free atomic pointer operations
- Added predefined state slices and inline methods to reduce allocations in critical paths
Reviewed Changes
Copilot reviewed 8 out of 8 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
| internal/semaphore.go | New FastSemaphore implementation using atomics for high-performance connection limiting |
| internal/pool/pool.go | Replaced channel-based queue with FastSemaphore and converted hook manager to atomic pointer |
| internal/pool/hooks_test.go | Updated tests to use atomic loads for hook manager access |
| internal/pool/hooks.go | Added Clone() method to support lock-free hook manager updates |
| internal/pool/export_test.go | Updated test helper to use semaphore instead of channel queue |
| internal/pool/conn_state.go | Added predefined state slices, optimized waiter tracking with atomic counter, and fast transition method |
| internal/pool/conn.go | Added inline TryAcquire() and Release() methods with direct state machine access for hot paths |
| internal/auth/streaming/pool_hook.go | Migrated from channel-based semaphore to FastSemaphore for re-auth worker limiting |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
ndyakov
force-pushed
the
ndyakov/pool-performance
branch
from
e825db8 to
da5fe33
Compare
ndyakov
force-pushed
the
ndyakov/pool-performance
branch
from
6928e52 to
471a828
Compare
htemelski-redis
approved these changes
Oct 27, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Working on #3559 I saw a degradation in the pool performance. In this PR I am trying to reduce the degradation a bit: