Fix #1881 | Make the connection factory pruning timer demand-driven - #4479
Fix #1881 | Make the connection factory pruning timer demand-driven#4479apoorvdeshmukh wants to merge 3 commits into
Conversation
SqlConnectionFactory is a process-wide singleton whose constructor unconditionally started a repeating timer (4 minute due time, 30 second period). PruneConnectionPoolGroups walks _poolsToRelease, _poolGroupsToRelease and _connectionPoolGroups and no-ops when all three are empty, but nothing ever checked for that condition, so the timer kept waking the process every 30 seconds forever - even after every SqlConnection had been disposed and ClearAllPools() had been called, and even when Pooling=False. The only _pruningTimer.Dispose() lived in Unload(), which was wrapped in #if NET and hooked solely to AssemblyLoadContext.Unloading, so it never ran in a normal application on .NET and had no counterpart at all on .NET Framework. The timer is now created disarmed and armed only on the idle -> active transition, mirroring the arm/disarm pattern already used by PoolPruner: - armed from the three paths that create pruning work: GetConnectionPoolGroup (new pool group), QueuePoolForRelease and QueuePoolGroupForRelease - disarmed at the tail of PruneConnectionPoolGroups once all three tracking collections have drained - both coordinated through a dedicated _pruningTimerLock, with a _pruningTimerDisposed guard so an unload racing with a new connection cannot throw ObjectDisposedException on a background thread Producers publish their work and release their own collection lock before calling the arm helper, and the disarm path re-reads all three collection counts while holding _pruningTimerLock, so "publish then arm" cannot race with "observe empty then disarm". Lock ordering is acyclic: arm takes only _pruningTimerLock; disarm takes _pruningTimerLock -> _poolsToRelease -> _poolGroupsToRelease and never takes lock (this). Also adds the missing .NET Framework unload hook (AppDomain.DomainUnload and ProcessExit), matching the existing pattern in SqlDiagnosticListener and SqlClientMetrics. Without it _pruningTimerDisposed was never assigned on net462 (CS0649), and the timer was never disposed there at all. Behaviour note: the 4 minute due time now applies per idle -> active transition rather than once per process. This is equivalent for any continuously active application and strictly better when idle. No public API, connection string keyword or AppContext switch changes. Adds SqlConnectionFactoryPruningTimerTest covering construction, arming, staying armed while draining, disarming once drained, re-arming afterwards, and arming via QueuePoolGroupForRelease. Uses a new protected constructor overload taking the due time and period, plus internal IsPruningTimerActive and RunPruningPass members, so the state machine can be asserted deterministically without waiting minutes. Verified against a customer-style repro (open/close a connection, call ClearAllPools, accelerate the timer by reflection and sample tick counts): before the fix the tick counter climbed indefinitely with no-op passes (+15 per 3s, both Pooling=True and Pooling=False); after the fix it goes flat once the pool groups drain and re-arms on the next connection. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 7c2f8d67-2f83-49c0-ae8b-707b713be8ff
There was a problem hiding this comment.
Pull request overview
This PR addresses #1881 by making SqlConnectionFactory’s connection-pool pruning timer demand-driven, so idle processes aren’t woken every pruning period once all pruning work has drained (including after ClearAllPools() and when Pooling=False). It also adds a .NET Framework shutdown hook to ensure the timer is disposed during app-domain/process shutdown, matching the disposal coverage that previously only existed for .NET AssemblyLoadContext unloading.
Changes:
- Create the pruning timer in a disarmed state and arm it only when pruning work is introduced (
GetConnectionPoolGroup,QueuePoolForRelease,QueuePoolGroupForRelease), then disarm it once all tracked pruning work drains. - Add a dedicated
_pruningTimerLockplus_pruningTimerDisposedguard to coordinate state transitions and avoidObjectDisposedExceptionraces. - Add unit tests that drive pruning deterministically via new internal test hooks (
IsPruningTimerActive,RunPruningPass()) and a protected constructor overload to control timer cadence.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
| src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/SqlConnectionFactory.cs | Refactors pruning timer lifecycle to arm/disarm on demand, adds disposal coordination and a .NET Framework unload hook, and introduces test hooks for deterministic validation. |
| src/Microsoft.Data.SqlClient/tests/UnitTests/SqlConnectionFactoryPruningTimerTest.cs | Adds unit tests covering timer initial state, arming on work publication, remaining armed while draining, disarming on quiescence, and re-arming on subsequent work. |
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #4479 +/- ##
==========================================
- Coverage 64.64% 62.61% -2.03%
==========================================
Files 288 283 -5
Lines 44009 66985 +22976
==========================================
+ Hits 28451 41945 +13494
- Misses 15558 25040 +9482
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
paulmedynski
left a comment
There was a problem hiding this comment.
This whole apparatus seems over-engineered, but that's a pre-existing concern. Your changes make sense superficially, with just one question about the tests.
The drain of a pool group that never had a pool created in it takes exactly three passes, so assert that instead of a range. Also assert the timer stays armed after every pass of the drain rather than only the first, so a regression that stalls pruning half-drained is caught. MaxPruningPasses is now only a hang guard for DrainPruningWork. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 7c2f8d67-2f83-49c0-ae8b-707b713be8ff
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 2 out of 2 changed files in this pull request and generated no new comments.
Comments suppressed due to low confidence (1)
src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/SqlConnectionFactory.cs:57
- The lock-ordering comment contradicts the implementation:
StopPruningTimerIfIdle/HasPruningWorkacquire_pruningTimerLockand then take the release-list locks, so it is possible to hold_pruningTimerLockwhile locking_poolsToRelease/_poolGroupsToRelease. The comment should be updated to reflect the actual ordering requirement (i.e.,_pruningTimerLockis outermost and should not be taken while holding the release-list locks) to avoid future deadlock-prone changes based on incorrect guidance.
// Lock ordering: this lock may be taken while holding "this", a pool group lock, or
// either release-list lock, but the reverse is never true. StopPruningTimerIfIdle takes
// _pruningTimerLock first and only then peeks at the release lists, and it deliberately
// reads _connectionPoolGroups without taking "this" (the dictionary is swapped wholesale,
// so reading the reference is safe). That keeps the lock graph acyclic.
mdaigle
left a comment
There was a problem hiding this comment.
One duplicate test, otherwise looks good.
Registering a pool group is the only path that introduces pruning work. Pools and pool groups are queued for release only while a group is still registered, and the timer cannot disarm until all three collections drain, so the arm calls in QueuePoolForRelease and QueuePoolGroupForRelease could never be the ones to arm the timer. Drop them and document the invariant at the remaining site. Also merge the two drain tests, which had become duplicates, and drop the test that only covered the removed arm call. Its coverage moves into the merged test, where the timer staying armed for a queued pool group now exercises HasPruningWork rather than a redundant arm call. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 7c2f8d67-2f83-49c0-ae8b-707b713be8ff
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 2 out of 2 changed files in this pull request and generated no new comments.
Comments suppressed due to low confidence (1)
src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/SqlConnectionFactory.cs:351
- QueuePoolForRelease can be invoked from DbConnectionPoolGroup.Clear() (e.g., via ClearPool(DbConnection) on a stale PoolGroup) even after the pruning timer has disarmed. In that case, the pool is added to _poolsToRelease but no background pruning pass will run to drain/dispose it unless the timer is re-armed when this work is published.
}
SqlClientDiagnostics.Metrics.EnterInactiveConnectionPool();
SqlClientDiagnostics.Metrics.ExitActiveConnectionPool();
}
Fixes #1881
SqlConnectionFactoryis a process-wide singleton whose constructor unconditionally started a repeating timer (4 min due time, 30 s period).PruneConnectionPoolGroupsno-ops when_poolsToRelease,_poolGroupsToReleaseand_connectionPoolGroupsare all empty, but nothing ever checked for that — so the process was woken every 30 s forever, even after everySqlConnectionwas disposed andClearAllPools()was called, and even withPooling=False.The only
_pruningTimer.Dispose()lived inUnload(), wrapped in#if NETand hooked solely toAssemblyLoadContext.Unloading, so it never ran in a normal app on .NET and had no counterpart at all on .NET Framework.Change
The timer is now created disarmed and armed only on the idle → active transition, mirroring the arm/disarm pattern already used by
PoolPruner:GetConnectionPoolGroup. Registering a pool group is the only way pruning work enters the factory — pools and pool groups are queued for release only while a group is still registered, and a disabled group can never re-acquire a pool.PruneConnectionPoolGroupsonce all three collections drain._pruningTimerLock(outermost; the disarm path re-reads all three counts while holding it) with a_pruningTimerDisposedguard, so an unload racing a new connection cannot throwObjectDisposedExceptionon a background thread.To make the state machine testable without waiting minutes, this adds a
protectedconstructor overload taking the due time and period, plusinternal IsPruningTimerActive/RunPruningPass().Worth calling out in review
AppDomain.DomainUnload+ProcessExit), matching the existing pattern inSqlDiagnosticListenerandSqlClientMetrics. Without it_pruningTimerDisposedis never assigned onnet462(CS0649 under warnings-as-errors), and the timer was never disposed there at all. Slightly beyond the minimal fix, but it closes the disposal gap named in the issue title.ClearAllPools()is intentionally not included here.No public API, connection string keyword or AppContext switch changes.
Validation
Repro: open/close a connection, call
ClearAllPools(), accelerate the timer by reflection, then sample tick counts._pruningTimerEnabledPooling=TruePooling=FalsePooling=TrueFalsePooling=FalseFalseThe pooled run drains
ReleasePool=24/ReleasePoolGroup=1before quiescing, so the release paths are exercised.Adds 4 unit tests covering the arm/disarm state machine. Full
UnitTestssuite on net9.0 shows the same pre-existing failures with and without the change (native SNI won't initialize in this environment). Builds clean on net8.0, net9.0 and net462.Suggested release notes