Skip to content
This repository has been archived by the owner on Jan 23, 2023. It is now read-only.

Fix new deadlock possibility in ReaderWriterLockSlim from a recent change #14337

Merged
merged 2 commits into from
Oct 5, 2017

Conversation

kouvel
Copy link
Member

@kouvel kouvel commented Oct 5, 2017

Fixed the following scenario that would lead to deadlock (rare, but frequently reproducible if code is changed):

  • Thread T0 signals the write waiter event or the upgradeable read waiter event to wake a waiter
  • There are no threads waiting on the event, but T1 is in WaitOnEvent() after exiting the spin lock and before actually waiting on the event (that is, it's recorded that there is one waiter for the event). It remains in this region for a while, in the repro case it typically gets context-switched out.
  • T2 acquires the RW lock in some fashion that blocks T0 or T3 from acquiring the RW lock
  • T0 or T3 fails to acquire the RW lock enough times for it to enter WaitOnEvent for the same event as T1
  • T0 or T3 resets the event
  • T2 releases the RW lock and does not wake a waiter because the reset at the previous step lost a signal but _waiterStates was not updated to reflect that
  • T1 and other threads begin waiting on the event, but there's no longer any thread that would wake them

@kouvel kouvel added area-System.Threading bug Product bug (most likely) labels Oct 5, 2017
@kouvel kouvel added this to the 2.1.0 milestone Oct 5, 2017
@kouvel kouvel self-assigned this Oct 5, 2017
@kouvel
Copy link
Member Author

kouvel commented Oct 5, 2017

There was no significant change to perf. With this fix though, PR #13495, while still a good change overall, is no longer sufficient to fix the scalability issue identified by issue https://github.com/dotnet/coreclr/issues/12780 by itself. It does reduce the degree of the original issue that led to that test case (a write lock holder was starved from releasing the write lock due to excessive spin lock contention, for long enough to appear as a hang), but the same issue could still occur with higher thread counts. I believe PR #13243 is required to properly fix the original issue and the test case.

…ange

Fixed the following scenario that would lead to deadlock (rare, but frequently reproducible if code is changed):
- Thread T0 signals the write waiter event or the upgradeable read waiter event to wake a waiter
- There are no threads waiting on the event, but T1 is in WaitOnEvent() after exiting the spin lock and before actually waiting on the event (that is, it's recorded that there is one waiter for the event). It remains in this region for a while, in the repro case it typically gets context-switched out.
- T2 acquires the RW lock in some fashion that blocks T0 or T3 from acquiring the RW lock
- T0 or T3 fails to acquire the RW lock enough times for it to enter WaitOnEvent for the same event as T1
- T0 or T3 resets the event
- T2 releases the RW lock and does not wake a waiter because the reset at the previous step lost a signal but _waiterStates was not updated to reflect that
- T1 and other threads begin waiting on the event, but there's no longer any thread that would wake them
@kouvel
Copy link
Member Author

kouvel commented Oct 5, 2017

@dotnet-bot test CentOS7.1 x64 Debug Build and Test
test Windows_NT x86 full_opt legacy_backend CoreCLR Perf Tests Correctness

@tarekgh
Copy link
Member

tarekgh commented Oct 5, 2017

@kouvel your description on this PR is good to include it as a comment on the code to help to avoid such problem in the future.

@tarekgh
Copy link
Member

tarekgh commented Oct 5, 2017

LGTM

@kouvel kouvel merged commit 7e43a10 into dotnet:master Oct 5, 2017
@kouvel kouvel deleted the RwLockFix branch October 5, 2017 19:06
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
area-System.Threading bug Product bug (most likely)
Projects
None yet
3 participants