Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

make Quic AcceptStreamAsync concurrent safe #56768

Merged
merged 2 commits into from
Aug 4, 2021
Merged

Conversation

wfurt
Copy link
Member

@wfurt wfurt commented Aug 3, 2021

I found this while investigating #55249
While I could rarely reproduce the reported issue, I would often see strange timeout.
With the current code, running AcceptStreamAsync in parallel is not safe.
Thanks to @stephentoub who pointed the right place when we would throw OperationCancelledException without token expiring.
With the change, I'm yet to see the InvalidOperationException. (not really sure why)

contributes to #55249
I'm not sure if this really matters to runtime tests but the local runs seems happier. (not enough samples yet)

cc: @JamesNK

@wfurt wfurt requested a review from a team August 3, 2021 06:45
@wfurt wfurt self-assigned this Aug 3, 2021
@ghost
Copy link

ghost commented Aug 3, 2021

Tagging subscribers to this area: @dotnet/ncl
See info in area-owners.md if you want to be subscribed.

Issue Details

I found this while investigating #55249
While I could rarely reproduce the reported issue, I would often see strange timeout.
With the current code, running AcceptStreamAsync in parallel is not safe.
Thanks to @stephentoub who pointed the right place when we would throw OperationCancelledException without token expiring.
With the change, I'm yet to see the InvalidOperationException. (not really sure why)

contributes to #55249
I'm not sure if this really matters to runtime tests but the local runs seems happier. (not enough samples yet)

cc: @JamesNK

Author: wfurt
Assignees: wfurt
Labels:

area-System.Net.Quic

Milestone: -

@JamesNK
Copy link
Member

JamesNK commented Aug 3, 2021

Ah, I didn't realize AcceptStreamAsync wasn't thread-safe. In the real-world, Kestrel will only have one call to AcceptStreamAsync active at a time.

In some errors from that test the exception said that multiple ReadAsync calls at a time weren't supported. I guess in that situation one QuicStream was being returned multiple times.

Up to you whether to make AcceptStreamAsync thread-safe or throw a more user-friendly error on multiple concurrent calls (like QuicStream.ReadAsync does). I can modify the test to be more real-world.

@JamesNK
Copy link
Member

JamesNK commented Aug 3, 2021

If I update the test to lock the code that calls AcceptStreamAsync then it passes 100%

await asyncLock.WaitAsync();
try
{
    serverStream = await requestState.ServerConnection.AcceptAsync();
}
finally
{
    asyncLock.Release();
}

@wfurt
Copy link
Member Author

wfurt commented Aug 3, 2021

If I update the test to lock the code that calls AcceptStreamAsync then it passes 100%

The accept failure should have no impact on the reading state. I will proceed with this change and I'll keep it watching for a while. I suspect that some other issue may be lurking out there - we are just lucky because of changed timing.

@wfurt wfurt merged commit c082af3 into dotnet:main Aug 4, 2021
@wfurt wfurt deleted the accept_55249 branch August 4, 2021 05:06
@CarnaViire
Copy link
Member

With old SingleReader=true, it still could not be that two concurrent readers would get the same stream instance from the queue, could it? 🤔

@wfurt
Copy link
Member Author

wfurt commented Aug 4, 2021

no, I don't think so @CarnaViire. This what the Channel is for to provide atomic operations.

@karelz karelz added this to the 6.0.0 milestone Aug 17, 2021
@ghost ghost locked as resolved and limited conversation to collaborators Sep 16, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

8 participants