Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Test failure: System.Net.Http.Functional.Tests.SocketsHttpHandlerTest_Http2.Http2_DataSentBeforeServerPreface_ProtocolError #56138

Closed
karelz opened this issue Jul 22, 2021 · 2 comments · Fixed by #56552
Closed

Test failure: System.Net.Http.Functional.Tests.SocketsHttpHandlerTest_Http2.Http2_DataSentBeforeServerPreface_ProtocolError #56138

karelz opened this issue Jul 22, 2021 · 2 comments · Fixed by #56552
Assignees
@alnikola
Labels
area-System.Net.Http os-linux Linux OS (any supported distro) test-run-core Test failures in .NET Core test runs
Milestone
6.0.0

Comments

@karelz
Copy link
Member

karelz commented Jul 22, 2021 •
edited
Loading

Test failures 5/22-7/31 (incl. PRs):

Day Run OS
6/8 PR #53851 Ubuntu.1804.Amd64.Open
7/16 PR #55842 Centos.8.Amd64.Open
7/19 Official run Ubuntu.1910.Amd64.Open
7/22 Official run Alpine.312.Amd64.Open
7/24 PR #56134 Ubuntu.1604.Amd64.Open
7/30 PR #56531 Ubuntu.1604.Amd64.Open

Looks like increased frequency after Preview 7 cut off on 7/15 - perhaps some regression?

Assert.Throws() Failure
Expected: typeof(System.Net.Http.HttpRequestException)
Actual:   typeof(System.Threading.Tasks.TaskCanceledException): The request was canceled due to the configured HttpClient.Timeout of 100 seconds elapsing.
---- System.Threading.Tasks.TaskCanceledException : The request was canceled due to the configured HttpClient.Timeout of 100 seconds elapsing.
-------- System.TimeoutException : A task was canceled.
------------ System.Threading.Tasks.TaskCanceledException : A task was canceled.

   at System.Net.Http.HttpClient.HandleFailure(Exception e, Boolean telemetryStarted, HttpResponseMessage response, CancellationTokenSource cts, CancellationToken cancellationToken, CancellationTokenSource pendingRequestsCts) in /_/src/libraries/System.Net.Http/src/System/Net/Http/HttpClient.cs:line 616
   at System.Net.Http.HttpClient.<SendAsync>g__Core|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken) in /_/src/libraries/System.Net.Http/src/System/Net/Http/HttpClient.cs:line 553
----- Inner Stack Trace -----
   at System.Net.Http.HttpClient.HandleFailure(Exception e, Boolean telemetryStarted, HttpResponseMessage response, CancellationTokenSource cts, CancellationToken cancellationToken, CancellationTokenSource pendingRequestsCts) in /_/src/libraries/System.Net.Http/src/System/Net/Http/HttpClient.cs:line 616
   at System.Net.Http.HttpClient.<SendAsync>g__Core|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken) in /_/src/libraries/System.Net.Http/src/System/Net/Http/HttpClient.cs:line 553
----- Inner Stack Trace -----

----- Inner Stack Trace -----
   at System.Threading.Tasks.TaskCompletionSourceWithCancellation`1.WaitWithCancellationAsync(CancellationToken cancellationToken) in /_/src/libraries/Common/src/System/Threading/Tasks/TaskCompletionSourceWithCancellation.cs:line 23
   at System.Net.Http.HttpConnectionPool.GetHttp2ConnectionAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken) in /_/src/libraries/System.Net.Http/src/System/Net/Http/SocketsHttpHandler/HttpConnectionPool.cs:line 789
   at System.Net.Http.HttpConnectionPool.TrySendUsingHttp2Async(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken) in /_/src/libraries/System.Net.Http/src/System/Net/Http/SocketsHttpHandler/HttpConnectionPool.cs:line 965
   at System.Net.Http.HttpConnectionPool.DetermineVersionAndSendAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken) in /_/src/libraries/System.Net.Http/src/System/Net/Http/SocketsHttpHandler/HttpConnectionPool.cs:line 1015
   at System.Net.Http.HttpConnectionPool.SendAndProcessAltSvcAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken) in /_/src/libraries/System.Net.Http/src/System/Net/Http/SocketsHttpHandler/HttpConnectionPool.cs:line 1028
   at System.Net.Http.HttpConnectionPool.SendWithRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken) in /_/src/libraries/System.Net.Http/src/System/Net/Http/SocketsHttpHandler/HttpConnectionPool.cs:line 1038
   at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken) in /_/src/libraries/System.Net.Http/src/System/Net/Http/SocketsHttpHandler/RedirectHandler.cs:line 30
   at System.Net.Http.HttpClient.<SendAsync>g__Core|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken) in /_/src/libraries/System.Net.Http/src/System/Net/Http/HttpClient.cs:line 553
@dotnet-issue-labeler dotnet-issue-labeler bot added area-System.Net.Http untriaged New issue has not been triaged by the area owner labels Jul 22, 2021
@ghost
Copy link

ghost commented Jul 22, 2021

Tagging subscribers to this area: @dotnet/ncl
See info in area-owners.md if you want to be subscribed.

Issue Details

Test failures 5/22-7/22 (incl. PRs):

Day Run OS
6/8 PR #53851 Ubuntu.1804.Amd64.Open
7/16 PR #55842 Centos.8.Amd64.Open
7/19 Official run Ubuntu.1910.Amd64.Open
7/22 Official run Alpine.312.Amd64.Open 
Assert.Throws() Failure
Expected: typeof(System.Net.Http.HttpRequestException)
Actual:   typeof(System.Threading.Tasks.TaskCanceledException): The request was canceled due to the configured HttpClient.Timeout of 100 seconds elapsing.
---- System.Threading.Tasks.TaskCanceledException : The request was canceled due to the configured HttpClient.Timeout of 100 seconds elapsing.
-------- System.TimeoutException : A task was canceled.
------------ System.Threading.Tasks.TaskCanceledException : A task was canceled.

   at System.Net.Http.HttpClient.HandleFailure(Exception e, Boolean telemetryStarted, HttpResponseMessage response, CancellationTokenSource cts, CancellationToken cancellationToken, CancellationTokenSource pendingRequestsCts) in /_/src/libraries/System.Net.Http/src/System/Net/Http/HttpClient.cs:line 616
   at System.Net.Http.HttpClient.<SendAsync>g__Core|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken) in /_/src/libraries/System.Net.Http/src/System/Net/Http/HttpClient.cs:line 553
----- Inner Stack Trace -----
   at System.Net.Http.HttpClient.HandleFailure(Exception e, Boolean telemetryStarted, HttpResponseMessage response, CancellationTokenSource cts, CancellationToken cancellationToken, CancellationTokenSource pendingRequestsCts) in /_/src/libraries/System.Net.Http/src/System/Net/Http/HttpClient.cs:line 616
   at System.Net.Http.HttpClient.<SendAsync>g__Core|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken) in /_/src/libraries/System.Net.Http/src/System/Net/Http/HttpClient.cs:line 553
----- Inner Stack Trace -----

----- Inner Stack Trace -----
   at System.Threading.Tasks.TaskCompletionSourceWithCancellation`1.WaitWithCancellationAsync(CancellationToken cancellationToken) in /_/src/libraries/Common/src/System/Threading/Tasks/TaskCompletionSourceWithCancellation.cs:line 23
   at System.Net.Http.HttpConnectionPool.GetHttp2ConnectionAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken) in /_/src/libraries/System.Net.Http/src/System/Net/Http/SocketsHttpHandler/HttpConnectionPool.cs:line 789
   at System.Net.Http.HttpConnectionPool.TrySendUsingHttp2Async(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken) in /_/src/libraries/System.Net.Http/src/System/Net/Http/SocketsHttpHandler/HttpConnectionPool.cs:line 965
   at System.Net.Http.HttpConnectionPool.DetermineVersionAndSendAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken) in /_/src/libraries/System.Net.Http/src/System/Net/Http/SocketsHttpHandler/HttpConnectionPool.cs:line 1015
   at System.Net.Http.HttpConnectionPool.SendAndProcessAltSvcAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken) in /_/src/libraries/System.Net.Http/src/System/Net/Http/SocketsHttpHandler/HttpConnectionPool.cs:line 1028
   at System.Net.Http.HttpConnectionPool.SendWithRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken) in /_/src/libraries/System.Net.Http/src/System/Net/Http/SocketsHttpHandler/HttpConnectionPool.cs:line 1038
   at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken) in /_/src/libraries/System.Net.Http/src/System/Net/Http/SocketsHttpHandler/RedirectHandler.cs:line 30
   at System.Net.Http.HttpClient.<SendAsync>g__Core|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken) in /_/src/libraries/System.Net.Http/src/System/Net/Http/HttpClient.cs:line 553
Author: karelz
Assignees: -
Labels:

area-System.Net.Http, untriaged
Milestone: -

@karelz karelz added the test-run-core Test failures in .NET Core test runs label Jul 22, 2021
@karelz karelz added this to the 6.0.0 milestone Jul 22, 2021
@karelz
Copy link
Member Author

karelz commented Jul 22, 2021

cc @geoffkizer in case it is related to ConnectionPool changes ...

@karelz karelz added os-linux Linux OS (any supported distro) and removed untriaged New issue has not been triaged by the area owner labels Jul 22, 2021
@alnikola alnikola mentioned this issue Jul 29, 2021
Merged
@alnikola alnikola self-assigned this Jul 29, 2021
alnikola added a commit that referenced this issue Jul 30, 2021
@alnikola
Check _abortException before checking _shutdown flag (#56552)
facbdce 
In case of an error aborting the connection, there is a race between a thread creating new `Http2Stream` to send a request and the thread looping in `ProcessIncomingFrames` that sets _shutdown flag and `_abortException`. If the request thread first sees `_shutdown == true`, then it won't see the `_abortException` even if it's set, so the request will be retried when it shouldn't.

This PR adds `_abortException` check just before the `_shutdown == true` check to make sure an abort exception is observed.

Fixes #1581 
Fixes #56138
Fixes #56026
@ghost ghost locked as resolved and limited conversation to collaborators Aug 30, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees

@alnikola alnikola

Labels
area-System.Net.Http os-linux Linux OS (any supported distro) test-run-core Test failures in .NET Core test runs
Projects
None yet
Milestone
6.0.0
Development

Successfully merging a pull request may close this issue.

Check _abortException before checking _shutdown flag alnikola/runtime
2 participants
@karelz @alnikola