System.Net.NameResolution.Tests.GetHostAddressesTest_Cancellation.DnsGetHostAddresses_PostCancelledToken_Throws failing in CI

[jakobbotsch]

Failure: https://dev.azure.com/dnceng-public/public/_build/results?buildId=94183&view=ms.vss-test-web.build-test-results-tab
Console log: https://helixre107v0xdeko0k025g8.blob.core.windows.net/dotnet-runtime-refs-pull-78879-merge-4f7487e318c446cc92/System.Net.NameResolution.Functional.Tests/3/console.b90e962b.log?helixlogtype=result

This has failed before: #69993. Back then it was disabled in jitstress (in #70089), but note that no stress modes are set in the scenario that failed.

[ghost]

Tagging subscribers to this area: @dotnet/ncl
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Failure: https://dev.azure.com/dnceng-public/public/_build/results?buildId=94183&view=ms.vss-test-web.build-test-results-tab
Console log: https://helixre107v0xdeko0k025g8.blob.core.windows.net/dotnet-runtime-refs-pull-78879-merge-4f7487e318c446cc92/System.Net.NameResolution.Functional.Tests/3/console.b90e962b.log?helixlogtype=result

This has failed before: #69993. Back then it was disabled in jitstress (in #70089), but note that no stress modes are set in the scenario that failed.

Author:	jakobbotsch
Assignees:	-
Labels:	area-System.Net
Milestone:	-

[jakobbotsch]

There's 2 failures in innerloop within the past 3 weeks: https://runfo.azurewebsites.net/search/timelines/?q=started%3A%7E21+definition%3Aruntime+message%3ADnsGetHostAddresses_PostCancelledToken_Throws

8 failures if you include libraries-jitstress, most often under tailcallstress: https://runfo.azurewebsites.net/search/timelines/?q=started%3A%7E21+message%3ADnsGetHostAddresses_PostCancelledToken_Throws

[jakobbotsch]

@rzikm @wfurt @antonfirsov If the flakiness cannot be fixed, can we add some kind of retry mechanism in the test, so that we only fail if e.g. 5/5 tries fail?

[rzikm]

There is global retry mechanism for all networking tests, the CI fails the test if it fails 3 times in a row.

[wfurt]

We should kick it to outerloop e.g. same as DnsGetHostEntry_PostCancelledToken_Throws.
Curiously the TestSettings.UncachedHost is used in multiple places so it may not work with negative caching.
retries may have similar problem.

[rzikm]

Triage: All tests touching external host should be moved to outerloop. We should have a look in 8.0 timeframe

[carlossanlop]

I'm seeing this test fail in 6.0. @antonfirsov can you please disable the test in the servicing branches?

Unfortunately the original issue has very old links which are already gone, so I can't tell for sure if the failure was the same.

The failure I am seeing in 6.0 is weird, because only the build output shows up, but is no test result link being generated.

• PR: [release/6.0-staging] Update dependencies from dotnet/arcade dotnet/emsdk #84705
• Queue: Libraries Test Run checked coreclr windows x64 Debug
• Job: https://dev.azure.com/dnceng-public/public/_build/results?buildId=236706&view=logs&j=a68d6dca-d11c-59af-6c4a-cf852cb20e3a&t=28d89dec-7e09-5293-5fd2-aeb03e603264

  Uploading payload for D:\a\_work\1\s\.packages\microsoft.dotnet.helix.sdk\6.0.0-beta.23211.7\tools\azure-pipelines\reporter
  Uploading payload for C:\Users\cloudtest\AppData\Local\Temp\014c4abfe6534f26a0f156d48655d35e
  Created job list at https://helixde107v0xdeko0k025g8.blob.core.windows.net/helix-job-9831ff69-d2cf-4bae-bfb9-946b611bcbbd77c2af293af4a51bf/job-list-c7ea2267-2b1c-4c0b-b4d3-47a0c2ad07ad.json
  Sent Helix Job; see work items at https://helix.dot.net/api/jobs/e1e13316-1c99-4285-8544-eff1b3ee83f0/workitems?api-version=2019-06-17
  Waiting for completion of job e1e13316-1c99-4285-8544-eff1b3ee83f0 on Windows.10.Amd64.Open
  Job e1e13316-1c99-4285-8544-eff1b3ee83f0 on Windows.10.Amd64.Open is completed with 277 finished work items.
  Stopping Azure Pipelines Test Run net6.0-windows-Debug-x64-CoreCLR_checked-Windows.10.Amd64.Open
D:\a\_work\1\s\.packages\microsoft.dotnet.helix.sdk\6.0.0-beta.23211.7\tools\azure-pipelines\AzurePipelines.MultiQueue.targets(43,5): error : Test System.Net.NameResolution.Tests.GetHostAddressesTest_Cancellation.DnsGetHostAddresses_PostCancelledToken_Throws has failed. Check the Test tab for details. [D:\a\_work\1\s\src\libraries\sendtohelixhelp.proj]
##[error].packages\microsoft.dotnet.helix.sdk\6.0.0-beta.23211.7\tools\azure-pipelines\AzurePipelines.MultiQueue.targets(43,5): error : (NETCORE_ENGINEERING_TELEMETRY=Test) Test System.Net.NameResolution.Tests.GetHostAddressesTest_Cancellation.DnsGetHostAddresses_PostCancelledToken_Throws has failed. Check the Test tab for details.

Build FAILED.