Intermittent failure in AddProvider_InvokeTypeMultipleTimes_Refreshes test and AddProviderTransparent_InvokeTypeMultipleTimes_Refreshes

[AntonLapounov]

AddProvider_InvokeTypeMultipleTimes_Refreshes test failed in runtime (Libraries Test Run checked coreclr OSX x64 Debug) leg:

  Starting:    System.ComponentModel.TypeConverter.Tests (parallel test collections = on, max threads = 4)
    System.ComponentModel.Tests.TypeDescriptorTests.AddProvider_InvokeTypeMultipleTimes_Refreshes [FAIL]
      Assert.Equal() Failure
      Expected: 2
      Actual:   3

Tagging subscribers to this area: @safern
See info in area-owners.md if you want to be subscribed.

Issue Details

---

AddProvider_InvokeTypeMultipleTimes_Refreshes test failed in runtime (Libraries Test Run checked coreclr OSX x64 Debug) leg:

  Starting:    System.ComponentModel.TypeConverter.Tests (parallel test collections = on, max threads = 4)
    System.ComponentModel.Tests.TypeDescriptorTests.AddProvider_InvokeTypeMultipleTimes_Refreshes [FAIL]
      Assert.Equal() Failure
      Expected: 2
      Actual:   3

Author:	AntonLapounov
Assignees:	-
Labels:	area-System.ComponentModel
Milestone:	-

[safern]

Happened again in: https://dev.azure.com/dnceng/public/_build/results?buildId=1107359&view=ms.vss-test-web.build-test-results-tab&runId=33804176&paneView=debug

[VincentBu]

Failed in runtime-coreclr libraries-jitstress 20210428.1

Failed test:

net6.0-Linux-Release-x64-CoreCLR_checked-jitstress2_tiered-Ubuntu.1804.Amd64.Open
 -System.ComponentModel.Tests.TypeDescriptorTests.AddProviderTransparent_InvokeTypeMultipleTimes_Refreshes

Error message:

Assert.Equal() Failure
Expected: 2
Actual:   3


Stack trace
   at System.ComponentModel.Tests.TypeDescriptorTests.AddProviderTransparent_InvokeTypeMultipleTimes_Refreshes() in /_/src/libraries/System.ComponentModel.TypeConverter/tests/TypeDescriptorTests.cs:line 359

[danmoseley]

Console log: 'System.ComponentModel.TypeConverter.Tests' from job 6a590814-12cb-4b2a-a6e7-7ebffe9e04be (ubuntu.1804.armarch.open) using docker image mcr.microsoft.com/dotnet-buildtools/prereqs:ubuntu-18.04-helix-arm32v7-bfcd90a-20200121150440 on ddvsotx2l087
running $HELIX_CORRELATION_PAYLOAD/scripts/0120b789f9164388b2230151ee1a8067/execute.sh in /home/helixbot/work/ABC70A03/w/B022096C/e max 2700 seconds
...
===========================================================================================================
/root/helix/work/workitem /root/helix/work/workitem
  Discovering: System.ComponentModel.TypeConverter.Tests (method display = ClassAndMethod, method display options = None)
  Discovered:  System.ComponentModel.TypeConverter.Tests (found 1371 of 1372 test cases)
  Starting:    System.ComponentModel.TypeConverter.Tests (parallel test collections = on, max threads = 4)
    System.ComponentModel.Tests.TypeDescriptorTests.AddProviderTransparent_InvokeTypeMultipleTimes_Refreshes [FAIL]
      Assert.Equal() Failure
      Expected: 3
      Actual:   4
      Stack Trace:
        /_/src/libraries/System.ComponentModel.TypeConverter/tests/TypeDescriptorTests.cs(365,0): at System.ComponentModel.Tests.TypeDescriptorTests.AddProviderTransparent_InvokeTypeMultipleTimes_Refreshes()
  Finished:    System.ComponentModel.TypeConverter.Tests
=== TEST EXECUTION SUMMARY ===

net6.0-Linux-Release-arm-CoreCLR_checked-(Ubuntu.1804.Arm32.Open)Ubuntu.1804.Armarch.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:ubuntu-18.04-helix-arm32v7-bfcd90a-20200121150440

[danmoseley]

https://engsrvprod.kusto.windows.net/engineeringdata?query=H4sIAAAAAAAEAI1QwUrDQBC9B%2fIPQ04p2NLWmxDBg4UIlVoDHkTKtjtttk1248ykMeLHu0mLeuhBGNgZ9s17b16GLEvkuhCGMPiCvTMWDsbqxFiLBC%2bODqlgyeDsz5Dqi9gHt%2b5h%2fk11B2hyJISZsYZz1HCbgFaCYkqMp%2bPpZHg9nMD4ZtzVIAyU1RDPUXKnIUkgutN6Qe5oNNIqtUd3wKytcO6tmqrAzLPwaolbQk%2fOETiCy8sZKcuVIrTyD56zkVMmPdVMmSLqzsEPQf%2b1Z2cTz8e46trY61RIYpAHf1DbQh29pQTEsZCxu7gDv0bPLfsER4tcMT6qEqO3fqsit8eNDFWj2l7hN785MqsdAosi4cZI7s9j9pKj%2b%2fdanbxxXZaKzCfCxtVW4gGsW3iqse5Frs52wiAMvgEbuumM8gEAAA%3d%3d&web=0

TestResults
| join kind=inner WorkItems on WorkItemId
| join kind=inner Jobs on JobId
| where Finished >= datetime(2021-3-1 0:00:00)
and (Method == "AddProvider_InvokeTypeMultipleTimes_Refreshes" or Method == "AddProviderTransparent_InvokeTypeMultipleTimes_Refreshes")
and Result == "Fail"
| extend json=parse_json(Properties)
| extend flavor = tostring(json["System.PhaseName"])
| project-away json
| where Message startswith "Assert.Equal"
| summarize count() by QueueName, flavor

QueueName	flavor	count_
ubuntu.1604.amd64.open.rt	libraries_test_run_release_coreclr_Linux_x64_Debug	3
debian.9.amd64.open.rt	libraries_test_run_release_mono_interpreter_Linux_x64_Release	1
ubuntu.1804.armarch.open	libraries_test_run_checked_coreclr_Linux_arm_Release	7
osx.1014.amd64.open	libraries_test_run_release_mono_OSX_x64_Release	1
ubuntu.1804.armarch.open	libraries_test_run_checked_coreclr_Linux_musl_arm_Release	3
ubuntu.1804.amd64.open.rt	libraries_test_run_checked_coreclr_Linux_x64_Release	2
ubuntu.1804.armarch.open	libraries_test_run_release_mono_Linux_arm64_Debug	3
ubuntu.1604.amd64.open.rt	libraries_test_run_release_mono_Linux_x64_Debug	4
ubuntu.1804.armarch.open	libraries_test_run_release_coreclr_Linux_arm64_Release	2
windows.7.amd64.open.rt	libraries_test_run_release_coreclr_windows_x86_Debug	2
osx.1013.amd64.open	libraries_test_run_checked_coreclr_OSX_x64_Debug	4
osx.1015.amd64.open	libraries_test_run_release_mono_OSX_x64_Debug	2
sles.15.amd64.open.rt	libraries_test_run_release_mono_Linux_x64_Debug	1
osx.1014.amd64.open	libraries_test_run_release_mono_OSX_x64_Debug	2
redhat.7.amd64.open.rt	libraries_test_run_release_mono_Linux_x64_Debug	1
osx.1013.amd64.open	libraries_test_run_checked_coreclr_OSX_x64_Release	1
ubuntu.1804.armarch.open	libraries_test_run_checked_coreclr_Linux_musl_arm64_Release	2
osx.1015.amd64.open	libraries_test_run_release_coreclr_OSX_x64_Debug	1
redhat.7.amd64.open.rt	libraries_test_run_release_coreclr_Linux_x64_Debug	1
windows.10.amd64.server19h1.es.open.rt	build_windows_x64_Release_AllSubsets_Mono	1
ubuntu.1604.amd64.open.rt	libraries_test_run_release_coreclr_Linux_x64_Release	1
ubuntu.1604.amd64.open.rt	libraries_test_run_release_coreclr_Linux_musl_x64_Debug	2
osx.1014.amd64.open	libraries_test_run_release_coreclr_OSX_x64_Debug	1
windows.10.amd64.server19h1.es.open.rt	libraries_test_run_release_coreclr_windows_x86_Release	1

[danmoseley]

Clearly it's not OS, runtime, or build flavor dependent.

@safern I don't know this code -- I don't see any obvious threads - maybe some static cache that other tests are interfering with concurrently? any theories?