CreateDefaultBuilder_SecretsDoesReload test failed in CI #48696

stephentoub · 2021-02-24T12:00:27Z

https://helixre8s23ayyeko0k025g8.blob.core.windows.net/dotnet-runtime-refs-pull-48684-merge-1dda5503a115420687/Microsoft.Extensions.Hosting.Unit.Tests/console.b5b90d65.log?sv=2019-07-07&se=2021-03-16T05%3A35%3A14Z&sr=c&sp=rl&sig=ytXzgYX%2F6x0X9rBYl8sm%2B1JpNoEfEPjQFQhlMtH69tU%3D

  Discovering: Microsoft.Extensions.Hosting.Unit.Tests (method display = ClassAndMethod, method display options = None)
  Discovered:  Microsoft.Extensions.Hosting.Unit.Tests (found 99 test cases)
  Starting:    Microsoft.Extensions.Hosting.Unit.Tests (parallel test collections = on, max threads = 2)
�[40m�[32minfo�[39m�[22m�[49m: Microsoft.Extensions.Hosting.Tests.HostTests[0]
      Request starting
    Microsoft.Extensions.Hosting.Tests.HostTests.CreateDefaultBuilder_SecretsDoesReload [FAIL]
      Assert.Equal() Failure
                                    â†“ (pos 20)
      Expected: Hello ASP.NET Core: f577b280d3814d6c935970a1aadbb8e6
      Actual:   Hello ASP.NET Core: 407a83ef08c04e1ca1e07690cb30dc0a
                                    â†‘ (pos 20)
      Stack Trace:
        /_/src/libraries/Microsoft.Extensions.Hosting/tests/UnitTests/HostTests.cs(264,0): at Microsoft.Extensions.Hosting.Tests.HostTests.CreateDefaultBuilder_SecretsDoesReload()
        --- End of stack trace from previous location ---
  Finished:    Microsoft.Extensions.Hosting.Unit.Tests

The text was updated successfully, but these errors were encountered:

ghost · 2021-02-24T12:00:30Z

Tagging subscribers to this area: @eerhardt, @maryamariyan
See info in area-owners.md if you want to be subscribed.

Issue Details

https://helixre8s23ayyeko0k025g8.blob.core.windows.net/dotnet-runtime-refs-pull-48684-merge-1dda5503a115420687/Microsoft.Extensions.Hosting.Unit.Tests/console.b5b90d65.log?sv=2019-07-07&se=2021-03-16T05%3A35%3A14Z&sr=c&sp=rl&sig=ytXzgYX%2F6x0X9rBYl8sm%2B1JpNoEfEPjQFQhlMtH69tU%3D

  Discovering: Microsoft.Extensions.Hosting.Unit.Tests (method display = ClassAndMethod, method display options = None)
  Discovered:  Microsoft.Extensions.Hosting.Unit.Tests (found 99 test cases)
  Starting:    Microsoft.Extensions.Hosting.Unit.Tests (parallel test collections = on, max threads = 2)
�[40m�[32minfo�[39m�[22m�[49m: Microsoft.Extensions.Hosting.Tests.HostTests[0]
      Request starting
    Microsoft.Extensions.Hosting.Tests.HostTests.CreateDefaultBuilder_SecretsDoesReload [FAIL]
      Assert.Equal() Failure
                                    â†“ (pos 20)
      Expected: Hello ASP.NET Core: f577b280d3814d6c935970a1aadbb8e6
      Actual:   Hello ASP.NET Core: 407a83ef08c04e1ca1e07690cb30dc0a
                                    â†‘ (pos 20)
      Stack Trace:
        /_/src/libraries/Microsoft.Extensions.Hosting/tests/UnitTests/HostTests.cs(264,0): at Microsoft.Extensions.Hosting.Tests.HostTests.CreateDefaultBuilder_SecretsDoesReload()
        --- End of stack trace from previous location ---
  Finished:    Microsoft.Extensions.Hosting.Unit.Tests

Author:	stephentoub
Assignees:	-
Labels:	`area-Extensions-Hosting`
Milestone:	-

eerhardt · 2021-02-24T14:55:57Z

Looking at the test log, it looks like that test took 1 minute to run:

Looking at the test, it looks like it is waiting for 1 minute:

runtime/src/libraries/Microsoft.Extensions.Hosting/tests/UnitTests/HostTests.cs

Lines 211 to 218 in 5a3d44e

    
                       config.GetReloadToken().RegisterChangeCallback(o => 
        
                       { 
        
                           configReloadedCancelTokenSource.Cancel(); 
        
                       }, null); 
        
                       // Wait for up to 1 minute, if config reloads at any time, cancel the wait. 
        
                       await Task.WhenAny(Task.Delay(TimeSpan.FromMinutes(1), configReloadedCancelToken)); // Task.WhenAny ignores the task throwing on cancellation. 
        
                       Assert.NotEqual(dynamicConfigMessage1, dynamicConfigMessage2); // Messages are different. 
        
                       Assert.Equal(dynamicConfigMessage2, config["Hello"]); // Config DID reload from disk

So this leads me to believe that the change callback was never invoked, which is why we are getting stale data in the config.

stephentoub · 2021-03-29T18:01:13Z

Failed again here:
https://dev.azure.com/dnceng/public/_build/results?buildId=1061100&view=ms.vss-test-web.build-test-results-tab&runId=32699320&paneView=attachments&resultId=181769
https://helixre8s23ayyeko0k025g8.blob.core.windows.net/dotnet-runtime-refs-pull-50357-merge-ede7a005545e42cf8b/Microsoft.Extensions.Hosting.Unit.Tests/console.39af16ec.log?sv=2019-07-07&se=2021-04-18T11%3A42%3A53Z&sr=c&sp=rl&sig=k%2BL7uF4zgHywaVYUegb2Pi%2F%2F7AIejrTog53v%2FftznF0%3D

  Discovering: Microsoft.Extensions.Hosting.Unit.Tests (method display = ClassAndMethod, method display options = None)
  Discovered:  Microsoft.Extensions.Hosting.Unit.Tests (found 100 test cases)
  Starting:    Microsoft.Extensions.Hosting.Unit.Tests (parallel test collections = on, max threads = 2)
    Microsoft.Extensions.Hosting.Tests.HostTests.CreateDefaultBuilder_SecretsDoesReload [FAIL]
      Assert.Equal() Failure
                                    â†“ (pos 20)
      Expected: Hello ASP.NET Core: 1466b2f4da5249bdaef71d3aeee50e47
      Actual:   Hello ASP.NET Core: 8ab58f24c0c34494a4a4969709dc8a26
                                    â†‘ (pos 20)
      Stack Trace:
        /_/src/libraries/Microsoft.Extensions.Hosting/tests/UnitTests/HostTests.cs(264,0): at Microsoft.Extensions.Hosting.Tests.HostTests.CreateDefaultBuilder_SecretsDoesReload()
        --- End of stack trace from previous location ---
�[40m�[32minfo�[39m�[22m�[49m: Microsoft.Extensions.Hosting.Tests.HostTests[0]
      Request starting
  Finished:    Microsoft.Extensions.Hosting.Unit.Tests

maryamariyan · 2021-03-31T00:55:44Z

The test was added in #43807

maryamariyan · 2021-04-06T16:25:56Z

Removed blocking-clean-ci as the ActiveIssue attribute is now added to the test.

Keeping issue open to track re-enabling it properly

There appeared to be two issues, where the config.GetReloadToken().RegisterChangeCallback was registered after the updates were made.

ViktorHofer · 2021-04-27T15:55:51Z

Happened again in https://dev.azure.com/dnceng/public/_build/results?buildId=1109669&view=ms.vss-test-web.build-test-results-tab&runId=33849264&resultId=182792&paneView=debug

  Discovering: Microsoft.Extensions.Hosting.Unit.Tests (method display = ClassAndMethod, method display options = None)
  Discovered:  Microsoft.Extensions.Hosting.Unit.Tests (found 108 of 109 test cases)
  Starting:    Microsoft.Extensions.Hosting.Unit.Tests (parallel test collections = on, max threads = 2)
    Microsoft.Extensions.Hosting.Tests.HostTests.CreateDefaultBuilder_SecretsDoesReload [FAIL]
      Assert.Equal() Failure
                                    â†“ (pos 20)
      Expected: Hello ASP.NET Core: fd88495f2f6f4395a50b8a8bb9002821
      Actual:   Hello ASP.NET Core: 96d471cc9668492aa1d5528c5d782693
                                    â†‘ (pos 20)
      Stack Trace:
        /_/src/libraries/Microsoft.Extensions.Hosting/tests/UnitTests/HostTests.cs(315,0): at Microsoft.Extensions.Hosting.Tests.HostTests.CreateDefaultBuilder_SecretsDoesReload()
        --- End of stack trace from previous location ---
�[40m�[32minfo�[39m�[22m�[49m: Microsoft.Extensions.Hosting.Tests.HostTests[0]
      Request starting
  Finished:    Microsoft.Extensions.Hosting.Unit.Tests

Configuration: net6.0-Linux-Debug-x64-CoreCLR_release-Ubuntu.1604.Amd64.Open

ViktorHofer · 2021-04-27T15:56:02Z

cc @IEvangelist

bartonjs · 2021-06-21T23:36:15Z

And again. https://dev.azure.com/dnceng/public/_build/results?buildId=1179018&view=ms.vss-test-web.build-test-results-tab&runId=35916356&resultId=117977&paneView=debug

/datadisks/disk1/work/C7C50B48/w/B068098D/e /datadisks/disk1/work/C7C50B48/w/B068098D/e
  Discovering: Microsoft.Extensions.Hosting.Unit.Tests (method display = ClassAndMethod, method display options = None)
  Discovered:  Microsoft.Extensions.Hosting.Unit.Tests (found 109 of 110 test cases)
  Starting:    Microsoft.Extensions.Hosting.Unit.Tests (parallel test collections = on, max threads = 2)
    Microsoft.Extensions.Hosting.Tests.HostTests.CreateDefaultBuilder_SecretsDoesReload [FAIL]
      Assert.Equal() Failure
                                    â†“ (pos 20)
      Expected: Hello ASP.NET Core: d05cbd4bd6cf4e89b5e5a16267814a60
      Actual:   Hello ASP.NET Core: 5d71a4fa37c348828f14c431f252b45e
                                    â†‘ (pos 20)
      Stack Trace:
        /_/src/libraries/Microsoft.Extensions.Hosting/tests/UnitTests/HostTests.cs(316,0): at Microsoft.Extensions.Hosting.Tests.HostTests.CreateDefaultBuilder_SecretsDoesReload()
        --- End of stack trace from previous location ---
�[40m�[32minfo�[39m�[22m�[49m: Microsoft.Extensions.Hosting.Tests.HostTests[0]
      Request starting
  Finished:    Microsoft.Extensions.Hosting.Unit.Tests
=== TEST EXECUTION SUMMARY ===
   Microsoft.Extensions.Hosting.Unit.Tests  Total: 115, Errors: 0, Failed: 1, Skipped: 0, Time: 3.639s

eerhardt · 2022-07-19T21:43:34Z

Closing as this hasn't failed in over a year. Please re-open if this happens again.

stephentoub added the area-Extensions-Hosting label Feb 24, 2021

dotnet-issue-labeler bot added the untriaged New issue has not been triaged by the area owner label Feb 24, 2021

ghost added this to Untriaged in ML, Extensions, Globalization, etc, POD. Feb 24, 2021

maryamariyan mentioned this issue Mar 1, 2021

Test failure: Microsoft.Extensions.Hosting.Tests.HostTests.CreateDefaultBuilder_ConfigJsonDoesReload #44280

Closed

maryamariyan removed the untriaged New issue has not been triaged by the area owner label Mar 1, 2021

maryamariyan added this to the 6.0.0 milestone Mar 1, 2021

ghost moved this from Untriaged to 6.0.0 in ML, Extensions, Globalization, etc, POD. Mar 1, 2021

maryamariyan self-assigned this Mar 1, 2021

This was referenced Mar 31, 2021

Add ConfigurationKeyNameAttribute to binder and updated tests. #50338

Merged

Added the ConfigureDefaults API, as an extension method of IHostBuilder #50447

Merged

safern added the blocking-clean-ci Blocking PR or rolling runs of 'runtime' or 'runtime-extra-platforms' label Mar 31, 2021

runfoapp bot mentioned this issue Mar 31, 2021

Infrastructure - Status/Health #702

Closed

maryamariyan mentioned this issue Mar 31, 2021

Unblock clean CI #50494

Merged

eerhardt mentioned this issue Apr 6, 2021

Microsoft.Extensions.Hosting.Tests.HostTests.CreateDefaultBuilder_SecretsDoesReload fails on Linux #50751

Closed

maryamariyan removed the blocking-clean-ci Blocking PR or rolling runs of 'runtime' or 'runtime-extra-platforms' label Apr 6, 2021

maryamariyan added the help wanted [up-for-grabs] Good issue for external contributors label Apr 6, 2021

IEvangelist added a commit that referenced this issue Apr 19, 2021

Fix issue #48696

44a0622

There appeared to be two issues, where the config.GetReloadToken().RegisterChangeCallback was registered after the updates were made.

IEvangelist mentioned this issue Apr 19, 2021

Fix issue #48696 #51520

Merged

ghost added the in-pr There is an active PR which will close this issue when it is merged label Apr 19, 2021

IEvangelist closed this as completed in #51520 Apr 21, 2021

ML, Extensions, Globalization, etc, POD. automation moved this from 6.0.0 to Done Apr 21, 2021

IEvangelist added a commit that referenced this issue Apr 21, 2021

Fix issue #48696 (#51520)

79fdeb8

There appeared to be two issues, where the config.GetReloadToken().RegisterChangeCallback was registered after the updates were made.

ghost removed the in-pr There is an active PR which will close this issue when it is merged label Apr 21, 2021

ViktorHofer reopened this Apr 27, 2021

ViktorHofer mentioned this issue Apr 27, 2021

Stop harvesting old frameworks in more libraries #51894

Merged

maryamariyan moved this from Done to 6.0.0 in ML, Extensions, Globalization, etc, POD. Apr 29, 2021

IEvangelist mentioned this issue May 7, 2021

Fix CreateDefaultBuilder_SecretsDoesReload test failure #52466

Closed

ghost added the in-pr There is an active PR which will close this issue when it is merged label May 7, 2021

ghost removed the in-pr There is an active PR which will close this issue when it is merged label Jun 7, 2021

eerhardt modified the milestones: 6.0.0, 7.0.0 Jul 22, 2021

ghost moved this from 6.0.0 to Untriaged in ML, Extensions, Globalization, etc, POD. Jul 22, 2021

maryamariyan moved this from Untriaged to 7.0.0 in ML, Extensions, Globalization, etc, POD. Aug 4, 2021

maryamariyan removed their assignment Feb 27, 2022

eerhardt closed this as completed Jul 19, 2022

ghost locked as resolved and limited conversation to collaborators Aug 19, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

CreateDefaultBuilder_SecretsDoesReload test failed in CI #48696

CreateDefaultBuilder_SecretsDoesReload test failed in CI #48696

stephentoub commented Feb 24, 2021

ghost commented Feb 24, 2021

eerhardt commented Feb 24, 2021

stephentoub commented Mar 29, 2021

maryamariyan commented Mar 31, 2021

maryamariyan commented Apr 6, 2021

ViktorHofer commented Apr 27, 2021

ViktorHofer commented Apr 27, 2021

bartonjs commented Jun 21, 2021

eerhardt commented Jul 19, 2022

CreateDefaultBuilder_SecretsDoesReload test failed in CI #48696

CreateDefaultBuilder_SecretsDoesReload test failed in CI #48696

Comments

stephentoub commented Feb 24, 2021

ghost commented Feb 24, 2021

eerhardt commented Feb 24, 2021

stephentoub commented Mar 29, 2021

maryamariyan commented Mar 31, 2021

maryamariyan commented Apr 6, 2021

ViktorHofer commented Apr 27, 2021

ViktorHofer commented Apr 27, 2021

bartonjs commented Jun 21, 2021

eerhardt commented Jul 19, 2022