Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Test failures: Microsoft.Extensions.Caching.Memory flakiness #33993

Closed
maryamariyan opened this issue Mar 24, 2020 · 10 comments · Fixed by #72821
Closed

Test failures: Microsoft.Extensions.Caching.Memory flakiness #33993

maryamariyan opened this issue Mar 24, 2020 · 10 comments · Fixed by #72821
Labels
area-Extensions-Caching blocking-clean-ci Blocking PR or rolling runs of 'runtime' or 'runtime-extra-platforms'
Milestone
Future

Comments

@maryamariyan
Copy link
Member

maryamariyan commented Mar 24, 2020 •
edited
Loading 

Microsoft.Extensions.Caching.Memory.MemoryCacheSetAndRemoveTests.GetAndSet_AreThreadSafe_AndUpdatesNeverLeavesNullValues [FAIL]
      Assert.False() Failure
      Expected: False
      Actual:   True
      Stack Trace:
        /_/src/libraries/Microsoft.Extensions.Caching.Memory/tests/MemoryCacheSetAndRemoveTests.cs(488,0): at Microsoft.Extensions.Caching.Memory.MemoryCacheSetAndRemoveTests.GetAndSet_AreThreadSafe_AndUpdatesNeverLeavesNullValues()
        /_/src/mono/netcore/System.Private.CoreLib/src/System/Reflection/RuntimeMethodInfo.cs(339,0): at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)

https://helix.dot.net/api/2019-06-17/jobs/e7ed99d0-5a25-4792-8b58-04c5a0009c69/workitems/Microsoft.Extensions.Caching.Memory.Tests/console

UPDATE:
Failures not just on Mono. The flakiness reported in #34128 and disabled in PR #34142
@Dotnet-GitSync-Bot Dotnet-GitSync-Bot added the untriaged New issue has not been triaged by the area owner label Mar 24, 2020
This was referenced Mar 26, 2020
Closed
Merged
@maryamariyan maryamariyan changed the title mono test failure: Microsoft.Extensions.Caching.Memory.MemoryCacheSetAndRemoveTests.GetAndSet_AreThreadSafe_AndUpdatesNeverLeavesNullValues Test failures: Microsoft.Extensions.Caching.Memory flakiness Mar 26, 2020
@maryamariyan maryamariyan mentioned this issue Mar 26, 2020
Closed
@ericstj
Copy link
Member

ericstj commented Mar 26, 2020

Duping this to #34128 since it has more data.

@ericstj ericstj closed this as completed Mar 26, 2020
@ericstj ericstj reopened this Mar 26, 2020
@ericstj
Copy link
Member

ericstj commented Mar 26, 2020

Reopen since tests are disabled against this

@jaredpar jaredpar mentioned this issue Mar 26, 2020
Merged
@ericstj
Copy link
Member

ericstj commented Mar 27, 2020 •
edited
Loading

There may be more tests failing than just these. I sampled some of the logs listed in #34128 and see a couple other tests failing:

https://helix.dot.net/api/2019-06-17/jobs/69862130-a475-4556-96d7-896801bfcfec/workitems/Microsoft.Extensions.Caching.Memory.Tests/files/console.ed98370f.log

Microsoft.Extensions.Caching.Memory.CapacityTests.CompactsToLessThanLowWatermarkUsingLRUWhenHighWatermarkExceeded [FAIL]
      Assert.True() Failure
      Expected: True
      Actual:   False
      Stack Trace:
        /_/src/libraries/Microsoft.Extensions.Caching.Memory/tests/CapacityTests.cs(378,0): at Microsoft.Extensions.Caching.Memory.CapacityTests.CompactsToLessThanLowWatermarkUsingLRUWhenHighWatermarkExceeded()

https://helix.dot.net/api/2019-06-17/jobs/7df59f03-64bf-479e-a50e-680591bfadde/workitems/Microsoft.Extensions.Caching.Memory.Tests/files/console.b7698877.log

    Microsoft.Extensions.Caching.Memory.CapacityTests.ExpiringEntryDecreasesCacheSize [FAIL]
      Assert.True() Failure
      Expected: True
      Actual:   False
      Stack Trace:
        /_/src/libraries/Microsoft.Extensions.Caching.Memory/tests/CapacityTests.cs(338,0): at Microsoft.Extensions.Caching.Memory.CapacityTests.ExpiringEntryDecreasesCacheSize()

I wonder if there is something broken with how we're testing these in dotnet/runtime? Maybe concurrency differences and shared state?

This was referenced Mar 27, 2020
Merged
Closed
@maryamariyan maryamariyan added this to the Future milestone Jun 12, 2020
@maryamariyan maryamariyan removed the untriaged New issue has not been triaged by the area owner label Jun 12, 2020
@BruceForstall BruceForstall mentioned this issue Jul 26, 2020
Closed
BruceForstall added a commit to BruceForstall/runtime that referenced this issue Jul 26, 2020
@BruceForstall
Disable Microsoft.Extensions.Caching.Memory.CapacityTests.DoNotAddIfS…
b6a3957 
…izeOverflows

Issue: dotnet#33993,
dotnet#39241
@BruceForstall BruceForstall mentioned this issue Jul 26, 2020
Merged
jkotas pushed a commit that referenced this issue Jul 26, 2020
@BruceForstall
Disable Microsoft.Extensions.Caching.Memory.CapacityTests.DoNotAddIfS…
0b16471 
…izeOverflows (#39930)

Issue: #33993,
#39241
@v-haren
Copy link

v-haren commented Jul 27, 2020

failed again in job: runtime-coreclr libraries-jitstress 20200726.1

failed test: Microsoft.Extensions.Caching.Memory.CapacityTests.DoNotAddIfSizeOverflows

Error message

Assert.True() Failure
Expected: True
Actual:   False


Stack trace
   at Microsoft.Extensions.Caching.Memory.CapacityTests.DoNotAddIfSizeOverflows() in /_/src/libraries/Microsoft.Extensions.Caching.Memory/tests/CapacityTests.cs:line 137
--- End of stack trace from previous location ---

Jacksondr5 pushed a commit to Jacksondr5/runtime that referenced this issue Aug 10, 2020
@BruceForstall @Jacksondr5
Disable Microsoft.Extensions.Caching.Memory.CapacityTests.DoNotAddIfS…
0fc001a 
…izeOverflows (dotnet#39930)

Issue: dotnet#33993,
dotnet#39241
@maryamariyan maryamariyan added this to Uncommitted in ML, Extensions, Globalization, etc, POD. via automation Oct 7, 2020
@maryamariyan maryamariyan self-assigned this Oct 7, 2020
@maryamariyan maryamariyan moved this from Uncommitted to Future in ML, Extensions, Globalization, etc, POD. Feb 3, 2021
@danmoseley
Copy link
Member

Duping arbitrarily #50270

ML, Extensions, Globalization, etc, POD. automation moved this from Future to Done Jun 22, 2021
@jkotas
Copy link
Member

jkotas commented Jun 22, 2021

We still have 7 tests disabled against this issue.

@jkotas jkotas reopened this Jun 22, 2021
@danmoseley
Copy link
Member

More commentary in #50270

This was referenced Jun 22, 2021
Closed
Merged
@maryamariyan maryamariyan removed their assignment Feb 27, 2022
@danmoseley danmoseley mentioned this issue Jul 26, 2022
Merged
@ghost ghost added the in-pr There is an active PR which will close this issue when it is merged label Jul 26, 2022
@ghost ghost removed the in-pr There is an active PR which will close this issue when it is merged label Jul 26, 2022
@jakobbotsch
Copy link
Member

Failed again in https://dev.azure.com/dnceng/public/_build/results?buildId=1903569&view=ms.vss-test-web.build-test-results-tab&runId=49497968&resultId=111528&paneView=dotnet-dnceng.dnceng-anon-build-release-tasks.helix-anon-test-information-tab after the reenabling

@jakobbotsch jakobbotsch reopened this Jul 26, 2022
@jakobbotsch jakobbotsch added the blocking-clean-ci Blocking PR or rolling runs of 'runtime' or 'runtime-extra-platforms' label Jul 26, 2022
@runfoapp runfoapp bot mentioned this issue Jul 26, 2022
Closed
@danmoseley
Copy link
Member

I'll disable that asap and create a new issue, for clarity, as this has covered other symptoms...

@danmoseley
Copy link
Member

#72879

@ghost ghost locked as resolved and limited conversation to collaborators Aug 26, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
area-Extensions-Caching blocking-clean-ci Blocking PR or rolling runs of 'runtime' or 'runtime-extra-platforms'
Projects
None yet
Milestone
Future
Development

Successfully merging a pull request may close this issue.

Fix MemoryCache test failures due to race danmoseley/runtime
7 participants
@maryamariyan @danmoseley @jkotas @jakobbotsch @ericstj @v-haren @Dotnet-GitSync-Bot