Update value under concurrency fails when a cancellation token is registered.

[troydai]

Scenario:

1. Set up a memory cache.
2. Add an entry with a cancellation token registered with it.
3. Start a child thread which keep reading the memory cache with the same key.
4. Reset the value of the entry from main thread.
5. Exception InvalidOperationException is thrown.

Issue repo:

https://github.com/troydai/CachingIssues/blob/2a85107b9b572c47d66d2a0303deafecaeaa3166/TestActions/ConcurrencyIssue184.cs#L17

Run the sample follow exception will be thrown from child thread after the value is updated by main thread.

Unhandled Exception: System.InvalidOperationException: Collection was modified; enumeration operation may not execute.
   at System.ThrowHelper.ThrowInvalidOperationException(ExceptionResource resource)
   at System.Collections.Generic.List`1.Enumerator.MoveNextRare()
   at Microsoft.Extensions.Caching.Memory.CacheEntry.PropageOptions(CacheEntry parent)
   at Microsoft.Extensions.Caching.Memory.MemoryCache.TryGetValue(Object key, Object& result)
   at Microsoft.Extensions.Caching.Memory.CacheExtensions.TryGetValue[TItem](IMemoryCache cache, Object key, TItem& value)
   at Microsoft.Extensions.Caching.Memory.CacheExtensions.Get[TItem](IMemoryCache cache, Object key)
   at ConsoleApplication.Program.<>c__DisplayClass5_0.<Monitor>b__0(Int32 round, DateTime beginning)
   at ConsoleApplication.Program.<>c__DisplayClass4_0.<Polling>b__0()
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)

[Eilon]

@troydai thanks - can you add a bit more info to this bug?

[troydai]

Yes. I'm adding except stack to the description. I did talk with @Tratcher and @sebastienros before I file the issue. He's has a rough idea what probably break this scneario.

[kevinchalet]

Since this ticket is marked for 1.0.0, I guess it was not fixed for RC2?

An unhandled exception occurred while processing the request.

InvalidOperationException: La collection a été modifiée ; l'opération d'énumération peut ne pas s'exécuter.
à System.ThrowHelper.ThrowInvalidOperationException(ExceptionResource resource)
à System.Collections.Generic.List`1.Enumerator.MoveNextRare()
à System.Collections.Generic.List`1.Enumerator.MoveNext()
à Microsoft.Extensions.Caching.Memory.CacheEntry.PropageOptions(CacheEntry parent)
à Microsoft.Extensions.Caching.Memory.CacheEntry.Dispose()
à Microsoft.Extensions.Caching.Memory.CacheExtensions.Set[TItem](IMemoryCache cache, Object key, TItem value, MemoryCacheEntryOptions options)
à Microsoft.AspNetCore.Mvc.Razor.RazorViewEngine.OnCacheMiss(ViewLocationExpanderContext expanderContext, ViewLocationCacheKey cacheKey)
à Microsoft.AspNetCore.Mvc.Razor.RazorViewEngine.LocatePageFromViewLocations(ActionContext actionContext, String pageName, Boolean isMainPage)
à Microsoft.AspNetCore.Mvc.Razor.RazorViewEngine.FindView(ActionContext context, String viewName, Boolean isMainPage)
à Microsoft.AspNetCore.Mvc.ViewEngines.CompositeViewEngine.FindView(ActionContext context, String viewName, Boolean isMainPage)
à Microsoft.AspNetCore.Mvc.ViewFeatures.Internal.ViewResultExecutor.FindView(ActionContext actionContext, ViewResult viewResult)
à Microsoft.AspNetCore.Mvc.ViewResult.<ExecuteResultAsync>d__26.MoveNext()

[Tratcher]

Correct. You managed to reproduce it in the wild? We only managed in contrived background threading scenarios.

[kevinchalet]

You managed to reproduce it in the wild?

Yep. Adding some "DB initialization" code at the end of Startup.Configure in a whole new MVC RC2 app seems enough to reproduce it almost consistently on my machine:

var factory = app.ApplicationServices.GetRequiredService<IServiceScopeFactory>();
using (var scope = factory.CreateScope()) {
    var context = scope.ServiceProvider.GetRequiredService<ApplicationDbContext>();
    context.Database.EnsureCreated();

    if (!context.Users.Any()) {
        // Skipped.
    }

    context.SaveChanges();
}

For reasons I can't explain (yet), this alternative pattern causes fewer exceptions:

var options = app.ApplicationServices.GetRequiredService<DbContextOptions<ApplicationDbContext>>();
using (var context = new ApplicationDbContext(options)) {
    context.Database.EnsureCreated();

    if (!context.Users.Any()) {
        // Skipped.
    }

    context.SaveChanges();
}

[kevinchalet]

Luckily, it seems to only impact the very first request, so it's probably not critical (just a bit annoying 😄)

[kevinchalet]

@Tratcher curious: can you reproduce it or is it just me? 😄

[sebastienros]

We were able to reproduce, we added some unit tests along with the fix if you want some ways to repro.
If you update to the latest version of Caching then you should be covered.

[kevinchalet]

If you update to the latest version of Caching then you should be covered.

Yep, but it would imply migrating to the nightly builds and this is something I'd really like to avoid for a public sample.

[gdoron]

I face it a lot, every first request after I start the application.
Can it happen in production IIS server or it's only with VisualStudio-IIS-Express?

[Tratcher]

@gdoron yes it can happen in production.