Skip to content

bug: concurrent /identity/connect/token crashes Identity on premium license cache race #8320

Description

@georglauterbach

Steps To Reproduce

  1. Run a self-hosted Bitwarden lite instance (v2026.8.1) with a premium user and a valid license at /etc/bitwarden/licenses/user/{selfHostedUserId}.json.
  2. Log in on a mobile client (reproduced with Bitwarden iOS 2026.8.0) so the client holds a refresh token.
  3. Trigger two overlapping POST /identity/connect/token requests for that user (refresh-token grant). This happens naturally when the iOS app retries / opens in the background; it can also be done with two parallel curls using the same refresh token.
  4. Observe Identity logs and the client receiving HTTP 504 / an unexpected error.

Expected Result

Token issuance succeeds (or returns a normal OAuth error). Concurrent refresh requests for the same premium user must not crash Identity.

Actual Result

The first overlapping request can succeed or fail; the second throws an unhandled exception while building the access token. Identity then serves 504s until the process restarts.

System.ArgumentException: An item with the same key has already been added. Key: {selfHostedUserId}
   at System.Collections.Generic.Dictionary`2.Add(...)
   at Bit.Core.Billing.Services.LicensingService.ValidateUserPremiumAsync(User user)
      in /source/src/Core/Billing/Services/Implementations/LicensingService.cs:line 249
   at Bit.Identity.IdentityServer.ProfileService.GetProfileDataAsync(...)
   at Duende.IdentityServer.ResponseHandling.TokenResponseGenerator.ProcessRefreshTokenRequestAsync(...)

Follow-up requests then hit:

System.InvalidOperationException: Operations that change non-concurrent collections must have exclusive access. A concurrent update was performed on this collection and corrupted its state.
   at Bit.Core.Billing.Services.LicensingService.ValidateUserPremiumAsync(User user)

This is not a duplicate license file. There is a single user/{selfHostedUserId}.json. The dictionary key is the self-hosted user id; the Id inside the JSON is the cloud user id (expected).

Screenshots or Videos

No response

Additional Context

LicensingService keeps an in-memory cache:

private IDictionary<Guid, DateTime> _userCheckCache = new Dictionary<Guid, DateTime>();

In ValidateUserPremiumAsync (still present on master):

if (_userCheckCache.TryGetValue(user.Id, out var lastCheck))
{
    // ...
}
else
{
    _userCheckCache.Add(user.Id, now); // race
}

Two concurrent token requests for the same premium user both miss the cache and both call .Add. The unhandled exception aborts /identity/connect/token.

Suggested fix: use ConcurrentDictionary<Guid, DateTime> (or IMemoryCache) and TryAdd / indexer assignment instead of Dictionary.Add.

Related: #4274 (same stack traces; closed as an email/license mismatch; the race was not fixed).

Build Version

Bitwarden lite 2026.8.1 (ghcr.io/bitwarden/lite). Reproduced against server tag v2026.8.1; the racy _userCheckCache.Add is still on master.

Environment

Self-Hosted

Environment Details

  • Deployment: Kubernetes, Bitwarden lite (single container)
  • Identity listens on http://+:5005
  • Premium individual license (not an organization)
  • PostgreSQL 18
  • ref License status corruption #4274

Issue Tracking Info

  • I understand that work is tracked outside of Github. A PR will be linked to this issue should one be opened to address it, but Bitwarden doesn't use fields like "assigned", "milestone", or "project" to track progress.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions