TokenCache expects that token providers send pre-sorted (alphabetically) scopes - Auth0 does not - which results in multiple_matching_tokens_detected

[torangel]

Library version used

4.5.7

.NET version

net4.8

Scenario

ConfidentialClient - web site (AcquireTokenByAuthCode)

Is this a new or an existing app?

This is a new app or experiment

Issue description and reproduction steps

MsalAccessTokenCacheItem seems to expect that the token returned from the token endpoint has the scopes ordered alphabetically.
The scopes are included in the cache-key - which results in cache miss if the scopes are sorted different in the response to AcquireTokenByAuthorizationCode vs the response to msal.AcquireTokenSilent(scopes, account).WithForceRefresh(true)

This in turn results in multiple access-tokens added to the cache - which becomes a problem when trying to obtain a token silently from the cache.

MSAL.Desktop.4.57.0.0.MsalClientException: ErrorCode: multiple_matching_tokens_detected Microsoft.Identity.Client.MsalClientException: The cache contains multiple tokens satisfying the requirements. Try to clear token cache. at Microsoft.Identity.Client.Internal.Requests.Silent.SilentRequest.d__5.MoveNext()

I have read through the Oauth2 spec without finding any requirements for how scopes should be ordered in the token response.

The picture below shows the responses from the code exchange and the refreshtoken exchange. The result of these two requests are two items in the token cache - with the same scopes, but with different order.

Relevant code snippets

//Setup application with "generic autority (auth0)
var msal = ConfidentialClientApplicationBuilder
                .Create(ClientId)
                .WithExperimentalFeatures(true)
                .WithRedirectUri(RedirectUri)
                .WithClientSecret(ClientSecret)
                .WithLegacyCacheCompatibility(false)
                .WithGenericAuthority($"https://{Tenant}")
                .WithLogging(new MyIdentityLogger(),true)
                .Build();

//Configure token caching using SQL server
msal.AddDistributedTokenCache(services => {
                services.AddDistributedTokenCaches();
                services.AddDistributedSqlServerCache(options => {
                    options.SchemaName = "dbo";
                    options.TableName = "TokenCache";
                    options.ConnectionString = TokenCacheSqlServerConnectionString;
                    options.DefaultSlidingExpiration = TimeSpan.FromDays(30);
                });
            });

//Exchange authcode with access-, id- and refreshtoken
await msal.AcquireTokenByAuthorizationCode(scopes, notification.ProtocolMessage.Code)
                        .ExecuteAsync();

//Force usage of refresh token
await msal.AcquireTokenSilent(scopes, account).WithForceRefresh(true).ExecuteAsync();

//try to aquire token from cache - this fails because of multiple access tokens in cache which have the same scopes
await msal.AcquireTokenSilent(scopes, account).ExecuteAsync()

Expected behavior

No response

Identity provider

Other

Regression

No response

Solution and workarounds

Not sure how this might affect existing cache-items, but either this library has to sort the returned scopes before creating the cache key, or the identity provider has to be consistent on the order of returned scopes.

Changing the order of the requested scopes does not seem to make any difference on the response from Auth0

[bgavrilMS]

MSAL's internal logic for looking in the cache treats scopes as a SET data structure. I think that comment in the code is just old..

https://github.com/AzureAD/microsoft-authentication-library-for-dotnet/blob/main/src/client/Microsoft.Identity.Client/TokenCache.ITokenCacheInternal.cs#L499

Can you please enable PII verbose logging and send us the logs? Happy to take them via email (bogavril at microsoft com) if posting them here is not acceptable. https://learn.microsoft.com/en-us/entra/msal/dotnet/advanced/exceptions/msal-logging?tabs=dotnet

Also https://learn.microsoft.com/en-us/entra/msal/dotnet/resources/handling-pii#what-msal-will-never-log

[torangel]

Email is sent :)

[bgavrilMS]

Got it. And it looks like you are correct, the token from auth_code has different scope order than the token from refresh_token and this leads to MSAL getting into a "multiple matching tokens" error.

AT Cache Key: auth0|sub-host-accesstoken---openid profile email all:catchreport offline_access
AT Cache Key: auth0|sub-host-accesstoken---email openid profile all:catchreport offline_access

Should be an easy fix.

[torangel]

Great :)

Thank you for your rapid response to this issue! Highly appreciated.

[ztepsic]

@torangel in his first post stated that scenario is: ConfidentialClient - web site (AcquireTokenByAuthCode), and I saw it in code snippets he provided that he is using WithGenericAuthority.

But based on this documentation This is an experimental API and only AcquireTokenForClient (client_credentials flow) has been implemented

AcquireTokenByAuthCode method is not supported with WithGenericAuthority.

Am I missing something here or?

[bgavrilMS]

@ztepsic - bad docs (now fixed, but the official docs will take some time to re-generate). All confidential client flows are supported included - web site (AcquireTokenByAuthCode), web api (AcquireTokenOnBehalfOf) and S2S (AcquireTokenForClient).

[CommCody]

Hi, we are experiencing the same issue in production.
When will this fix be released?

[bgavrilMS]

We plan to release MSAL 4.59 in the next few days, with the fix.

[CommCody]

Ok, thank you. Are there any intermediate fixes or workarounds we can apply? (We use Microsoft.Identity.Web)