Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

AADSTS54005: OAuth2 Authorization code was already redeemed, please retry with a new valid code or use an existing refresh token. #2612

Closed
nromano32 opened this issue Dec 7, 2023 · 16 comments
Labels
question Further information is requested

Comments

@nromano32
Copy link

nromano32 commented Dec 7, 2023

Microsoft.Identity.Web Library

Microsoft.Identity.Web

Microsoft.Identity.Web version

2.11.0

Web app

Sign-in users

Web API

Protected web APIs (validating tokens)

Token cache serialization

Distributed caches

Description

We have a Microsoft Identity app and a Google Identity app running as independent sites on an AWS Elastic Beanstalk linux Instance. The Microsoft app hits a multi tenant Graph app. Each app uses a unique redirect url (i.e microsoft-signin-oidc and google-signin-oidc). We have configured the nginx proxy at 5000 for Microsoft and 5200 for google. We also are using Data protection to ensure the sites stay separated.

services.AddDataProtection()
.SetApplicationName(string.Concat("Identity.Microsoft.", EnvSettings.env))
.PersistKeysToAWSSystemsManager(string.Concat("/Identity/Microsoft/", EnvSettings.env));

Forward headers are added

services.Configure(options =>
{
options.ForwardedHeaders =
ForwardedHeaders.XForwardedFor | ForwardedHeaders.XForwardedProto;
});

The app works fine until it does not. It is like the app get in a weird state were regardless of user we try we still get the same error. Even users that hav not been used before. Just to test we put code to send a message to slack when each handle is entered we received the below

🔴 HandleOnRedirectToIdentityProvider
4:53
🔴 HandleOnRedirectToIdentityProvider
4:53
🔴 HandleOnRedirectToIdentityProvider
4:53
🔴 HandleOnRedirectToIdentityProvider
4:53
🔴 HandleOnRedirectToIdentityProvider
4:53
🔴 HandleOnRedirectToIdentityProvider
4:53
🔴 HandleOnAuthenticationFailed
4:53
🔴 HandleOnAuthenticationFailed
4:53
🔴 HandleOnAuthenticationFailed

From this it looks like the auth call is being run multiple times during a single user login.

This system is in production but not yet adopted. Our sales and development teams hit the site throughout the day so it is possible that two people from two different ip's might be accessing the site at the same time.

This app really does little more then get the token for backend processing, which seems to not be effected by this error. Literally all the app does is log in save the token and present the welcome screen. It calls the following o n the HomeController.

BaseBearerTokenAuthenticationProvider authenticationProvider =
new BaseBearerTokenAuthenticationProvider(new TokenProvider(_tokenAcquisition, new string[] { Base.Constants.ScopeUserRead }));

GraphServiceClient graphServiceClient = new GraphServiceClient(authenticationProvider);

User me = graphServiceClient.Me.GetAsync().Result

We tried to generate the codeChallenge and codeVerifier and add them to the Auth call but this has know effect. We trapped the call sent from the framework

https://login.microsoftonline.com/common/oauth2/v2.0/authorize?client_id=xxxxxxxx&redirect_uri=https%3A%2F%2Flocalhost%2Fmicrosoft-signin-oidc&response_type=code&scope=openid%20profile%20offline_access%20user.read%20MailboxSettings.ReadWrite&**code_challenge=MeZ6h09gO-heuRP1zLevDe6VURBTpbnAoWM_P9CXuJM&code_challenge_method=S256**&response_mode=form_post&nonce=638375799326446837.ZWFhOGRmMmEtMzZkYS00MGZjLWJhMjAtODNjZDY3OWIwYzQzYTNlNjhlYjAtNWE5ZS00MjM2LTgzODEtMzE1MTBkMzFiNTlk&client_info=1&x-client-brkrver=IDWeb.2.11.0.0&state=CfDJ8OeX6EEAQktPhE3LOQ-h3_vcIWWR6AO3E0gJ21ekdGkEPfSWXRp9woCvnZo9Ruj_hjuqECAhOtwo8F_k8TXKE23_z5Zg2WT6u6IGInpAPhO4DIYNozI7vkB1z-9EUHat1mGdT-_jO4-tobU91J8Flam5x8dbfcTPyKLaIjI0d2ZDq_OtHLb1oTx095SZfs-g2k66s6XwQaDTtA_glkx-GzmOFHht5LPZOIy-l0xa6hEpq6DGLOw1ZGDVKAyN3Sk9ynZUdFx1YVzCPUfZFg3pUSRqYF_Ah8eRmR3qPqaCYH42F50aJ9utxU8g8oQDOjV77RQbZUaldCKV24wh99YAYdS2lUU3Ez0Rz_wlG0zzjKjXsegrtx_RVOYwSouHWhyU5w&x-client-SKU=ID_NET6_0&x-client-ver=7.0.3.0

So it is obviously using PKCE. So in the error state we trapped two different Auth call to see the codeChallenge and codeVerifier change and they do.

https://login.microsoftonline.com/common/oauth2/v2.0/authorize?client_id=xxxxxxx&redirect_uri=https%3A%2F%2Fident-staging.xxxxxx.com%2Fmicrosoft-signin-oidc&response_type=code&scope=openid%20profile%20offline_access%20user.read%20MailboxSettings.ReadWrite&code_challenge=-yCoWJWrRn54ke8pvBc2LcSTTLJ0F9Xw4OtTwncIxPU&code_challenge_method=S256&response_mode=form_post&nonce=638375805082914538.YTEyNjMxMzEtYzU4ZC00MDExLTliOTUtNGJlMjA5OTYwMWQxYzc3ZWJiM2EtM2IyNC00MjI4LThiZmEtMjdkMWQ3N2NmNWVi&client_info=1&x-client-brkrver=IDWeb.2.11.0.0&state=CfDJ8Mup8ueqXQ5Fi4VDrywijDbktJ-c1JmFKM26cKVT9O5r46W1L3yBPn33Tx-LtStqdJzEIKA3ZPr9mLIuWEqB5GMRTx568QvMKo7zqz4_xdvl_IDP0xtbAKmsDeLM6xM7BTDpBKfj1vze5SVu7TKT-yh4jHJdrIKsw1BKjcYU1Je97j8poZkxhutjjGaE7rvZiIR3CU7tycEJj_q0TJtUkzhrtfxE05PD82Na2npcM3RUr_RmveE8SFpxgRCzGFfLe7ritl54eXMupr8KKYvbwLO_sg-hmIpwzr1JzhXNBDV2maw45LB4hFlq8VmBEenaBloroGb-L2TmqQVpOB57_KffX5BC7j0ixx4x2gpB0m72Cv0iJ8LO-WUk5b4yuulRFByulMG2kdaG5gpEZHRbPCE&x-client-SKU=ID_NET6_0&x-client-ver=7.0.3.0

https://login.microsoftonline.com/common/oauth2/v2.0/authorize?client_id=xxxxxxx&redirect_uri=https%3A%2F%2Fident-staging.xxxxxx.com%2Fmicrosoft-signin-oidc&response_type=code&scope=openid%20profile%20offline_access%20user.read%20MailboxSettings.ReadWrite&code_challenge=TuPqCCQoRI2rexcuJYit7ElBW7Kl8rodxy7mMpa4AiY&code_challenge_method=S256&response_mode=form_post&nonce=638375807209772604.YTI5ZDAzYzUtMWM3NC00YWQ2LTk1YmItZTg0ZDQ0MTUxM2Y4ZmYwMGIxNGEtMTc0Yi00ZGM3LWIxNDItNzQ3YjBmMjZlNzk1&client_info=1&x-client-brkrver=IDWeb.2.11.0.0&state=CfDJ8Mup8ueqXQ5Fi4VDrywijDa486AHI8qsNI7gmpSe1WQY-RT2hhK4HRUpcwOZ62zC2r3o2iJgWOA5k57OqJ19XobiU_mjmydVQywKB8POu9FPm-1KXo4fwEQ4d4_UgZ8Z3LgIKAp2U4FJKVq-soZh9Hn6Q9jcEg-wvQNRFH38Xnvr_7lOR3pr9LNlkTENOV4UJg5wgFC-urdycYEozUIzIXGe-tsmnVJHQS8jFMuSe4W3_FBf4IKKKlL6CUnU18ExKX7z0B4rFGVX_5gzfU4gMYbvW0s3Bw4vFPV_ttyUoAWPBp2vQqNqVZCUoLofeJk3FclgS865_t07itua8LifLA93bhT_VqGTasgli7VoPtQbkTWEKiSBrQpc7rueSQFGkDZVXSJaJejX3StKycX6rzQ&x-client-SKU=ID_NET6_0&x-client-ver=7.0.3.0

Our development environments do not see this issue. Happy to answer any questions. We are completely stumped.

Reproduction steps

I honest do not know ... my best guess is it is a time or user collision issue.

Error message

An unhandled exception occurred while processing the request.
MsalUiRequiredException: AADSTS54005: OAuth2 Authorization code was already redeemed, please retry with a new valid code or use an existing refresh token. Trace ID: f1bfb317-8f31-4e83-bc10-69173a8c4b00 Correlation ID: 9d70e458-5694-45d7-a914-34a63e0898a0 Timestamp: 2023-12-07 20:07:42Z
Microsoft.Identity.Client.OAuth2.OAuth2Client.ThrowServerException(HttpResponse response, RequestContext requestContext)

Exception: An error was encountered while handling the remote login.
Microsoft.AspNetCore.Authentication.RemoteAuthenticationHandler.HandleRequestAsync()

Stack Query Cookies Headers Routing
MsalUiRequiredException: AADSTS54005: OAuth2 Authorization code was already redeemed, please retry with a new valid code or use an existing refresh token. Trace ID: f1bfb317-8f31-4e83-bc10-69173a8c4b00 Correlation ID: 9d70e458-5694-45d7-a914-34a63e0898a0 Timestamp: 2023-12-07 20:07:42Z
Microsoft.Identity.Client.OAuth2.OAuth2Client.ThrowServerException(HttpResponse response, RequestContext requestContext)
Microsoft.Identity.Client.OAuth2.OAuth2Client.CreateResponse(HttpResponse response, RequestContext requestContext)
Microsoft.Identity.Client.OAuth2.OAuth2Client.ExecuteRequestAsync(Uri endPoint, HttpMethod method, RequestContext requestContext, bool expectErrorsOn200OK, bool addCommonHeaders, Func<OnBeforeTokenRequestData, Task> onBeforePostRequestData)
Microsoft.Identity.Client.OAuth2.OAuth2Client.GetTokenAsync(Uri endPoint, RequestContext requestContext, bool addCommonHeaders, Func<OnBeforeTokenRequestData, Task> onBeforePostRequestHandler)
Microsoft.Identity.Client.OAuth2.TokenClient.SendHttpAndClearTelemetryAsync(string tokenEndpoint, ILoggerAdapter logger)
Microsoft.Identity.Client.OAuth2.TokenClient.SendHttpAndClearTelemetryAsync(string tokenEndpoint, ILoggerAdapter logger)
Microsoft.Identity.Client.OAuth2.TokenClient.SendTokenRequestAsync(IDictionary<string, string> additionalBodyParameters, string scopeOverride, string tokenEndpointOverride, CancellationToken cancellationToken)
Microsoft.Identity.Client.Internal.Requests.RequestBase.SendTokenRequestAsync(IDictionary<string, string> additionalBodyParameters, CancellationToken cancellationToken)
Microsoft.Identity.Client.Internal.Requests.ConfidentialAuthCodeRequest.ExecuteAsync(CancellationToken cancellationToken)
Microsoft.Identity.Client.Internal.Requests.RequestBase.RunAsync(CancellationToken cancellationToken)
Microsoft.Identity.Client.ApiConfig.Executors.ConfidentialClientExecutor.ExecuteAsync(AcquireTokenCommonParameters commonParameters, AcquireTokenByAuthorizationCodeParameters authorizationCodeParameters, CancellationToken cancellationToken)
Microsoft.Identity.Web.TokenAcquisition.AddAccountToCacheFromAuthorizationCodeAsync(AuthCodeRedemptionParameters authCodeRedemptionParameters)
Microsoft.Identity.Web.TokenAcquisitionAspNetCore.AddAccountToCacheFromAuthorizationCodeAsync(AuthorizationCodeReceivedContext context, IEnumerable scopes, string authenticationScheme)
Microsoft.Identity.Web.MicrosoftIdentityWebAppAuthenticationBuilder+<>c__DisplayClass11_1+<b__1>d.MoveNext()
Microsoft.Identity.Web.MicrosoftIdentityWebAppAuthenticationBuilder+<>c__DisplayClass11_1+<b__1>d.MoveNext()
Microsoft.AspNetCore.Authentication.OpenIdConnect.OpenIdConnectHandler.RunAuthorizationCodeReceivedEventAsync(OpenIdConnectMessage authorizationResponse, ClaimsPrincipal user, AuthenticationProperties properties, JwtSecurityToken jwt)
Microsoft.AspNetCore.Authentication.OpenIdConnect.OpenIdConnectHandler.HandleRemoteAuthenticateAsync()

Show raw exception details
MSAL.NetCore.4.58.0.0.MsalUiRequiredException:
ErrorCode: invalid_grant
Microsoft.Identity.Client.MsalUiRequiredException: AADSTS54005: OAuth2 Authorization code was already redeemed, please retry with a new valid code or use an existing refresh token. Trace ID: f1bfb317-8f31-4e83-bc10-69173a8c4b00 Correlation ID: 9d70e458-5694-45d7-a914-34a63e0898a0 Timestamp: 2023-12-07 20:07:42Z
at Microsoft.Identity.Client.OAuth2.OAuth2Client.ThrowServerException(HttpResponse response, RequestContext requestContext)
at Microsoft.Identity.Client.OAuth2.OAuth2Client.CreateResponse[T](HttpResponse response, RequestContext requestContext)
at Microsoft.Identity.Client.OAuth2.OAuth2Client.ExecuteRequestAsync[T](Uri endPoint, HttpMethod method, RequestContext requestContext, Boolean expectErrorsOn200OK, Boolean addCommonHeaders, Func2 onBeforePostRequestData) at Microsoft.Identity.Client.OAuth2.OAuth2Client.GetTokenAsync(Uri endPoint, RequestContext requestContext, Boolean addCommonHeaders, Func2 onBeforePostRequestHandler)
at Microsoft.Identity.Client.OAuth2.TokenClient.SendHttpAndClearTelemetryAsync(String tokenEndpoint, ILoggerAdapter logger)
at Microsoft.Identity.Client.OAuth2.TokenClient.SendHttpAndClearTelemetryAsync(String tokenEndpoint, ILoggerAdapter logger)
at Microsoft.Identity.Client.OAuth2.TokenClient.SendTokenRequestAsync(IDictionary2 additionalBodyParameters, String scopeOverride, String tokenEndpointOverride, CancellationToken cancellationToken) at Microsoft.Identity.Client.Internal.Requests.RequestBase.SendTokenRequestAsync(IDictionary2 additionalBodyParameters, CancellationToken cancellationToken)
at Microsoft.Identity.Client.Internal.Requests.ConfidentialAuthCodeRequest.ExecuteAsync(CancellationToken cancellationToken)
at Microsoft.Identity.Client.Internal.Requests.RequestBase.RunAsync(CancellationToken cancellationToken)
at Microsoft.Identity.Client.ApiConfig.Executors.ConfidentialClientExecutor.ExecuteAsync(AcquireTokenCommonParameters commonParameters, AcquireTokenByAuthorizationCodeParameters authorizationCodeParameters, CancellationToken cancellationToken)
at Microsoft.Identity.Web.TokenAcquisition.AddAccountToCacheFromAuthorizationCodeAsync(AuthCodeRedemptionParameters authCodeRedemptionParameters)
at Microsoft.Identity.Web.TokenAcquisitionAspNetCore.AddAccountToCacheFromAuthorizationCodeAsync(AuthorizationCodeReceivedContext context, IEnumerable`1 scopes, String authenticationScheme)
at Microsoft.Identity.Web.MicrosoftIdentityWebAppAuthenticationBuilder.<>c__DisplayClass11_1.<b__1>d.MoveNext()
--- End of stack trace from previous location ---
at Microsoft.Identity.Web.MicrosoftIdentityWebAppAuthenticationBuilder.<>c__DisplayClass11_1.<b__1>d.MoveNext()
--- End of stack trace from previous location ---
at Microsoft.AspNetCore.Authentication.OpenIdConnect.OpenIdConnectHandler.RunAuthorizationCodeReceivedEventAsync(OpenIdConnectMessage authorizationResponse, ClaimsPrincipal user, AuthenticationProperties properties, JwtSecurityToken jwt)
at Microsoft.AspNetCore.Authentication.OpenIdConnect.OpenIdConnectHandler.HandleRemoteAuthenticateAsync()
StatusCode: 400
ResponseBody: {"error":"invalid_grant","error_description":"AADSTS54005: OAuth2 Authorization code was already redeemed, please retry with a new valid code or use an existing refresh token. Trace ID: f1bfb317-8f31-4e83-bc10-69173a8c4b00 Correlation ID: 9d70e458-5694-45d7-a914-34a63e0898a0 Timestamp: 2023-12-07 20:07:42Z","error_codes":[54005],"timestamp":"2023-12-07 20:07:42Z","trace_id":"f1bfb317-8f31-4e83-bc10-69173a8c4b00","correlation_id":"9d70e458-5694-45d7-a914-34a63e0898a0"}
Headers: Cache-Control: no-store, no-cache
Pragma: no-cache
Strict-Transport-Security: max-age=31536000; includeSubDomains
X-Content-Type-Options: nosniff
P3P: CP="DSP CUR OTPi IND OTRi ONL FIN"
client-request-id: 9d70e458-5694-45d7-a914-34a63e0898a0
x-ms-request-id: f1bfb317-8f31-4e83-bc10-69173a8c4b00
x-ms-ests-server: 2.1.16878.5 - NCUS ProdSlices
x-ms-clitelem: 1,54005,0,,
X-XSS-Protection: 0
Set-Cookie: fpc=Ajbm2CNrDklEjlibyHTg9VKjzM1WAgAAAN4cBN0OAAAAJuNecAYAAAD5HATdDgAAAA; expires=Sat, 06-Jan-2024 20:07:42 GMT; path=/; secure; HttpOnly; SameSite=None, x-ms-gateway-slice=estsfd; path=/; secure; httponly
Date: Thu, 07 Dec 2023 20:07:42 GMT
Exception: An error was encountered while handling the remote login.
Microsoft.AspNetCore.Authentication.RemoteAuthenticationHandler.HandleRequestAsync()
Microsoft.AspNetCore.Authentication.AuthenticationMiddleware.Invoke(HttpContext context)
xxxx.Microsoft.Identity.Startup+<>c+<b__7_0>d.MoveNext() in C:\Users\xxxxx\source\repos\xxxxx_core\xxxxx.Microsoft.Identiy\Startup.cs
Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddleware.Invoke(HttpContext context)

Error message in the Azure Portal of the testing tenant
Silent interrupt required to recognize browser capabilities. Used to differentiate between Safari running in iPadOS or Mac.

Id Web logs

No response

Relevant code snippets

services.Configure<ForwardedHeadersOptions>(options =>
{
    options.ForwardedHeaders =
        ForwardedHeaders.XForwardedFor | ForwardedHeaders.XForwardedProto;
});
               
// Configure the dependency injection for IMsalAccountActivityStore to use a SQL Server to store the entity MsalAccountActivity.
// You might want to customize this class, or implement our own, with logic that fits your business need.
services.AddScoped<IMicrosoftUserTokenStore, MicrosoftUserTokenStore>();
services.AddScoped<IMicrosoftAppCredStore, MicrosoftAppCredStore>();
services.AddScoped<IManagedDomainStore, ManagedDomainStore>();

// Add Postgres as Token cache store
services.AddDbContext<IntegratedTokenCacheDbContext>(options =>
    options.UseNpgsql(EnvSettings.WriteConnectionString));

services.Configure<CookieAuthenticationOptions>(CookieAuthenticationDefaults.AuthenticationScheme, options =>
{
    options.Cookie.SameSite = SameSiteMode.Lax;
    options.Cookie.SecurePolicy = CookieSecurePolicy.Always;
    options.Cookie.HttpOnly = true;
    options.Cookie.IsEssential = true;

    options.Events.OnSigningIn = async (context) =>
    {
        context.Properties.IsPersistent = false;

        await Task.CompletedTask;
    };

    options.AccessDeniedPath = new PathString("/End/Index");
});

services.Configure<CookiePolicyOptions>(options =>
{
    options.MinimumSameSitePolicy = SameSiteMode.Lax;
    // Handling SameSite cookie according to https://docs.microsoft.com/en-us/aspnet/core/security/samesite?view=aspnetcore-3.1
    options.HandleSameSiteCookieCompatibility();
});

// Sign-in users with the Microsoft identity platform
// Configures the web app to call a web api (Ms Graph)
// Sets the IMsalTokenCacheProvider to be the IntegratedTokenCacheAdapter
services.AddAuthentication(OpenIdConnectDefaults.AuthenticationScheme)
    .AddMicrosoftIdentityWebApp(options =>
    {
        Configuration.Bind("AzureAd", options);
        options.Events.OnTokenValidated = HandleOnTokenValidated;
        options.Events.OnAuthenticationFailed = HandleOnAuthenticationFailed;
        options.Events.OnRedirectToIdentityProviderForSignOut = HandleOnRedirectToIdentityProviderForSignOut;
        options.Events.OnRedirectToIdentityProvider = HandleOnRedirectToIdentityProvider;
     })
    .EnableTokenAcquisitionToCallDownstreamApi(initialScopes)
    .AddDistributedTokenCaches();

services.AddIntegratedUserTokenCache();

services.AddDistributedPostgreSqlCache(setup =>
{
    setup.ConnectionString = EnvSettings.WriteConnectionString;
    setup.SchemaName = "public";
    setup.TableName = "microsoft_token_cache";
    setup.CreateInfrastructure = false;
    setup.DefaultSlidingExpiration = TimeSpan.FromDays(14);
});

if (!baseURI.ToLower().Contains("localhost"))
{
    AWSOptions awsOptions = new AWSOptions
    {
        Credentials = new BasicAWSCredentials(DataProtection.GetDataValue("AWSAccessKey"),
            DataProtection.GetDataValue("AWSAccessSecret"))
    };

    services.AddDefaultAWSOptions(awsOptions);

    services.AddDataProtection()
       .SetApplicationName(string.Concat("Identity.Microsoft.", EnvSettings.env))
       .PersistKeysToAWSSystemsManager(string.Concat("/Identity/Microsoft/", EnvSettings.env));
}

services.AddControllersWithViews(options =>
{
    var policy = new AuthorizationPolicyBuilder()
        .RequireAuthenticatedUser()
        .Build();
    options.Filters.Add(new AuthorizeFilter(policy));
}).AddMicrosoftIdentityUI();

services.AddRazorPages();

The handler configurations
private Task HandleOnTokenValidated(TokenValidatedContext context)
{
    ErrorHelper.SendAlertToSlack("HandleOnTokenValidated");
    string tenantId = context.SecurityToken.Claims.FirstOrDefault(x => x.Type == "tid" || x.Type == "http://schemas.microsoft.com/identity/claims/tenantid")?.Value;

    if (string.IsNullOrWhiteSpace(tenantId))
        throw new UnauthorizedAccessException("Unable to get tenantId from token.");

    IMicrosoftAppCredStore dbContext = context.HttpContext.RequestServices.GetRequiredService<IMicrosoftAppCredStore>();

    var authorizedTenant = dbContext.GetAppCredForTenantID(new Guid(tenantId)).Result;

    if (authorizedTenant == null)
        throw new UnauthorizedTenantException("This tenant is not authorized");

    return Task.FromResult(0);
}

private Task HandleOnAuthenticationFailed(AuthenticationFailedContext context)
{
    ErrorHelper.SendAlertToSlack("HandleOnAuthenticationFailed");
    if (context.Exception != null && context.Exception is UnauthorizedTenantException)
    {
        context.Response.Redirect("/Onboarding/UnauthorizedTenant");
        context.HandleResponse(); // Suppress the exception
    }

    return Task.FromResult(0);
}

private Task HandleOnRedirectToIdentityProviderForSignOut(RedirectContext context)
{
    ErrorHelper.SendAlertToSlack("HandleOnRedirectToIdentityProviderForSignOut");
    context.ProtocolMessage.PostLogoutRedirectUri = FixRedirectURL("End/Index/");

    IMicrosoftUserTokenStore dbContext = context.HttpContext.RequestServices.GetRequiredService<IMicrosoftUserTokenStore>();

    dbContext.DeleteUserToken(context.HttpContext.User.Identities.FirstOrDefault().Name.ToLower());

    return Task.FromResult(0);
}

private Task HandleOnRedirectToIdentityProvider(RedirectContext context)
{
    ErrorHelper.SendAlertToSlack("HandleOnRedirectToIdentityProvider");
    if (context.ProtocolMessage.RedirectUri.Contains("microsoft-signin-oidc"))
        context.ProtocolMessage.RedirectUri = FixRedirectURL("microsoft-signin-oidc");
    else if (context.ProtocolMessage.RedirectUri.Contains("Onboarding/ProcessCode"))
        context.ProtocolMessage.RedirectUri = FixRedirectURL("Onboarding/ProcessCode");

    return Task.FromResult(0);
}

Regression

No response

Expected behavior

Login in

@nromano32
Copy link
Author

So I've been trying to solve the issue by looking at the handler registration. If I understand the code correctly

options.Events.OnTokenValidated = HandleOnTokenValidated. (i.e using =) should register and deregister the Handler
options.Events.OnTokenValidated += HandleOnTokenValidated. (i.e using +=) should register but not deregister the Handler
options.Events.OnTokenValidated -= HandleOnTokenValidated. (i.e using +=) should only deregister the Handler

if the above is correct the above multiple calls may be a false positive as I was testing +=. With = i see the following and it looks as thought the auth call is executed only once.

🔴 HandleOnRedirectToIdentityProviderForSignOut
New
12:57
🔴 HandleOnRedirectToIdentityProvider
12:57
🔴 HandleOnTokenValidated
12:57
🔴 In
12:57
🔴 BaseBearerTokenAuthenticationProvider
12:57
🔴 GraphServiceClient
12:57
🔴 User

The other thing I did was to make all the Handlers operate async await. I not sure I understand but now it seems to run correctly. Does that make sense? I had been trying to run the handlers synchronously. Might this be my issue. Sorry I'm new to async await coding.

Thanks for you help

@nromano32
Copy link
Author

nromano32 commented Dec 10, 2023

Ok more testing this AM. We put in an environment variable to turn on of off the slack message sends. Super strange but without the slack message sends we get the error. Conversely, with the slack message sends the code works correctly (same code no deployments between runs). My best guess is the delay caused by sending the slack message slows the process sufficiently to allow the order of operation to process correctly. Without the slack message send delay the handlers might be firing out of order. Really strange ... should we make the handlers all synchronous. I hope that helps.

@jmprieur
Copy link
Collaborator

jmprieur commented Jan 2, 2024

are you using the latest version of Microsoft.Identity.Web?

@jmprieur jmprieur added question Further information is requested and removed untriaged needs attention labels Jan 2, 2024
@nromano32
Copy link
Author

nromano32 commented Jan 3, 2024

Yes Version 2.16.0. However, we has tried and number of different version. We are running on .NET 6.0.25 in AWS Elastic Beanstalk (on Linux)

We have been investigating if the AWS Elastic BeanStalk health check is contributing to the issue. We noticed after adding the write statements that the heath check was triggering authentication. The health check page has the attribute [AllowAnonymous], which we thought would exclude the authentication calls.

The health check is set to run every 15 seconds (AWS default)

Screenshot 2024-01-03 at 6 41 18 AM

So we aded the following code, which did not work and are still looking for a solution.

private static bool SkipAuthorization(HttpContext context)
{

try
{
    var endpoint = context.GetEndpoint();
    if (endpoint?.Metadata?.GetMetadata<IAllowAnonymous>() is object)
    {
        ErrorHelper.SendAlertToSlack("True");
        return true;
    }
}
catch (Exception ex) { ErrorHelper.WriteException(ex, "SkipAuthorization", ex.InnerException.Message); }

ErrorHelper.SendAlertToSlack("False");
return false;

}

The write messages are sporadic

:58
🔴 False
5:58
🔴 False
5:58
🔴 False
5:59
🔴 False
5:59
🔴 False
5:59
🔴 False

next run
🔴 False
6:12
🔴 False
6:12
🔴 False
6:12
🔴 False
6:12
🔴 False
6:12
🔴 False

Is it possible that we are not skipping the authenticate calls and are hitting some kind of per use limit that is causing the sporadic behavior?

Thanks for the response. Nick

@nromano32
Copy link
Author

We added a Health Check Page using
services.AddHealthChecks();
And it appears the system is not trying to authorize during the health check. We will give it a couple of days to see if that caused our problem.

@JoshLozensky
Copy link
Contributor

Hi @nromano32 has the AddHealthChecks() solved this issue for you?

@nromano32
Copy link
Author

nromano32 commented Jan 10, 2024

It did not solve the main issue. It did remove the health check authorization calls. At this point, we are out of ideas. We can't go to .net8.0 until AWS updates Elastic beanstalk. I would really appreciate any guidance or ideas.

@jmprieur
Copy link
Collaborator

Would you be able to share with us the code for a small repro?

@jmprieur
Copy link
Collaborator

Also for the usage of Microsoft graph, I recommend not using BaseBearerTokenAuthenticationProvider but just use

services.AddMicrosoftGraph()

and inject the MicrosoftGraph GraphServiceClient in your controller.

See https://learn.microsoft.com/en-us/entra/identity-platform/scenario-web-app-call-api-call-api?tabs=aspnetcore#option-1-call-microsoft-graph-with-the-sdk

@nromano32
Copy link
Author

Happy to send you the code. I'll put it up in a separate repo tomorrow. As for services.AddMicrosoftGraph() We tried that but could not get past a compile error

<title></title> <style type="text/css"> p.p1 {margin: 0.0px 0.0px 0.0px 0.0px; font: 9.0px Helvetica} table.t1 {border-style: solid; border-width: 0.5px 0.5px 0.5px 0.5px; border-color: #bfbfbf #bfbfbf #bfbfbf #bfbfbf; border-collapse: collapse} td.td1 {border-style: solid; border-width: 0.5px 0.5px 0.5px 0.5px; border-color: #7f7f7f #7f7f7f #7f7f7f #7f7f7f; padding: 0.0px 5.0px 0.0px 5.0px} </style>
Severity Code Description Project File Line Suppression State
Error CS7069 Reference to type 'IAuthenticationProvider' claims it is defined in 'Microsoft.Graph.Core', but it could not be found xxxxxMicrosoft.Identity C:\Users\xxxxx\source\repos\xxxxx_core\xxxxxx.Microsoft.Identiy\Startup.cs 116 Active

Microsoft.Graph.Core is added to the project

@nromano32
Copy link
Author

Just circling back here. I got wrapped up in 2023 taxes for a bit and then had the idea to port everything to .net 8.0 (to see if the issue would go way). We know can use .AddMicrosoftGraph(Configuration.GetSection("AzureAd")) at least we can compile it. Working to get the debug environment up with IIS Express. We use Mac M1's and parallels so this is proving to be a more difficult task then we realized. More to come.

@LGSIMM
Copy link

LGSIMM commented Jan 22, 2024

Hi @nromano32 I work for an organisation also experiencing this issue since Nov23.

We opened a support case with MS back then, and after lots of troubleshooting and sharing of logs, they are now pointing us to this thread as being the same issue as ours.

I'm curious if any technicians from MS are assisting you in some capacity with this issue or not?

@nromano32
Copy link
Author

nromano32 commented Jan 23, 2024 via email

@nromano32
Copy link
Author

Ok so in .net 6.0 we updated the code and deployed to production using .AddMicrosoftGraph(Configuration.GetSection("AzureAd")). The trick was to reference Microsoft.Identity.Web.GraphServiceClient instead of Microsoft.Identity.Web.GraphClient. Please give me a week or two to see if the error reappears.

            services.AddAuthentication(OpenIdConnectDefaults.AuthenticationScheme)
                .AddMicrosoftIdentityWebApp(options =>
                {
                    Configuration.Bind("AzureAd", options);
                    options.Events.OnTokenValidated = HandleOnTokenValidated;
                    options.Events.OnAuthenticationFailed = HandleOnAuthenticationFailed;
                    options.Events.OnRedirectToIdentityProviderForSignOut = HandleOnRedirectToIdentityProviderForSignOut;
                    options.Events.OnRedirectToIdentityProvider = HandleOnRedirectToIdentityProvider;
                 })
                .EnableTokenAcquisitionToCallDownstreamApi(initialScopes)
                .AddMicrosoftGraph(Configuration.GetSection("AzureAd"))
                .AddDistributedTokenCaches();

We also changed how we get the GraphClient to reflect the injection

public HomeController(GraphServiceClient graphClient, IMicrosoftAppCredStore microsoftAppCredStore,
IMicrosoftUserTokenStore microsoftUserTokenStore, IManagedDomainStore managedDomainStore)
{
_graphClient = graphClient;
_managedDomainStore = managedDomainStore;
_microsoftAppCredStore = microsoftAppCredStore;
_microsoftUserTokenStore = microsoftUserTokenStore;
}

Best - Nick

@nromano32
Copy link
Author

Well that did not work .. still got the error. I did read this issue a bit ago #1995 detailing a race condition with .net6.0 and .net7.0.

We have complied the project in .net8.0 and deployed the code to Prod and Staging. Prod was failing with AADSTS54005 at the time and immediately worked correctly.

All of our debug messages fired in order and not multiple times. I'll report back in a couple of days on the systems state.

Best - Nick

@nromano32
Copy link
Author

We have seen no issue since the upgrade. I suspect that the above issue was my problem. I think you can close this case now.

Thanks for the help - Nick

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
question Further information is requested
Projects
None yet
Development

No branches or pull requests

4 participants