Strange behavior with AddJwtBearer authentication policy that was working in .NET8

[glb-cblin]

Is there an existing issue for this?

• I have searched the existing issues

Describe the bug

The following code was working with .NET8

auth.AddJwtBearer(IdentityAuthentication.Scheme, options =>
    {
        options.RequireHttpsMetadata = false; //do not bother with that, it is infra related not code related
        options.SaveToken = true; //to be able to send the token to another service                                          
        options.Authority = "http://localhost:5117/identity-api";
        options.TokenValidationParameters = new TokenValidationParameters
        {                
            ValidateIssuer = true, 
            ValidIssuer = "http://localhost:5117/identity-api",
            ValidateAudience = true,
            ValidAudience = IdentityAuthentication.Audience,
            ValidateLifetime = true,
            ClockSkew = TimeSpan.Zero, //default is 5 minutes, too long for a JWT which is valid 10 minutes ...
        };
        options.MapInboundClaims = false; //by default, .NET brutally maps some claims (https://learn.microsoft.com/en-us/aspnet/core/security/authentication/claims?view=aspnetcore-6.0#claims-namespaces-default-namespaces)
    });

When using the exact same code with .NET9, I am now facing this error

www-authenticate: Bearer error="invalid_token",error_description="The signature key was not found",Bearer error="invalid_token",error_description="The signature key was not found" 

And in the logs, I have

info: Microsoft.AspNetCore.Authentication.JwtBearer.JwtBearerHandler[1]
      Failed to validate the token.
      Microsoft.IdentityModel.Tokens.SecurityTokenSignatureKeyNotFoundException: IDX10500: Signature validation failed. No security keys were provided to validate the signature.
         at Microsoft.IdentityModel.JsonWebTokens.JsonWebTokenHandler.ValidateSignature(JsonWebToken jwtToken, TokenValidationParameters validationParameters, BaseConfiguration configuration)
         at Microsoft.IdentityModel.JsonWebTokens.JsonWebTokenHandler.ValidateSignatureAndIssuerSecurityKey(JsonWebToken jsonWebToken, TokenValidationParameters validationParameters, BaseConfiguration configuration)
         at Microsoft.IdentityModel.JsonWebTokens.JsonWebTokenHandler.ValidateJWSAsync(JsonWebToken jsonWebToken, TokenValidationParameters validationParameters, BaseConfiguration configuration)
info: Microsoft.AspNetCore.Authentication.JwtBearer.JwtBearerHandler[7]
      Identity was not authenticated. Failure message: IDX10500: Signature validation failed. No security keys were provided to validate the signature.

Here is the relevant part of http://localhost:5117/identity-api/.well-known/openid-configration

{  
  "issuer": "http://localhost:5117/identity-api",
  "jwks_uri": "http://localhost:5117/identity-api/.well-known/jwks.json",
}

and http://localhost:5117/identity-api/.well-known/jwks.json

{
  "keys": [
    {
      "alg": "RS256",
      "e": "AQAB",
      "key_ops": [],
      "kid": "1",
      "kty": "RSA",
      "n": "xxx",
      "oth": [],
      "use": "sig",
      "x5c": []
    }
  ]
}

What changed between .NET8 and .NET9 ?

Under .NET8, the http://localhost:5117/identity-api/.well-known/openid-configration AND http://localhost:5117/identity-api/.well-known/jwks.json are called

Under .NET9, only http://localhost:5117/identity-api/.well-known/openid-configration is called (and so obviously it complains that there is no keys ... they were not fetched)

Expected Behavior

I expect that the jwks.json would also be fetched to populate the keys and that after that the JWT will be considered valid

Steps To Reproduce

create a first service to serve the .well-known opeind urls

create a second service with a jwtbearer

Exceptions (if any)

No response

.NET Version

.net9

Anything else?

No response