SecurityTokenMalformedException after updating to .NET 8

[Nefcanto]

Is there an existing issue for this?

• I have searched the existing issues

Describe the bug

I know #52191 is about the same error. But in my case, my JWT is created via Keycloak and I can verify it at https://jwt.io/.

This is the error I get after upgrading to .NET 8 and upgrading my package reference to 8.0.0:

info: Microsoft.AspNetCore.Authentication.JwtBearer.JwtBearerHandler[1]
      Failed to validate the token.
      Microsoft.IdentityModel.Tokens.SecurityTokenMalformedException: IDX14100: JWT is not well formed, there are no dots (.).
      The token needs to be in JWS or JWE Compact Serialization Format. (JWS): 'EncodedHeader.EndcodedPayload.EncodedSignature'. (JWE): 'EncodedProtectedHeader.EncodedEncryptedKey.EncodedInitializationVector.EncodedCiphertext.EncodedAuthenticationTag'.
       ---> System.ArgumentException: IDX14102: Unable to decode the header '[Security Artifact of type 'Microsoft.IdentityModel.Logging.SecurityArtifact' is hidden. For more details, see https://aka.ms/IdentityModel/SecurityArtifactLogging.]' as Base64Url encoded string.
       ---> System.ArgumentOutOfRangeException: Specified argument was out of the range of valid values. (Parameter 'IDX10820: Invalid character found in Base64UrlEncoding. Character: '32', Encoding: 'Bearer eyJhbGciOiJSUzI1NiI8OeCv-m8PNIUHSFy39iLfOw3vkcA'.')

Expected Behavior

No response

Steps To Reproduce

No response

Exceptions (if any)

No response

.NET Version

No response

Anything else?

No response

[martincostello]

Looks like a duplicate of #52099.

[Nefcanto]

Based on #52099 I went to Security token events return a JsonWebToken and added options.UseSecurityTokenValidators = true; to my code. But it had no effect. This is my code:

        services.AddAuthentication(options =>
        {
            options.DefaultAuthenticateScheme = JwtBearerDefaults.AuthenticationScheme;
            options.DefaultChallengeScheme = JwtBearerDefaults.AuthenticationScheme;
        })
        .AddJwtBearer(jwtOptions =>
        {
            jwtOptions.UseSecurityTokenValidators = true;
            jwtOptions.Authority = @$"{ApiConfig.GetSetting("KeycloakUrl").Trim('/')}/realms/{ApiConfig.GetSetting("Realm")}";
            jwtOptions.Audience = "account";
            jwtOptions.Events = new JwtBearerEvents()
            {
                OnAuthenticationFailed = c =>
                {
                    c.NoResult();

                    Logger.LogException(c.Exception);

                    c.Response.StatusCode = 500;
                    c.Response.ContentType = "text/plain";
                    if (ApiConfig.IsDeveloping)
                    {
                        return c.Response.WriteAsync(c.Exception.ToString());
                    }
                    return c.Response.WriteAsync("An error occured processing your authentication.");
                },
                OnTokenValidated = c =>
                {
                    if (c.Principal.IsInRole("SuperAdmin"))
                    {
                        c.HttpContext.Items["IsSuperAdmin"] = true;
                    }
                    return System.Threading.Tasks.Task.CompletedTask;
                }
            };
        });

@martincostello Any ideas why the recommended option does not work?

[martincostello]

I'm afraid not - maybe it's a bug in IdentityModel. See #52099 (comment).

[Nefcanto]

@martincostello can you guide me to a sample code that shows me how to rewrite my entire security to match .NET 8? I mean, instead of me trying to make .NET 8 work for my old way, I want to upgrade to the new JsonWebToken. But I can't find a page on the docs with a sample on what should I change.

[martincostello]

Sorry, I'm not aware of such a sample - someone from the ASP.NET Core team should be able to help you when they come online later today.

[davidfowl]

cc @brentschmaltz

[brentschmaltz]

@Nefcanto would it be possible to obtain a sample token that is failing?
What IdentityProvider is creating the token?
Normally tokens contain sensitive information, can a token be crafted without sensitive information?

[Nefcanto]

@brentschmaltz we're using Keycloak, and we tested its token using jwt.io, and it was valid. I can create a test user and create a token to paste here. Actually I had pasted a token in the original post and it's in the edit history. Can you see it?

[brentschmaltz]

@martincostello @Nefcanto #52099 is clearly an issue which needs to be address.

[brentschmaltz]

@martincostello I was able to obtain the token from history.

[brentschmaltz]

@martincostello i was able to create a new JsonWebToken using the jwt i found, so the jwt is indeed well-formed.
The error message ids (IDX10820, IDX14102) indicate an error in parsing the jwt header.
The error message contains: 'Bearer eyJhbGciOiJSUzI1NiI8OeCv-m8PNIUHSFy39iLfOw3vkcA', which seems to indicate the token passed to IdentityModel has the prefix 'Bearer '.

[Nefcanto]

@brentschmaltz thank you for helping. I think it's an internal thing related to the .NET itself. We don't have Bearer hardcoded anywhere in our code.

[bayardwest]

I'm running into the exact same error.

[dzerenus]

I faced the same problem, and now I will describe my solution to the problem.
Apparently, this is a bug in the Microsoft.AspNetCore.Authentication.JwtBearer library at 8.0.1 version.

If you try to validate the token using the standard means that the library implies, you will get an error.

builder.Services.AddAuthentication(JwtBearerDefaults.AuthenticationScheme).AddJwtBearer(options =>
{
    options.TokenValidationParameters = new TokenValidationParameters
    {
        ValidateIssuer = false,
        ValidateAudience = false,
        ValidateLifetime = false,,
    };
});

I decided this because the main method of token validation, which the library assumes, produces a similar error when trying to validate a valid JWT token.

var jwtTokenString = "paste_your_correct_token_here";
var validationParams = new TokenValidationParameters
{
    ValidateIssuer = false,
    ValidateAudience = false,
    ValidateLifetime = false,
};

var tokenHandler = new JwtSecurityTokenHandler();
await tokenHandler.ValidateTokenAsync(jwtTokenString, validationParams); 
// Exception: IDX14100: JWT is not well formed, there are no dots (.)

Based on what I was able to find on the Internet, the problem manifests itself on .NET Core 8.0, and it can be solved if you use an older version of System.Text library in the project, but I found another solution.
We can write our own validator and use it instead of the standard one.

The validator code that I wrote:

public class BearerTokenHandler : TokenHandler
{
    private readonly JwtSecurityTokenHandler _tokenHandler = new();

    public override Task<TokenValidationResult> ValidateTokenAsync(string token, TokenValidationParameters validationParameters)
    {
        try
        {
            _tokenHandler.ValidateToken(token, validationParameters, out var validatedToken);

            if (validatedToken is not JwtSecurityToken jwtSecurityToken)
                return Task.FromResult(new TokenValidationResult() { IsValid = false });

            return Task.FromResult(new TokenValidationResult
            {
                IsValid = true,
                ClaimsIdentity = new ClaimsIdentity(jwtSecurityToken.Claims, JwtBearerDefaults.AuthenticationScheme),

                // If you do not add SecurityToken to the result, then our validator will fire, return a positive result, 
                // but the authentication, in general, will fail.
                SecurityToken = jwtSecurityToken,
            });
        }

        catch (Exception e)
        {
            return Task.FromResult(new TokenValidationResult 
            {
                IsValid = false,
                Exception = e,
            });
        }
    }
}

In the end, you only need to connect your own validator.

builder.Services.AddAuthentication(JwtBearerDefaults.AuthenticationScheme).AddJwtBearer(options =>
{
    options.TokenValidationParameters = new TokenValidationParameters
    {
        ValidateIssuer = false,
        ValidateAudience = false,
        ValidateLifetime = false,
    };
    options.TokenHandlers.Add(new BearerTokenHandler()); // <---
});

Thus, we bypassed the error and did not use the deprecated UseSecurityTokenValidators.

Write your thoughts on this, and excuse my English (I used a translator).

[jhelmink]

Just to add a couple of thoughts here, as I'm battling with JWT authentication since upgrading to 8.x this week.

There doesn't seem to be any complete end to end official examples of how to implement JWT using the newer JsonWebToken

The breaking changes from 7.x -> 8.x were documented, with workarounds (that did not work) and did not contain enough information for users to create an updated end to end solution.

Coupled with the difficulty in debugging the built in solution and the end result is a large number of developers creating workarounds and custom implementations for a complex security protocol, which is supposed to be standardised to stop buggy "roll your owns".

I'll be reverting to 7.x after creating my own 8.x implementation, only to find it can't validate a perfectly valid token (as confirmed by jwt.io). Dead ends everywhere SMH.

[jhelmink]

I should provide an example for repro;

        public JsonWebTokenHandler TokenHandler { get; set; } = new JsonWebTokenHandler();

        public AuthenticationResponse CreateToken(AppUser user)
        {
            var expiration = DateTime.UtcNow.AddMinutes(TokenExpirationMinutes);

            var token = new SecurityTokenDescriptor
            {
                Issuer = _configuration["Jwt:Issuer"],
                Audience = _configuration["Jwt:Audience"],
                // Claims = User based Claims Removed for debugging
                Expires = expiration,
                SigningCredentials = new SigningCredentials(
                    new SymmetricSecurityKey(
                        Encoding.UTF8.GetBytes(_configuration["Jwt:Key"]!)
                    ),
                    SecurityAlgorithms.HmacSha256
                )
            };

            return new AuthenticationResponse
            {
                Token = TokenHandler.CreateToken(token),
                Expiration = expiration
            };
        }

        public async Task<bool> ValidateTokenAsync(string tokenString)
        {
            var validationParameters = new TokenValidationParameters
            {
                LogValidationExceptions = false, // Will log as Info not Error
                ValidateIssuer = true,
                ValidateAudience = true,
                ValidateLifetime = true,
                ValidateIssuerSigningKey = true,
                ValidIssuer = _configuration["Jwt:Issuer"],
                ValidAudience = _configuration["Jwt:Audience"],
                IssuerSigningKey = new SymmetricSecurityKey(
                    Encoding.UTF8.GetBytes(_configuration["Jwt:Key"]!)
                )
            };

            try
            {
                var validationResult = await TokenHandler.ValidateTokenAsync(tokenString, validationParameters);
                return validationResult.IsValid;
            }
            catch // Any validation failure will throw an exception
            {
                return false;
            }
        }

[migajek]

also getting JWT is not well formed, there are no dots (.) - the part about missing dots is extremely misleading.

the inner exception is more accurate on this - in my case, the issue being the several fields (such as sub) are numeric, not string.

[tylerlantern]

I was able to solve this by updating this code

var signingKey = new SymmetricSecurityKey(
      Encoding.UTF8.GetBytes(Configuration["JWT:Secret"])
  );

to

 var signingKey = new SymmetricSecurityKey(
      Encoding.ASCII.GetBytes(Configuration["JWT:Secret"])
  );

and here is all dependencies package relate to jwt

    <PackageReference Include="Microsoft.AspNetCore.Authentication.JwtBearer" Version="8.0.2" />
    <PackageReference Include="Microsoft.IdentityModel.JsonWebTokens" Version="7.4.0" />
    <PackageReference Include="Microsoft.IdentityModel.Tokens" Version="7.4.0" />

[jpdh9881]

I was able to solve this along @migajek lines:

also getting JWT is not well formed, there are no dots (.) - the part about missing dots is extremely misleading.

the inner exception is more accurate on this - in my case, the issue being the several fields (such as sub) are numeric, not string.

I was setting the issued at (iat) claim using DateTime.UtcNow.ToString(). This seemed fine for https://jwt.io but .NET was expecting a number (epoch time). This is in line with the RFC.

[cactusoft]

I faced the same problem, and now I will describe my solution to the problem. Apparently, this is a bug in the Microsoft.AspNetCore.Authentication.JwtBearer library at 8.0.1 version.

I am struggling with this issue. I checked in nuget and I have 8.0.2. So I am guessing this is not fixed yet?

In my case, my PWA WASM blazor app talks fine with jwt to my server API. The problem is I have javascript code too in the PWA, but when I post the exact same jwt string in that, i get an auth error. Have quadruple checked and all looks good. If I remove the [Authorize] from my controller, the javascript works fine. I have tried with and without "Bearer " in front of token, still doesn't work.

So I am guessing there is something wrong in the format of the header, maybe it changed, but since I have .NET 8 on pwa and the server API, they cancel each other out by both writing and understanding bad header syntax, but the header on the javascript doesn't work because it's correct.

[cactusoft]

Just to update, I finally figured out my issue. It seemed that when I retrieve the token string from localstorage in javascript, it comes out with a double quote at start and finish. It seems the PWA retrieving from local storage doesn't get that.

So I simply added code in my javascript to remove double quotes from the string:

// Retrieve JWT token from storage, remove double quotes which
// seem to end up in there when we get the token from storage
const jwtToken = localStorage.getItem('strToken').replace(/"/g, '');

Hopefully this might help someone else.

[mkArtakMSFT]

Thanks everyone for all your input. It looks like too many though related things have been discussed here, and it's not clear at this point what is pending to be done and if there is any pending issue remaining here.
If you're still blocked, can you please drop a comment with your issue that is still unresolved, and we'll review it. thanks.

[runxc1]

I seem to be running into this as well. I have a server that is running OpenIdDict and I have multiple applications all running .net 6.0 authenticating against it without any issues. I upgraded one of them to .net 8.0 and am now getting the following errors.

An unhandled exception occurred while processing the request.
JsonException: IDX11020: The JSON value of type: 'StartArray', could not be converted to 'JsonTokenType.String or JsonTokenType.Number'. Reading: 'Microsoft.IdentityModel.JsonWebTokens.JsonWebToken.sub', Position: '7', CurrentDepth: '1', BytesConsumed: '8'.
Microsoft.IdentityModel.Tokens.Json.JsonSerializerPrimitives.ReadStringOrNumberAsString(ref Utf8JsonReader reader, string propertyName, string className, bool read)

ArgumentException: IDX14101: Unable to decode the payload '[PII of type 'Microsoft.IdentityModel.Logging.SecurityArtifact' is hidden. For more details, see https://aka.ms/IdentityModel/PII.]' as Base64Url encoded string.
Microsoft.IdentityModel.JsonWebTokens.JsonWebToken.ReadToken(string encodedJson)

SecurityTokenMalformedException: IDX14100: JWT is not well formed, there are no dots (.).
The token needs to be in JWS or JWE Compact Serialization Format. (JWS): 'EncodedHeader.EndcodedPayload.EncodedSignature'. (JWE): 'EncodedProtectedHeader.EncodedEncryptedKey.EncodedInitializationVector.EncodedCiphertext.EncodedAuthenticationTag'.
Microsoft.AspNetCore.Authentication.OpenIdConnect.OpenIdConnectHandler.ValidateTokenUsingHandlerAsync(string idToken, AuthenticationProperties properties, TokenValidationParameters validationParameters)

AuthenticationFailureException: An error was encountered while handling the remote login.
Microsoft.AspNetCore.Authentication.RemoteAuthenticationHandler<TOptions>.HandleRequestAsync()

[runxc1]

also getting JWT is not well formed, there are no dots (.) - the part about missing dots is extremely misleading.

the inner exception is more accurate on this - in my case, the issue being the several fields (such as sub) are numeric, not string.

I am seeing that sub is an array in my case as it looks like it was supplied twice. This didn't cause any issue previously but now seems to throw the exception

[natelowry]

Not sure if it's the exact same issue, but I was able to resolve this by adding the Microsoft.IdentityModel.JsonWebTokens package.

I originally only had Microsoft.IdentityModel.Tokens and Microsoft.AspNetCore.Authentication.JwtBearer. With all 3 packages I have joy 🎉

[kokosky93]

same issue happened to me

[Risanshita]

Certainly! When upgrading from .NET 6 to .NET 8, I encountered an issue related to setting the "issued at" (iat) claim. Initially, I was using DateTime.UtcNow.ToString(), but I switched to DateTimeOffset.Now.ToUnixTimeSeconds().ToString() to address the issue. 🚀🕰️

also getting JWT is not well formed, there are no dots (.) - the part about missing dots is extremely misleading.

the inner exception is more accurate on this - in my case, the issue being the several fields (such as sub) are numeric, not string.

[judilsteve]

This is a really nasty issue. Our tokens (which validated fine in jwt.io) were failing to validate for seemingly no reason after upgrading from dotnet 7 to dotnet 8. Even with JwtBearerOptions.IncludeErrorDetails set to true the only info we got was www-authenticate: Bearer error="invalid_token". Finally, after setting verbose auth logging in appsettings.json:

{
  "Logging": {
    "LogLevel": {
      "Microsoft.AspNetCore.Authentication": "Information",
    }
  }
}

We were able to see the following incredibly misleading message:

Microsoft.AspNetCore.Authentication.JwtBearer.JwtBearerHandler: Bearer was not authenticated. Failure message: IDX14100: JWT is not well formed, there are no dots (.).
The token needs to be in JWS or JWE Compact Serialization Format. (JWS): 'EncodedHeader.EndcodedPayload.EncodedSignature'. (JWE): 'EncodedProtectedHeader.EncodedEncryptedKey.EncodedInitializationVector.EncodedCiphertext.EncodedAuthenticationTag'.

This was despite the fact that our tokens definitely had dots. I even double-checked to make sure it wasn't some weird unicode character that just looked like a dot. Searching the internet for IDX14100 led us to to a couple Stackoverflow posts and eventually this issue page.

Stackoverflow suggested setting JwtBearerOptions.UseSecurityTokenValidators to true, which got our tokens validating again:

builder.Services.AddAuthentication(JwtBearerDefaults.AuthenticationScheme)
       .AddJwtBearer(opts =>
       {
            opts.UseSecurityTokenValidators = true;
       });

However, the docs suggest this isn't as performant as the default of false.

@natelowry 's solution of installing Microsoft.AspNetCore.Authentication.JwtBearer (which I do not recall being required in dotnet 7) fixed the issue without requiring any changes to the JwtBearerOptions:

dotnet add package Microsoft.IdentityModel.JsonWebTokens

I think some sort of check needs to be added on startup to make sure that the appropriate package is installed when JwtBearerOptions.UseSecurityTokenValidators is set to false, so that others don't fall into this trap. We wasted days debugging this.

[long29103107]

Not sure if it's the exact same issue, but I was able to resolve this by adding the Microsoft.IdentityModel.JsonWebTokens package.

I originally only had Microsoft.IdentityModel.Tokens and Microsoft.AspNetCore.Authentication.JwtBearer. With all 3 packages I have joy 🎉

Thanks, it's work in my project

[gentcod]

public class BearerTokenHandler : TokenHandler
{
    private readonly JwtSecurityTokenHandler _tokenHandler = new();

    public override Task<TokenValidationResult> ValidateTokenAsync(string token, TokenValidationParameters validationParameters)
    {
        try
        {
            _tokenHandler.ValidateToken(token, validationParameters, out var validatedToken);

            if (validatedToken is not JwtSecurityToken jwtSecurityToken)
                return Task.FromResult(new TokenValidationResult() { IsValid = false });

            return Task.FromResult(new TokenValidationResult
            {
                IsValid = true,
                ClaimsIdentity = new ClaimsIdentity(jwtSecurityToken.Claims, JwtBearerDefaults.AuthenticationScheme),

                // If you do not add SecurityToken to the result, then our validator will fire, return a positive result, 
                // but the authentication, in general, will fail.
                SecurityToken = jwtSecurityToken,
            });
        }

        catch (Exception e)
        {
            return Task.FromResult(new TokenValidationResult 
            {
                IsValid = false,
                Exception = e,
            });
        }
    }
}

@dzerenus This works fine, having a custom validator solves the problem. Thanks for sharing. It is important to point out though that the SecurityToken should be the validatedToken. That is SecurityToken = validatedToken.