OIDC token refresh fails with 401, if user info is used and not available in the cache (anymore)

[sschellh]

Describe the bug

We are using Quarkus OIDC and for our clientId the following settings are configured with the Identity Provider:

OIDC Token Lifetime (Seconds): 300
Access Token Lifetime (Seconds): 300
Refresh Token Lifetime (Seconds): 600

(small numbers for testing only!)

We have configured Quarkus OIDC with following settings:

quarkus.oidc.enabled=true
quarkus.oidc.application-type=web-app
quarkus.oidc.authentication.user-info-required=true
quarkus.oidc.token.refresh-expired=true
quarkus.oidc.authentication.session-age-extension=PT300S
quarkus.oidc.token-cache.max-size=1000

After a users logs in, the q_session lifetime is set to the duration of the OIDC token plus the session-age-extension. In our case the session lifetime is exactly as long as the refresh token lifetime.

Expected behavior

When a user makes a request after the id token and access token expired, but before the refresh token and the session cookie expired, then the refresh token is used to retrieve a new pair of access token, id token and refresh token from the identity provider.

Actual behavior

When a user makes a request after the id token and access token expired, but before the refresh token and the session cookie expired, then an HTTP 401 exception is returned.

In the logs we can find the following DEBUG and ERROR statements:

2023-03-09 09:26:56,817 DEBUG [io.qua.oid.run.OidcIdentityProvider] (vert.x-eventloop-thread-3) Requesting UserInfo
2023-03-09 09:26:56,818 DEBUG [io.qua.oid.run.OidcProviderClient] (vert.x-eventloop-thread-3) Get UserInfo on: https://.../userinfo auth: Bearer xxx-XX-XXXXXXXX_XXXXX
2023-03-09 09:26:56,906 DEBUG [io.qua.oid.run.OidcProviderClient] (vert.x-eventloop-thread-3) Request has failed: status: 401, error message: {"error_description":"The access token provided is expired, revoked, malformed, or invalid for other reasons.","error":"invalid_token"}
2023-03-09 09:26:56,908 ERROR [io.qua.oid.run.CodeAuthenticationMechanism] (vert.x-eventloop-thread-3) ID token verification failure: io.quarkus.oidc.OIDCException: {"error_description":"The access token provided is expired, revoked, malformed, or invalid for other reasons.","error":"invalid_token"}

How to Reproduce?

Steps to reproduce

1. Start the server in DEV mode
2. Open the web application and login
3. Restart the server (in order to make sure there are not tokens cached anymore). This mimics the situation in a cloud environment where you have many pods running and they might restart during the lifespan of a user session (which will be in our case over 10h in reality)
4. Perform an activity in your open session in the web app
5. You see an HTTP 401

Output of uname -a or ver

No response

Output of java -version

No response

GraalVM version (if different from Java)

No response

Quarkus version or git rev

2.16.3.Final

Build tool (ie. output of mvnw --version or gradlew --version)

Gradle 7.3.3

Additional information

No response

[quarkus-bot]

/cc @gwenneg (cache), @pedroigor (oidc), @sberyozkin (oidc)

[sberyozkin]

@sschellh I think this issue was already fixed, can you please add quarkus.oidc.authentication.verify-access-token=true ?

[sberyozkin]

The code flow access token is not used by Quarkus itself, it is expected the downstream service like the actual OIDC provider in case of the UserInfo request will verify it.
quarkus.oidc.authentication.verify-access-token=true is needed for the code flow access token only if the endpoint needs to do something with it as well, in addition to ID token. By default it is problematic, some providers return binary tokens but have no introspection endpoint

[sberyozkin]

@sschellh FYI, #30485 (UserInfo precondition there is that if the access token is binary and no introspection point exists in which case requesting userinfo is the only way to verify the binary token)

[sschellh]

Hi @sberyozkin

Many thanks for your response and your suggestion.

From what I see setting quarkus.oidc.authentication.verify-access-token=true does not help. It results now in the following error:

2023-03-09 14:31:10,702 DEBUG [io.qua.oid.run.OidcIdentityProvider] (vert.x-eventloop-thread-3) Starting the opaque token introspection
2023-03-09 14:31:10,703 DEBUG [io.qua.oid.run.OidcProviderClient] (vert.x-eventloop-thread-3) Get token on: https://.../access_token params: token=xxxxxxxxxxxxxx
token_type_hint=access_token
client_id=xxxxxxxxxxxx
client_secret=xxxxxxxxxxxxxxxx
 headers: user-agent=Vert.x-WebClient/4.3.7
content-type=application/x-www-form-urlencoded
accept=application/json

2023-03-09 14:31:10,770 DEBUG [io.qua.oid.run.OidcProviderClient] (vert.x-eventloop-thread-3) Request succeeded: {"active":false}
2023-03-09 14:31:10,786 DEBUG [io.qua.oid.run.OidcProvider] (vert.x-eventloop-thread-3) Token issued to client xxxxxxxxxxxxx is not active
2023-03-09 14:31:10,786 ERROR [io.qua.oid.run.CodeAuthenticationMechanism] (vert.x-eventloop-thread-3) ID token verification failure: io.quarkus.security.AuthenticationFailedException

Are there any other settings you want me to verify?

[sberyozkin]

@sschellh Thanks for verifying it. Hmm, so you have a binary access token, can you please check if the introspection response contains an expiry date for the inactive token (I can only think of putting a breakpoint in OidcProvider ? Right now if the introspection response says the token is inactive, 401 is immediately returned, but we can tweak it and return an exception requiring a refresh if it can be confirmed the token is inactive because it has expired

[sberyozkin]

Or may be we just can add a property always requiring a refresh whatever the verification failure is

[sschellh]

Hello @sberyozkin

If the access token is expired, then the introspection response is just like that: { "active": false}.

I would expect following behavior (not sure however if this is possible):

• If the id_token is expired (information contained in the token), but the session is still valid (i.e. not expired yet), then the refresh token is used to refresh the access_token and the id_token.
• If the refresh process fails, then the user receives 401 and the q_session is removed.

Ideally that process should be independent from the question whether access token validation is activated or not.
(i.e. quarkus.oidc.authentication.verify-access-token shall not have any relevance in that process as the dependency is not really obvious and intuitive).

Another question I ask myself in this context is why the session duration is derived from the id_token lifetime and not from the maximum out of id_token lifetime and refresh token lifetime). Because as long as the refresh token is valid, the session can be kept active as one can always ask for a new access_token and id_token.

[sberyozkin]

@sschellh

Ideally that process should be independent from the question whether access token validation is activated or not.

But the whole problem is that your application requires UserInfo which can only be acquired with the active access token. For access tokens in JWT format it has been fixed - i.e, we can locally detect it has expired and if yes, refresh it.
In your case we can't do it because the token is binary, and we don't know why exactly the token is not active, may be it was removed by the admin

Another question I ask myself in this context is why the session duration is derived from the id_token lifetime and not from the maximum out of id_token lifetime and refresh token lifetime)

See #30766

[sberyozkin]

The ID token verification failure in this context if this is what is logged is confusing and will have to be fixed, the code authentication mechanism dealing with the session is not aware why the OIDC identity provider failed the authentication request, so it should log a more precise message that ID token / session is valid but the request can not continue due to this or that failure

[sberyozkin]

@sschellh, more about

Another question I ask myself in this context is why the session duration is derived from the id_token lifetime and not from the maximum out of id_token lifetime and refresh token lifetime)

So I linked to #30766, but does your provider return refresh token in JWT format ? How will Quarkus know what the refresh token lifespan is ?

[sschellh]

@sberyozkin

So I linked to #30766, but does your provider return refresh token in JWT format ? How will Quarkus know what the refresh token lifespan is ?

The introspection endpoint returns the expire information of the refresh token.

[sschellh]

The ID token verification failure in this context if this is what is logged is confusing and will have to be fixed, the code authentication mechanism dealing with the session is not aware why the OIDC identity provider failed the authentication request, so it should log a more precise message that ID token / session is valid but the request can not continue due to this or that failure

And the token refresh process should be started (if there is a refresh token).

[sberyozkin]

@sschellh

The introspection endpoint returns the expire information of the refresh token.

Are you sure ? Did you mean a code exchange response and refresh_token_expires_in property ? Refresh tokens are never introspected by Quarkus.

And the token refresh process should be started (if there is a refresh token).

Only if it is proven that the ID or access token has expired. In your case we don't have any proof the access token has expired. If the introspection response was returning the access token expiry time then it would prove it.

Without this proof the DDOS surface for the binary tokens increases dramatically, anyone can send random blobs and Quarkus will be stressing the OIDC provider with the bound to fail introspection and/or refresh attempts, etc

However if the code exchange response returns expires_in for the binary token then this property can be used however the challenge there is how to retain this property in the session cookie - it has to be secured to be trusted to use its value to decide if the refresh is required. I need to think about it, how to secure the expires_in property for the binary tokens, may be I can get it signed with the client secret

[sschellh]

Are you sure ? Did you mean a code exchange response and refresh_token_expires_in property ? Refresh tokens are never introspected by Quarkus.

No. The code exchange response does not contain this information. Only the introspection endpoint.

[sberyozkin]

@sschellh

No. The code exchange response does not contain this information. Only the introspection endpoint.

I'm sorry, but if you'd like the refresh token lifetime be automatically used to to set the session lifespan then it will need to be returned from the code exchange response. I know Keycloak does it, Microsoft (https://learn.microsoft.com/en-us/linkedin/shared/authentication/programmatic-refresh-tokens) and possibly others. Requesting a remote refresh token introspection just to get its expiry time seems too expensive.

That is though is not really related to this issue. Can you give me a favor and confirm your provider returns expires_in for the access token, either in the code exchange or the access token introspection response (as we need to introspect the binary access token anyway) ? If it does I can try to build a fix for this issue around that.
Thanks

[sberyozkin]

@sschellh Re the refresh token, that said, since auto-extending the session to the lifetime of RT will be optional, i.e, require a user approval (because of the RT lifetime is 1 month and ID token lifetime is 1 day then I don't think it will be wise for Quarkus support the session for 1 month out of the box - in real deployments users may not want that and may prefer re-authentication), I may as well try a falback with a refresh token introspection. I'd like quarkus-oidc support all sort of variations, I'll see how that goes when I get a chance to look at that issue.
For now lets finalize the details related to the access token expiry time, see my prev comment

[sschellh]

That is though is not really related to this issue. Can you give me a favor and confirm your provider returns expires_in for the access token, either in the code exchange or the access token introspection response (as we need to introspect the binary access token anyway) ? If it does I can try to build a fix for this issue around that. Thanks

Yes the code exchange and the access token introspection response contain the expires_in

[sberyozkin]

@sschellh sounds good. Thanks

[sberyozkin]

@sschellh FYI, #31811, that update is all what should be required to get this issue fixed, I won't have time right now to add a test as we have another test for exactly the same scenario for JWT access tokens without the introspection, have a look please at this PR if you can to confirm it resolves the problem. Note the (code flow) access token verification should be enabled as discussed above.

[srozange]

@sberyozkin I have a similar issue (401 when requesting user info and (opaque) access token expired).
I added verify-access-token=true and used your PR but I found out that my provider does not return the "exp" field (introspectionResult only contains the active field). I don't know if there is a solution to this issue (I asked the provider if can return this field but I have little hope).

[sberyozkin]

@srozange Sorry I missed your ping, I'm actually trying to complete that PR right now, just struggling a bit with the test.

We need to take it step by step, some providers will return that in the introspection response, so the PR will work for those cases. The next option is to retain the binary access token expiry date when it is provided in the code exchange response. I'll look into it at some point after the current PR is complete

[sberyozkin]

@srozange Please create another issue to have access token expiry saved un a cookie, it will be tricky and likely require encryption, but let's track it, thanks