Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

OAuth2 access token response parsing fails with nested JSON object #6463

Closed
buckett opened this issue Jan 21, 2019 · 22 comments
Closed

OAuth2 access token response parsing fails with nested JSON object #6463

buckett opened this issue Jan 21, 2019 · 22 comments

Comments

@buckett
Copy link

@buckett buckett commented Jan 21, 2019

Summary

When parsing OAuth2 access token response a nested JSON object causes the response parsing to fail.

Actual Behavior

When attempting to use Spring Security OAuth to allow logins against a provider that responds with objects in their access token reponse an error message is shown:

An error occurred reading the OAuth 2.0 Access Token Response: JSON parse error: Cannot deserialize instance of `java.lang.String` out of START_OBJECT token; nested exception is com.fasterxml.jackson.databind.exc.MismatchedInputException: Cannot deserialize instance of `java.lang.String` out of START_OBJECT token
 at [Source: (ByteArrayInputStream); line: 7, column: 14] (through reference chain: java.util.LinkedHashMap["object"]); nested exception is org.springframework.http.converter.HttpMessageNotReadableException: JSON parse error: Cannot deserialize instance of `java.lang.String` out of START_OBJECT token; nested exception is com.fasterxml.jackson.databind.exc.MismatchedInputException: Cannot deserialize instance of `java.lang.String` out of START_OBJECT token
 at [Source: (ByteArrayInputStream); line: 7, column: 14] (through reference chain: java.util.LinkedHashMap["object"])

Expected Behavior

According to the OAuth spec https://tools.ietf.org/html/rfc6749#section-5.1 clients must ignore values they don't understand. The value should either end up in the additionalParameters of the OAuth2AccessTokenResponse or it should be ignored.

Configuration

Jackson is being used to parse the JSON response (seems to be default in my spring-boot application).

Version

Spring Security 5.1.3, issue also looks to be present on master.

Sample

You can see a test case that currently fails in: https://github.com/spring-projects/spring-security/compare/master...buckett:oauth-response?expand=1

@buckett
Copy link
Author

@buckett buckett commented Jan 21, 2019

An example response that triggers this bug is:

{
   "access_token": "access-token-1234",
   "token_type": "bearer",
   "expires_in": "3600",
   "scope": "read write",
   "refresh_token": "refresh-token-1234",
   "object": {"param": "value" },
   "custom_parameter": "custom-value"
}

@buckett
Copy link
Author

@buckett buckett commented Jan 21, 2019

To workaround this I copied OAuth2AccessTokenResponseHttpMessageConverter into my own project and updated the tokenResponseParameters to be a Map<String, Object>. Here's a gist of the file: https://gist.github.com/buckett/7fe338901b4fcfefb6cfce637629af3f

Then I just connect this converter up to the RestTemplate that's used by the OAuth 2 code:

        .oauth2Login().tokenEndpoint().accessTokenResponseClient(accessTokenResposeClient());

[..snipped..]

    private OAuth2AccessTokenResponseClient<OAuth2AuthorizationCodeGrantRequest> accessTokenResposeClient() {
        DefaultAuthorizationCodeTokenResponseClient client = new DefaultAuthorizationCodeTokenResponseClient();
        RestTemplate restTemplate = new RestTemplate(Arrays.asList(
                new FormHttpMessageConverter(), new OAuth2AccessTokenResponseHttpMessageConverter()));
        HttpClient requestFactory = HttpClientBuilder.create().build();
        restTemplate.setRequestFactory(new HttpComponentsClientHttpRequestFactory(requestFactory));
        restTemplate.setErrorHandler(new OAuth2ErrorResponseErrorHandler());
        client.setRestOperations(restTemplate);
        return client;
    }

This isn't the actual code I'm using as I've removed other changes that aren't needed so it may not work as expected.

@jgrandja jgrandja self-assigned this Jan 22, 2019
@jgrandja
Copy link
Contributor

@jgrandja jgrandja commented Jan 22, 2019

@buckett The solution you implemented with providing your own AbstractHttpMessageConverter<OAuth2AccessTokenResponse> is exactly what you would do for these kind of custom token responses. This is the main extension point for RestTemplate, the HttpMessageConverter for request/response serializing/deserializing. OAuth2AccessTokenResponseHttpMessageConverter is simply a default implementation.

It's not clear to me if you're still having a problem or are you expecting a different way of implementing your setup?

As per spec:

The parameters are serialized into a JavaScript Object Notation (JSON)
structure by adding each parameter at the highest structure level.
Parameter names and string values are included as JSON strings.
Numerical values are included as JSON numbers. The order of
parameters does not matter and can vary.

Based on my understanding how the spec reads, each parameter must be at the highest (root) structure level and are either strings or numbers. However, your example token response has a JSON object value at the root level with the parameter names/values at the next level below the object. So therefore the value of the root level parameter is an object and not a string or number as the spec dictates.

Does this make sense?

@buckett
Copy link
Author

@buckett buckett commented Jan 22, 2019

When I read the spec I was reading that paragraph as applying to the previously specified parameters (access_token, token_type, expires_in, refresh_token, scope).

It also goes on to say that the "the client MUST ignore unrecognized values names in the response", which again suggests we should be a little more forgiving in how we deal with unexpected value names.

If nothing else it would have been helpful if this was easier to code around as this took me a little while to debug (I came across this upgrading from Spring 5 to Spring 5.1, Spring 5 was forgiving of extra values I think).

@jgrandja
Copy link
Contributor

@jgrandja jgrandja commented Jan 22, 2019

As per spec:

The client MUST ignore unrecognized value names in the response. The
sizes of tokens and other values received from the authorization
server are left undefined. The client should avoid making
assumptions about value sizes.
The authorization server SHOULD
document the size of any value it issues.

I see your point based on the reference in bold.

If nothing else it would have been helpful if this was easier to code around

Our goal is to make it easy for the user so this is valuable feedback for us. Do you have a suggestion on improving? Are you possibly interested in submitting a PR for this improvement?

@Gyurmatag
Copy link

@Gyurmatag Gyurmatag commented Jan 25, 2020

When will this be fixed?

@jgrandja jgrandja added this to the 5.3.x milestone Jan 29, 2020
@jgrandja
Copy link
Contributor

@jgrandja jgrandja commented Jan 29, 2020

@Gyurmatag We'll do our best to fix this soon.

@jgrandja jgrandja changed the title When parsing OAuth2 access token response a nested JSON object causes the response parsing to fail OAuth2 access token response parsing fails with nested JSON object Feb 24, 2020
@jgrandja jgrandja self-assigned this Feb 24, 2020
@jgrandja jgrandja removed this from the 5.3.x milestone Feb 24, 2020
@jgrandja jgrandja added this to the 5.3.0 milestone Feb 24, 2020
@jgrandja jgrandja closed this in fb2bbd7 Feb 24, 2020
@jscoder1009
Copy link

@jscoder1009 jscoder1009 commented Feb 25, 2020

when will this change be available? I checked my project maven dependencies and I am on spring-security-oauth2-core-5.2.1.RELEASE.jar. Once the change is live which version will have this change? thanks!

@jscoder1009
Copy link

@jscoder1009 jscoder1009 commented Feb 25, 2020

To workaround this I copied OAuth2AccessTokenResponseHttpMessageConverter into my own project and updated the tokenResponseParameters to be a Map<String, Object>. Here's a gist of the file: https://gist.github.com/buckett/7fe338901b4fcfefb6cfce637629af3f

Then I just connect this converter up to the RestTemplate that's used by the OAuth 2 code:

        .oauth2Login().tokenEndpoint().accessTokenResponseClient(accessTokenResposeClient());

[..snipped..]

    private OAuth2AccessTokenResponseClient<OAuth2AuthorizationCodeGrantRequest> accessTokenResposeClient() {
        DefaultAuthorizationCodeTokenResponseClient client = new DefaultAuthorizationCodeTokenResponseClient();
        RestTemplate restTemplate = new RestTemplate(Arrays.asList(
                new FormHttpMessageConverter(), new OAuth2AccessTokenResponseHttpMessageConverter()));
        HttpClient requestFactory = HttpClientBuilder.create().build();
        restTemplate.setRequestFactory(new HttpComponentsClientHttpRequestFactory(requestFactory));
        restTemplate.setErrorHandler(new OAuth2ErrorResponseErrorHandler());
        client.setRestOperations(restTemplate);
        return client;
    }

This isn't the actual code I'm using as I've removed other changes that aren't needed so it may not work as expected.

how are you able to cast "client" to OAuth2AccessTokenResponseClient? any help is greatly appreciated.

@rwinch
Copy link
Member

@rwinch rwinch commented Feb 26, 2020

The change is in 5.3.0.BUILD-SNAPSHOT and will be in 5.3.0.RELEASE as indicated by the milestone selected with this issue.

@jscoder1009
Copy link

@jscoder1009 jscoder1009 commented Feb 26, 2020

The change is in 5.3.0.BUILD-SNAPSHOT and will be in 5.3.0.RELEASE as indicated by the milestone selected with this issue.

thank you for the update.

@jscoder1009
Copy link

@jscoder1009 jscoder1009 commented Feb 26, 2020

The change is in 5.3.0.BUILD-SNAPSHOT and will be in 5.3.0.RELEASE as indicated by the milestone selected with this issue.

I as was just wondering if there any options to reference to the snapshot version ? our app is live but broken for facebook login. I greatly appreciate your response.

@rwinch
Copy link
Member

@rwinch rwinch commented Feb 26, 2020

Please see the relevant section of the reference documentation https://docs.spring.io/spring-security/site/docs/5.3.0.BUILD-SNAPSHOT/reference/html5/#getting

@jscoder1009
Copy link

@jscoder1009 jscoder1009 commented Mar 7, 2020

I upgraded to 5.3.0 for org.springframework.security.oauth2.core and still facing the issue. I am using facebook OAuth2 for user authentication.

org.springframework.http.converter.HttpMessageNotReadableException: An error occurred reading the OAuth 2.0 Error: Invalid JSON input: Cannot deserialize instance of java.lang.String out of START_OBJECT token; nested exception is com.fasterxml.jackson.databind.exc.MismatchedInputException: Cannot deserialize instance of java.lang.String out of START_OBJECT token
at [Source: (sun.net.www.protocol.http.HttpURLConnection$HttpInputStream); line: 1, column: 10] (through reference chain: java.util.LinkedHashMap["error"]); nested exception is org.springframework.http.converter.HttpMessageNotReadableException: Invalid JSON input: Cannot deserialize instance of java.lang.String out of START_OBJECT token; nested exception is com.fasterxml.jackson.databind.exc.MismatchedInputException: Cannot deserialize instance of java.lang.String out of START_OBJECT token
at [Source: (sun.net.www.protocol.http.HttpURLConnection$HttpInputStream); line: 1, column: 10] (through reference chain: java.util.LinkedHashMap["error"])
at org.springframework.security.oauth2.core.http.converter.OAuth2ErrorHttpMessageConverter.readInternal(OAuth2ErrorHttpMessageConverter.java:78) ~[spring-security-oauth2-core-5.3.0.RELEASE.jar!/:5.3.0.RELEASE]

@jgrandja
Copy link
Contributor

@jgrandja jgrandja commented Mar 9, 2020

@jscoder1009 This ticket is related to a parsing error for a Successful Token Response. However, based on the stacktrace, the issue you are having happens during an attempt to parse an Error Token Response.

org.springframework.security.oauth2.core.http.converter.OAuth2ErrorHttpMessageConverter.readInternal(OAuth2ErrorHttpMessageConverter.java:78)

It looks like Facebook does not return a correctly formatted OAuth 2.0 Error as defined in 5.2. Error Response.

You can override the default OAuth2ErrorResponseErrorHandler assigned to RestTemplate.setErrorHandler() in the associated OAuth2AccessTokenResponseClient (eg. DefaultAuthorizationCodeTokenResponseClient) to work around this issue with Facebook. Please see the ref doc for further details on custom configuration.

@josephbleau
Copy link

@josephbleau josephbleau commented Mar 14, 2020

I landed here after a very long excursion! I wanted to report that Strava is also returning an object in one of their response attributes. Upgrading to 5.3.0 resolved the issue for me (was on 5.2.x).

https://developers.strava.com/docs/authentication/

Example Response

{
  "token_type": "Bearer",
  "expires_at": 1568775134,
  "expires_in": 21600,
  "refresh_token": "e5n567567...",
  "access_token": "a4b945687g...",
  "athlete": {
    #{summary athlete representation}
  }
}

Thanks everyone for their hard work!

@jscoder1009
Copy link

@jscoder1009 jscoder1009 commented Mar 22, 2020

spring:
security:
oauth2:
client:
registration:
facebook:
client-id: xxddd...
clientSecret: secrettt....
redirect-uri: https://example.com/login/oauth2/code/facebook

Issue was redirect-uri was going as http. I explicitly added callback URL with https and this fixed my issue.

@keyuls
Copy link

@keyuls keyuls commented Jun 21, 2020

@jscoder1009 where did you add callback URL with https?

@jmgomez77
Copy link

@jmgomez77 jmgomez77 commented Oct 20, 2021

Dear all,

It seems the bug appears again in 5.4.6. Is this possible? As a result, the upgrade of Keycloak from 10.0 to 15.0 fails, since new values were added to the openid well-known configuration.

Below is the stack.

Kind regards.


Caused by: java.lang.RuntimeException: com.nimbusds.oauth2.sdk.ParseException: Unexpected type of JSON object member with key mtls_endpoint_aliases
	at org.springframework.security.oauth2.client.registration.ClientRegistrations.parse(ClientRegistrations.java:232) ~[spring-security-oauth2-client-5.4.6.jar:5.4.6]
	at org.springframework.security.oauth2.client.registration.ClientRegistrations.lambda$oidc$0(ClientRegistrations.java:157) ~[spring-security-oauth2-client-5.4.6.jar:5.4.6]
	at org.springframework.security.oauth2.client.registration.ClientRegistrations.getBuilder(ClientRegistrations.java:209) ~[spring-security-oauth2-client-5.4.6.jar:5.4.6]
	... 90 common frames omitted
Caused by: com.nimbusds.oauth2.sdk.ParseException: Unexpected type of JSON object member with key mtls_endpoint_aliases
	at com.nimbusds.oauth2.sdk.util.JSONObjectUtils.getGeneric(JSONObjectUtils.java:161) ~[oauth2-oidc-sdk-8.36.jar:8.36]
	at com.nimbusds.oauth2.sdk.util.JSONObjectUtils.getJSONObject(JSONObjectUtils.java:827) ~[oauth2-oidc-sdk-8.36.jar:8.36]
	at com.nimbusds.oauth2.sdk.as.AuthorizationServerMetadata.parse(AuthorizationServerMetadata.java:2042) ~[oauth2-oidc-sdk-8.36.jar:8.36]
	at com.nimbusds.openid.connect.sdk.op.OIDCProviderMetadata.parse(OIDCProviderMetadata.java:1362) ~[oauth2-oidc-sdk-8.36.jar:8.36]
	at org.springframework.security.oauth2.client.registration.ClientRegistrations.parse(ClientRegistrations.java:229) ~[spring-security-oauth2-client-5.4.6.jar:5.4.6]
	... 92 common frames omitted
Caused by: com.nimbusds.oauth2.sdk.ParseException: Unexpected type: class java.util.LinkedHashMap
	at com.nimbusds.oauth2.sdk.util.JSONUtils.to(JSONUtils.java:98) ~[oauth2-oidc-sdk-8.36.jar:8.36]
	at com.nimbusds.oauth2.sdk.util.JSONObjectUtils.getGeneric(JSONObjectUtils.java:159) ~[oauth2-oidc-sdk-8.36.jar:8.36]
	... 96 common frames omitted

@simonpai
Copy link

@simonpai simonpai commented Oct 26, 2021

@jmgomez77
It seems to be a different issue caused by a bug in an underlying library com.nimbusds:oauth2-oidc-sdk:8.x:
https://bitbucket.org/connect2id/oauth-2.0-sdk-with-openid-connect-extensions/issues/377/nimbus-8-prevent-auto-discover-mechanism

It has been fixed in 9.x and I think you can work around it with maven/gradle dependency management.

@yomal
Copy link

@yomal yomal commented Nov 9, 2021

Hi all, I am facing the same issue using 5.1.7, and i am using this as for the client credentials flow.. Is there any workaround for this?

@laurocesar
Copy link

@laurocesar laurocesar commented Dec 26, 2021

I have the same issue in version 5.4.5 and Keycloak 16.1.0:

Caused by: java.lang.RuntimeException: com.nimbusds.oauth2.sdk.ParseException: Unexpected type of JSON object member with key mtls_endpoint_aliases
        at org.springframework.security.oauth2.client.registration.ClientRegistrations.parse(ClientRegistrations.java:232)
        at org.springframework.security.oauth2.client.registration.ClientRegistrations.lambda$oidc$0(ClientRegistrations.java:157)
        at org.springframework.security.oauth2.client.registration.ClientRegistrations.getBuilder(ClientRegistrations.java:209)
        ... 113 common frames omitted
Caused by: com.nimbusds.oauth2.sdk.ParseException: Unexpected type of JSON object member with key mtls_endpoint_aliases
        at com.nimbusds.oauth2.sdk.util.JSONObjectUtils.getGeneric(JSONObjectUtils.java:161)
        at com.nimbusds.oauth2.sdk.util.JSONObjectUtils.getJSONObject(JSONObjectUtils.java:827)
        at com.nimbusds.oauth2.sdk.as.AuthorizationServerMetadata.parse(AuthorizationServerMetadata.java:2042)
        at com.nimbusds.openid.connect.sdk.op.OIDCProviderMetadata.parse(OIDCProviderMetadata.java:1362)
        at org.springframework.security.oauth2.client.registration.ClientRegistrations.parse(ClientRegistrations.java:229)
        ... 115 common frames omitted
Caused by: com.nimbusds.oauth2.sdk.ParseException: Unexpected type: class java.util.LinkedHashMap
        at com.nimbusds.oauth2.sdk.util.JSONUtils.to(JSONUtils.java:100)
        at com.nimbusds.oauth2.sdk.util.JSONObjectUtils.getGeneric(JSONObjectUtils.java:159)
        ... 119 common frames omitted

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet