New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Google+ shutdown will break OAuth provider #6069

Closed
Tratcher opened this Issue Dec 20, 2018 · 40 comments

Comments

@Tratcher
Copy link
Member

Tratcher commented Dec 20, 2018

The Authentication.Google package implements OAuth2 with Google services. However, it uses Google+ to fetch additional user information.

public static readonly string UserInformationEndpoint = "https://www.googleapis.com/plus/v1/people/me";

ClaimActions.MapJsonKey(ClaimTypes.NameIdentifier, "id");
ClaimActions.MapJsonKey(ClaimTypes.Name, "displayName");
ClaimActions.MapJsonSubKey(ClaimTypes.GivenName, "name", "givenName");
ClaimActions.MapJsonSubKey(ClaimTypes.Surname, "name", "familyName");
ClaimActions.MapJsonKey("urn:google:profile", "url");
ClaimActions.MapCustomJson(ClaimTypes.Email, GoogleHelper.GetEmail);

"The Google+ Sign-in feature is fully deprecated and is being shut down on March 7, 2019. This will be a progressive shutdown, with intermittent failures starting as early as January 28, 2019. Developers should migrate to the more comprehensive Google Sign-in authentication system." ~https://developers.google.com/+/web/signin/

This is a patch candidate all the way down to 1.0 and Katana. @muratg @blowdart

Proposals:

  • Find a new API that will give us basic information like name, e-mail, etc.. It's unlikely the transition would be seamless.
  • Deprecate the provider and show people how to use OpenIdConnect. This has the benefit of being a docs only change. It may not work for Katana though, we'll have to see if it supported enough ODIC features.
@jamesgurung

This comment has been minimized.

Copy link

jamesgurung commented Dec 21, 2018

The scopes requested by default are:

It seems like it's just the last one which is connected with Google+. Is the fix as simple as removing this scope? I don't think the default auth flow even uses it?

This would be a breaking change, but most common use cases are covered by the other scopes - and it's going to break anyway when Google+ is shut down.

@Tratcher

This comment has been minimized.

Copy link
Member

Tratcher commented Dec 21, 2018

It's the call out to https://www.googleapis.com/plus/v1/people/me that's really going to break. That's where we get names, e-mails, etc.. We need to find a replacement.

@hempels

This comment has been minimized.

Copy link

hempels commented Dec 21, 2018

I expect you can simply replace the call to /plus/v1/people/me with a call to /userinfo/v2/me.

It should return a response comparable to the one returned by the deprecated + API and not require any further changes (aside from removing the plus.me scope.)

https://developers.google.com/apis-explorer/#search/userinfo/m/oauth2/v2/oauth2.userinfo.v2.me.get

Alternatively, you can skip that call if using their TokenInfo endpoint to handle token validation. If the email and profile scopes are included, it automatically returns those fields.

@rhubrightj

This comment has been minimized.

Copy link

rhubrightj commented Dec 21, 2018

Pardon my ignorance on the subject.... but where do I find this call in an asp.net web api project? I'm only using token validation.

I know I'm using it somewhere, according to google I'm sending calls to plus.people.get

I do have reference to Microsoft.Owin.Security.Google

@theaswanson

This comment has been minimized.

Copy link

theaswanson commented Dec 21, 2018

Pardon my ignorance on the subject.... but where do I find this call in an asp.net web api project? I'm only using token validation.

I know I'm using it somewhere, according to google I'm sending calls to plus.people.get

I do have reference to Microsoft.Owin.Security.Google

If you're using ASP.Net Core 2.2, you might have the following code in your Startup.cs file which makes use of the OAuth provider (taken from https://docs.microsoft.com/en-us/aspnet/core/security/authentication/social/google-logins?view=aspnetcore-2.2):

services.AddAuthentication().AddGoogle(googleOptions => { googleOptions.ClientId = Configuration["Authentication:Google:ClientId"]; googleOptions.ClientSecret = Configuration["Authentication:Google:ClientSecret"]; });

I'm currently using this and am affected by this issue.

@rhubrightj

This comment has been minimized.

Copy link

rhubrightj commented Dec 21, 2018

Definitely a breaking change on my site... I hope it's as simple as just updating Microsoft.Owin.Security.Google NuGet pkg when they are able to update it.

@ThoughtHopper

This comment has been minimized.

Copy link

ThoughtHopper commented Dec 21, 2018

Yup this will break our stuff.
Google has really screwed us up by pushing the date to end of march.

I tried changing the endpoints to the ones given back by the well-known configuration:
https://accounts.google.com/.well-known/openid-configuration

The current code works until it tries to retrieve the userinfo.
The response is not the same format as the response given by google+ apis.

For users like us, we do not really need to poke the userinfo endpoint, the id_token already contains information about the user. I guess in some cases it may be easier to poke the user endpoint than to actually verify the id_token.

I hope there is an answer soon from the owners of the project, Google has threaten to start failing requests as early as January 28,2019.

@muratg muratg added this to the 2.2.2 milestone Dec 21, 2018

@muratg

This comment has been minimized.

Copy link
Member

muratg commented Dec 21, 2018

@4deeptech

This comment has been minimized.

Copy link

4deeptech commented Dec 21, 2018

I confirmed that manually disabling the Google+ API in their developer console does indeed break the process. I tried enabling their People API hoping they may do some magic on their side to redirect the request but no luck there. According to their documentation, the People API allows for the profile, email, and some other related profile scopes: https://developers.google.com/people/v1/how-tos/authorizing#OAuth2Authorizing

@Tratcher

This comment has been minimized.

Copy link
Member

Tratcher commented Dec 21, 2018

Workaround, all the user info has changed format so you need to remap everything:

    .AddGoogle(o =>
            {
                o.ClientId = Configuration["google:clientid"];
                o.ClientSecret = Configuration["google:clientsecret"];
                o.UserInformationEndpoint = "https://www.googleapis.com/oauth2/v2/userinfo";
                o.ClaimActions.Clear();
                o.ClaimActions.MapJsonKey(ClaimTypes.NameIdentifier, "id");
                o.ClaimActions.MapJsonKey(ClaimTypes.Name, "name");
                o.ClaimActions.MapJsonKey(ClaimTypes.GivenName, "given_name");
                o.ClaimActions.MapJsonKey(ClaimTypes.Surname, "family_name");
                o.ClaimActions.MapJsonKey("urn:google:profile", "link");
                o.ClaimActions.MapJsonKey(ClaimTypes.Email, "email");
                o.ClaimActions.MapJsonKey("urn:google:image", "picture");
            })

[Edit 1/3/19]: Updated to use a different endpoint.

This should work with ASP.NET Core 2.0 and later.

Does anybody here have interest in 1.0 or 1.1?

I'll follow up on the the Microsoft.Owin components.

@trurl123

This comment has been minimized.

Copy link

trurl123 commented Dec 21, 2018

Workaround, all the user info has changed format so you need to remap everything:

This does not work.
"Error loading external login information. "

@theaswanson

This comment has been minimized.

Copy link

theaswanson commented Dec 21, 2018

Workaround, all the user info has changed format so you need to remap everything:

This does not work.
"Error loading external login information. "

Can confirm: I get the same error on my end. Using ASP.NET Core 2.1.

@Tratcher

This comment has been minimized.

Copy link
Member

Tratcher commented Dec 21, 2018

@HaoK is Identity looking for the NameIdentifier claim? I wonder if sub is the same as the old Id claim.

What if you add:
o.ClaimActions.MapJsonKey(ClaimTypes.NameIdentifier, "sub");

Ah, yes, there it is. https://github.com/aspnet/Identity/blob/fcc02103aa10dcdd8759e0463cac2717114f3c1e/src/Identity/SignInManager.cs#L611

Edit: I updated my sample above.

@theaswanson

This comment has been minimized.

Copy link

theaswanson commented Dec 21, 2018

@HaoK is Identity looking for the NameIdentifier claim? I wonder if sub is the same as the old Id claim.

What if you add:
o.ClaimActions.MapJsonKey(ClaimTypes.NameIdentifier, "sub");

Ah, yes, there it is. https://github.com/aspnet/Identity/blob/fcc02103aa10dcdd8759e0463cac2717114f3c1e/src/Identity/SignInManager.cs#L611

Adding o.ClaimActions.MapJsonKey(ClaimTypes.NameIdentifier, "sub"); did the trick! Login is working now with Google+ API disabled.

@Tratcher

This comment has been minimized.

Copy link
Member

Tratcher commented Dec 21, 2018

Did it recognize you as the same user, or as a new user?

@theaswanson

This comment has been minimized.

Copy link

theaswanson commented Dec 21, 2018

Same user. I used two different accounts and both logged in correctly.

@HaoK

This comment has been minimized.

Copy link
Member

HaoK commented Dec 21, 2018

Yeah we use the Name identifier to use as the key to associate logins in identity, and the email which we use as the user name

@Tratcher

This comment has been minimized.

Copy link
Member

Tratcher commented Dec 22, 2018

Updates for Microsoft.Owin.Security.Google are being tracked at aspnet/AspNetKatana#251. I've posted a temporary workaround there.

@dwdickens

This comment has been minimized.

Copy link

dwdickens commented Dec 24, 2018

some of us are still using asp.net core 1.1 - interested if there is a simple work around for us as well. Thanks for the short notice google!

@Tratcher

This comment has been minimized.

Copy link
Member

Tratcher commented Dec 24, 2018

@dwdickens the Microsoft.Owin example I posted here should easily adapt for Asp.Net Core 1.0 and 1.1. Let me know if it gives you any trouble.

@ADefWebserver

This comment has been minimized.

Copy link

ADefWebserver commented Dec 26, 2018

the Microsoft.Owin example I posted here should easily adapt for Asp.Net Core 1.0 and 1.1. Let me know if it gives you any trouble.

I can confirm it works in .Net 4.5.2 if you update Microsoft.Owin.Security.Google to version 4.0.0

@earllocker

This comment has been minimized.

Copy link

earllocker commented Dec 28, 2018

Workaround, all the user info has changed format so you need to remap everything:

    .AddGoogle(o =>
            {
                o.ClientId = Configuration["google:clientid"];
                o.ClientSecret = Configuration["google:clientsecret"];
                o.UserInformationEndpoint = "https://openidconnect.googleapis.com/v1/userinfo";
                o.ClaimActions.Clear();
                o.ClaimActions.MapJsonKey(ClaimTypes.NameIdentifier, "sub");
                o.ClaimActions.MapJsonKey(ClaimTypes.Name, "name");
                o.ClaimActions.MapJsonKey(ClaimTypes.GivenName, "given_Name");
                o.ClaimActions.MapJsonKey(ClaimTypes.Surname, "family_Name");
                o.ClaimActions.MapJsonKey("urn:google:profile", "profile");
                o.ClaimActions.MapJsonKey(ClaimTypes.Email, "email");
                o.ClaimActions.MapJsonKey("urn:google:image", "picture");
            })

This should work with ASP.NET Core 2.0 and later.

Does anybody here have interest in 1.0 or 1.1?

I'll follow up on the the Microsoft.Owin components.

This worked great. Just an FYI that given_name and family_name should be all lowercase otherwise it won't pull that information correctly. Thanks.

@Tratcher

This comment has been minimized.

Copy link
Member

Tratcher commented Dec 28, 2018

@earllocker I updated the sample, thanks.

@kanadaj

This comment has been minimized.

Copy link

kanadaj commented Jan 1, 2019

@Tratcher the hotfix works for some users, but for some others it returns with a

System.Exception: The oauth state was missing or invalid.

Not sure why exactly. Maybe missing profile picture? I don't have enough info to figure out right now.

@Tratcher

This comment was marked as off-topic.

Copy link
Member

Tratcher commented Jan 1, 2019

That error doesn't seem directly related. It may be a secondary error. Turn on you application logs to see if there's anything else. Also capture a fiddler trace file.

@alanleouk

This comment has been minimized.

Copy link

alanleouk commented Jan 3, 2019

I modified my code to use the endpoints provided by https://accounts.google.com/.well-known/openid-configuration but hit a problem where the response from the user information endpoint wasn't deserialising. I modified my user information endpoint to https://www.googleapis.com/oauth2/v2/userinfo which seems to be compatible with the old mappings. I have also taken out the profile scope for my implementation.

Shouldn't this library get it's configuration from https://accounts.google.com/.well-known/openid-configuration in the future?

authenticationBuilder.AddGoogle(options =>
{
	options.ClientId = Configuration["auth:google:clientid"];
	options.ClientSecret = Configuration["auth:google:clientsecret"];

	options.AuthorizationEndpoint = "https://accounts.google.com/o/oauth2/v2/auth";
	options.TokenEndpoint = "https://oauth2.googleapis.com/token";
	options.UserInformationEndpoint = "https://www.googleapis.com/oauth2/v2/userinfo";
	
	options.Scope.Clear();
	options.Scope.Add("openid");
	options.Scope.Add("email");

	options.ClaimActions.Clear();
	options.ClaimActions.MapJsonKey(ClaimTypes.NameIdentifier, "id");
	options.ClaimActions.MapCustomJson(ClaimTypes.Email, GoogleHelper.GetEmail);
});
@kanadaj

This comment was marked as off-topic.

Copy link

kanadaj commented Jan 3, 2019

That error doesn't seem directly related. It may be a secondary error. Turn on you application logs to see if there's anything else. Also capture a fiddler trace file.

There is no secondary error. Also, don't have Fiddler in the pipeline so can't really make logs with that.

Generally speaking though, I find that External Login errors seem very hard to debug since the errors are very generic. "Correlation failed" and "The oauth state was missing or invalid." aren't really actionable. A more detailed logging setup built into the pipeline would make things much easier...

@Tratcher

This comment was marked as off-topic.

Copy link
Member

Tratcher commented Jan 3, 2019

@kanadaj agreed, though the trouble with those situations is that the error usually occurs on the client or IDP, the server doesn't know why it happened.

@Tratcher

This comment has been minimized.

Copy link
Member

Tratcher commented Jan 3, 2019

@alanleouk the .well-known endpoint is for OpenIdConnect, not OAuth2 (though the two are related). There is a proposal to replace this library with the existing OpenIdConnect implementation in a future release.

@thomaslevesque

This comment has been minimized.

Copy link
Contributor

thomaslevesque commented Jan 3, 2019

Using OpenID Connect is a good alternative anyway. It's just as easy to setup, you don't need a client secret, and you can get all the information from the id_token without requiring an extra API call (just add email to the requested scopes to get the email address)

@kanadaj

This comment was marked as off-topic.

Copy link

kanadaj commented Jan 3, 2019

@Tratcher The problem is that even if Sentry (what I use) picks up the request details, it's usually still just encrypted tokens and such. I imagine it'd be the same with Fiddler. So it'd be helpful to just log everything that the server actually knows about the login attempt.

@Tratcher

This comment has been minimized.

Copy link
Member

Tratcher commented Jan 3, 2019

@thomaslevesque you don't need a client secret? Last I checked Google only supported the code response type so the client secrete was required to exchange the code for an idtoken. I double checked and they do support token flows, but only for the client.

https://developers.google.com/identity/protocols/OpenIDConnect#authenticationuriparameters

response_type | (Required) | If the value is code, launches a Basic flow, requiring a POST to the token endpoint to obtain the tokens. If the value is token id_token or id_token token, launches an Implicit flow, requiring the use of Javascript at the redirect URI to retrieve tokens from the URI #fragment.
@Tratcher

This comment was marked as off-topic.

Copy link
Member

Tratcher commented Jan 3, 2019

@kanadaj let's take that discussion to another thread.

@thomaslevesque

This comment has been minimized.

Copy link
Contributor

thomaslevesque commented Jan 3, 2019

@thomaslevesque you don't need a client secret? Last I checked Google only supported the code response type so the client secrete was required to exchange the code for an idtoken. I double checked and they do support token flows, but only for the client.

You can get an id_token directly, without needing an authorization code. Just use id_token as the response_type (this is the default when you use AddOpenIdConnect). In this case you don't need the client secret.

@thomaslevesque

This comment has been minimized.

Copy link
Contributor

thomaslevesque commented Jan 3, 2019

@Tratcher I see the Google documentation is misleading. It says:

If the value is token id_token or id_token token, launches an Implicit flow, requiring the use of Javascript at the redirect URI to retrieve tokens from the URI #fragment.

But I just tried it in my app, and it works just fine. Probably because it's using form_post as the response_mode, not fragment.

@Tratcher

This comment has been minimized.

Copy link
Member

Tratcher commented Jan 3, 2019

@thomaslevesque Ah, it does work. Well there's an undocumented feature for us.

Tratcher added a commit that referenced this issue Jan 3, 2019

@Tratcher

This comment has been minimized.

Copy link
Member

Tratcher commented Jan 3, 2019

See #6338 for the proposed patch. Note this uses a different endpoint than proposed earlier. I've updated the mitigation above to match.

@AnthonyMascia

This comment has been minimized.

Copy link

AnthonyMascia commented Jan 3, 2019

See #6338 for the proposed patch. Note this uses a different endpoint than proposed earlier. I've updated the mitigation above to match.

Is there any specific reason you changed the new endpoint? Is it because the current OpenID Connect endpoint might be replaced in the future?

@Tratcher

This comment has been minimized.

Copy link
Member

Tratcher commented Jan 3, 2019

@AnthonyMascia this one seemed more appropriate for an OAuth2 component, and it's also a later version of the API. The prior one was an OpenIdConnect specific endpoint and an older API version.

@gilm0079

This comment has been minimized.

Copy link

gilm0079 commented Jan 8, 2019

@Tratcher Thanks for submitting the fix for this. I'll keep an eye on the pull request. #6338 and re-post a question in IdentityServer/IdentityServer4#2931 to see if the IdentityServer4 build will need to reference the new core update.

Tratcher added a commit that referenced this issue Jan 14, 2019

Update Google Auth UserInfo endpoint (#6338)
* Update Google Auth UserInfo endpoint #6069

* Add Google to PatchConfig

@Tratcher Tratcher added the 3 - Done label Jan 15, 2019

@Tratcher Tratcher closed this Jan 15, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment