brokenAuthHeaderProviders - time to try something else?

[philpearl]

Perhaps add a BrokenAuthHeader to EndPoint?

The server bug behind this seems pretty common (it's also in Shopify, with domain names [shop].myshopify.com), and it seems unfortunate to need to update this library every time someone finds one. I'd bet 90% of the forks are to add another broken provider.

Let me know if you might accept a change like this and I'll work on it.

[rakyll]

The exported APIs shouldn't reflect that we have workarounds for buggy implementations. If there are any alternative solutions we can consider without touching the APIs, I would like to hear. A wildcard match might more be useful, why don't we convert the brokenAuthHeaderProviders into a regex list?

[philpearl]

Unfortunately the world is buggy... I'd think the API should reflect whatever is useful to operate in the world as we find it. I don't think a regex list would help particularly - you'd still need a code change to deal with a new site.

I can't see why you're happier to make code changes for every site with this bug than to make a single small code change to cope with all of them.

[rakyll]

you'd still need a code change to deal with a new site.

We're fine with maintaining the list for now.

The world may be buggy, but the users don't have to know about it. This package is a higher-level implementation of OAuth 2.0. Most of our users don't even truly know the spec and how OAuth 2.0 works. Exporting an API to turn on/off a workaround is unlikely to be user-friendly.

[philpearl]

I in no way agree with you, but its your project and I'm very grateful for it.

Please add [shop].myshopify.com to the list. The host name varies by shop, but no other change is required to get it to work.

[rakyll]

cc/ @bradfitz

Brad, do you think it might be worthy to export the busted provider's list, so the users can modify it?

Additional to the current bug report, our current solution doesn't help for those who can't upstream their domain names to the list for various reasons such as confidentiality.

[bradfitz]

I like the List Of Shame and a simpler API.

The latency to add to the List Of Shame should put pressure (directly or indirectly) on servers to comply with specs.

[flexd]

This is a very common problem sadly.. please add slack.com to the list of badly implemented APIs.

I like the clean API, but being able to append a host to the list at runtime or toggle the authheader manually would make this library usable.
Off-topic:
Alernatively, @rcrowley could you or someone else at Slack just fix your OAuth implementation?

[philpearl]

I think that's a Shame. And one of the most charmingly passive-aggressive attempts to get something fixed I've seen in a long time. Are you lobbying other OAUTH implementations to add a similar hidden list? Or perhaps your scheme is further along than I imagine and every language has an OAUTH implementation with a secret internal list of naughty hosts? That will show them.

Unfortunately, for the shorter, unqualified entries, once a service is on the list they can't then fix their implementation without adding a new domain name. Perhaps you should at least ask for prefixes with some amount of path.

[bradfitz]

Maybe the community should do something like publicsuffix.org with a shared list of sites which have quirks. I don't know.

The Go community values simplicity. This is an experiment to see if we can have both simplicity and still work. It's an ongoing experiment.

[philpearl]

Moving this configuration to a flag in the Endpoint doesn't feel to me less simple than a hard-coded list. I tried it in my own fork (https://github.com/philpearl/oauth2) and it feels quite nice. If the Endpoints are defined in the repo like the github, etc, ones are then the end-user interface is no more complex.

[ernesto-jimenez]

The latency to add to the List Of Shame should put pressure (directly or indirectly) on servers to comply with specs.

@bradfitz @rakyll unfortunately, I think it puts more pressure on the people consuming non-compliant servers than on the servers themselves.

This is penalising the package users, rather than the servers.

With that being said, I understand your point for not adding workarounds for non-compliant APIs, but at least this should be documented and warned in the package. Current usability means developers try to use the package, it fails, and then they dig into the package's source and find the internal whitelist.

If we are talking about better developer experience, in my opinion it would be better to clearly document the issue. I would also suggest for dropping the internal whitelist at all since the same non-compliant implementation might work or not depending on an internal whitelist most developers are not aware of.

[bradfitz]

Has anybody explored auto-detecting this condition?

[ernesto-jimenez]

@bradfitz IMHO, auto-detecting the condition would enable people building non-compliant providers. I think it should be very clear when a provider is broken rather than working around those silently.

@rakyll @bradfitz

I agree it should be very clear when a provider is broken so we can put some pressure on the server. However, in my opinion OAuth consumers should not be forced to fork the package in order to consume APIs from non-compliant providers.

I would suggest:

• Adding an exported field Config to assign function that builds the http.Request to be used when retrieving a token
• Give it a long name that highlights using that config option means the provider is broken. e.g: BrokenProviderRetrieveTokenHTTPRequest
• When config.BrokenProviderRetrieveTokenHTTPRequest is nil, use the spec-compliant http request.

This way, anybody working around a non-compliant provider can clearly see it's broken and we are not making it easy for them to do such integration. That would make it very clear the provider is broken and might help putting some pressure on the providers.

Ideally, the current internal whitelist would be removed and there would be no exception. Unfortunately, that would probably break applications depending on this package.

Here's a quick implementation

[philpearl]

@ernesto-jimenez it makes more sense to have a flag on the Endpoint as this behaviour varies with the Endpoint, not a particular configuration using/connecting to the Endpoint

[ernesto-jimenez]

@philpearl I didn't pay much attention to naming and where in the Config the field should be. I agree it would probably make sense to have it in Endpoint.

In my opinion, it might still be better to make integrating non-compliant providers a bit awkward rather than just setting up a flag. Integrating a provider that follows the spec would be seamless, but integrating one that doesn't would feel a bit more wrong and the person integrating it would need to learn what the difference is.

[philpearl]

I can't agree with making something more difficult for no reason other than
what is a very subjective "moral" judgement.

On Sat, 11 Apr 2015 00:29 Ernesto Jiménez notifications@github.com wrote:

@philpearl https://github.com/philpearl I didn't pay much attention to
naming and where in the Config the field should be. I agree it would
probably make sense to have it in Endpoint.

In my opinion, it might still be better to make integrating non-compliant
providers a bit awkward rather than just setting up a flag. Integrating a
provider that follows the spec would be seamless, but integrating one that
doesn't would feel a bit more wrong and the person integrating it would
need to learn what the difference is.

—
Reply to this email directly or view it on GitHub
#111 (comment).

[rakyll]

@bradfitz, server implementations differ. There is no standard response for the basic-auth-not-supported case. We can't fallback to the query parameters every time we receive a non-200 response from Basic Authorization.

[jfcote87]

According to RFC6749 section 2.3.1 page 16

Including the client credentials in the request-body using the two
parameters is NOT RECOMMENDED and SHOULD be limited to clients unable
to directly utilize the HTTP Basic authentication scheme (or other
password-based HTTP authentication schemes).  The parameters can only
be transmitted in the request-body and MUST NOT be included in the
request URI.

The text then proceeds to give an example of sending the client id and secret in a refresh request.

I don't see where using Basic Authentication is required.

[tt]

@jfcote87, the first paragraph of that section says:

The authorization server MUST support the HTTP Basic authentication scheme for authenticating clients that were issued a client password.

However, that section is about exchanging a client password for a token.

[tt]

The appropriate section describing client authentication for the token endpoint, section 3.2.1, does not require basic authentication (except in the aforementioned scenario):

Confidential clients or other clients issued client credentials MUST authenticate with the authorization server as described in Section 2.3 when making requests to the token endpoint.

Section 2.3 says:

The authorization server MAY support any suitable HTTP authentication scheme matching its security requirements.

Section 2.3.2 says:

If the client type is confidential, the client and authorization server establish a client authentication method suitable for the security requirements of the authorization server. The authorization server MAY accept any form of client authentication meeting its security requirements.

This seems to indicate that the current behavior is broken except for PasswordCredentialsToken.

[ernesto-jimenez]

@tt, how is the current behaviour broken? The package currently uses Basic auth to authenticate with the server.

    if !bustedAuth {
        req.SetBasicAuth(c.ClientID, c.ClientSecret)
    }

The issue is with the servers that rely on sending the client_secret in the request-body rather than using basic auth. HTTP Basic authentication is required when using a client password.

From the Section 2.3.1 you quoted (emphasis mine):

The authorization server MUST support the HTTP Basic authentication scheme for authenticating clients that were issued a client password.
[...]
Alternatively, the authorization server MAY support including the client credentials in the request-body
[...]
Including the client credentials in the request-body using the two parameters is NOT RECOMMENDED and SHOULD be limited to clients unable to directly utilize the HTTP Basic authentication scheme (or other password-based HTTP authentication schemes).

Which menas that servers that require receiving the client_secret in the request-body instead of using HTTP Basic authentication are not following the spec.

Section 2.3.2 about "Other Authentication Methods" is there to make the spec extensible and allow other means of authentication such as certificates. However, password authentication is clearly specified.

@tt, also, regarding:

This seems to indicate that the current behavior is broken except for PasswordCredentialsToken.

How is PasswordCredentialsToken OK but Exchange is broken when both use retrieveToken and authenticate the client the same way?

[tt]

@ernesto-jimenez, you're right. The language confused me. I interpreted the "password" in section 2.3.1 to relate to cases where you exchange a password for a token.

[bradfitz]

Yeah, maybe we can let callers do:

func init() {
   oauth2.RegisterBrokenAuthHeaderHost("http://prefix.com")
}

So then we can occasionally grep all of Github and find the public ones to add to our central list in batches.

That might be a good middle ground.

[adg]

I'm happy with that.

On 21 May 2015 at 07:20, Brad Fitzpatrick notifications@github.com wrote:

Yeah, maybe we can let callers do:

func init() {
oauth2.RegisterBrokenAuthHeaderHost("http://prefix.com")
}

So then we can occasionally grep all of Github and find the public ones to
add to our central list in batches.

That might be a good middle ground.

—
Reply to this email directly or view it on GitHub
#111 (comment).

[ernesto-jimenez]

So then we can occasionally grep all of Github and find the public ones to add to our central list in batches.

Once there's an option to enable the non-compliant providers. Why maintain a central list?

In my opinion, the biggest issue with the current implementation is inconsistency. As a developer, I might use this package to integrate with Google or Dropbox, then have to integrate another provider with the same API, but the code would fail because of some internal whitelist I'm not aware of. Same API, same library, same code, different results.

[bradfitz]

@ernesto-jimenez, if you're going to be using providers A, B, and C, with the "same" API but different interpretations/buggy implementations of OAuth2, the workaround config has to live somewhere. What I don't want is for that workaround config to be part of the OAuth config in user code. I'd like to flag buggy providers centrally so things work for most people by default, but new buggy sites discovered can be quickly fixed by users while waiting for us to update the central list.

Why I opposed adding this to Endpoint earlier was because it brings this too close to the user as a concept they need to be aware of. In general they should not be aware of this. We should do our best to maintain the bad list automatically.

[ernesto-jimenez]

@bradfitz:

if you're going to be using providers A, B, and C, with the "same" API but different interpretations/buggy implementations of OAuth2, the workaround config has to live somewhere.

But we are not talking abut different implementations, are we? After all in many cases the request is to simply add a new provider to the list.

Imagine the following situation:

• Provider A follows the spec: the package works OK.
• Provider B has a buggy implementation of OAuth2, but it's in the central list: the package works OK transparently.
• Provider C has the same buggy implementation as provider B, but it's not in the central list: the package fails.

If I integrate provider B, and then provider C fails, I would be very confused.

Just my opinion :)

Re: Keeping the workaround away from the user's OAuth config.
👍 understood, makes sense.

[bradfitz]

Provider A follows the spec: the package works OK.
Provider B has a buggy implementation of OAuth2, but it's in the central list: the package works OK transparently.
Provider C has the same buggy implementation as provider B, but it's not in the central list: the package fails.
If I integrate provider B, and then provider C fails, I would be very confused.

You would be confused in that case regardless of whether this package had any knowledge of this workaround, and regardless of whether it exposed an API. It sucks, because people suck at writing and implementing specs. Fact of life.

This whole bug is purely about how we handle the confusion and where we shove it.

[adg]

On 21 May 2015 at 15:41, Nikolay Turpitko notifications@github.com wrote:

Wouldn't it be simpler

Everything that follows this is way more complicated than it needs to be,
IMO.
It's a fun idea, but the reality is that the providers are not interested
in fixing the bugs.
I, as one of the maintainers of this library, don't want to make a big deal
out of it.
I just want the library to work well for most people.
The status quo means that anyone using any of the already-listed providers
do not even need to know about the bugs.

Integration with a new provider usually requires more work from app
developer, then just throw in a new Endpoint, anyway

What, why?

[adg]

On 22 May 2015 at 03:25, Nikolay Turpitko notifications@github.com wrote:

To simplify contributions to this list a bit (within boundaries of the
current approach), this list could be extracted into separate go file
within oauth2 project, so that it would be simply located, fixed and
reviewed. Location of this file should be documented. That's it. No big
deal at all.

That sounds like a reasonable move to me.

[williamjoy]

Confirmed Github Enterprise 2.1 does not work with Auth Header, and it is not possible to regex github enterprise URL or even whitelist it

It might be better to give user a chance to register RegisterBrokenAuthHeaderHost

[abourget]

+1 on this.. or add slack.com .. just like api.netatmo.net was added.. what's the special process to get one in ? I'd be for RegisterBrokenAuthHeaderHost or something alike.

[abourget]

Sorry, going back to #129 .. I wasn't set up to do Gerrit reviews.. so I dropped it there.. but I'd prefer a Register call nonetheless.

[rakyll]

I'd prefer we export the list. There is less harm in exporting BrokenAuthHeaderHosts than a Register function. Registration makes broken implementations more legitimate. An exported list with an awful name that helps the programs reading who is not implementing the spec fully is a safer option.

[giganteous]

NEWS from the github enterprise side:

https://enterprise.github.com/releases/2.3.4/notes

Perhaps thanks to some pushing as a Github:E user.

[bradfitz]

@giganteous, excellent.

[bradfitz]

I'm still okay with a Register function, but not with exporting the list.

[jerbob92]

I appreciate you guys want this to work for the most common providers by default. I also understand that you don't want to be burdened by the mistakes of the providers.

But I also feel like you are unnecessarily punishing developers that want to implement some other provider that isn't following the specs. It's not the developer that just picked a random provider to implement that didn't follow the specs. Most providers won't comply and you'll have to add it to your blacklist which is too much work IMHO. When I want to add the current provider to the list, I'd have to add 6 prefixes to the list, with possibly more to come (they use a different domain for every country).

It would be great if we get a workaround. My suggestion is adding a HasBrokenAuthHeader field on the oauth2.Endpoint type.
Another option is adding a AuthorizeLocation field on the oauth2.Endpoint type, the value could be header or query (I have seen some OAuth implementations that do this).

We can still keep our bad provider list to make sure the most common providers keep working.

[rakyll]

+1 to the Register function, will send a CL soon.

[bradfitz]

I'm going to reopen this.

I've started to work on this in https://go-review.googlesource.com/c/oauth2/+/157820

I want it to just be automatic with no configuration or required maintenance of registries of quirks.