Experimental: OAuth Token Retriever #702

andresmgot · 2023-06-12T14:52:21Z

What this PR does / why we need it:

Will use this as a library for grafana/grafana-plugin-examples#138

Which issue(s) this PR fixes:

Special notes for your reviewer:

marefr · 2023-06-15T00:31:38Z

Feels like this might be provided by a 3rd party, something like github.com/go-jose/go-jose maybe - we're already using that within Grafana. Or something already available in the SDK perhaps, like https://github.com/grafana/grafana-plugin-sdk-go/blob/main/experimental/authclient/oauth2.go.

andresmgot · 2023-06-15T11:02:33Z

Feels like this might be provided by a 3rd party, something like github.com/go-jose/go-jose maybe - we're already using that within Grafana.

Some Internet search revealed nothing regarding a library that already does this. I checked go-jose but that is related to JSON Web Signatures which I believe is different from what we need here. I tried and signatures generated with that package doesn't work (maybe I am missing something, this goes beyond my cryptographic knowledge). @mgyongyosi do you have any input?

Or something already available in the SDK perhaps, like https://github.com/grafana/grafana-plugin-sdk-go/blob/main/experimental/authclient/oauth2.go.

That does not cover signatures.

marefr · 2023-06-15T12:54:53Z

Some Internet search revealed nothing regarding a library that already does this. I checked go-jose but that is related to JSON Web Signatures which I believe is different from what we need here. I tried and signatures generated with that package doesn't work (maybe I am missing something, this goes beyond my cryptographic knowledge). @mgyongyosi do you have any input?

https://github.com/go-jose/go-jose/blob/6531eff483988d182103292d4348e075a6090c9e/jose-util/generator/utils.go#L71 for loading private key. Found some others as well.

That does not cover signatures.

I just noticed usage of a private key below and we're basically looking for creating an OAuth client, right?

grafana-plugin-sdk-go/experimental/authclient/oauth2.go

Line 66 in e8cf32a

PrivateKey: options.PrivateKey,

That seems to call this to parse the private key internally:
https://github.com/golang/oauth2/blob/2323c81c8dba82e8650ed3a24a1a5667e293af38/internal/oauth2.go#L20

Regarding signatures, what about https://pkg.go.dev/gopkg.in/go-jose/go-jose.v2?utm_source=godoc#NewSigner ? Some example usages in Grafana https://github.com/search?q=repo%3Agrafana%2Fgrafana%20NewSigner&type=code

marefr

LGTM. My common impression though is that OAuth is standardized and therefore should already be libraries available to do all this. but I might be wrong/misunderstand. This is not a blocker for now since this is in experimental why I approve, but would be good to figure out 3rd party libraries etc.

marefr · 2023-06-15T12:55:14Z

experimental/testdata/folder.golden.txt

@@ -29,4 +29,4 @@ Dimensions: 2 Fields by 15 Rows


 ====== TEST DATA RESPONSE (arrow base64) ======
-FRAME=QVJST1cxAAD/////yAEAABAAAAAAAAoADgAMAAsABAAKAAAAFAAAAAAAAAEEAAoADAAAAAgABAAKAAAACAAAALgAAAADAAAATAAAACgAAAAEAAAAwP7//wgAAAAMAAAAAAAAAAAAAAAFAAAAcmVmSWQAAADg/v//CAAAAAwAAAAAAAAAAAAAAAQAAABuYW1lAAAAAAD///8IAAAAUAAAAEQAAAB7InR5cGUiOiJkaXJlY3RvcnktbGlzdGluZyIsInR5cGVWZXJzaW9uIjpbMCwwXSwicGF0aFNlcGFyYXRvciI6Ii8ifQAAAAAEAAAAbWV0YQAAAAACAAAAeAAAAAQAAACi////FAAAADwAAAA8AAAAAAAABTgAAAABAAAABAAAAJD///8IAAAAEAAAAAYAAABzdHJpbmcAAAYAAAB0c3R5cGUAAAAAAACI////CgAAAG1lZGlhLXR5cGUAAAAAEgAYABQAAAATAAwAAAAIAAQAEgAAABQAAABEAAAASAAAAAAAAAVEAAAAAQAAAAwAAAAIAAwACAAEAAgAAAAIAAAAEAAAAAYAAABzdHJpbmcAAAYAAAB0c3R5cGUAAAAAAAAEAAQABAAAAAQAAABuYW1lAAAAAP/////YAAAAFAAAAAAAAAAMABYAFAATAAwABAAMAAAAgAEAAAAAAAAUAAAAAAAAAwQACgAYAAwACAAEAAoAAAAUAAAAeAAAAA8AAAAAAAAAAAAAAAYAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABAAAAAAAAAAEAAAAAAAAAAvwAAAAAAAAAAAQAAAAAAAAAAAAAAAAAAAAEAAAAAAABAAAAAAAAAAEABAAAAAAAAPwAAAAAAAAAAAAAAAgAAAA8AAAAAAAAAAAAAAAAAAAAPAAAAAAAAAAAAAAAAAAAAAAAAAAkAAAAQAAAAGgAAAB0AAAAoAAAAOAAAAEcAAABbAAAAdQAAAJQAAACfAAAApQAAAKkAAAC3AAAAvwAAAFJFQURNRS5tZGFjdGlvbnNhdXRoY2xpZW50ZTJlZmlsZWluZm8uZ29maWxlaW5mb190ZXN0LmdvZnJhbWVfc29ydGVyLmdvZnJhbWVfc29ydGVyX3Rlc3QuZ29nb2xkZW5fcmVzcG9uc2VfY2hlY2tlci5nb2dvbGRlbl9yZXNwb25zZV9jaGVja2VyX3Rlc3QuZ29odHRwX2xvZ2dlcm1hY3Jvc21vY2tyZXN0X2NsaWVudC5nb3Rlc3RkYXRhAAAAAAAAAAAACQAAABIAAAAbAAAAGwAAABsAAAAbAAAAGwAAABsAAAAbAAAAJAAAAC0AAAA2AAAANgAAAD8AAABkaXJlY3RvcnlkaXJlY3RvcnlkaXJlY3RvcnlkaXJlY3RvcnlkaXJlY3RvcnlkaXJlY3RvcnlkaXJlY3RvcnkAEAAAAAwAFAASAAwACAAEAAwAAAAQAAAALAAAADwAAAAAAAQAAQAAANgBAAAAAAAA4AAAAAAAAACAAQAAAAAAAAAAAAAAAAAAAAAAAAAACgAMAAAACAAEAAoAAAAIAAAAuAAAAAMAAABMAAAAKAAAAAQAAADA/v//CAAAAAwAAAAAAAAAAAAAAAUAAAByZWZJZAAAAOD+//8IAAAADAAAAAAAAAAAAAAABAAAAG5hbWUAAAAAAP///wgAAABQAAAARAAAAHsidHlwZSI6ImRpcmVjdG9yeS1saXN0aW5nIiwidHlwZVZlcnNpb24iOlswLDBdLCJwYXRoU2VwYXJhdG9yIjoiLyJ9AAAAAAQAAABtZXRhAAAAAAIAAAB4AAAABAAAAKL///8UAAAAPAAAADwAAAAAAAAFOAAAAAEAAAAEAAAAkP///wgAAAAQAAAABgAAAHN0cmluZwAABgAAAHRzdHlwZQAAAAAAAIj///8KAAAAbWVkaWEtdHlwZQAAAAASABgAFAAAABMADAAAAAgABAASAAAAFAAAAEQAAABIAAAAAAAABUQAAAABAAAADAAAAAgADAAIAAQACAAAAAgAAAAQAAAABgAAAHN0cmluZwAABgAAAHRzdHlwZQAAAAAAAAQABAAEAAAABAAAAG5hbWUAAAAA+AEAAEFSUk9XMQ==
+FRAME=QVJST1cxAAD/////yAEAABAAAAAAAAoADgAMAAsABAAKAAAAFAAAAAAAAAEEAAoADAAAAAgABAAKAAAACAAAALgAAAADAAAATAAAACgAAAAEAAAAwP7//wgAAAAMAAAAAAAAAAAAAAAFAAAAcmVmSWQAAADg/v//CAAAAAwAAAAAAAAAAAAAAAQAAABuYW1lAAAAAAD///8IAAAAUAAAAEQAAAB7InR5cGUiOiJkaXJlY3RvcnktbGlzdGluZyIsInR5cGVWZXJzaW9uIjpbMCwwXSwicGF0aFNlcGFyYXRvciI6Ii8ifQAAAAAEAAAAbWV0YQAAAAACAAAAeAAAAAQAAACi////FAAAADwAAAA8AAAAAAAABTgAAAABAAAABAAAAJD///8IAAAAEAAAAAYAAABzdHJpbmcAAAYAAAB0c3R5cGUAAAAAAACI////CgAAAG1lZGlhLXR5cGUAAAAAEgAYABQAAAATAAwAAAAIAAQAEgAAABQAAABEAAAASAAAAAAAAAVEAAAAAQAAAAwAAAAIAAwACAAEAAgAAAAIAAAAEAAAAAYAAABzdHJpbmcAAAYAAAB0c3R5cGUAAAAAAAAEAAQABAAAAAQAAABuYW1lAAAAAP/////YAAAAFAAAAAAAAAAMABYAFAATAAwABAAMAAAAoAEAAAAAAAAUAAAAAAAAAwQACgAYAAwACAAEAAoAAAAUAAAAeAAAABAAAAAAAAAAAAAAAAYAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABEAAAAAAAAAEgAAAAAAAAAwwAAAAAAAAAQAQAAAAAAAAAAAAAAAAAAEAEAAAAAAABEAAAAAAAAAFgBAAAAAAAASAAAAAAAAAAAAAAAAgAAABAAAAAAAAAAAAAAAAAAAAAQAAAAAAAAAAAAAAAAAAAAAAAAAAkAAAAQAAAAGgAAAB0AAAAoAAAAOAAAAEcAAABbAAAAdQAAAJQAAACfAAAApQAAAKkAAAC3AAAAuwAAAMMAAAAAAAAAUkVBRE1FLm1kYWN0aW9uc2F1dGhjbGllbnRlMmVmaWxlaW5mby5nb2ZpbGVpbmZvX3Rlc3QuZ29mcmFtZV9zb3J0ZXIuZ29mcmFtZV9zb3J0ZXJfdGVzdC5nb2dvbGRlbl9yZXNwb25zZV9jaGVja2VyLmdvZ29sZGVuX3Jlc3BvbnNlX2NoZWNrZXJfdGVzdC5nb2h0dHBfbG9nZ2VybWFjcm9zbW9ja3Jlc3RfY2xpZW50Lmdvc2lnbnRlc3RkYXRhAAAAAAAAAAAAAAAAAAkAAAASAAAAGwAAABsAAAAbAAAAGwAAABsAAAAbAAAAGwAAACQAAAAtAAAANgAAADYAAAA/AAAASAAAAAAAAABkaXJlY3RvcnlkaXJlY3RvcnlkaXJlY3RvcnlkaXJlY3RvcnlkaXJlY3RvcnlkaXJlY3RvcnlkaXJlY3RvcnlkaXJlY3RvcnkQAAAADAAUABIADAAIAAQADAAAABAAAAAsAAAAPAAAAAAABAABAAAA2AEAAAAAAADgAAAAAAAAAKABAAAAAAAAAAAAAAAAAAAAAAAAAAAKAAwAAAAIAAQACgAAAAgAAAC4AAAAAwAAAEwAAAAoAAAABAAAAMD+//8IAAAADAAAAAAAAAAAAAAABQAAAHJlZklkAAAA4P7//wgAAAAMAAAAAAAAAAAAAAAEAAAAbmFtZQAAAAAA////CAAAAFAAAABEAAAAeyJ0eXBlIjoiZGlyZWN0b3J5LWxpc3RpbmciLCJ0eXBlVmVyc2lvbiI6WzAsMF0sInBhdGhTZXBhcmF0b3IiOiIvIn0AAAAABAAAAG1ldGEAAAAAAgAAAHgAAAAEAAAAov///xQAAAA8AAAAPAAAAAAAAAU4AAAAAQAAAAQAAACQ////CAAAABAAAAAGAAAAc3RyaW5nAAAGAAAAdHN0eXBlAAAAAAAAiP///woAAABtZWRpYS10eXBlAAAAABIAGAAUAAAAEwAMAAAACAAEABIAAAAUAAAARAAAAEgAAAAAAAAFRAAAAAEAAAAMAAAACAAMAAgABAAIAAAACAAAABAAAAAGAAAAc3RyaW5nAAAGAAAAdHN0eXBlAAAAAAAABAAEAAQAAAAEAAAAbmFtZQAAAAD4AQAAQVJST1cx


Hmm why did this change?

everytime there is a new package, this test fails so the snapshot here needs to be updated 🤷

mgyongyosi · 2023-06-15T13:16:55Z

Feels like this might be provided by a 3rd party, something like github.com/go-jose/go-jose maybe - we're already using that within Grafana.

Some Internet search revealed nothing regarding a library that already does this. I checked go-jose but that is related to JSON Web Signatures which I believe is different from what we need here. I tried and signatures generated with that package doesn't work (maybe I am missing something, this goes beyond my cryptographic knowledge). @mgyongyosi do you have any input?

Or something already available in the SDK perhaps, like https://github.com/grafana/grafana-plugin-sdk-go/blob/main/experimental/authclient/oauth2.go.

That does not cover signatures.

@andresmgot JSON Web Signature is what we need. The go-jose library can be used to sign a JWT (example: https://github.com/grafana/grafana/blob/21f8dd9599693597524aa55b5f14d76926bc70f5/pkg/services/authn/clients/ext_jwt_test.go#L539)

Apologies for the confusion I caused, I had a custom implementation before I realized that go-jose should contain everything we need.

CLAassistant · 2023-06-21T08:31:21Z

All committers have signed the CLA.

andresmgot · 2023-06-21T15:44:13Z

Thanks to the changes from @mgyongyosi I was able to higly simplify the sign code and simply use go-jose so what I did next was to extract more (generic?) logic from the plugin code. Now the sign function is not exposed, only a function to retrieve a token so the interface looks something like:

		tr, err := oauthtokenretriever.New(
			app.httpClient,
			app.grafanaAppURL,
			os.Getenv("GF_PLUGIN_APP_CLIENT_ID"),
			os.Getenv("GF_PLUGIN_APP_CLIENT_SECRET"),
			os.Getenv("GF_PLUGIN_APP_PRIVATE_KEY"),
		)
... 
	token, err := tr.GetExternalServiceToken(req.FormValue("userID"))
... 
	proxyReq.Header.Set("Authorization", token)

WDYT?

marefr

Given this is in experimental I approve this. However, added some nits and suggestions.

In general, would be helpful to see how grafana/grafana-plugin-examples#138 would implement this before merge. You can do that in examples repo with go mod edit -replace=github.com/grafana/grafana-plugin-sdk-go=github.com/grafana/grafana-plugin-sdk-go@experimentalSign. If I understand it correctly it's up to each consumer (plugin) to retrieve client id, secret and private key from environment variables - we probably want to hide this from plugin authors within the SDK eventually?

In regards to oauth2 package nits/suggestions. As noted earlier this can happen at a later stage, but if we cannot use those out of the box, would then suggest implement the oauth2.TokenSource interface to make it compatible with the oauth2 package. The oauth2 client would possible be a nice abstraction to hide the retrieval of client id, secret and private key for plugin authors within the SDK.

marefr · 2023-06-22T11:42:45Z

experimental/oauthtokenretriever/token.go

+}
+
+// retrieveSelfToken returns a JWT bearer token for the service account created with the app.
+func (t *tokenRetriever) retrieveSelfToken() (string, error) {


Nit. Can't let go of this since exactly this seems to be provided by the oauth2 package: https://cs.opensource.google/go/x/oauth2/+/refs/tags/v0.9.0:clientcredentials/clientcredentials.go

Doesn't need to be fixed now, but seems like what we might want in the future since it provides standard things plus automatic reuse of token until expiration and automatically refreshes when expired.

plus automatic reuse of token until expiration and automatically refreshes when expired

For the record, I checked this with @mgyongyosi and for the moment, the token returned does not contain a refresh token so it cannot be automatically refreshed.

marefr · 2023-06-22T11:50:05Z

experimental/oauthtokenretriever/token.go

+}
+
+// retrieveJWTBearerToken returns a JWT bearer token for the given user ID.
+func (t *tokenRetriever) retrieveJWTBearerToken(userID string) (string, error) {


Nit. Can't let go of this since exactly this seems to be provided by the oauth2 package: https://cs.opensource.google/go/x/oauth2/+/refs/tags/v0.9.0:jwt/jwt.go

Doesn't need to be fixed now, but seems like what we might want in the future since it provides standard things plus automatic reuse of token until expiration and automatically refreshes when expired.

in this case, I was not able to use the jwt package since I couldn't find a way to set the necessary client_id and client_secret (the endpoint returned an error because that info was missing).

another reason: that library only supports RS256 keys, in our case the default is ES256:

https://github.com/golang/oauth2/blob/master/internal/oauth2.go#L20

Thanks for looking into this ❤️

experimental/oauthtokenretriever/sign.go

experimental/oauthtokenretriever/token.go

andresmgot · 2023-06-23T10:46:24Z

In general, would be helpful to see how grafana/grafana-plugin-examples#138 would implement this before merge. You can do that in examples repo with go mod edit -replace=github.com/grafana/grafana-plugin-sdk-go=github.com/grafana/grafana-plugin-sdk-go@experimentalSign. If I understand it correctly it's up to each consumer (plugin) to retrieve client id, secret and private key from environment variables - we probably want to hide this from plugin authors within the SDK eventually?

yes, I am keeping that up to date with the changes in the rest of repos. See how it's reading env vars:

https://github.com/grafana/grafana-plugin-examples/pull/138/files#diff-41427faf689ff7375a1ea69d019375d414befb4b1e175b3806c39f807a791f39R64

It makes sense to also hide the envvar management to the SDK 🤔

In regards to oauth2 package nits/suggestions. As noted earlier this can happen at a later stage, but if we cannot use those out of the box, would then suggest implement the oauth2.TokenSource interface to make it compatible with the oauth2 package. The oauth2 client would possible be a nice abstraction to hide the retrieval of client id, secret and private key for plugin authors within the SDK.

Let me also check if that works 👍

experimental/oauthtokenretriever/token.go

* add test manage func * flesh out * fix linter * adjust test to use experimental package * regenerate * delete dir * Experimental: OAuth Token Retriever (#702) Co-authored-by: Mihaly Gyongyosi <mgyongyosi@users.noreply.github.com> * fix golden file * update package name * fix test * fix golden file * undo commit * fix comment * shutdown for plugin --------- Co-authored-by: Andres Martinez Gotor <andres.martinez@grafana.com> Co-authored-by: Mihaly Gyongyosi <mgyongyosi@users.noreply.github.com>

Experimental: Sign package

94d1752

andresmgot requested a review from a team as a code owner June 12, 2023 14:52

andresmgot requested review from wbrowne and marefr and removed request for a team June 12, 2023 14:52

andresmgot added 2 commits June 12, 2023 16:57

Merge branch 'main' into experimentalSign

24bddab

update test

1600c4a

andresmgot mentioned this pull request Jun 13, 2023

Example app with external service registration grafana/grafana-plugin-examples#138

Merged

andresmgot requested a review from mgyongyosi June 15, 2023 11:02

marefr approved these changes Jun 15, 2023

View reviewed changes

Sign JWT with go-jose v3

53e2043

OAuthTokenRetriever

3bd6f6c

andresmgot changed the title ~~Experimental: Sign package~~ Experimental: OAuth Token Retriever Jun 21, 2023

andresmgot added 2 commits June 21, 2023 17:44

rename folder

31eda0c

fix test

d80b7f8

andresmgot requested a review from marefr June 21, 2023 15:50

marefr approved these changes Jun 22, 2023

View reviewed changes

andresmgot added 2 commits June 23, 2023 17:31

use upstream oauth2 library

d49e78e

use clientcredentials

17c0ee6

marefr approved these changes Jun 26, 2023

View reviewed changes

experimental/oauthtokenretriever/token.go Outdated Show resolved Hide resolved

experimental/oauthtokenretriever/token.go Outdated Show resolved Hide resolved

andresmgot added 2 commits June 26, 2023 13:14

forward context

703cef1

fix test

7ebe196

andresmgot merged commit 6be9232 into main Jun 26, 2023
2 checks passed

andresmgot deleted the experimentalSign branch June 26, 2023 11:23

wbrowne mentioned this pull request Jun 29, 2023

Reapply 6be923230b09c4cf2f3cb4c9ac2665fa9593ed46 #710

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Experimental: OAuth Token Retriever #702

Experimental: OAuth Token Retriever #702

andresmgot commented Jun 12, 2023

marefr commented Jun 15, 2023

andresmgot commented Jun 15, 2023

marefr commented Jun 15, 2023

marefr left a comment

marefr Jun 15, 2023

andresmgot Jun 21, 2023

mgyongyosi commented Jun 15, 2023 •

edited

CLAassistant commented Jun 21, 2023 •

edited

andresmgot commented Jun 21, 2023

marefr left a comment

marefr Jun 22, 2023

andresmgot Jun 23, 2023

marefr Jun 22, 2023

andresmgot Jun 23, 2023

andresmgot Jun 26, 2023

marefr Jun 26, 2023

andresmgot commented Jun 23, 2023

Experimental: OAuth Token Retriever #702

Experimental: OAuth Token Retriever #702

Conversation

andresmgot commented Jun 12, 2023

marefr commented Jun 15, 2023

andresmgot commented Jun 15, 2023

marefr commented Jun 15, 2023

marefr left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mgyongyosi commented Jun 15, 2023 • edited

CLAassistant commented Jun 21, 2023 • edited

andresmgot commented Jun 21, 2023

marefr left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

andresmgot commented Jun 23, 2023

mgyongyosi commented Jun 15, 2023 •

edited

CLAassistant commented Jun 21, 2023 •

edited