Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add authn for graphql and http admin endpoints #5162

Merged
merged 30 commits into from
May 19, 2020

Conversation

abhimanyusinghgaur
Copy link
Contributor

@abhimanyusinghgaur abhimanyusinghgaur commented Apr 10, 2020

Description.

This PR adds authentication to following endpoints:

  • /admin/backup (http & graphql)
  • /admin/config/lru_mb (http [GET & PUT] & graphql [query & mutation])
  • /admin/draining (http & graphql)
  • /admin/export (http & graphql)
  • /admin/shutdown (http & graphql)
  • /admin/restore (graphql only)
  • /admin/listBackups (graphql only)

Now, all the above http endpoints and their corresponding graphql versions have following kinds of auth:

  • IP White-listing, if --whitelist flag is passed to alpha
  • Poor-man's auth, if --auth_token flag is passed to alpha
  • Guardian only access, if ACL is enabled

This PR also adds query for config in graphql admin, as it was missing earlier.

In addition to above points:

  • All the /admin endpoints apply Poor-man's auth check at http level itself, while other auth checks are routed through graphql resolvers.
  • GraphQL Resolvers for health/state and the ones related to ACL User/Group have IP whitelisting middleware applied, while dgraph handles Guardian auth for them.
  • /alter has the existing behaviour of checking only Poor-man's and Guardian auth.
  • GraphQL Resolvers related to schema don't apply IP whitelisting as to keep them in sync with /alter. They do apply Guardian auth.
  • Any GraphQL admin introspection queries don't require IP whitelisting or Guardian auth.

GitHub Issue or Jira number.

Fixes #4758.

Other components or 3rd party tools affected (or regression areas).

No

Affected releases.

> 20.03.0

Changelog tags.

added authentication for admin endpoints

Please indicate if this is a breaking change.

No

Please indicate if this is an enterprise feature.

No

Please indicate if documentation needs to be updated.

Yes

Please indicate if end to end testing is needed.

Yes, for the whitelisting and poor-man's ACL part.


This change is Reviewable

Docs Preview: Dgraph Preview

Copy link
Contributor

@pawanrawal pawanrawal left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work, this mostly looks good. I am wondering if we can avoid duplication by checking poor man's auth and guardian access in the functions that are finally called by both GraphQL and HTTP.
Also needs a bunch of negative test cases which show access is restricted.

Reviewed 11 of 11 files at r1.
Reviewable status: all files reviewed, 2 unresolved discussions (waiting on @abhimanyusinghgaur)


graphql/admin/config.go, line 58 at r1 (raw file):

	query *gql.GraphQuery,
	mutations []*dgoapi.Mutation) (map[string]string, map[string]interface{}, error) {
	err := edgraph.AuthorizeGuardians(ctx)

Is the poor man's auth applied here or in any of the GraphQL endpoints?


graphql/admin/export.go, line 59 at r1 (raw file):

	query *gql.GraphQuery,
	mutations []*dgoapi.Mutation) (map[string]string, map[string]interface{}, error) {
	err := edgraph.AuthorizeGuardians(ctx)

How about the poor man's auth here?

@abhimanyusinghgaur abhimanyusinghgaur added the area/enterprise/acl Related to Access Control Lists label Apr 14, 2020
@abhimanyusinghgaur abhimanyusinghgaur marked this pull request as ready for review April 15, 2020 10:18
@abhimanyusinghgaur abhimanyusinghgaur changed the title graphql: Add authn for graphql and http admin endpoints [BREAKING] Add authn for graphql and http admin endpoints Apr 15, 2020
Copy link
Contributor

@manishrjain manishrjain left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the auth stuff is getting copied in many different places. Instead, it should be done in one place, before the corresponding handler gets called.

Reviewable status: 10 of 15 files reviewed, 2 unresolved discussions (waiting on @abhimanyusinghgaur, @manishrjain, and @pawanrawal)

@abhimanyusinghgaur abhimanyusinghgaur changed the title [BREAKING] Add authn for graphql and http admin endpoints Add authn for graphql and http admin endpoints Apr 16, 2020
…dmin-auth

# Conflicts:
#	ee/acl/acl_test.go
#	graphql/admin/backup.go
#	graphql/admin/config.go
#	graphql/admin/draining.go
#	graphql/admin/export.go
#	graphql/admin/shutdown.go
…dmin-auth

# Conflicts:
#	ee/acl/acl_test.go
#	testutil/graphql.go
Copy link
Contributor Author

@abhimanyusinghgaur abhimanyusinghgaur left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added test cases for both positive (wherever possible) and negative cases.
Using middlewares now for Guardian auth in graphql; Poormans's and whitelisting are applied directly in http handler for /admin. So, no duplication now.

Reviewable status: 3 of 13 files reviewed, 2 unresolved discussions (waiting on @pawanrawal)


graphql/admin/config.go, line 58 at r1 (raw file):

Previously, pawanrawal (Pawan Rawal) wrote…

Is the poor man's auth applied here or in any of the GraphQL endpoints?

Using middlewares now for Guardian auth in graphql, Poormans's and whitelisting are applied directly in http handler for /admin


graphql/admin/export.go, line 59 at r1 (raw file):

Previously, pawanrawal (Pawan Rawal) wrote…

How about the poor man's auth here?

Using middlewares now for Guardian auth in graphql, Poormans's and whitelisting are applied directly in http handler for /admin

Copy link
Contributor

@gja gja left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Small changes requested

// handlerInit does some standard checks. Returns false if something is wrong.
func handlerInit(w http.ResponseWriter, r *http.Request, allowedMethods map[string]bool) bool {
func handlerInit(w http.ResponseWriter, r *http.Request, allowedMethods map[string]bool,
allowOnlyGuardians bool) bool {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we be clubbing allowedMethods and allowOnlyGuardians into a type?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, can we rename handlerInit to something like checkAuth?

// handlerInit does some standard checks. Returns false if something is wrong.
func handlerInit(w http.ResponseWriter, r *http.Request, allowedMethods map[string]bool) bool {
func handlerInit(w http.ResponseWriter, r *http.Request, allowedMethods map[string]bool,
allowOnlyGuardians bool) bool {
if _, ok := allowedMethods[r.Method]; !ok {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can't we move these allowedMethods type checks into some sort of router or something? I know we do use httprouter, why aren't 405s moved over there?

if _, ok := allowedMethods[r.Method]; !ok {
x.SetStatus(w, x.ErrorInvalidMethod, "Invalid method")
w.WriteHeader(http.StatusMethodNotAllowed)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isn't this a bit strange? Middlewares should either do the return true/false pattern, or they should call next() on success. This feels like it's mixing both

Copy link
Contributor

@pawanrawal pawanrawal left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems like poorman's auth and IP whitelisting are applied to all endpoints except for /alter. Should /alter have Poorman's auth?

And then guardian ACL check is applied to all endpoints. One question is between the consistency of checks between /alter and updateGQLSchema. updateGQLSchema has IP whitelisting whereas /alter doesn't. With the way things are structured right now, I guess it would be hard to remove IP whitelisting just from updateGQLSchema to keep both endpoints consistent?

Reviewed 12 of 15 files at r2.
Reviewable status: all files reviewed, 7 unresolved discussions (waiting on @abhimanyusinghgaur)


dgraph/cmd/alpha/http.go, line 584 at r2 (raw file):

	}

	if !checkIpIsWhitelisted(w, r) || !checkPoormansAcl(w, r) {

Since IP whitelisting is not applied to schema alter above, we should keep it consistent for the graphql schema alter as well. Is the poor man's auth applied to schema alter.


graphql/admin/admin.go, line 404 at r2 (raw file):

	adminMutationResolvers := map[string]resolve.MutationResolverFunc{
		"backup":   resolve.CommonMutationMiddlewares.Then(resolveBackup),

Instead of this then pattern here, would have liked for this just to be an slice of functions which are applied in order.


graphql/resolve/middlewares.go, line 70 at r2 (raw file):

// resolveAuth returns a Resolved with error if the context doesn't contain any Guardian auth,
// otherwise it returns nil
func resolveAuth(ctx context.Context, f schema.Field) *Resolved {

This doesn't seem to apply poor man's auth. Is that applied earlier along with IP whitelisting?


graphql/resolve/middlewares.go, line 71 at r2 (raw file):

// otherwise it returns nil
func resolveAuth(ctx context.Context, f schema.Field) *Resolved {
	if err := edgraph.AuthorizeGuardians(ctx); err != nil {

I am assuming that AuthorizeGuardians returns true for oss builds?

Copy link
Contributor Author

@abhimanyusinghgaur abhimanyusinghgaur left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: 5 of 22 files reviewed, 7 unresolved discussions (waiting on @gja and @pawanrawal)


dgraph/cmd/alpha/admin.go, line 55 at r2 (raw file):

Previously, gja (Tejas Dinkar) wrote…

Also, can we rename handlerInit to something like checkAuth?

Done.


dgraph/cmd/alpha/admin.go, line 56 at r2 (raw file):

Previously, gja (Tejas Dinkar) wrote…

can't we move these allowedMethods type checks into some sort of router or something? I know we do use httprouter, why aren't 405s moved over there?

I couldn't find the use of httprouter in our codebase.
For now, I am letting adminAuthHandler do these checks for admin endpoints.


dgraph/cmd/alpha/admin.go, line 58 at r2 (raw file):

Previously, gja (Tejas Dinkar) wrote…

Isn't this a bit strange? Middlewares should either do the return true/false pattern, or they should call next() on success. This feels like it's mixing both

Done.


dgraph/cmd/alpha/http.go, line 584 at r2 (raw file):

Previously, pawanrawal (Pawan Rawal) wrote…

Since IP whitelisting is not applied to schema alter above, we should keep it consistent for the graphql schema alter as well. Is the poor man's auth applied to schema alter.

Done.


graphql/admin/admin.go, line 404 at r2 (raw file):

Previously, pawanrawal (Pawan Rawal) wrote…

Instead of this then pattern here, would have liked for this just to be an slice of functions which are applied in order.

Done.


graphql/resolve/middlewares.go, line 70 at r2 (raw file):

Previously, pawanrawal (Pawan Rawal) wrote…

This doesn't seem to apply poor man's auth. Is that applied earlier along with IP whitelisting?

Poor-man's auth is handled at http level by adminAuthHandler for /admin


graphql/resolve/middlewares.go, line 71 at r2 (raw file):

Previously, pawanrawal (Pawan Rawal) wrote…

I am assuming that AuthorizeGuardians returns true for oss builds?

Yes, it returns nil error for oss builds.

Copy link
Contributor

@MichaelJCompton MichaelJCompton left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I haven't reviewed the actual auth that is applied at each point. This is a breaking change so for that, I think we should take the behaviour to Manish+Daniel in a demo to check through all the options and make sure it's making expected changes. I'd say we should hold off tipping this into master until that has gone through.

Looks in general to be a nice simplification/generalisation of how these auth rules are applied as well as adding features, so looks good.

I had a bunch of questions about the GraphQL middleware. I quite like the pattern of wrapping resolvers in other resolvers that apply things like auth. I expect we'll add more to this: like logging in admin, maybe query complexity in /graphql, and also maybe open up custom logic to allow users to give custom pre processing.

In thinking about the answers to the questions and the future stuff we might add, like logging in admin (which we'll add soon) make sure the pattern is general enough that we can play with it a bit and not have to rip it apart as soon as we add more things.

Reviewed 1 of 18 files at r3.
Reviewable status: 5 of 22 files reviewed, 12 unresolved discussions (waiting on @abhimanyusinghgaur, @gja, and @pawanrawal)


ee/acl/acl_test.go, line 1927 at r3 (raw file):

	assertNonGuardianFailure(t, "export", true, params)

	// assert non-ACL error for guardians

I don't quite get these cases in each of these tests ... why do we need to test this? Aren't there existing tests that show that this works for guardians? So what's really being tested here?


graphql/admin/admin.go, line 414 at r3 (raw file):

		"backup":   resolve.AdminMutationMWs.Then(resolve.MutationResolverFunc(resolveBackup)),
		"config":   resolve.AdminMutationMWs.Then(resolve.MutationResolverFunc(resolveConfig)),
		"draining": resolve.AdminMutationMWs.Then(resolve.MutationResolverFunc(resolveDraining)),

The 'Then' thing looks like a cute pattern, but is it really needed here? It doesn't look like we use it for anything other than applying the same two middlewares, so what benefit do we get from the generic pattern?

Having it require the complexity of writing it, plus writing tests for it. I'd say think carefully if the pattern is required and if we are using it now, or intend to use it really soon.


graphql/admin/admin.go, line 621 at r3 (raw file):

			}).
		// for queries and mutations related to User/Group, dgraph handles Guardian auth,
		// so need to apply GuardianAuth Middleware

does it matter though? We already have the resolve.AdminMutationMWs.Then pattern ... do we need to not apply the guardian middleware?

If we don't, it would be ok to remove the extra fns, and just apply the standard ones?

Also, if this is really required to only call the one middleware, you have this 'Then' pattern, so what would be the reason to have a specific IpWhitelistingMW4Mutation function and not use 'Then' on a list of middleware that just has the ip whitelisting? Looks to me like it adds a specific function that's already handled by the general case.


graphql/resolve/middlewares.go, line 17 at r3 (raw file):

 */

package resolve

does this belong in 'resolve' looks like this and the test belong in admin


graphql/resolve/middlewares.go, line 144 at r3 (raw file):

// GuardianAuthMW4Query blocks the resolution of resolverFunc if there is no Guardian auth
// present in context, otherwise it lets the resolverFunc resolve the query.
func GuardianAuthMW4Query(resolver QueryResolver) QueryResolver {

it's a bit sad that we need versions for query and for mutation ... that's a bit of a structural error in how the resolvers work for each case

http.HandleFunc("/admin/export", exportHandler)
http.HandleFunc("/admin/config/lru_mb", memoryLimitHandler)
http.Handle("/admin/shutdown", adminAuthHandler(http.HandlerFunc(shutDownHandler),
adminAuthOptions{allowedMethods: map[string]bool{
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

change the type for allowedMethods to a alias type

"login": resolveLogin,
"restore": resolveRestore,
"shutdown": resolveShutdown,
adminMutationResolvers := map[string]resolve.MutationResolver{
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please make this configuration into a map (or even a configuration file).

"backup": applyMiddleWare(resolveBackup, middleware["backups"])

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

middleware := {
	"backup": &MiddlewareOptions{
		poorman: true,
		ipwhitelist: true
	}
}

http.handle("/backup", AdminAuthHandler(handleBackup, middleware["backups"]))
foo.AddMutationResolver(AdminAuthResolver(someResolver(qtx, etc...), middleware["backups"]))

Copy link
Contributor

@gja gja left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@@ -1821,6 +1801,219 @@ func TestHealthForAcl(t *testing.T) {
}
}

func TestAclForAdminEndpoints(t *testing.T) {
tcases := []struct {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

move the constant out of the function

Copy link
Contributor Author

@abhimanyusinghgaur abhimanyusinghgaur left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: 3 of 24 files reviewed, 15 unresolved discussions (waiting on @gja, @MichaelJCompton, and @pawanrawal)


ee/acl/acl_test.go, line 1927 at r3 (raw file):

Previously, MichaelJCompton (Michael Compton) wrote…

I don't quite get these cases in each of these tests ... why do we need to test this? Aren't there existing tests that show that this works for guardians? So what's really being tested here?

There were no such existing tests. We have limited the admin endpoint access to Guardians only in this PR. So, what I am checking here is two things:

  1. When non-guardians try to access, they always get permission denied error
  2. When guardians try to access, they get either success or some other error not related to permission

ee/acl/acl_test.go, line 1805 at r4 (raw file):

Previously, gja (Tejas Dinkar) wrote…

move the constant out of the function

Done.


graphql/admin/admin.go, line 411 at r3 (raw file):

Previously, gja (Tejas Dinkar) wrote…
middleware := {
	"backup": &MiddlewareOptions{
		poorman: true,
		ipwhitelist: true
	}
}

http.handle("/backup", AdminAuthHandler(handleBackup, middleware["backups"]))
foo.AddMutationResolver(AdminAuthResolver(someResolver(qtx, etc...), middleware["backups"]))

Done.


graphql/admin/admin.go, line 414 at r3 (raw file):

Previously, MichaelJCompton (Michael Compton) wrote…

The 'Then' thing looks like a cute pattern, but is it really needed here? It doesn't look like we use it for anything other than applying the same two middlewares, so what benefit do we get from the generic pattern?

Having it require the complexity of writing it, plus writing tests for it. I'd say think carefully if the pattern is required and if we are using it now, or intend to use it really soon.

Now middlewares are applied while querying the resolver for a query/mutation. Then is being used at that time only. Having it makes it easier to read through the code in resolver.go.


graphql/admin/admin.go, line 621 at r3 (raw file):

Previously, MichaelJCompton (Michael Compton) wrote…

does it matter though? We already have the resolve.AdminMutationMWs.Then pattern ... do we need to not apply the guardian middleware?

If we don't, it would be ok to remove the extra fns, and just apply the standard ones?

Also, if this is really required to only call the one middleware, you have this 'Then' pattern, so what would be the reason to have a specific IpWhitelistingMW4Mutation function and not use 'Then' on a list of middleware that just has the ip whitelisting? Looks to me like it adds a specific function that's already handled by the general case.

Refactored middlewares as a configuration now.


graphql/resolve/middlewares.go, line 71 at r2 (raw file):

Previously, abhimanyusinghgaur (Abhimanyu Singh Gaur) wrote…

Yes, it returns nil error for oss builds.

Done.


graphql/resolve/middlewares.go, line 17 at r3 (raw file):

Previously, MichaelJCompton (Michael Compton) wrote…

does this belong in 'resolve' looks like this and the test belong in admin

Its being user in resolver.go


graphql/resolve/middlewares.go, line 144 at r3 (raw file):

Previously, MichaelJCompton (Michael Compton) wrote…

it's a bit sad that we need versions for query and for mutation ... that's a bit of a structural error in how the resolvers work for each case

Done.

Copy link
Contributor

@pawanrawal pawanrawal left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:lgtm:

Reviewed 7 of 18 files at r3, 12 of 14 files at r4, 2 of 2 files at r5.
Reviewable status: all files reviewed, 14 unresolved discussions (waiting on @gja, @MichaelJCompton, @pawanrawal, and @vvbalaji-dgraph)

@abhimanyusinghgaur abhimanyusinghgaur added the area/graphql Issues related to GraphQL support on Dgraph. label May 19, 2020
@abhimanyusinghgaur abhimanyusinghgaur merged commit 8992238 into master May 19, 2020
@abhimanyusinghgaur abhimanyusinghgaur deleted the abhimanyu/admin-auth branch May 19, 2020 12:00
dna2github pushed a commit to dna2fork/dgraph that referenced this pull request Jul 18, 2020
Fixes dgraph-io#4758.
This PR adds authentication to following endpoints:

/admin/backup (http & graphql)
/admin/config/lru_mb (http [GET & PUT] & graphql [query & mutation])
/admin/draining (http & graphql)
/admin/export (http & graphql)
/admin/shutdown (http & graphql)
/admin/restore (graphql only)
/admin/listBackups (graphql only)
Now, all the above http endpoints and their corresponding graphql versions have following kinds of auth:

IP White-listing, if --whitelist flag is passed to alpha
Poor-man's auth, if --auth_token flag is passed to alpha
Guardian only access, if ACL is enabled
This PR also adds query for config in graphql admin, as it was missing earlier.

In addition to above points:

All the /admin endpoints apply Poor-man's auth check at http level itself, while other auth checks are routed through graphql resolvers.
GraphQL Resolvers for health/state and the ones related to ACL User/Group have IP whitelisting middleware applied, while dgraph handles Guardian auth for them.
/alter has the existing behaviour of checking only Poor-man's and Guardian auth.
GraphQL Resolvers related to schema don't apply IP whitelisting as to keep them in sync with /alter. They do apply Guardian auth.
Any GraphQL admin introspection queries don't require IP whitelisting or Guardian auth.
@danielmai danielmai changed the title Add authn for graphql and http admin endpoints [BREAKING] Add authn for graphql and http admin endpoints Oct 26, 2020
@danielmai danielmai changed the title [BREAKING] Add authn for graphql and http admin endpoints Add authn for graphql and http admin endpoints Oct 26, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/enterprise/acl Related to Access Control Lists area/graphql Issues related to GraphQL support on Dgraph.
Development

Successfully merging this pull request may close these issues.

Export endpoint doesn't require authentication
5 participants