New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Don't let users create endpoints inside ClusterNetworkCIDR #9383

Merged
merged 2 commits into from Jul 13, 2016

Conversation

Projects
None yet
10 participants
@danwinship
Contributor

danwinship commented Jun 16, 2016

If a service has a selector then it's pointless to manually create endpoints for it, because the endpoints controller will (eventually) just overwrite your manually-created endpoints with the correct value according to the selector. ("Eventually" meaning 30 seconds at the most, or sooner if something related to the service changes before then.) Nonetheless, it's currently possible for a user to temporarily override the Endpoints with an incorrect value. This patch adds an admission controller that prevents that.

This is part of the fix for #9255; the only reason openshift-sdn needs to track IP-to-pod/namespace mappings is because it can't 100% reliably distinguish endpoints-that-point-to-pod-IPs that were added by the endpoints controller (which are always safe) from endpoints-that-point-to-pod-IPs that were added by users (which probably indicate an attempt to bypass isolation), so it has to check that each endpoint points into the right namespace. With the admission controller in place, we can just make it accept all auto-created endpoints and reject all manually-created ClusterNetwork-internal endpoints.

(#9255 talked about making the admission controller do the pointing-into-ClusterNetworkCIDR check itself, and then getting rid of openshift-sdn's endpoint filter entirely. However, it looks like we're going to need to keep the endpoint filter around for 1.3 to do egress-firewall-related endpoint filtering anyway, so we might as well keep the other endpoint filtering there as well and just go with the simpler admission controller at least for now. The discussion there also suggested allowing cluster admins to override the admission controller, but I didn't bother to do that here, since there isn't much point in allowing even admins to manually create these Endpoints given that they'll just get overwritten anyway.)

Not sure about the use of a.GetUserInfo().GetName() == bootstrappolicy.MasterUsername to identify the endpoints controller... (It works but I don't know if it's "right".)

@smarterclayton @openshift/networking PTAL

@knobunc

This comment has been minimized.

Show comment
Hide comment
@knobunc

knobunc Jun 16, 2016

Contributor

I like the concept and implementation.

Contributor

knobunc commented Jun 16, 2016

I like the concept and implementation.

return apierrs.NewForbidden(a.GetResource().GroupResource(), ep.Name, fmt.Errorf("can't add explicit Endpoints to Service with a selector"))
}
return nil

This comment has been minimized.

@liggitt

liggitt Jun 16, 2016

Contributor

I'm not seeing how this prevents a user from updating an endpoint to include questionable IPs...

@liggitt

liggitt Jun 16, 2016

Contributor

I'm not seeing how this prevents a user from updating an endpoint to include questionable IPs...

This comment has been minimized.

@smarterclayton

smarterclayton Jun 16, 2016

Member

Unfortunately this is racy and can't guarantee you observe the right
service state. A user can flip the service state rapidly and inject
endpoints. We probably have to do the whitelist on endpoints plus control
who can set them.

On Jun 16, 2016, at 12:07 PM, Jordan Liggitt notifications@github.com
wrote:

In pkg/service/admission/endpoint_admission.go
#9383 (comment):

  • // The Endpoint controller is always allowed to add Endpoints
  • if a.GetUserInfo().GetName() == bootstrappolicy.MasterUsername {
  • return nil
    
  • }
    +
  • svc, err := r.kclient.Core().Services(a.GetNamespace()).Get(ep.Name)
  • if err != nil {
  • return apierrs.NewNotFound(a.GetResource().GroupResource(), ep.Name)
    
  • }
    +
  • if svc.Spec.Selector != nil {
  • return apierrs.NewForbidden(a.GetResource().GroupResource(), ep.Name, fmt.Errorf("can't add explicit Endpoints to Service with a selector"))
    
  • }
    +
  • return nil

I'm not seeing how this prevents a user from updating an endpoint to
include questionable IPs...


You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/openshift/origin/pull/9383/files/b21ea8af78d57db74065dacec6a4bc8f93496912#r67373975,
or mute the thread
https://github.com/notifications/unsubscribe/ABG_p9qRkygJOBp7B5uhAc7VyHn3PZ2Xks5qMXSsgaJpZM4I3hI-
.

@smarterclayton

smarterclayton Jun 16, 2016

Member

Unfortunately this is racy and can't guarantee you observe the right
service state. A user can flip the service state rapidly and inject
endpoints. We probably have to do the whitelist on endpoints plus control
who can set them.

On Jun 16, 2016, at 12:07 PM, Jordan Liggitt notifications@github.com
wrote:

In pkg/service/admission/endpoint_admission.go
#9383 (comment):

  • // The Endpoint controller is always allowed to add Endpoints
  • if a.GetUserInfo().GetName() == bootstrappolicy.MasterUsername {
  • return nil
    
  • }
    +
  • svc, err := r.kclient.Core().Services(a.GetNamespace()).Get(ep.Name)
  • if err != nil {
  • return apierrs.NewNotFound(a.GetResource().GroupResource(), ep.Name)
    
  • }
    +
  • if svc.Spec.Selector != nil {
  • return apierrs.NewForbidden(a.GetResource().GroupResource(), ep.Name, fmt.Errorf("can't add explicit Endpoints to Service with a selector"))
    
  • }
    +
  • return nil

I'm not seeing how this prevents a user from updating an endpoint to
include questionable IPs...


You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/openshift/origin/pull/9383/files/b21ea8af78d57db74065dacec6a4bc8f93496912#r67373975,
or mute the thread
https://github.com/notifications/unsubscribe/ABG_p9qRkygJOBp7B5uhAc7VyHn3PZ2Xks5qMXSsgaJpZM4I3hI-
.

This comment has been minimized.

@smarterclayton

smarterclayton Jun 16, 2016

Member
@smarterclayton

smarterclayton via email Jun 16, 2016

Member

This comment has been minimized.

@danwinship

danwinship Jun 16, 2016

Contributor

I'm not seeing how this prevents a user from updating an endpoint to include questionable IPs...

The admission controller runs on both creates and updates, though I guess the error message text doesn't reflect that

@danwinship

danwinship Jun 16, 2016

Contributor

I'm not seeing how this prevents a user from updating an endpoint to include questionable IPs...

The admission controller runs on both creates and updates, though I guess the error message text doesn't reflect that

This comment has been minimized.

@danwinship

danwinship Jun 16, 2016

Contributor

Unfortunately this is racy and can't guarantee you observe the right service state. A user can flip the service state rapidly and inject endpoints.

My original idea had been to have the endpoints controller set a particular annotation on the Endpoints object, and then enforce that only the endpoints controller is allowed to set that annotation, and openshift-sdn would check for that to know whether to trust it. Is that any better?

@danwinship

danwinship Jun 16, 2016

Contributor

Unfortunately this is racy and can't guarantee you observe the right service state. A user can flip the service state rapidly and inject endpoints.

My original idea had been to have the endpoints controller set a particular annotation on the Endpoints object, and then enforce that only the endpoints controller is allowed to set that annotation, and openshift-sdn would check for that to know whether to trust it. Is that any better?

This comment has been minimized.

@liggitt

liggitt Jun 16, 2016

Contributor

I wasn't expecting a user to be able to point to any IP they want just because the service doesn't have a selector. I was expecting something to prevent regular users from setting endpoint IPs within the service or pod CIDRs

@liggitt

liggitt Jun 16, 2016

Contributor

I wasn't expecting a user to be able to point to any IP they want just because the service doesn't have a selector. I was expecting something to prevent regular users from setting endpoint IPs within the service or pod CIDRs

This comment has been minimized.

@smarterclayton

smarterclayton Jun 17, 2016

Member

Yeah, that's the safer and more compact change - doesn't require us to
solve races.

On Jun 16, 2016, at 12:45 PM, Jordan Liggitt notifications@github.com
wrote:

In pkg/service/admission/endpoint_admission.go
#9383 (comment):

  • // The Endpoint controller is always allowed to add Endpoints
  • if a.GetUserInfo().GetName() == bootstrappolicy.MasterUsername {
  • return nil
    
  • }
    +
  • svc, err := r.kclient.Core().Services(a.GetNamespace()).Get(ep.Name)
  • if err != nil {
  • return apierrs.NewNotFound(a.GetResource().GroupResource(), ep.Name)
    
  • }
    +
  • if svc.Spec.Selector != nil {
  • return apierrs.NewForbidden(a.GetResource().GroupResource(), ep.Name, fmt.Errorf("can't add explicit Endpoints to Service with a selector"))
    
  • }
    +
  • return nil

I wasn't expecting a user to be able to point to any IP they want just
because the service doesn't have a selector. I was expecting something to
prevent regular users from setting endpoint IPs within the service or pod
CIDRs


You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/openshift/origin/pull/9383/files/b21ea8af78d57db74065dacec6a4bc8f93496912#r67381183,
or mute the thread
https://github.com/notifications/unsubscribe/ABG_p3vGWwVyHjYYmBgNTWvd3SzrrzHTks5qMX20gaJpZM4I3hI-
.

@smarterclayton

smarterclayton Jun 17, 2016

Member

Yeah, that's the safer and more compact change - doesn't require us to
solve races.

On Jun 16, 2016, at 12:45 PM, Jordan Liggitt notifications@github.com
wrote:

In pkg/service/admission/endpoint_admission.go
#9383 (comment):

  • // The Endpoint controller is always allowed to add Endpoints
  • if a.GetUserInfo().GetName() == bootstrappolicy.MasterUsername {
  • return nil
    
  • }
    +
  • svc, err := r.kclient.Core().Services(a.GetNamespace()).Get(ep.Name)
  • if err != nil {
  • return apierrs.NewNotFound(a.GetResource().GroupResource(), ep.Name)
    
  • }
    +
  • if svc.Spec.Selector != nil {
  • return apierrs.NewForbidden(a.GetResource().GroupResource(), ep.Name, fmt.Errorf("can't add explicit Endpoints to Service with a selector"))
    
  • }
    +
  • return nil

I wasn't expecting a user to be able to point to any IP they want just
because the service doesn't have a selector. I was expecting something to
prevent regular users from setting endpoint IPs within the service or pod
CIDRs


You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/openshift/origin/pull/9383/files/b21ea8af78d57db74065dacec6a4bc8f93496912#r67381183,
or mute the thread
https://github.com/notifications/unsubscribe/ABG_p3vGWwVyHjYYmBgNTWvd3SzrrzHTks5qMX20gaJpZM4I3hI-
.

@bmeng

This comment has been minimized.

Show comment
Hide comment
@bmeng

bmeng Jun 17, 2016

Contributor

In some of our storage testing, we will create the endpoints manually to point to the external/internal glusterfs server. Seems this kind of testing will be blocked by the change. And not sure if the real users will also have this kind of requirement.

Our testing step like this:
oc create -f https://raw.githubusercontent.com/openshift-qe/v3-testfiles/master/persistent-volumes/gluster/security/endpoints_2.json

Contributor

bmeng commented Jun 17, 2016

In some of our storage testing, we will create the endpoints manually to point to the external/internal glusterfs server. Seems this kind of testing will be blocked by the change. And not sure if the real users will also have this kind of requirement.

Our testing step like this:
oc create -f https://raw.githubusercontent.com/openshift-qe/v3-testfiles/master/persistent-volumes/gluster/security/endpoints_2.json

@danwinship

This comment has been minimized.

Show comment
Hide comment
@danwinship

danwinship Jun 17, 2016

Contributor

@bmeng This only prevents you from creating endpoints on services that have a "selector" field; on those services, openshift will overwrite your endpoints at some unpredictable point in the future anyway, so that wouldn't be suitable for testing.

Contributor

danwinship commented Jun 17, 2016

@bmeng This only prevents you from creating endpoints on services that have a "selector" field; on those services, openshift will overwrite your endpoints at some unpredictable point in the future anyway, so that wouldn't be suitable for testing.

@smarterclayton

This comment has been minimized.

Show comment
Hide comment
@smarterclayton

smarterclayton Jun 17, 2016

Member

So the other proposal (which is not racy) is to make this two parts:

  1. Check whether endpoints is over a "protected" range of IPs (disjoint sets, including ClusterNetworkCIDR and ServiceNetworkCIDR, but potentially others).
  2. If the endpoints are protected, do a LSAR on a virtual subresource of endpoints (like "POST" on "endpoints/internal" or something, @liggitt can recommend better). See pkg/build/admission/strategyrestrictions/admission.go#checkBuildConfigAuthorization for a rough analogue.
  3. Give the endpoints controller that "endpoints/internal" "POST" permission.

That ensures that admins can set any external endpoints they like, admins can grant to specific users, and regular users can't set.

Member

smarterclayton commented Jun 17, 2016

So the other proposal (which is not racy) is to make this two parts:

  1. Check whether endpoints is over a "protected" range of IPs (disjoint sets, including ClusterNetworkCIDR and ServiceNetworkCIDR, but potentially others).
  2. If the endpoints are protected, do a LSAR on a virtual subresource of endpoints (like "POST" on "endpoints/internal" or something, @liggitt can recommend better). See pkg/build/admission/strategyrestrictions/admission.go#checkBuildConfigAuthorization for a rough analogue.
  3. Give the endpoints controller that "endpoints/internal" "POST" permission.

That ensures that admins can set any external endpoints they like, admins can grant to specific users, and regular users can't set.

@marun

This comment has been minimized.

Show comment
Hide comment
@marun

marun Jun 17, 2016

Member

@smarterclayton Can you explain 'LSAR' and how it would be used in this context? I'm not familiar with it and I'm having trouble finding a good explanation.

Member

marun commented Jun 17, 2016

@smarterclayton Can you explain 'LSAR' and how it would be used in this context? I'm not familiar with it and I'm having trouble finding a good explanation.

@smarterclayton

This comment has been minimized.

Show comment
Hide comment
@smarterclayton

smarterclayton Jun 17, 2016

Member

It's a local subject access review - subject access review is basically a
removable "ACL check" that the component invokes to check access to
something.

On Fri, Jun 17, 2016 at 11:07 AM, Maru Newby notifications@github.com
wrote:

@smarterclayton https://github.com/smarterclayton Can you explain
'LSAR' and how it would be used in this context? I'm not familiar with it
and I'm having trouble finding a good explanation.


You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#9383 (comment),
or mute the thread
https://github.com/notifications/unsubscribe/ABG_p6nXkMHpxTHh8EAQqW9Xoxic6eEEks5qMrgmgaJpZM4I3hI-
.

Member

smarterclayton commented Jun 17, 2016

It's a local subject access review - subject access review is basically a
removable "ACL check" that the component invokes to check access to
something.

On Fri, Jun 17, 2016 at 11:07 AM, Maru Newby notifications@github.com
wrote:

@smarterclayton https://github.com/smarterclayton Can you explain
'LSAR' and how it would be used in this context? I'm not familiar with it
and I'm having trouble finding a good explanation.


You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#9383 (comment),
or mute the thread
https://github.com/notifications/unsubscribe/ABG_p6nXkMHpxTHh8EAQqW9Xoxic6eEEks5qMrgmgaJpZM4I3hI-
.

@danwinship danwinship changed the title from Don't let users manually create endpoints for services with selectors to Don't let users create endpoints inside ClusterNetworkCIDR Jun 21, 2016

@danwinship

This comment has been minimized.

Show comment
Hide comment
@danwinship

danwinship Jun 21, 2016

Contributor

OK, branch updated (and PR renamed):

  • It now works as originally suggested, blocking unauthorized users from adding endpoints in ClusterNetworkCIDR and ServiceNetworkCIDR.
    • There's currently no config option to add other CIDR blocks to this list. I could add it, but what's the use case? It seems like if a user can connect to the IP directly, then they should be able to create a service pointing to it as well; and if they can't connect to the IP directly (for some reason) then the admin shouldn't have to configure any additional thing to make it so they can't connect via a service...
  • The endpoints controller is explicitly given permission to bypass the check.
    • Do I need to do something to upgrade existing clusters to add this permission?
    • Although I didn't add it on purpose, cluster admins seem to get to bypass the check by default too. (I guess because the ClusterAdminRoleName role applies to Resources("*")? So this is a feature, not a bug?)
  • Existing endpoints are not revalidated if you change ClusterNetworkCIDR, since we can't tell at that point whether the person who added them would have had permission. So that should be documented.
  • No tests yet but I'll add some once you're happy with the rest of it
Contributor

danwinship commented Jun 21, 2016

OK, branch updated (and PR renamed):

  • It now works as originally suggested, blocking unauthorized users from adding endpoints in ClusterNetworkCIDR and ServiceNetworkCIDR.
    • There's currently no config option to add other CIDR blocks to this list. I could add it, but what's the use case? It seems like if a user can connect to the IP directly, then they should be able to create a service pointing to it as well; and if they can't connect to the IP directly (for some reason) then the admin shouldn't have to configure any additional thing to make it so they can't connect via a service...
  • The endpoints controller is explicitly given permission to bypass the check.
    • Do I need to do something to upgrade existing clusters to add this permission?
    • Although I didn't add it on purpose, cluster admins seem to get to bypass the check by default too. (I guess because the ClusterAdminRoleName role applies to Resources("*")? So this is a feature, not a bug?)
  • Existing endpoints are not revalidated if you change ClusterNetworkCIDR, since we can't tell at that point whether the person who added them would have had permission. So that should be documented.
  • No tests yet but I'll add some once you're happy with the rest of it
@@ -70,6 +70,8 @@ const (
BuildStrategySourceRoleName = "system:build-strategy-source"
BuildStrategyJenkinsPipelineRoleName = "system:build-strategy-jenkinspipeline"
RestrictedEndpointsAdmissionRoleName = "system:endpoints-admission"

This comment has been minimized.

@liggitt

liggitt Jun 21, 2016

Contributor

instead of this, we want a role, associated with a service account, that actually includes all the permissions the endpoints controller needs. I assume that is get/list/watch access to services and pods, full access to endpoints, and now create access to our virtual endpoints/restricted resource (or whatever we call it)

@liggitt

liggitt Jun 21, 2016

Contributor

instead of this, we want a role, associated with a service account, that actually includes all the permissions the endpoints controller needs. I assume that is get/list/watch access to services and pods, full access to endpoints, and now create access to our virtual endpoints/restricted resource (or whatever we call it)

@@ -607,6 +607,14 @@ func GetBootstrapClusterRoles() []authorizationapi.ClusterRole {
authorizationapi.NewRule("get").Groups(projectGroup).Resources("projects").RuleOrDie(),
},
},
{

This comment has been minimized.

@liggitt

liggitt Jun 21, 2016

Contributor

move the definition and binding to infra_sa_policy.go, and follow the pattern there... we define a service account name, role name, and permissions we expect that service account to have

@liggitt

liggitt Jun 21, 2016

Contributor

move the definition and binding to infra_sa_policy.go, and follow the pattern there... we define a service account name, role name, and permissions we expect that service account to have

}
resource := kapi.Resource(authorizationapi.RestrictedEndpointsResource)
subjectAccessReview := authorizationapi.AddUserToLSAR(a.GetUserInfo(),

This comment has been minimized.

@liggitt

liggitt Jun 21, 2016

Contributor

since 99% of restricted writes will be from the endpoints controller it would be nice to avoid checking this permission on the same user super frequently... @deads2k, do we have an LRU/expiring cache around SAR results anywhere else?

@liggitt

liggitt Jun 21, 2016

Contributor

since 99% of restricted writes will be from the endpoints controller it would be nice to avoid checking this permission on the same user super frequently... @deads2k, do we have an LRU/expiring cache around SAR results anywhere else?

This comment has been minimized.

@deads2k

deads2k Jun 24, 2016

Contributor

since 99% of restricted writes will be from the endpoints controller it would be nice to avoid checking this permission on the same user super frequently... @deads2k, do we have an LRU/expiring cache around SAR results anywhere else?

No, we don't. What about using the WantsAuthorizer at least to avoid the API server call and keep everything in-cache?

@deads2k

deads2k Jun 24, 2016

Contributor

since 99% of restricted writes will be from the endpoints controller it would be nice to avoid checking this permission on the same user super frequently... @deads2k, do we have an LRU/expiring cache around SAR results anywhere else?

No, we don't. What about using the WantsAuthorizer at least to avoid the API server call and keep everything in-cache?

This comment has been minimized.

@liggitt

liggitt Jun 24, 2016

Contributor

+1, forgot about that

@danwinship, have your admission plugin implement a method like this:

func (o *podNodeConstraints) SetAuthorizer(a authorizer.Authorizer) {
    o.authorizer = a
}

add a check to make sure you conform to the WantsAuthorizer interface:

var _ = oadmission.WantsAuthorizer(&restrictedEndpointsAdmission{})

then, rather than an API call to do the access check, you can check against the same authorizer the API guards itself with (see podNodeConstraints#checkPodsBindAccess for an example how to do this)

@liggitt

liggitt Jun 24, 2016

Contributor

+1, forgot about that

@danwinship, have your admission plugin implement a method like this:

func (o *podNodeConstraints) SetAuthorizer(a authorizer.Authorizer) {
    o.authorizer = a
}

add a check to make sure you conform to the WantsAuthorizer interface:

var _ = oadmission.WantsAuthorizer(&restrictedEndpointsAdmission{})

then, rather than an API call to do the access check, you can check against the same authorizer the API guards itself with (see podNodeConstraints#checkPodsBindAccess for an example how to do this)

@liggitt

This comment has been minimized.

Show comment
Hide comment
@liggitt

liggitt Jun 21, 2016

Contributor

Do I need to do something to upgrade existing clusters to add this permission?

If you define it as an infra service account role, the new role, role binding, and service account will be created automatically on startup

Contributor

liggitt commented Jun 21, 2016

Do I need to do something to upgrade existing clusters to add this permission?

If you define it as an infra service account role, the new role, role binding, and service account will be created automatically on startup

@liggitt

This comment has been minimized.

Show comment
Hide comment
@liggitt

liggitt Jun 21, 2016

Contributor

Although I didn't add it on purpose, cluster admins seem to get to bypass the check by default too. (I guess because the ClusterAdminRoleName role applies to Resources("*")? So this is a feature, not a bug?)

That's a feature... * on * means root access. For this, we'll want to switch the endpoints controller to run using a service account (new controllers all get their own service account, and we've been switching old ones piecemill)

Contributor

liggitt commented Jun 21, 2016

Although I didn't add it on purpose, cluster admins seem to get to bypass the check by default too. (I guess because the ClusterAdminRoleName role applies to Resources("*")? So this is a feature, not a bug?)

That's a feature... * on * means root access. For this, we'll want to switch the endpoints controller to run using a service account (new controllers all get their own service account, and we've been switching old ones piecemill)

@danwinship

This comment has been minimized.

Show comment
Hide comment
@danwinship

danwinship Jun 22, 2016

Contributor

@liggitt updated to use a service account

Contributor

danwinship commented Jun 22, 2016

@liggitt updated to use a service account

// Managing endpoints
{
APIGroups: []string{kapi.GroupName},
Verbs: sets.NewString("get", "list", "create", "update", "delete"),

This comment has been minimized.

@liggitt

liggitt Jun 22, 2016

Contributor

do we know if the endpoints controller uses patch?

@liggitt

liggitt Jun 22, 2016

Contributor

do we know if the endpoints controller uses patch?

This comment has been minimized.

@danwinship

danwinship Jun 22, 2016

Contributor

It doesn't. If the endpoint exists but is wrong it just overwrites it with Update().

@danwinship

danwinship Jun 22, 2016

Contributor

It doesn't. If the endpoint exists but is wrong it just overwrites it with Update().

Verbs: sets.NewString("create"),
Resources: sets.NewString("endpoints/restricted"),
},
},

This comment has been minimized.

@liggitt

liggitt Jun 22, 2016

Contributor

does the endpoints controller need permissions to create/get events?

@liggitt

liggitt Jun 22, 2016

Contributor

does the endpoints controller need permissions to create/get events?

This comment has been minimized.

@danwinship

danwinship Jun 22, 2016

Contributor

I don't think so? There's nothing relevant about "events" in endpoints_controller.go, and it doesn't log any errors with the current set of permissions.

@danwinship

danwinship Jun 22, 2016

Contributor

I don't think so? There's nothing relevant about "events" in endpoints_controller.go, and it doesn't log any errors with the current set of permissions.

func (r *restrictedEndpointsAdmission) Admit(a kadmission.Attributes) error {
if a.GetResource().GroupResource() != kapi.Resource("endpoints") {
return nil
}

This comment has been minimized.

@pravisankar

pravisankar Jun 23, 2016

Contributor

We want only create and update endpoints to perform these checks?

    if (a.GetOperation() != admission.Create) && (a.GetOperation() != admission.Update)  {
        return nil
    }
@pravisankar

pravisankar Jun 23, 2016

Contributor

We want only create and update endpoints to perform these checks?

    if (a.GetOperation() != admission.Create) && (a.GetOperation() != admission.Update)  {
        return nil
    }

This comment has been minimized.

@pravisankar

pravisankar Jun 23, 2016

Contributor

Ignore, I see L49: kadmission.NewHandler(kadmission.Create, kadmission.Update) does the trick

@pravisankar

pravisankar Jun 23, 2016

Contributor

Ignore, I see L49: kadmission.NewHandler(kadmission.Create, kadmission.Update) does the trick

subjectAccessReview := authorizationapi.AddUserToLSAR(a.GetUserInfo(),
&authorizationapi.LocalSubjectAccessReview{
Action: authorizationapi.AuthorizationAttributes{
Verb: "create",

This comment has been minimized.

@pravisankar

pravisankar Jun 23, 2016

Contributor

This could be create or update...Verb: strings.ToLower(a.GetOperation())?

@pravisankar

pravisankar Jun 23, 2016

Contributor

This could be create or update...Verb: strings.ToLower(a.GetOperation())?

This comment has been minimized.

@danwinship

danwinship Jun 23, 2016

Contributor

No, this isn't a real resource that actually gets created/updated; it's a fake resource that exists only to have permissions checked against it. (This is basically copied from the build strategy admission controller.)

@danwinship

danwinship Jun 23, 2016

Contributor

No, this isn't a real resource that actually gets created/updated; it's a fake resource that exists only to have permissions checked against it. (This is basically copied from the build strategy admission controller.)

// Permission for RestrictedEndpointsAdmission
{
APIGroups: []string{kapi.GroupName},
Verbs: sets.NewString("create"),

This comment has been minimized.

@pravisankar

pravisankar Jun 23, 2016

Contributor

Don't we want to allow "update" here?

@pravisankar

pravisankar Jun 23, 2016

Contributor

Don't we want to allow "update" here?

const RestrictedEndpointsPluginName = "RestrictedEndpointsAdmission"
func init() {
kadmission.RegisterPlugin("RestrictedEndpointsAdmission", func(client clientset.Interface, config io.Reader) (kadmission.Interface, error) {

This comment has been minimized.

@danwinship

danwinship Jun 23, 2016

Contributor

I don't know. Possibly we don't, but all of the other explicitly-created admission controller types do it too.

@danwinship

danwinship Jun 23, 2016

Contributor

I don't know. Possibly we don't, but all of the other explicitly-created admission controller types do it too.

@dcbw

This comment has been minimized.

Show comment
Hide comment
@dcbw

dcbw Jun 23, 2016

Member

LGTM, seems like all the other comments have already been addressed with code or replies, right?

Member

dcbw commented Jun 23, 2016

LGTM, seems like all the other comments have already been addressed with code or replies, right?

@liggitt

This comment has been minimized.

Show comment
Hide comment
@liggitt

liggitt Jun 23, 2016

Contributor

there's the question of perf impact of doing an API call on every update by the endpoints controller: #9383 (comment)

Contributor

liggitt commented Jun 23, 2016

there's the question of perf impact of doing an API call on every update by the endpoints controller: #9383 (comment)

@danwinship

This comment has been minimized.

Show comment
Hide comment
@danwinship

danwinship Jun 27, 2016

Contributor

@liggitt updated to use authorizer.Authorize()

Contributor

danwinship commented Jun 27, 2016

@liggitt updated to use authorizer.Authorize()

@liggitt

This comment has been minimized.

Show comment
Hide comment
@liggitt

liggitt Jun 27, 2016

Contributor

Thanks, looks close. This will prevent a user from making an unrelated change (like adding an annotation) to an Endpoints object created by the endpoints controller. To avoid blocking that, I think we'd need to pay attention to the existing addresses on the old object during an update, and only restrict new/modified addresses. That depends on an enhancement to the info available to admission which we'll get in the next rebase, so I think a follow-up would be ok there.

Can you add an integration test that makes sure the cluster admin and endpoints controller service account can create endpoints with whatever addresses they want, and a project admin is denied permission for addresses in the service/cluster ranges, and allowed outside those ranges?

Contributor

liggitt commented Jun 27, 2016

Thanks, looks close. This will prevent a user from making an unrelated change (like adding an annotation) to an Endpoints object created by the endpoints controller. To avoid blocking that, I think we'd need to pay attention to the existing addresses on the old object during an update, and only restrict new/modified addresses. That depends on an enhancement to the info available to admission which we'll get in the next rebase, so I think a follow-up would be ok there.

Can you add an integration test that makes sure the cluster admin and endpoints controller service account can create endpoints with whatever addresses they want, and a project admin is denied permission for addresses in the service/cluster ranges, and allowed outside those ranges?

@liggitt

This comment has been minimized.

Show comment
Hide comment
@liggitt

liggitt Jun 27, 2016

Contributor

also needs hack/update-generated-docs.sh run

Contributor

liggitt commented Jun 27, 2016

also needs hack/update-generated-docs.sh run

}
}
func TestEndpointAdmissionClusterAdmin(t *testing.T) {

This comment has been minimized.

@liggitt

liggitt Jun 29, 2016

Contributor

test setup is a little expensive (~7-10 seconds per test on Jenkins), so I would unify the tests to a single Test* method and do this as a table test instead

@liggitt

liggitt Jun 29, 2016

Contributor

test setup is a little expensive (~7-10 seconds per test on Jenkins), so I would unify the tests to a single Test* method and do this as a table test instead

@liggitt

This comment has been minimized.

Show comment
Hide comment
@liggitt

liggitt Jun 29, 2016

Contributor

LGTM pending integration test combining (and needs a rebase). can you open a follow up issue to track allowing updates unrelated to address changes

Contributor

liggitt commented Jun 29, 2016

LGTM pending integration test combining (and needs a rebase). can you open a follow up issue to track allowing updates unrelated to address changes

@@ -10,4 +10,6 @@ const (
NodeMetricsResource = "nodes/metrics"
NodeStatsResource = "nodes/stats"
NodeLogResource = "nodes/log"
RestrictedEndpointsResource = "endpoints/restricted"

This comment has been minimized.

@liggitt

liggitt Jun 29, 2016

Contributor

@deads2k any better ideas for the virtual resource to do authz checks against to cover restricted endpoint addresses?

@liggitt

liggitt Jun 29, 2016

Contributor

@deads2k any better ideas for the virtual resource to do authz checks against to cover restricted endpoint addresses?

This comment has been minimized.

@deads2k

deads2k Jun 29, 2016

Contributor

@deads2k any better ideas for the virtual resource to do authz checks against to cover restricted endpoint addresses?

This seems fine.

@deads2k

deads2k Jun 29, 2016

Contributor

@deads2k any better ideas for the virtual resource to do authz checks against to cover restricted endpoint addresses?

This seems fine.

@liggitt

This comment has been minimized.

Show comment
Hide comment
@liggitt

liggitt Jun 29, 2016

Contributor

do we want this enabled by default? (I think we do)
do we need a way to disable it? (not sure we do)

we need release notes calling out the behavior change (some users might not be able to create the same endpoints they used to be able to, without additional permission grants), and documentation about how to give users the ability to create restricted endpoints

Contributor

liggitt commented Jun 29, 2016

do we want this enabled by default? (I think we do)
do we need a way to disable it? (not sure we do)

we need release notes calling out the behavior change (some users might not be able to create the same endpoints they used to be able to, without additional permission grants), and documentation about how to give users the ability to create restricted endpoints

@openshift-bot

This comment has been minimized.

Show comment
Hide comment
@openshift-bot

openshift-bot Jun 30, 2016

[Test]ing while waiting on the merge queue

openshift-bot commented Jun 30, 2016

[Test]ing while waiting on the merge queue

@danwinship

This comment has been minimized.

Show comment
Hide comment
@danwinship

danwinship Jul 1, 2016

Contributor

failures appear to be spurious network/jenkins issues

Contributor

danwinship commented Jul 1, 2016

failures appear to be spurious network/jenkins issues

@knobunc

This comment has been minimized.

Show comment
Hide comment
@knobunc

knobunc Jul 8, 2016

Contributor

LGTM, re[merge]

Contributor

knobunc commented Jul 8, 2016

LGTM, re[merge]

@danwinship

This comment has been minimized.

Show comment
Hide comment
@danwinship

danwinship Jul 11, 2016

Contributor

test_pull_requests_origin_check failure is #9371
test_pull_requests_origin_conformance failure is #9681
re-[merge]

Contributor

danwinship commented Jul 11, 2016

test_pull_requests_origin_check failure is #9371
test_pull_requests_origin_conformance failure is #9681
re-[merge]

@danwinship

This comment has been minimized.

Show comment
Hide comment
@danwinship

danwinship Jul 11, 2016

Contributor

9681 again... [test]

Contributor

danwinship commented Jul 11, 2016

9681 again... [test]

@smarterclayton

This comment has been minimized.

Show comment
Hide comment
@smarterclayton

smarterclayton Jul 11, 2016

Member

Flake was #9780

Member

smarterclayton commented Jul 11, 2016

Flake was #9780

@smarterclayton

This comment has been minimized.

Show comment
Hide comment
@smarterclayton

smarterclayton Jul 11, 2016

Member

Admission now has the appropriate info for mutations (saw some of the earlier comments).

Member

smarterclayton commented Jul 11, 2016

Admission now has the appropriate info for mutations (saw some of the earlier comments).

@danwinship

This comment has been minimized.

Show comment
Hide comment
@danwinship

danwinship Jul 12, 2016

Contributor

rebased, and added a fixup commit (to be squashed) using OldObject to allow non-IP-related endpoint modifications, and a corresponding update to the integration test

Contributor

danwinship commented Jul 12, 2016

rebased, and added a fixup commit (to be squashed) using OldObject to allow non-IP-related endpoint modifications, and a corresponding update to the integration test

@smarterclayton

This comment has been minimized.

Show comment
Hide comment
@smarterclayton
Member

smarterclayton commented Jul 12, 2016

Flake in #9803

@smarterclayton

This comment has been minimized.

Show comment
Hide comment
@smarterclayton

smarterclayton Jul 12, 2016

Member

Another flake #9804

Member

smarterclayton commented Jul 12, 2016

Another flake #9804

Add an admission controller to block illegal service endpoints
Add an admission controller to allow blocking service endpoints in
particular CIDR ranges (by default, the ClusterNetworkCIDR and
ServiceNetworkCIDR) and give the endpoints controller service account
permission to bypass it.
@openshift-bot

This comment has been minimized.

Show comment
Hide comment
@openshift-bot

openshift-bot Jul 12, 2016

continuous-integration/openshift-jenkins/merge SUCCESS (https://ci.openshift.redhat.com/jenkins/job/test_pr_origin/6169/) (Image: devenv-rhel7_4576)

openshift-bot commented Jul 12, 2016

continuous-integration/openshift-jenkins/merge SUCCESS (https://ci.openshift.redhat.com/jenkins/job/test_pr_origin/6169/) (Image: devenv-rhel7_4576)

@openshift-bot

This comment has been minimized.

Show comment
Hide comment
@openshift-bot

openshift-bot Jul 12, 2016

Evaluated for origin merge up to 6afa180

openshift-bot commented Jul 12, 2016

Evaluated for origin merge up to 6afa180

@openshift-bot

This comment has been minimized.

Show comment
Hide comment
@openshift-bot

openshift-bot Jul 12, 2016

Evaluated for origin test up to 6afa180

openshift-bot commented Jul 12, 2016

Evaluated for origin test up to 6afa180

@openshift-bot

This comment has been minimized.

Show comment
Hide comment
@openshift-bot

openshift-bot Jul 12, 2016

continuous-integration/openshift-jenkins/test SUCCESS (https://ci.openshift.redhat.com/jenkins/job/test_pr_origin/6169/)

openshift-bot commented Jul 12, 2016

continuous-integration/openshift-jenkins/test SUCCESS (https://ci.openshift.redhat.com/jenkins/job/test_pr_origin/6169/)

@openshift-bot openshift-bot merged commit f7638df into openshift:master Jul 13, 2016

3 checks passed

continuous-integration/openshift-jenkins/merge Passed
Details
continuous-integration/openshift-jenkins/test Passed
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
@danwinship

This comment has been minimized.

Show comment
Hide comment
@danwinship

danwinship Jul 13, 2016

Contributor

um... I had assumed that pushing new commits would cancel the openshift-bot merge request... so anyway, this got merged with the change to allow unpriviliged users to still make non-IP-address-related updates to restricted Endpoints objects, but I don't think anyone ever reviewed that new code. FYI.

Contributor

danwinship commented Jul 13, 2016

um... I had assumed that pushing new commits would cancel the openshift-bot merge request... so anyway, this got merged with the change to allow unpriviliged users to still make non-IP-address-related updates to restricted Endpoints objects, but I don't think anyone ever reviewed that new code. FYI.

@danwinship danwinship deleted the danwinship:endpoint-admission branch Jul 13, 2016

@liggitt

This comment has been minimized.

Show comment
Hide comment
@liggitt

liggitt Jul 13, 2016

Contributor

it's just the 3-4 lines that sees if we have an old object with identical subsets? that seems fine, but a test to make sure non-admins can annotate/label an endpoints object like that would be good

Contributor

liggitt commented Jul 13, 2016

it's just the 3-4 lines that sees if we have an old object with identical subsets? that seems fine, but a test to make sure non-admins can annotate/label an endpoints object like that would be good

@danwinship

This comment has been minimized.

Show comment
Hide comment
@liggitt

This comment has been minimized.

Show comment
Hide comment
@liggitt

liggitt Jul 13, 2016

Contributor

ah, missed that. looks fine

Contributor

liggitt commented Jul 13, 2016

ah, missed that. looks fine

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment