Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add protection for reserved API groups #79992

Merged
merged 1 commit into from Jul 24, 2019

Conversation

@deads2k
Copy link
Contributor

commented Jul 10, 2019

Implementation of kubernetes/enhancements#1111

CRDs under k8s.io and kubernetes.io must have the "api-approved.kubernetes.io" set to either `unapproved.*` or a link to the pull request approving the schema.  See https://github.com/kubernetes/enhancements/pull/1111 for more details.

/priority important-soon
/kind feature
@kubernetes/sig-api-machinery-api-reviews

/hold
Holding for more discussion in the KEP

/cc @kubernetes/api-approvers
/assign @liggitt @sttts @lavalamp

lastSeen, seenBefore := c.lastSeenProtectedAnnotation[inCustomResourceDefinition.Name]
c.lastSeenProtectedAnnotationLock.Unlock()
if seenBefore && protectionAnnotationValue == lastSeen {
return nil

This comment has been minimized.

Copy link
@lavalamp

lavalamp Jul 10, 2019

Member

I think this is premature optimization. I think it also means someone could delete the condition and it wouldn't be put back. I would just remove this, it should be pretty cheap to recompute the condition. You already protect against unnecessary writes below.

This comment has been minimized.

Copy link
@lavalamp

lavalamp Jul 10, 2019

Member

I guess since this just seems to be a stopgap and the validation criteria will do the work in the future, maybe this isn't too important.

This comment has been minimized.

Copy link
@deads2k

deads2k Jul 12, 2019

Author Contributor

I think this is premature optimization. I think it also means someone could delete the condition and it wouldn't be put back. I would just remove this, it should be pretty cheap to recompute the condition. You already protect against unnecessary writes below.

We've seen this particular area fight over messages and reasons. This was inspired by the non-structural condition which operates similarly.

This comment has been minimized.

Copy link
@liggitt

liggitt Jul 12, 2019

Member

yeah, this is a simple solution to avoid dueling HA apiservers, which is important

}

// store annotation in order to avoid repeated updates for the same annotation (and potential
// fights of API server in HA environments).

This comment has been minimized.

Copy link
@lavalamp

lavalamp Jul 10, 2019

Member

Yeah apiserver fights are problematic if you do what I said in the prior comment. But there's no way to tell the difference between an apiserver and a user.

What if we versioned the condition somehow, such that apiservers using the same version are running the same code and therefore won't fight, and apiservers using older versions will avoid writing over newer versions?

This comment has been minimized.

Copy link
@deads2k

deads2k Jul 12, 2019

Author Contributor

Yeah apiserver fights are problematic if you do what I said in the prior comment. But there's no way to tell the difference between an apiserver and a user.

That seems like overkill for an informational condition. If someone twiddles this, they're only lying to themselves. I'd prefer to avoid the risk the apiserver fights and take the chance that a cluster-admin intentionally deceives themselves. Also, that cluster-admin needs therapy.

klog.Errorf("Couldn't get object from tombstone %#v", obj)
return
}
castObj, ok = tombstone.Obj.(*apiextensions.CustomResourceDefinition)

This comment has been minimized.

Copy link
@lavalamp

lavalamp Jul 10, 2019

Member

You only need the name, which is in the tombstone already IIRC? You might not need this second cast.

This comment has been minimized.

Copy link
@deads2k

deads2k Jul 12, 2019

Author Contributor

You only need the name, which is in the tombstone already IIRC? You might not need this second cast.

The name is used a few lines down. I'd prefer to keep it because I think it reads easier unless you feel strongly and prefer the idea of depending on the shape of a key to match a map set from a different value. I don't think future code archeologists would appreciate having to know it.

This comment has been minimized.

Copy link
@lavalamp

lavalamp Jul 23, 2019

Member

My point was you can still do what you want to even if this cast fails. I don't think it's too important.

return field.ErrorList{field.Invalid(field.NewPath("metadata", "annotations").Key(v1beta1.KubeAPIApprovedAnnotation), newCRD.Annotations[v1beta1.KubeAPIApprovedAnnotation], reason)}
case apihelpers.APIApprovalMissing:
return field.ErrorList{field.Required(field.NewPath("metadata", "annotations").Key(v1beta1.KubeAPIApprovedAnnotation), reason)}
case apihelpers.APIApproved, apihelpers.APIUnapproved:

This comment has been minimized.

Copy link
@lavalamp

lavalamp Jul 10, 2019

Member

This reads a little funny, maybe instead of "APIUnapproved" we can have a name something more like

  • APIApprovalOverridden
  • APIApprovalBypassed
  • APIKnowinglyUnapproved

Something like that?

@liggitt liggitt added this to Required for GA, in progress in Custom Resource Definitions Jul 11, 2019

@deads2k deads2k force-pushed the deads2k:crd-protection branch from 86f249f to 652489a Jul 22, 2019

@deads2k

This comment has been minimized.

Copy link
Contributor Author

commented Jul 22, 2019

/retest

@deads2k

This comment has been minimized.

Copy link
Contributor Author

commented Jul 22, 2019

/hold cancel

@fejta-bot

This comment has been minimized.

Copy link

commented Jul 22, 2019

This PR may require API review.

If so, when the changes are ready, complete the pre-review checklist and request an API review.

Status of requested reviews is tracked in the API Review project.

1 similar comment
@fejta-bot

This comment has been minimized.

Copy link

commented Jul 22, 2019

This PR may require API review.

If so, when the changes are ready, complete the pre-review checklist and request an API review.

Status of requested reviews is tracked in the API Review project.

@@ -304,6 +309,11 @@ const (
NonStructuralSchema CustomResourceDefinitionConditionType = "NonStructuralSchema"
// Terminating means that the CustomResourceDefinition has been deleted and is cleaning up.
Terminating CustomResourceDefinitionConditionType = "Terminating"
// KubeAPIApproved indicates that an API in *.k8s.io or *.kubernetes.io is or is not approved. For CRDs

This comment has been minimized.

Copy link
@lavalamp

lavalamp Jul 23, 2019

Member

I think you can't use the name "...Approved" to also mean "...or is not approved", that will confuse people. Maybe "ApprovalStatusKnown"? "ApprovalPolicyConformant"?

Also no abbreviations, s/Kube/Kubernetes/

@deads2k deads2k force-pushed the deads2k:crd-protection branch 2 times, most recently from eb3e23e to 410528f Jul 23, 2019

@lavalamp

This comment has been minimized.

Copy link
Member

commented Jul 23, 2019

I'm kinda sorry I asked for such a long name after seeing the controller, haha.

/approve

Does the merge bot squash for us now?

@k8s-ci-robot

This comment has been minimized.

Copy link
Contributor

commented Jul 23, 2019

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: deads2k, lavalamp

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@deads2k deads2k force-pushed the deads2k:crd-protection branch from 410528f to aa84028 Jul 24, 2019

@sttts

This comment has been minimized.

Copy link
Contributor

commented Jul 24, 2019

/lgtm

@k8s-ci-robot k8s-ci-robot added the lgtm label Jul 24, 2019

@k8s-ci-robot k8s-ci-robot merged commit e0ff963 into kubernetes:master Jul 24, 2019

23 checks passed

cla/linuxfoundation deads2k authorized
Details
pull-kubernetes-bazel-build Job succeeded.
Details
pull-kubernetes-bazel-test Job succeeded.
Details
pull-kubernetes-conformance-image-test Skipped.
pull-kubernetes-cross Skipped.
pull-kubernetes-dependencies Job succeeded.
Details
pull-kubernetes-e2e-gce Job succeeded.
Details
pull-kubernetes-e2e-gce-100-performance Job succeeded.
Details
pull-kubernetes-e2e-gce-csi-serial Skipped.
pull-kubernetes-e2e-gce-device-plugin-gpu Job succeeded.
Details
pull-kubernetes-e2e-gce-iscsi Skipped.
pull-kubernetes-e2e-gce-iscsi-serial Skipped.
pull-kubernetes-e2e-gce-storage-slow Skipped.
pull-kubernetes-godeps Skipped.
pull-kubernetes-integration Job succeeded.
Details
pull-kubernetes-kubemark-e2e-gce-big Job succeeded.
Details
pull-kubernetes-local-e2e Skipped.
pull-kubernetes-node-e2e Job succeeded.
Details
pull-kubernetes-node-e2e-containerd Job succeeded.
Details
pull-kubernetes-typecheck Job succeeded.
Details
pull-kubernetes-verify Job succeeded.
Details
pull-publishing-bot-validate Skipped.
tide In merge pool.
Details

@liggitt liggitt moved this from Required for GA, in progress to Complete in Custom Resource Definitions Jul 25, 2019

@liggitt liggitt added this to the v1.16 milestone Aug 6, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.