KEP for promoting seccomp to GA #1148

tallclair · 2019-07-17T23:57:53Z

This is a proposal to upgrade the seccomp annotation on pods & pod security policies to a field, and
mark the feature as GA. This proposal aims to do the bare minimum to clean up the feature, without
blocking future enhancements.

/sig-node
/sig-auth

/priority important-longterm

/assign @liggitt @dchen1107 @derekwaynecarr
/cc @jessfraz

keps/sig-node/20190717-seccomp-ga.md

dchen1107 · 2019-07-23T18:12:47Z

/lgtm but leaving it to @liggitt and @derekwaynecarr for the final approval.

nit: Per our offline discussion, can you make downgrade workflow more explicit here?

liggitt · 2019-07-27T19:40:53Z

keps/sig-node/20190717-seccomp-ga.md

+
+- Declare seccomp GA
+- Fully document and formally spec the feature support
+- Migrate the annotations to standard API fields


suggest using language like "add equivalent API fields" rather than "migrate" (which sounds like we will move data, which we know doesn't work well)

edit: if the migration only occurs in pod objects where the fields are immutable after creation, this can work... the description below should be explicit that no data movement is done in pod template objects embedded in other types

edit 2: you were already explicit about pod templates... nevermind

If the pod fields are immutable, is it possible to still set them on update as part of the storage flow? The criteria would be:

The seccomp annotations are set, and do not change as part of the update.

The seccomp fields are not set.

I guess the first criteria is problematic, since the seccomp fields could be changed in the update, and then a subsequent update could leave them be, so maybe that doesn't matter.

keps/sig-node/20190717-seccomp-ga.md

liggitt · 2019-07-27T19:43:42Z

keps/sig-node/20190717-seccomp-ga.md

+
+type SeccompOptions struct {
+    // The seccomp profile to run with.
+    SeccompProfile


was this intended to be embedded rather than something like Profile SeccompProfile? what other options do you anticipate?

Yes, I modeled it off the Volume type.

The only option that comes to mind is the fallback strategy. Currently, if the seccomp options can't be enforced the node silently ignores them. In contrast, when AppArmor can't be enforced the pod is held in a Blocked state.

From a security perspective, silently failing could leave a potential security hole. From a portability perspective, it's nice to be able to choose the best practices options with a best-effort approach. Hence it might make sense to give the user the choice.

what happens for nodes/runtimes that don't enable seccomp is worth documenting, at least

maybe just securityContext.seccompProfile instead of embedding/nesting (can prefix other seccomp fields with seccomp if they are needed in the future

liggitt · 2019-07-27T19:47:12Z

keps/sig-node/20190717-seccomp-ga.md

+}
+
+// Only one profile source may be set.
+// +union


cc @apelisse ... don't we want to add a discriminator here? what's the current approach to doing that?

Are discriminators required if the structure is immutable? I guess we'd still need it for pod templates...

Does the discriminator field need to be a 1:1 mapping with union fields? If not, the API could look like this:

type SeccompProfile struct { // +unionDiscriminator // +optional Type *SeccompProfileType // +optional LocalhostProfile *string } type SeccompProfileType string const ( SeccompProfileUnconfined SeccompProfileType = "Unconfined" SeccompProfileRuntimeDefault SeccompProfileType = "RuntimeDefault" SeccompProfileLocalhost SeccompProfileType = "Localhost" )

In other words, it seems redundant to have a the type indicate a boolean value that must be set to true. In those cases, the discriminator should be sufficient to indicate the option.

Does the discriminator field need to be a 1:1 mapping with union fields?

No, a union type without an associated member field is permitted

keps/sig-node/20190717-seccomp-ga.md

liggitt · 2019-07-27T19:49:13Z

keps/sig-node/20190717-seccomp-ga.md

+type SeccompProfileSet struct {
+    // Whether the unconfined profile is included in this set.
+    // +optional
+    Unconfined *bool


is false different from null?

Probably not. Maybe this should just be required? Is it permissible to have an optional non-pointer bool if the false & nil are equivalent? I suppose a reason to have nil is to allow for defaulting in mutating admission

keps/sig-node/20190717-seccomp-ga.md

liggitt · 2019-07-27T19:51:54Z

keps/sig-node/20190717-seccomp-ga.md

+    // Load a profile defined in static file on the node.
+    // The profile must be preconfigured on the node to work.
+    // +optional
+    LocalhostProfile *string


do the current annotations contain sufficient detail to know whether they should convert to a runtime or localhost profile?

Yes, currently the legal values for the annotations are:

unconfined # the real default docker/default # equivalent to runtime/default runtime/default localhost/<path> # where path is relative to the node's configured seccomp root

spec out how annotation -> field conversion would work in the docker/default case (and how we'd validate annotation and field matched in that case... probably a special case)

would we use API defaulting to set RuntimeProfile to "default" if type is set to "Runtime"? if so, would we clear RuntimeProfile on update if Type was changed to something other than "Runtime"?

tallclair · 2019-07-30T18:25:06Z

Note to reviewers: this PR is for 1.17 - please prioritize reviewing 1.16 PRs over this.

AkihiroSuda · 2019-07-30T18:42:18Z

keps/sig-node/20190717-seccomp-ga.md

+    // The allowed localhostProfiles. Values may end in '*' to include all
+    // localhostProfiles with a prefix.
+    // +optional
+    LocalhostProfiles []string


Can we use Config object for storing profiles across the nodes? Can be another KEP though.

I definitely agree this would be useful, but implementing it is a big project that I don't want to block GA on. See non-goals:

Providing mechanisms for loading profiles from outside the static seccomp node directory

AkihiroSuda · 2019-07-30T18:46:38Z

keps/sig-node/20190717-seccomp-ga.md

+All API skew is resolved in the API server. New Kubelets will only use the seccomp values specified
+in the fields, and ignore the annotations.
+
+#### Pod Creation


What is the expected behavior if the CRI runtime doesn't support seccomp? The pod should be created without seccomp (as in the current alpha annotation) or should result in an error?

I think we need to maintain the backwards compatible behavior of ignoring it (I'll add a line to be explicit about this). Going forward, I left space in the API for a FailureStrategy which could dictate how to handle that situation (e.g. some applications may depend on seccomp application to run securely, while others might accept a best-effort best-practices approach).

No application should depend on a policy; a policy can be applied as a non privileged operation by any code, so if it requires it then it can just apply it.

mayakacz · 2019-08-01T03:12:06Z

keps/sig-node/20190717-seccomp-ga.md

+harder to make some of the enhancements listed under [Non-Goals](#non-goals). Since the current
+behavior is unguarded, I think we already need to treat the behavior as GA (which is why it's been
+so hard to change the default profile), so I do not think these changes will actually increase the
+friction.


This is my biggest concern, but I think the arguments you've laid out are correct.
This is a necessary step to get to the good stuff :)

Thanks so much Tim!

keps/sig-node/20190717-seccomp-ga.md

mrunalp · 2019-08-06T15:43:41Z

This looks fine for promoting what we have right now. However, I think that we would want to follow up with owning/defining seccomp policies by k8s better for workload portability as well as handling of failure scenarios.

tallclair · 2020-01-17T02:29:05Z

Sorry it took so long to iterate on this. It is not currently a top priority for me, and may not make the v1.18 cutoff if it needs much more work.

There are 3 open issues:

how to handle warnings, I proposed an annotation in the updated KEP
how to handle runtime/default
how to handle localhost profiles

palnabarun · 2020-03-06T05:42:41Z

Sorry it took so long to iterate on this. It is not currently a top priority for me, and may not make the v1.18 cutoff if it needs much more work.

Removing the 1.18 milestone in that case.

palnabarun · 2020-03-06T05:42:56Z

/milestone clear

tallclair · 2020-04-07T21:41:16Z

@pjbgf - Any chance you'd be interested in picking up this proposal to get it across the finish line? I think the most complicated of the open questions is what to do with localhost profiles. There's a bit of a chicken-and-egg dilemma there:

Don't want to promote localhost profiles to GA and immediately deprecate
Don't want to deprecate localhost profiles before there's a replacement
Don't want to build the replacement until seccomp goes to GA

pjbgf · 2020-04-09T08:41:18Z

@tallclair I am indeed quite keen on pushing this forward. But will only have availability again from the end of the month. Will get back to you then.

pjbgf · 2020-04-30T14:31:22Z

Don't want to promote localhost profiles to GA and immediately deprecate

Don't want to deprecate localhost profiles before there's a replacement

Don't want to build the replacement until seccomp goes to GA

@tallclair I had a catch-up today (w/ @saschagrunert, @evrardjp) around this. We will present the current plan of action on the next sig-node meeting, which in summary is:

Keep this KEP as is - to decrease amount of changes required.
Drop the two KEPs: Built-in profiles, complain mode, enable seccomp by default and using ConfigMap to store profiles.
The end-to-end seccomp experience will be evolved out-of-tree with the seccomp-operator. Features developed there would potentially be pushed in-tree as and when it make sense.

tallclair · 2020-05-05T17:48:38Z

/retest

tallclair · 2020-05-05T17:59:12Z

@derekwaynecarr @liggitt - Can we merge this as-is (provisional, with unresolved sections), so that I can handoff to @pjbgf to resolve the remaining issues?

liggitt · 2020-05-05T18:31:27Z

if the open discussions are captured in unresolved sections with links to the thread in this PR and/or the people involved (e.g. https://github.com/kubernetes/enhancements/blame/master/keps/NNNN-kep-template/README.md#L38-L40) that seems ok

tallclair · 2020-05-05T20:18:30Z

Updated the unresolved section format to match the official standard. I don't think there is more context to add on these.

derekwaynecarr · 2020-05-06T20:58:58Z

I am fine merging this as-is, and iterating on the unresolved sections per discussion in sig-node on 5/4.

/approve
/lgtm

k8s-ci-robot · 2020-05-06T20:59:15Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: derekwaynecarr, tallclair

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~keps/sig-node/OWNERS~~ [derekwaynecarr]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

* KEP for promoting seccomp to GA * Add section about PSP application * Address feedback * Update seccomp GA API * Fill in approvers/reviewers * Mark KEP as implementable * Iterate on KEP review * revert to provisional status * Regenerate TOC * Update unresolved section format

KEP for promoting seccomp to GA

0e21d25

k8s-ci-robot assigned dchen1107, derekwaynecarr and liggitt Jul 17, 2019

k8s-ci-robot added the priority/important-longterm Important over the long term, but may not be staffed and/or may need multiple releases to complete. label Jul 17, 2019

k8s-ci-robot requested a review from jessfraz July 17, 2019 23:57

Add section about PSP application

82f393e

PatrickLang reviewed Jul 23, 2019

View reviewed changes

keps/sig-node/20190717-seccomp-ga.md Outdated Show resolved Hide resolved

PatrickLang reviewed Jul 23, 2019

View reviewed changes

keps/sig-node/20190717-seccomp-ga.md Show resolved Hide resolved

k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jul 23, 2019

liggitt reviewed Jul 27, 2019

View reviewed changes

keps/sig-node/20190717-seccomp-ga.md Outdated Show resolved Hide resolved

liggitt reviewed Jul 27, 2019

View reviewed changes

liggitt reviewed Jul 30, 2019

View reviewed changes

tallclair added this to the v1.17 milestone Jul 30, 2019

Address feedback

8d37151

tallclair force-pushed the seccomp-ga branch from fbab931 to 8d37151 Compare July 30, 2019 18:34

AkihiroSuda reviewed Jul 30, 2019

View reviewed changes

mayakacz reviewed Aug 1, 2019

View reviewed changes

mrunalp reviewed Aug 6, 2019

View reviewed changes

keps/sig-node/20190717-seccomp-ga.md Show resolved Hide resolved

tallclair mentioned this pull request Aug 12, 2019

TOB-K8S-002: Seccomp is disabled by default kubernetes/kubernetes#81115

Closed

liggitt moved this from In progress to Assigned in API Reviews Feb 17, 2020

liggitt moved this from Assigned to Backlog in API Reviews Feb 27, 2020

k8s-ci-robot removed this from the v1.18 milestone Mar 6, 2020

JAORMX mentioned this pull request Apr 10, 2020

seccomp: change empty profile behavior cri-o/cri-o#3342

Closed

revert to provisional status

da1bbc6

k8s-ci-robot added sig/architecture Categorizes an issue or PR as relevant to SIG Architecture. and removed approved Indicates a PR has been approved by an approver from all required OWNERS files. labels May 5, 2020

Regenerate TOC

81059c3

tallclair added the tide/merge-method-squash Denotes a PR that should be squashed by tide when it merges. label May 5, 2020

Update unresolved section format

b32550c

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label May 6, 2020

k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label May 6, 2020

k8s-ci-robot merged commit 2c681f9 into kubernetes:master May 6, 2020

k8s-ci-robot added this to the v1.19 milestone May 6, 2020

pjbgf mentioned this pull request May 7, 2020

Seccomp GA #1747

Merged

saschagrunert mentioned this pull request May 12, 2020

REQUEST: New repository kubernetes-sigs/seccomp-operator kubernetes/org#1873

Closed

KEP for promoting seccomp to GA #1148

KEP for promoting seccomp to GA #1148

Conversation

tallclair commented Jul 17, 2019

dchen1107 commented Jul 23, 2019

liggitt Jul 27, 2019 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tallclair Jul 30, 2019 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tallclair Jul 30, 2019 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tallclair commented Jul 30, 2019

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mrunalp commented Aug 6, 2019

tallclair commented Jan 17, 2020

palnabarun commented Mar 6, 2020

palnabarun commented Mar 6, 2020

tallclair commented Apr 7, 2020

pjbgf commented Apr 9, 2020

pjbgf commented Apr 30, 2020

tallclair commented May 5, 2020

tallclair commented May 5, 2020

liggitt commented May 5, 2020

tallclair commented May 5, 2020

derekwaynecarr commented May 6, 2020

k8s-ci-robot commented May 6, 2020

liggitt Jul 27, 2019 •

edited

tallclair Jul 30, 2019 •

edited

tallclair Jul 30, 2019 •

edited