Add support for seccomp #20870

vishh · 2016-02-09T00:41:22Z

Starting from v1.10 docker daemon will run with a restrictive seccomp profile turned on by default.

Users will have to include a custom seccomp profile in addition to capabilities.

Until we figure out a sane way to add support for seccomp, I propose disabling (unconfined) seccomp profiles by default.
This will allow users to upgrade to docker v1.10 seamlessly.

cc @kubernetes/goog-node @bgrant0607

The text was updated successfully, but these errors were encountered:

vishh · 2016-02-09T00:54:38Z

cc @erictune @kubernetes/rh-platform-management

bgrant0607 · 2016-02-09T06:31:55Z

cc @pmorie @pweil-

ncdc · 2016-02-09T14:50:38Z

cc @kubernetes/rh-cluster-infra

rhatdan · 2016-02-24T15:55:48Z

Is the problem here that you are worried about seccomp breaking certain actions? I hate to turn off a key security feature of docker for the masses of users of k8s.

vishh · 2016-02-24T18:57:26Z

@rhatdan: moby/moby#20082

erictune · 2016-03-28T22:46:43Z

Do these make sense as requirements for kubernetes support for seccomp filters:

we, the project, pre-defines names for certain "canned" seccomp filters with sensible defaults
- probably the same as Docker's 1.10 default, and unrestricted.
cluster admins can define their own custom filters too.
it should be possible to define a seccomp filter and a list of allowed capabilities in the same policy object, so that both are set on a pod/container at once, to avoid user confusion about the need to set both.
users can select the policy that a pod/container uses by giving the short name of a policy object, rather than inlining the entire seccomp filter text into the pod spec.
admins can change the default policy for pods that don't say what to do as regards seccomp.
the default default, on a new/upgraded cluster does not change, without some affirmative user action.
if a user writes a pod spec that asks for a level of security that is not supported by the node (e.g. they ask for a restrictive seccomp profile, and the node is running docker 1.9) then pod creation fails, rather then the user silently getting no security.

erictune · 2016-03-28T22:49:05Z

@kubernetes/kube-iam

erictune · 2016-03-28T22:49:10Z

@pmorie

eparis · 2016-03-29T16:46:00Z

@erictune my thoughts are here (although disorganized as usual) https://trello.com/c/1pn9IN3m/320-better-seccomp-support But I think you nailed it.

erictune · 2016-03-29T18:04:55Z

I didn't realize that only increasingly more restrictive seccomp profiles can be loaded, if I am reading the comments on https://trello.com/c/1pn9IN3m/320-better-seccomp-support correctly. That makes it much less flexible that I had imagined.

eparis · 2016-03-29T18:21:08Z

docker only lets us load a single profile, so that part isn't relevant today (I think RH is doing OCI work to make it a list of profiles but who knows if that ever gets plumbed up into docker)

If you load 2 profiles they will be executed in order and all of the profiles must 'approve' the syscall. If profile 1 says
deny mount; allow all
and profile 2 says
deny module_init; allow mount; allow all

Then BOTH mount and module_init will get denied.

eparis · 2016-03-29T18:22:52Z

So the profiles themselves do not have to be of increasing restrictivity (restrictivity is a word, right?) but the combination will always be at least as restrictive as either parent.

jessfraz · 2016-04-05T02:16:05Z

What if instead of turning it off completely kubernetes loaded a custom default, w mount and whatever else you all need

jessfraz · 2016-04-05T02:16:54Z

and if the user loads one it would override your default, just like how it works in docker.

jessfraz · 2016-04-05T03:17:44Z

I'd be happy to contribute the patch depending on what syscalls you all need added

vishh · 2016-04-05T03:55:40Z

If users have a mechanism to override the default profile, I guess docker's default profile should itself work.

jessfraz · 2016-04-05T05:01:27Z

So is the problem exposing the custom profile in the kubernetes API in a
time constraint? Just want to understand so I can help

On Monday, April 4, 2016, Vish Kannan notifications@github.com wrote:

If users have a mechanism to override the default profile, I guess
docker's default profile should itself work.

—
You are receiving this because you commented.
Reply to this email directly or view it on GitHub
#20870 (comment)

Jessie Frazelle
4096R / D4C4 DD60 0D66 F65A 8EFC 511E 18F3 685C 0022 BFF3
pgp.mit.edu http://pgp.mit.edu/pks/lookup?op=get&search=0x18F3685C0022BFF3

justincormack · 2016-04-05T14:45:33Z

@erictune are you explicitly referring to Linux capabilities there? One option would be to tie the default Docker seccomp filter to the allowed capabilities so that adding say CAP_SYS_ADMIN allows some more syscalls, while --cap-drop-all removes a few more potentially. In particular adding CAP_SYS_ADMIN would allow unshare and mount, allowing systemd to run.

Getting admins to write seccomp filters is somewhat unpleasant work, the aim with the Docker seccomp support has been to avoid it as much as possible.

thockin · 2016-04-05T14:49:48Z

Is this manifesting as a real problem right now? I am wild-guessing that it shows up as containers that fail (syscall blocked) with no remediation (since it didn't make it into our API) ?

It seems like an overly large thing to cherry pick, and yet not having it seems bad. Can we maybe have an annotation that we pass through to docker, and add proper profiles and fields in 1.3?

rhatdan · 2016-04-05T15:04:06Z

I tried to add this pull request to docker, but it was closed because docker wants the tighter security

projectatomic/docker#80

I believe that we should attempt to separate different security mechanisms from each other. For example. SELinux can block Capabilities, but I allow all Caps from an SELinux point of view and rely on the Capabilities Subsystem to manage caps, Seccomp did not follow this standard, at least to a few syscalls that are controlled by SYS_ADMIN. We saw these when trying to run systemd in a container.
We are working on patches to allow the addition of syscalls to the white list as well as changing the default way that seccomp is enforced, so that users would be allowed to specify KILL versus ERRNO. KILL Is the only seccomp action that outputs blocked syscalls to /var/log/audit/audit.log, which could give the user a clue on why he is getting permission denied.

Allowing tools to specify Whitelist or blacklist syscalls would help. It would also help if the syscall table that docker is using was provided externally to the tool, not compiled in. Then the admin or tools could just modify the default json and pass it to docker daemon on on the docker run command to specify the syscalls to be used by the container(s).

jessfraz · 2016-04-05T15:13:25Z

If the default profile is the problem it could easily be fixed by my
suggestion of kubes passing its own w whatever other syscalls they need
added to the whitelist, fwict the problem is more time adding it to the API
unless I am mistaken

On Tuesday, April 5, 2016, Daniel J Walsh notifications@github.com wrote:

I tried to add this pull request to docker, but it was closed because
docker wants the tighter security

projectatomic/docker#80 projectatomic/docker#80

I believe that we should attempt to separate different security mechanisms
from each other. For example. SELinux can block Capabilities, but I allow
all Caps from an SELinux point of view and rely on the Capabilities
Subsystem to manage caps, Seccomp did not follow this standard, at least to
a few syscalls that are controlled by SYS_ADMIN. We saw these when trying
to run systemd in a container.

We are working on patches to allow the addition of syscalls to the white
list as well as changing the default way that seccomp is enforced, so that
users would be allowed to specify KILL versus ERRNO. KILL Is the only
seccomp action that outputs blocked syscalls to /var/log/audit/audit.log,
which could give the user a clue on why he is getting permission denied.

Allowing tools to specify Whitelist or blacklist syscalls would help. It
would also help if the syscall table that docker is using was provided
externally to the tool, not compiled in. Then the admin or tools could just
modify the default json and pass it to docker daemon on on the docker run
command to specify the syscalls to be used by the container(s).

—
You are receiving this because you commented.
Reply to this email directly or view it on GitHub
#20870 (comment)

Jessie Frazelle
4096R / D4C4 DD60 0D66 F65A 8EFC 511E 18F3 685C 0022 BFF3
pgp.mit.edu http://pgp.mit.edu/pks/lookup?op=get&search=0x18F3685C0022BFF3

eparis · 2016-04-05T15:44:23Z

In 1.2 we explicitly disable the seccomp profiles (aka we disable the docker default). So the 'manifestation' today is that kubernetes users cannot make use of this security feature provided by the underlying container backend.

The default docker seccomp profile would be great 99% of the time (the exact same % of the time that it would be great for docker). But until we plumb through some way to control the seccomp profile there wasn't a better alternative. This issue requires someone to plumb through the kube API a way to represent seccomp profiles and how to disable them.

I think there are a lot of ideas above how to plumb seccomp through the API, some form of which needs to be done before 1.3...

bgrant0607 · 2016-04-06T03:54:02Z

We can't break existing applications running in existing clusters.

Users also expect to be able to delete and re-create their clusters across releases with identical behavior, and we have the official policy that we don't change default behavior of APIs without introducing a new API version.

erictune · 2016-04-06T04:06:32Z

@jfrazelle we don't know what calls our users might be doing, so we don't want to change the default and break them.

I see that before 1.10 you tested all the official docker images and didn't see any issues. Since then, have there been any user reports of images not working after upgrading to 1.10 due to seccomp?

thockin · 2016-04-20T16:37:50Z

You could argue it was an oversight, but it's a stretch. I was hoping
someone would convince me, but ...

On Wed, Apr 20, 2016 at 8:15 AM, Jess Frazelle notifications@github.com
wrote:

I think it would be a good idea to not include it in say a .1 or patch
release because it is kinda changing behavior and is better for a .0, if
feel bad breaking semver

On Wednesday, April 20, 2016, Andy Goldstein notifications@github.com
wrote:

I think we're probably going to have to defer a real solution until 1.3.
I
know @pmorie https://github.com/pmorie is planning on writing up a
proposal.

—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
<
#20870 (comment)

Jessie Frazelle
4096R / D4C4 DD60 0D66 F65A 8EFC 511E 18F3 685C 0022 BFF3
pgp.mit.edu <
http://pgp.mit.edu/pks/lookup?op=get&search=0x18F3685C0022BFF3>

—
You are receiving this because you commented.
Reply to this email directly or view it on GitHub
#20870 (comment)

jessfraz · 2016-04-21T17:12:28Z

don't worry I'll be the pusher for 1.3 😇

thockin · 2016-04-21T17:14:26Z

I'mma hold you to that.

On Thu, Apr 21, 2016 at 10:13 AM, Jess Frazelle notifications@github.com
wrote:

don't worry I'll be the pusher for 1.3 [image: :innocent:]

—
You are receiving this because you commented.
Reply to this email directly or view it on GitHub
#20870 (comment)

pmorie · 2016-04-26T20:33:31Z

@jfrazelle

don't worry I'll be the pusher for 1.3

👍

Automatic merge from submit-queue Seccomp Proposal WIP proposal to address #20870 @kubernetes/kube-api @kubernetes/sig-node  --- This change is [<img src="http://reviewable.k8s.io/review_button.svg" height="35" align="absmiddle" alt="Reviewable"/>](http://reviewable.k8s.io/reviews/kubernetes/kubernetes/24602)

dims · 2016-06-08T01:02:05Z

@thockin @vishh We can close this now (Since PR has merged aa8c72a)

ncdc · 2016-06-08T01:08:46Z

Not yet - we need #26710 to merge first

dchen1107 · 2016-06-08T18:13:05Z

Moved this to v1.3 for the proper feature tracking.

Automatic merge from submit-queue Seccomp Proposal WIP proposal to address kubernetes#20870 @kubernetes/kube-api @kubernetes/sig-node  --- This change is [<img src="http://reviewable.k8s.io/review_button.svg" height="35" align="absmiddle" alt="Reviewable"/>](http://reviewable.k8s.io/reviews/kubernetes/kubernetes/24602)

bgrant0607 · 2017-05-30T16:42:27Z

Feature issue: kubernetes/enhancements#135

tallclair · 2017-09-23T00:35:18Z

Closing in favor of kubernetes/enhancements#135

vishh added sig/node Categorizes an issue or PR as relevant to SIG Node. team/api priority/backlog Higher priority than priority/awaiting-more-evidence. labels Feb 9, 2016

bgrant0607 added this to the next-candidate milestone Feb 9, 2016

vishh mentioned this issue Feb 23, 2016

Disable seccomp policy by default on docker versions >= v1.10 #21790

Merged

erictune added the area/security label Mar 28, 2016

pmorie mentioned this issue Apr 21, 2016

Seccomp Proposal #24602

Merged

pmorie mentioned this issue Apr 21, 2016

Seccomp umbrella issue #24611

Closed

2 tasks

dchen1107 modified the milestones: v1.3, next-candidate Jun 8, 2016

goltermann modified the milestones: v1.3, next-candidate Jun 13, 2016

jessfraz mentioned this issue Jul 7, 2016

Add PSP support for seccomp profiles #28300

Merged

lblackstone mentioned this issue Oct 12, 2016

Default seccomp profile cannot be configured without hardcoding it in Kubelet #34649

Closed

l0rd mentioned this issue May 15, 2017

Che not communicating with GDB properly, failing to insert breakpoints eclipse/che#4284

Closed

jessfraz mentioned this issue May 30, 2017

seccomp: add default profile kubernetes/community#660

Closed

wongma7 mentioned this issue Jun 19, 2017

seccomp when running NFS-provisioner with the k8s Deployment kubernetes-retired/external-storage#176

Closed

tallclair closed this as completed Sep 23, 2017

hkolvenbach mentioned this issue Oct 25, 2017

Add support for Docker SecurityOpt (allowing GDB Debugging on unprivileged docker containers) eclipse/che#6856

Merged

thaJeztah mentioned this issue Jan 3, 2018

Add kubernetes support to docker/cli docker/cli#721

Merged

cji mentioned this issue Aug 8, 2019

TOB-K8S-002: Seccomp is disabled by default #81115

Closed

This was referenced Sep 25, 2019

[WIP] Add KEP to create seccomp built-in profiles and add complain mode kubernetes/enhancements#1257

Closed

Add ConfigMap support for seccomp custom profiles kubernetes/enhancements#1269

Closed

Add support for seccomp #20870

Add support for seccomp #20870

vishh commented Feb 9, 2016

vishh commented Feb 9, 2016

bgrant0607 commented Feb 9, 2016

ncdc commented Feb 9, 2016

rhatdan commented Feb 24, 2016

vishh commented Feb 24, 2016

erictune commented Mar 28, 2016

erictune commented Mar 28, 2016

erictune commented Mar 28, 2016

eparis commented Mar 29, 2016

erictune commented Mar 29, 2016

eparis commented Mar 29, 2016

eparis commented Mar 29, 2016

jessfraz commented Apr 5, 2016

jessfraz commented Apr 5, 2016

jessfraz commented Apr 5, 2016

vishh commented Apr 5, 2016

jessfraz commented Apr 5, 2016

justincormack commented Apr 5, 2016

thockin commented Apr 5, 2016

rhatdan commented Apr 5, 2016

jessfraz commented Apr 5, 2016

eparis commented Apr 5, 2016

bgrant0607 commented Apr 6, 2016

erictune commented Apr 6, 2016

thockin commented Apr 20, 2016

jessfraz commented Apr 21, 2016

thockin commented Apr 21, 2016

pmorie commented Apr 26, 2016

dims commented Jun 8, 2016

ncdc commented Jun 8, 2016

dchen1107 commented Jun 8, 2016

bgrant0607 commented May 30, 2017

tallclair commented Sep 23, 2017

Add support for seccomp #20870

Add support for seccomp #20870

Comments

vishh commented Feb 9, 2016

vishh commented Feb 9, 2016

bgrant0607 commented Feb 9, 2016

ncdc commented Feb 9, 2016

rhatdan commented Feb 24, 2016

vishh commented Feb 24, 2016

erictune commented Mar 28, 2016

erictune commented Mar 28, 2016

erictune commented Mar 28, 2016

eparis commented Mar 29, 2016

erictune commented Mar 29, 2016

eparis commented Mar 29, 2016

eparis commented Mar 29, 2016

jessfraz commented Apr 5, 2016

jessfraz commented Apr 5, 2016

jessfraz commented Apr 5, 2016

vishh commented Apr 5, 2016

jessfraz commented Apr 5, 2016

justincormack commented Apr 5, 2016

thockin commented Apr 5, 2016

rhatdan commented Apr 5, 2016

jessfraz commented Apr 5, 2016

eparis commented Apr 5, 2016

bgrant0607 commented Apr 6, 2016

erictune commented Apr 6, 2016

thockin commented Apr 20, 2016

jessfraz commented Apr 21, 2016

thockin commented Apr 21, 2016

pmorie commented Apr 26, 2016

dims commented Jun 8, 2016

ncdc commented Jun 8, 2016

dchen1107 commented Jun 8, 2016

bgrant0607 commented May 30, 2017

tallclair commented Sep 23, 2017