allow setting container capabilities in challenge.yaml

[sroettger]

We were always overwriting the capabilities of the challenge container so it was not possible to add custom caps.
Change this by checking for existence before overwriting.

Note that I'm not merging the Capabilities.Add list, instead the challenge author will have to add SYS_ADMIN manually if they want to change it.

[sirdarckcat]

This probably needs documentation (that cap_sys_admin is required by the template nsjail and not enforced by the operator) and some testing for troubleshooting. Specially the security consequences of it.

Also, this is more of a feature, so maybe better to push on v1.1? Unless we consider the old behavior a bug.

[sroettger]

It's really just a bug fix, we just ignored the value in the template before.
Also writing documentation for this seems overkill since I don't expect this feature to be used much and the error when you don't have sys_admin should be pretty straightforward to figure out. (mount will fail with EPERM)

[sirdarckcat]

It does look easy to troubleshoot https://www.google.com/search?q=mount+eperm - but it is surprising that adding a capability actually removes a capability.

Given that the podTemplate behavior has other cases like these (setting cpu limits is enforced to the config and volume mounts appends). Before we could say that in order to ensure compatibility we always enforce configurations that work for limits, mounts and capabilities, but now we can't say that (some times we overwrite them, sometimes we let them overwrite the configuration).

It would be nice to add a comment here:

kctf/kctf-operator/pkg/apis/kctf/v1/challenge_types.go

Lines 99 to 103 in 2011ea4

	// PodTemplate is used to set the template for the deployment's pod,
	// so that an author can add volumeMounts and other extra features
	PodTemplate *corev1.PodTemplate `json:"podTemplate"`

So at least it makes sense why we are inconsistent

[sroettger]

setting cpu limits is enforced to the config
but that's just another bug. Why would we force CPU limits, we have no idea about the requirements of the challenge

[sirdarckcat]

Hmm yea, I don't remember why we set them to 0.9/0.45 - looks like it was meant to be for the template task, but we ended up putting it on the base, which then made it all the way to the operator.

3ede18e
https://github.com/google/kctf/blob/alpha/base/k8s/deployment/containers.yaml#L35

Appending looks good though, maybe we should just delete the lines that set the limits, but that can be done separately.

[sroettger]

at first I didn't append since I thought it would break if you append the same capability twice. But just tried it out and seems to work fine. So this should be what users would expect it to do.