PSP/SCC for Beats

[david-kow]

Some of the Beats need additional permissions to succesfully run. Eg.: Filebeat needs to access hostpath for other pods logs and Metricbeat needs to access /proc for obtaining process information.

In clusters with security enabled, Beat pods need to have permission to use a PSP/SCC that allows that.

Given the above, we can either:

1. do nothing in the code, document the need for correct PSP to be existing and specifying it for Beats pod templates
   • simple for us
   • start up experience not as nice as it could be
2. create the necessary PSP/SCC during ECK installation (behind a flag in the eck manifest generation) and use it by default
   • might be surprising for the users
   • requires user to specify whether openshift is a target cluster or not during manifest generation
   • implementation is fairly simple
   • start up experience is nice
3. have the controller create and manage the necessary PSP/SCC.
   • will only create the additional resources when necessary
   • implementation seems complicated and requires a lot of new permissions for the controller

I think I'd be opting for 2, but wanted to discuss broadly. Thougths?

[pebrc]

2. sounds good to me too.

[barkbay]

Regarding the second option, creating the SCC is interesting but the user still have to add the right ServiceAccounts to it.

For example with something like:
> oc adm policy add-scc-to-user elastic-beats -z elastic-operator-beat-autodiscover-filebeat-sample -n elastic-monitoring
(where elastic-beats is the name of the SCC)

[david-kow]

I need to take back my original suggestion.

• we don't provide PSPs today for other resources (correct PSP and SecurityContext need to be provided by the user)
• the required PSP will vary a lot depending on Beat type and its config
• as we can't precreate SCC, due to what @barkbay pointed out. Users would have a different experience on OpenShift or we would need to heavily special case it in the operator code

Given that, I'd propose the following:

• we expect users to provide correct PSP/SCC, but we offer samples (we will need them anyway for testing)
• the operator creates a ServiceAccount with a well-known name template that users can target with SCC or bind to using separate ClusterRoleBinding

This way we are consistent and operator logic stays fairly simple while users have available example PSPs/SCCs that we know work.

[sebgl]

@david-kow your last suggestion makes sense to me. I think it's better for us to start with less magic and maybe add some magic later than the other way around.

IIUC our Beats Quickstart docs would look like the following:

1. Deploy this Filebeat/Metricbeat yaml file
2. If your cluster is running with the PSP feature enabled, or if you're running with Openshift see [these instructions]

PSP/SCC instructions:

• if you're running on Openshift, apply this SCC in which you must include the serviceAccount name we create for your beat resource
• if you're running Kubernetes with PSP enabled, apply this PSP and this ClusterRoleBinding in which you must include the serviceAccount name we create for your beat resource

[david-kow]

This is implemented with #3041. Docs issue will come separately.