Allow configuration of ContainerSecurityContext for XCom Sidecar (KubernetesPodOperator)

[DjVinnii]

Description

Allow configuring the Container Security Context for the XCom sidecar. It would be great if this can be set with a default the Airflow Deployment Managers and if needed overridden by the Dag Authors.

Use case/motivation

It might be impossible to use the XCom sidecar when using the KubernetesPodOperator in a strictly regulated environment with for example OPA policies. It is for example not possible to configure the Container Security Context for the ingested sidecar when certain Container Security Context settings are expected such as the error bellow mentions:

HTTP response body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"admission webhook \"validation.gatekeeper.sh\" denied the request: [deny-privilege-escalation] Privilege escalation in container is not allowed: airflow-xcom-sidecar\n[ro-rootfs-constraint] Read-only root filesystem is required: airflow-xcom-sidecar\n[k8sseccomp] Seccomp profile 'not configured' is not allowed for container 'airflow-xcom-sidecar'. Found at: no explicit profile found. Allowed profiles: {\"RuntimeDefault\", \"docker/default\", \"runtime/default\"}","reason":"Forbidden","code":403}

Related issues

No response

Are you willing to submit a PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[insomnes]

Regardless of the triage, you may find the way to configure described in #10605 useful to work around your problem:

#  You can redefine class vars of this class in your airflow_local_settings.py
class PodDefaults:
    """Static defaults for Pods."""

    XCOM_MOUNT_PATH = "/airflow/xcom"
    SIDECAR_CONTAINER_NAME = "airflow-xcom-sidecar"
    XCOM_CMD = 'trap "exit 0" INT; while true; do sleep 1; done;'
    VOLUME_MOUNT = k8s.V1VolumeMount(name="xcom", mount_path=XCOM_MOUNT_PATH)
    VOLUME = k8s.V1Volume(name="xcom", empty_dir=k8s.V1EmptyDirVolumeSource())
    # Here, the full spec is configurable for the sidecar container used
    SIDECAR_CONTAINER = k8s.V1Container(
        name=SIDECAR_CONTAINER_NAME,
        command=["sh", "-c", XCOM_CMD],
        image="alpine",
        volume_mounts=[VOLUME_MOUNT],
        resources=k8s.V1ResourceRequirements(
            requests={
                "cpu": "1m",
                "memory": "10Mi",
            },
        ),
    )

[DjVinnii]

Regardless of the triage, you may find the way to configure described in #10605 useful to work around your problem:

I tried the proposed solution/workaround by adding the code below. Unfortunately, the Security Context is not injected.

SIDECAR_CONTAINER = k8s.V1Container(
...
  security_context=k8s.V1SecurityContext(
    allow_privilege_escalation=False,
    read_only_root_filesystem=True,
    seccomp_profile=k8s.V1SeccompProfile(
      type="RuntimeDefault"
    ),
 ),
)

The complete airflow_local_settings.py looks like this:

from kubernetes.client import models as k8s

class PodDefaults:
  """Static defaults for Pods."""

  XCOM_MOUNT_PATH = "/airflow/xcom"
  SIDECAR_CONTAINER_NAME = "airflow-xcom-sidecar"
  XCOM_CMD = 'trap "exit 0" INT; while true; do sleep 1; done;'
  VOLUME_MOUNT = k8s.V1VolumeMount(name="xcom", mount_path=XCOM_MOUNT_PATH)
  VOLUME = k8s.V1Volume(name="xcom", empty_dir=k8s.V1EmptyDirVolumeSource())
  SIDECAR_CONTAINER = k8s.V1Container(
    name=SIDECAR_CONTAINER_NAME,
    command=["sh", "-c", XCOM_CMD],
    image="alpine",
    volume_mounts=[VOLUME_MOUNT],
    resources=k8s.V1ResourceRequirements(
      requests={
          "cpu": "1m",
          "memory": "10Mi",
      },
    ),
    security_context = k8s.V1SecurityContext(
      allow_privilege_escalation=False,
      read_only_root_filesystem=True,
      seccomp_profile=k8s.V1SeccompProfile(
        type="RuntimeDefault"
      ),
     ),
  )

[insomnes]

I am sorry for not providing the proper example in my first comment. As mentioned in the issue redifinition requires you to change the class variables of the existing class via imports in your airflow_local_settings.py:

from airflow.providers.cncf.kubernetes.utils.xcom_sidecar import PodDefaults

PodDefaults.SIDECAR_CONTAINER.image = "docker.very.private.repo.net/alpine:3.10"

So, your security context re-configuration should look like this, I suppose:

from airflow.providers.cncf.kubernetes.utils.xcom_sidecar import PodDefaults
from kubernetes.client import models as k8s

PodDefaults.SIDECAR_CONTAINER.security_context=k8s.V1SecurityContext(
    allow_privilege_escalation=False,
    read_only_root_filesystem=True,
    seccomp_profile=k8s.V1SeccompProfile(
        type="RuntimeDefault"
    ),
) 

[DjVinnii]

Thanks @insomnes! This indeed works like a charm. Is this something we could add in the provider docs? I'm willing to add this if needed.

[insomnes]

I don't know what the proper solution to this problem is.

Even with the current intended resource and image configuration approach via the k8s connection (or hack I mentioned), it is not clear to me. I think that the default XCOM sidecar container spec should be fully configurable, and we should have an easier way to reconfigure it for specific KubernetesPodOperator too. But I can't imagine how common / uncommon this requirement to tune sidecar is.

The current approach of digging through issues and finding this "arcane solution" is not sustainable for me either. Maybe you can try to make a PR with a paragraph in docs and see what committers would say on this, or wait for the issue triage.