Error: container has runAsNonRoot and image will run as root

[msschl]

Deploying jaeger results in the following error:

Error: container has runAsNonRoot and image will run as root

Deploy script:

helm upgrade jaeger vendor/jaeger-helm-charts/charts/jaeger \
    -n monitoring \
    -f vendor/jaeger-helm-charts/charts/jaeger/values.yaml \
    -f src/monitoring/jaeger/values.yaml \
    -i $@

src/monitoring/jaeger/values.yaml:

# Default values for jaeger are specified in
# vendor/jaeger-helm-charts/charts/jaeger/values.yaml

provisionDataStore:
  cassandra: false
  elasticsearch: false
  kafka: false

storage:
  type: elasticsearch
  elasticsearch:
    scheme: http
    # NOTE(msschl): cluster internal only. Not reachable from outside...
    host: elasticsearch-master.elasticsearch.svc.k8s-cluster.company.xyz
    port: 9200
    usePassword: false

query:
  ingress:
    enabled: true
    hosts:
      - tracing.kube-dev.company.xyz
    annotations:
      kubernetes.io/ingress.class: "nginx-ingress"
    tls:
      - secretName: "public/company-tls"
        hosts:
          - tracing.kube-dev.company.xyz

# QUESTION(msschl): Enable?
esIndexCleaner:
  enabled: false

We do have a default psp policy in place and its bound to the deployed jaeger pods:

apiVersion: policy/v1beta1
kind: PodSecurityPolicy
metadata:
  name: default-restricted
  annotations:
    seccomp.security.alpha.kubernetes.io/allowedProfileNames: 'runtime/default'
    apparmor.security.beta.kubernetes.io/allowedProfileNames: 'runtime/default'
    seccomp.security.alpha.kubernetes.io/defaultProfileName:  'runtime/default'
    apparmor.security.beta.kubernetes.io/defaultProfileName:  'runtime/default'
spec:
  allowedCapabilities: []
  privileged: false
  allowPrivilegeEscalation: false
  requiredDropCapabilities:
    - ALL
  volumes:
    - 'configMap'
    - 'secret'
    - 'persistentVolumeClaim'
  hostNetwork: false
  hostIPC: false
  hostPID: false
  runAsUser:
    rule: 'MustRunAsNonRoot'
  seLinux:
    rule: 'RunAsAny'
  supplementalGroups:
    rule: 'MustRunAs'
    ranges:
      - min: 1
        max: 65535
  fsGroup:
    rule: 'MustRunAs'
    ranges:
      - min: 1
        max: 65535
  readOnlyRootFilesystem: true

Full description from agent pod:

$ kubectl -n monitoring describe po jaeger-agent-ndzwz
Name:         jaeger-agent-ndzwz
Namespace:    monitoring
Priority:     0
Node:         lxkubenode02/172.18.2.93
Start Time:   Fri, 15 May 2020 17:36:33 +0200
Labels:       app.kubernetes.io/component=agent
              app.kubernetes.io/instance=jaeger
              app.kubernetes.io/name=jaeger
              controller-revision-hash=6ff9dcbc57
              pod-template-generation=1
Annotations:  cni.projectcalico.org/podIP: 10.244.206.69/32
              cni.projectcalico.org/podIPs: 10.244.206.69/32
              container.apparmor.security.beta.kubernetes.io/jaeger-agent: runtime/default
              kubernetes.io/psp: default-restricted
              seccomp.security.alpha.kubernetes.io/pod: runtime/default
Status:       Pending
IP:           10.244.206.69
IPs:
  IP:           10.244.206.69
Controlled By:  DaemonSet/jaeger-agent
Containers:
  jaeger-agent:
    Container ID:
    Image:          jaegertracing/jaeger-agent:1.17.1
    Image ID:
    Ports:          5775/UDP, 6831/UDP, 6832/UDP, 5778/TCP, 14271/TCP
    Host Ports:     0/UDP, 0/UDP, 0/UDP, 0/TCP, 0/TCP
    State:          Waiting
      Reason:       CreateContainerConfigError
    Ready:          False
    Restart Count:  0
    Liveness:       http-get http://:admin/ delay=0s timeout=1s period=10s #success=1 #failure=3
    Readiness:      http-get http://:admin/ delay=0s timeout=1s period=10s #success=1 #failure=3
    Environment:
      REPORTER_GRPC_HOST_PORT:  jaeger-collector:14250
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from jaeger-agent-token-s2hzt (ro)
Conditions:
  Type              Status
  Initialized       True
  Ready             False
  ContainersReady   False
  PodScheduled      True
Volumes:
  jaeger-agent-token-s2hzt:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  jaeger-agent-token-s2hzt
    Optional:    false
QoS Class:       BestEffort
Node-Selectors:  <none>
Tolerations:     node.kubernetes.io/disk-pressure:NoSchedule
                 node.kubernetes.io/memory-pressure:NoSchedule
                 node.kubernetes.io/not-ready:NoExecute
                 node.kubernetes.io/pid-pressure:NoSchedule
                 node.kubernetes.io/unreachable:NoExecute
                 node.kubernetes.io/unschedulable:NoSchedule
Events:
  Type     Reason     Age               From                   Message
  ----     ------     ----              ----                   -------
  Normal   Scheduled  <unknown>         default-scheduler      Successfully assigned monitoring/jaeger-agent-ndzwz to lxkubenode02
  Normal   Pulling    6s (x4 over 25s)  kubelet, lxkubenode02  Pulling image "jaegertracing/jaeger-agent:1.17.1"
  Normal   Pulled     5s (x4 over 24s)  kubelet, lxkubenode02  Successfully pulled image "jaegertracing/jaeger-agent:1.17.1"
  Warning  Failed     5s (x4 over 24s)  kubelet, lxkubenode02  Error: container has runAsNonRoot and image will run as root

[naseemkullah]

Hi @msschl we can set some for the various components' securityContext values.

#41 is an issue open to decide what we should do, running all containers as non root is probably a good idea. I guess user 1000?

https://github.com/jaegertracing/jaeger/blob/master/cmd/agent/Dockerfile shows that no additional user is created.

[msschl]

Yes, user 1000 would be fine.

[naseemkullah]

Alright, I think in this case we should test and then add default securityContext for every pod via the various values

helm-charts/charts/jaeger/values.yaml

Line 233 in d4042b2

securityContext: {}

to runAsNonRoot and runAsUser: 1000

[msschl]

When I set runAsUser to 1000 in podSecurityContext and securityContext it starts wroking.

[naseemkullah]

Great! I think you can get by with just specifying one of the two btw.

[msschl]

Yes, thanks for the hint. 👍

[haljin]

So many of the templates allow specifying podSecurityContext and securityContext noiw, but the cassandra-schema Job does not. With that we cannot deploy the chart, as we also do not allow root containers in our cluster. Is there any reason why that one cannot be configured or was it simply not done? If so I could maybe do a PR to add those capabilities.

[mehta-ankit]

@haljin I added context for some components in this PR: https://github.com/jaegertracing/helm-charts/pull/37/files
Not sure why it was not added to, cassandra-schema.
Please can go ahead and make a PR i would say.

[naseemkullah]

On another note I think it's time we set runAsNonRoot as default for all components... with say user 1000, related issue: #41

[mehta-ankit]

On another note I think it's time we set runAsNonRoot as default for all components... with say user 1000, related issue: #41

I agree, since most people have to set it anyways (giving them these defaults sounds correct)

[flickerfly]

As user 1000 is commonly used, perhaps a more random one would be appropriate. Otherwise, 1000 becomes the new root and breaking the context of a container as user 1000 means potentially access into the other containers running as 1000 and removes the benefit of the not running as root to some degree.

[m00nyONE]

still no update on this?

I'm trying to deploy jaeger AIO with opentelemetry-demo and it does not work on our cluster because it's restriced to only allow non-root users to run containers

[yurishkuro]

Is this still an issue? jaegertracing/jaeger#2466 was merged 3yrs ago (but only covers the agent)