Getting Permission Denied when running etcd container as non root user : securityContext

[Ajith61]

Hi,

We are getting permission denied when running etcd container as non root user. There was no issue when i tried to run as root user. Could you please guide me to fix this.

ETCD image : quay.io/coreos/etcd:v3.5.4
SecurityContext in Deployment yaml :

      securityContext:
        runAsUser: 1000
        runAsGroup: 2000
        fsGroup: 2000

Events:

Events:
Type Reason Age From Message

---

Normal Scheduled 2m7s default-scheduler Successfully assigned postgres-etcd1/etcd-2 to lxk8s-carbon-sadc-sbx-postgres-v1-np1-w1c2-worker-nph6q-pwd9t
Normal Pulled 33s (x5 over 119s) kubelet Container image "docker.com/postgres/carbon/etcd/postgresetcd:v3.5.4" already present on machine
Normal Created 33s (x5 over 119s) kubelet Created container: etcd
Normal Started 33s (x5 over 119s) kubelet Started container etcd
Warning BackOff 7s (x10 over 118s) kubelet Back-off restarting failed container etcd in pod etcd-2_postgres-etcd1(eaf08aa4-049d-4b68-ab4e-fcd0d43377ec)

Error Log

{"level":"warn","ts":"2025-04-23T12:43:39.294Z","caller":"flags/flag.go:93","msg":"unrecognized environment variable","environment-variable":"ETCD_PORT_2380_TCP_ADDR=192.168.133.173"}
{"level":"warn","ts":"2025-04-23T12:43:39.294Z","caller":"flags/flag.go:93","msg":"unrecognized environment variable","environment-variable":"ETCD_SERVICE_HOST=192.168.133.173"}
{"level":"warn","ts":"2025-04-23T12:43:39.294Z","caller":"flags/flag.go:93","msg":"unrecognized environment variable","environment-variable":"ETCD_PORT_2380_TCP_PORT=2380"}
{"level":"warn","ts":"2025-04-23T12:43:39.294Z","caller":"flags/flag.go:93","msg":"unrecognized environment variable","environment-variable":"ETCD_PORT_2380_TCP_PROTO=tcp"}
{"level":"warn","ts":"2025-04-23T12:43:39.294Z","caller":"flags/flag.go:93","msg":"unrecognized environment variable","environment-variable":"ETCD_SERVICE_PORT_CLIENT=2379"}
{"level":"warn","ts":"2025-04-23T12:43:39.294Z","caller":"flags/flag.go:93","msg":"unrecognized environment variable","environment-variable":"ETCD_PORT_2379_TCP_ADDR=192.168.133.173"}
{"level":"warn","ts":"2025-04-23T12:43:39.294Z","caller":"flags/flag.go:93","msg":"unrecognized environment variable","environment-variable":"ETCD_SERVICE_PORT=2379"}
{"level":"warn","ts":"2025-04-23T12:43:39.294Z","caller":"flags/flag.go:93","msg":"unrecognized environment variable","environment-variable":"ETCD_PORT=tcp://192.168.133.173:2379"}
{"level":"warn","ts":"2025-04-23T12:43:39.294Z","caller":"flags/flag.go:93","msg":"unrecognized environment variable","environment-variable":"ETCD_PORT_2379_TCP_PORT=2379"}
{"level":"warn","ts":"2025-04-23T12:43:39.294Z","caller":"flags/flag.go:93","msg":"unrecognized environment variable","environment-variable":"ETCD_PORT_2380_TCP=tcp://192.168.133.173:2380"}
{"level":"warn","ts":"2025-04-23T12:43:39.294Z","caller":"flags/flag.go:93","msg":"unrecognized environment variable","environment-variable":"ETCD_PORT_2379_TCP_PROTO=tcp"}
{"level":"warn","ts":"2025-04-23T12:43:39.294Z","caller":"flags/flag.go:93","msg":"unrecognized environment variable","environment-variable":"ETCD_SERVICE_PORT_PEER=2380"}
{"level":"warn","ts":"2025-04-23T12:43:39.294Z","caller":"flags/flag.go:93","msg":"unrecognized environment variable","environment-variable":"ETCD_PORT_2379_TCP=tcp://192.168.133.173:2379"}
{"level":"info","ts":"2025-04-23T12:43:39.294Z","caller":"etcdmain/etcd.go:73","msg":"Running: ","args":["etcd","--name","etcd-2","--listen-peer-urls","http://0.0.0.0:2380","--listen-client-urls","http://0.0.0.0:2379","--advertise-client-urls","http://etcd.postgres-etcd1.svc.cluster.local:2379","--initial-advertise-peer-urls","http://etcd-2:2380","--initial-cluster-token","etcd-cluster-1","--initial-cluster","etcd-0=http://etcd-0.etcd:2380,etcd-1=http://etcd-1.etcd:2380,etcd-2=http://etcd-2.etcd:2380","--initial-cluster-state","new","--data-dir","/var/run/etcd/default.etcd","--enable-v2=true"]}
{"level":"fatal","ts":"2025-04-23T12:43:39.294Z","caller":"etcdmain/etcd.go:435","msg":"failed to list data directory","dir":"/var/run/etcd/default.etcd","error":"open /var/run/etcd/default.etcd: permission denied","stacktrace":"go.etcd.io/etcd/server/v3/etcdmain.identifyDataDirOrDie\n\t/go/src/go.etcd.io/etcd/release/etcd/server/etcdmain/etcd.go:435\ngo.etcd.io/etcd/server/v3/etcdmain.startEtcdOrProxyV2\n\t/go/src/go.etcd.io/etcd/release/etcd/server/etcdmain/etcd.go:114\ngo.etcd.io/etcd/server/v3/etcdmain.Main\n\t/go/src/go.etcd.io/etcd/release/etcd/server/etcdmain/main.go:40\nmain.main\n\t/go/src/go.etcd.io/etcd/release/etcd/server/main.go:32\nruntime.main\n\t/go/gos/go1.16.15/src/runtime/proc.go:225"}

[jmhbnz]

Discussed during sig-etcd triage meeting - thanks for your question @Ajith61. In order for us to assist can you provide the full deployment spec (redacted where neccessary) so we can attempt to reproduce the issue easily?

Additionally what flavour of k8s are you attempting to run on?

Note: it appears you are using etcd v3.5.4, this release is over three years old and not recommended for production. We have fixed countless bugs and security vulnerabilities since then. Please update as soon as possible.