New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

etcd-wrapper propagates its own environmental knobs to etcd #2500

Open
saj opened this Issue Sep 4, 2018 · 0 comments

Comments

Projects
None yet
2 participants
@saj

saj commented Sep 4, 2018

Issue Report

Bug

Container Linux Version

# cat /etc/os-release
NAME="Container Linux by CoreOS"
ID=coreos
VERSION=1855.2.0
VERSION_ID=1855.2.0
BUILD_ID=2018-08-15-2250
PRETTY_NAME="Container Linux by CoreOS 1855.2.0 (Rhyolite)"
ANSI_COLOR="38;5;75"
HOME_URL="https://coreos.com/"
BUG_REPORT_URL="https://issues.coreos.com"
COREOS_BOARD="amd64-usr"

Environment

AWS EC2, HVM

Expected Behavior

No spurious messages from the flags package are logged on etcd startup.

Actual Behavior

The following spurious messages are logged on etcd startup:

pkg/flags: recognized and used environment variable ETCD_DATA_DIR=/var/lib/etcd
pkg/flags: unrecognized environment variable ETCD_IMAGE_URL=docker://<REDACTED>
pkg/flags: recognized environment variable ETCD_NAME, but unused: shadowed by corresponding flag
pkg/flags: unrecognized environment variable ETCD_USER=etcd
pkg/flags: unrecognized environment variable ETCD_IMAGE_TAG=v3.3.9

Reproduction Steps

Start etcd using CoreOS' etcd-wrapper. I think the message for ETCD_USER will be logged on all CoreOS etcd installations; the other variables, like ETCD_IMAGE_URL and ETCD_IMAGE_TAG, we supply as systemd environmental overrides (see below).

Other Information

I would suppose the fundamental problem is that etcd-wrapper and etcd have overlapping configuration space. Knobs like ETCD_IMAGE_TAG are clearly intended for the former, though they are propagated through to the latter where their values may be misinterpreted.

The names of these knobs cannot be changed without breaking backwards compatibility.

Would it be possible to unexport ETCD_IMAGE_TAG and friends before the etcd container is started?

# systemctl cat etcd-member.service
# /usr/lib/systemd/system/etcd-member.service
[Unit]
Description=etcd (System Application Container)
Documentation=https://github.com/coreos/etcd
Wants=network-online.target network.target
After=network-online.target
Conflicts=etcd.service
Conflicts=etcd2.service

[Service]
Type=notify
Restart=on-failure
RestartSec=10s
TimeoutStartSec=0
LimitNOFILE=40000

Environment="ETCD_IMAGE_TAG=v3.3.9"
Environment="ETCD_NAME=%m"
Environment="ETCD_USER=etcd"
Environment="ETCD_DATA_DIR=/var/lib/etcd"
Environment="RKT_RUN_ARGS=--uuid-file-save=/var/lib/coreos/etcd-member-wrapper.uuid"

ExecStartPre=/usr/bin/mkdir --parents /var/lib/coreos
ExecStartPre=-/usr/bin/rkt rm --uuid-file=/var/lib/coreos/etcd-member-wrapper.uuid
ExecStart=/usr/lib/coreos/etcd-wrapper $ETCD_OPTS
ExecStop=-/usr/bin/rkt stop --uuid-file=/var/lib/coreos/etcd-member-wrapper.uuid

[Install]
WantedBy=multi-user.target

# /etc/systemd/system/etcd-member.service.d/20-clct-etcd-member.conf
[Service]
Environment="ETCD_IMAGE_TAG=v3.3.9"
ExecStart=
ExecStart=/usr/lib/coreos/etcd-wrapper $ETCD_OPTS \
  --name="<REDACTED>" \
  --heartbeat-interval=200 \
  --election-timeout=2000 \
  --listen-peer-urls="https://0.0.0.0:2380" \
  --listen-client-urls="https://0.0.0.0:2379" \
  --initial-advertise-peer-urls="https://<REDACTED>:2380" \
  --initial-cluster-state="new" \
  --initial-cluster-token="<REDACTED>" \
  --advertise-client-urls="https://<REDACTED>:2379" \
  --discovery-srv="<REDACTED>" \
  --discovery-fallback="exit" \
  --auto-compaction-retention="24h" \
  --auto-compaction-mode="periodic" \
  --cert-file="/etc/ssl/certs/etcd/cert.pem" \
  --key-file="/etc/ssl/certs/etcd/key.pem" \
  --client-cert-auth=true \
  --trusted-ca-file="/etc/ssl/certs/etcd/ca.pem" \
  --peer-cert-file="/etc/ssl/certs/etcd/peer-cert.pem" \
  --peer-key-file="/etc/ssl/certs/etcd/peer-key.pem" \
  --peer-client-cert-auth=true \
  --peer-trusted-ca-file="/etc/ssl/certs/etcd/peer-ca.pem"
# /etc/systemd/system/etcd-member.service.d/50-<REDACTED>.conf
[Service]
# ETCD_IMAGE_URL is consumed by /usr/lib/coreos/etcd-wrapper
Environment="ETCD_IMAGE_URL=docker://<REDACTED>.dkr.ecr.<REDACTED>.amazonaws.com/<REDACTED>"

Environment="RKT_GLOBAL_ARGS=--user-config=/root/.rkt --insecure-options=image"
ExecStartPre=/usr/bin/env RKT_AUTHN_ECR_REGION=<REDACTED> RKT_AUTHN_ECR_REGISTRY=<REDACTED> /opt/bin/rkt-authn-ecr
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment