Skip to content
This repository has been archived by the owner on Oct 16, 2020. It is now read-only.

flannel-docker-opts.service produces empty flannel_docker_opts.env #1801

Closed
ghost opened this issue Feb 8, 2017 · 13 comments
Closed

flannel-docker-opts.service produces empty flannel_docker_opts.env #1801

ghost opened this issue Feb 8, 2017 · 13 comments
Labels
area/usability component/flannel kind/bug team/os

Comments

@ghost
Copy link

ghost commented Feb 8, 2017

Issue Report

Bug

Container Linux Version

$ cat /etc/os-release
NAME="Container Linux by CoreOS"
ID=coreos
VERSION=1235.9.0
VERSION_ID=1235.9.0
BUILD_ID=2017-02-02-0235
PRETTY_NAME="Container Linux by CoreOS 1235.9.0 (Ladybug)"
ANSI_COLOR="38;5;75"
HOME_URL="https://coreos.com/"
BUG_REPORT_URL="https://github.com/coreos/bugs/issues"

Environment

What hardware/cloud provider/hypervisor is being used to run Container Linux?

bare-metal

Expected Behavior

flannel-docker-opts.service should succeed and /run/flannel/flannel_docker_opts.env should not be empty.

Actual Behavior

/run/flannel/flannel_docker_opts.env is empty.

Reproduction Steps

  1. Place the following in /etc/systemd/system/docker.service.d/40-flannel.conf
# /etc/systemd/system/docker.service.d/40-flannel.conf
[Unit]
Requires=flanneld.service
After=flanneld.service
  1. Try to start docker.

Other Information

$ grep flannel.service /usr/lib/systemd/system/flannel-docker-opts.service
After=flannel.service                 
PartOf=flannel.service

It should probably be flanneld.service.

My assumption is that flanneld.service, which should be started Before=flannel-docker-opts.service, somehow is not yet up when the latter starts, hence /run/flannel/subnet.env is not yet present and thus /run/flannel/flannel_docker_opts.env is not filled with anything.

Later, when I log into the node, /run/flannel/subnet.env contains values for FLANNEL_NETWORK, FLANNEL_SUBNET, as well as FLANNEL_MTU=1500 and FLANNEL_IPMASQ=true. The latter should have caused /opt/bin/mk-docker-opts.sh from quay.io/coreos/flannel:v0.6.2 to write DOCKER_OPT_IPMASQ="--ip-masq=false". Since this was not written, I must assume $FLANNEL_IPMASQ was missing from /run/flannel/subnet.env, when /opt/bin/mk-docker-opts.sh looked at it.
@crawford
Copy link
Contributor

crawford commented Mar 1, 2017

This was fixed in 1248.0.0.

@ghost
Copy link
Author

ghost commented Mar 1, 2017

Could this be backported to stable, please?

@dm0-
Copy link

dm0- commented Mar 1, 2017

Stable is 1298.5.0, so this should be fixed in all channels. Let us know if that is not the case.

@dm0- dm0- closed this as completed Mar 1, 2017
@eghobo
Copy link

eghobo commented Mar 3, 2017

@dm0- I don't think 1298.5.0 has this fixed

cat /etc/os-release
NAME="Container Linux by CoreOS"
ID=coreos
VERSION=1298.5.0
VERSION_ID=1298.5.0
BUILD_ID=2017-02-28-0013
PRETTY_NAME="Container Linux by CoreOS 1298.5.0 (Ladybug)"
ANSI_COLOR="38;5;75"
HOME_URL="https://coreos.com/"
BUG_REPORT_URL="https://github.com/coreos/bugs/issues"

systemctl cat flannel-docker-opts.service
# /usr/lib/systemd/system/flannel-docker-opts.service
[Unit]
Description=flannel docker export service - Network fabric for containers (System Application Container)
Documentation=https://github.com/coreos/flannel
PartOf=flanneld.service
Before=docker.service

[Service]
Type=oneshot
TimeoutStartSec=60

Environment="FLANNEL_IMAGE_TAG=v0.6.2"
Environment="RKT_RUN_ARGS=--uuid-file-save=/var/lib/coreos/flannel-wrapper2.uuid"
Environment="FLANNEL_IMAGE_ARGS=--exec=/opt/bin/mk-docker-opts.sh"

ExecStartPre=-/usr/bin/rkt rm --uuid-file=/var/lib/coreos/flannel-wrapper2.uuid
ExecStart=/usr/lib/coreos/flannel-wrapper -d /run/flannel/flannel_docker_opts.env -i
ExecStop=-/usr/bin/rkt stop --uuid-file=/var/lib/coreos/flannel-wrapper2.uuid

[Install]
WantedBy=multi-user.target

It looks like After=flanneld.service is missing.

@dm0-
Copy link

dm0- commented Mar 3, 2017

@eghobo See https://github.com/coreos/coreos-overlay/blob/build-1298/app-admin/flannel-wrapper/files/flanneld.service#L5 .

@eghobo
Copy link

eghobo commented Mar 3, 2017

@dm0- thx, I also found this after comment. But we still this issue (specially at slow environments like vagrant, where pulling flannel image take time)

journalctl --no-pager -u flanneld
-- Logs begin at Fri 2017-03-03 19:18:01 UTC, end at Fri 2017-03-03 19:37:41 UTC. --
Mar 03 19:18:13 kub-master systemd[1]: Starting flannel - Network fabric for containers (System Application Container)...
Mar 03 19:18:13 kub-master rkt[1433]: rm: unable to resolve UUID from file: open /var/lib/coreos/flannel-wrapper.uuid: no such file or directory
Mar 03 19:18:13 kub-master rkt[1433]: rm: failed to remove one or more pods
Mar 03 19:18:13 kub-master etcdctl[1451]: etcd cluster has no published client endpoints.
Mar 03 19:18:13 kub-master etcdctl[1451]: Try '--no-sync' if you want to access non-published client endpoints(http://127.0.0.1:2379,http://127.0.0.1:4001).
Mar 03 19:18:13 kub-master etcdctl[1451]: Error:  client: no endpoints available
Mar 03 19:18:13 kub-master systemd[1]: flanneld.service: Control process exited, code=exited status=4
Mar 03 19:18:13 kub-master systemd[1]: Failed to start flannel - Network fabric for containers (System Application Container).
Mar 03 19:18:13 kub-master systemd[1]: flanneld.service: Unit entered failed state.
Mar 03 19:18:13 kub-master systemd[1]: flanneld.service: Failed with result 'exit-code'.
Mar 03 19:18:24 kub-master systemd[1]: flanneld.service: Service hold-off time over, scheduling restart.
Mar 03 19:18:24 kub-master systemd[1]: Stopped flannel - Network fabric for containers (System Application Container).
Mar 03 19:18:24 kub-master systemd[1]: Starting flannel - Network fabric for containers (System Application Container)...
Mar 03 19:18:24 kub-master rkt[1557]: rm: unable to resolve UUID from file: open /var/lib/coreos/flannel-wrapper.uuid: no such file or directory
Mar 03 19:18:24 kub-master rkt[1557]: rm: failed to remove one or more pods
Mar 03 19:18:24 kub-master etcdctl[1564]: {"Network": "10.2.0.0/16", "Backend": {"Type": "udp"}}
Mar 03 19:18:24 kub-master flannel-wrapper[1572]: + exec /usr/bin/rkt run --insecure-options=image --mount volume=notify,target=/run/systemd/notify --volume notify,kind=host,source=/run/systemd/notify --set-env=NOTIFY_SOCKET=/run/systemd/notify --net=host --volume run-flannel,kind=host,source=/run/flannel,readOnly=false --volume etc-ssl-certs,kind=host,source=/usr/share/ca-certificates,readOnly=true --volume usr-share-certs,kind=host,source=/usr/share/ca-certificates,readOnly=true --volume etc-hosts,kind=host,source=/etc/hosts,readOnly=true --volume etc-resolv,kind=host,source=/etc/resolv.conf,readOnly=true --mount volume=run-flannel,target=/run/flannel --mount volume=etc-ssl-certs,target=/etc/ssl/certs --mount volume=usr-share-certs,target=/usr/share/ca-certificates --mount volume=etc-hosts,target=/etc/hosts --mount volume=etc-resolv,target=/etc/resolv.conf --inherit-env --stage1-from-dir=stage1-fly.aci docker://quay.io/coreos/flannel:v0.6.2 -- --ip-masq=true
Mar 03 19:18:27 kub-master flannel-wrapper[1572]: Downloading sha256:9495713bb01 104 KB / 2.2 MB
Mar 03 19:18:27 kub-master flannel-wrapper[1572]: Downloading sha256:a3ed95caeb0 32 B / 32 B
Mar 03 19:18:27 kub-master flannel-wrapper[1572]: Downloading sha256:cfaba0bcdd4 17 KB / 53.3 KB
Mar 03 19:18:27 kub-master flannel-wrapper[1572]: Downloading sha256:a3ed95caeb0 32 B / 32 B
Mar 03 19:18:27 kub-master flannel-wrapper[1572]: Downloading sha256:a3ed95caeb0 32 B / 32 B
Mar 03 19:18:27 kub-master flannel-wrapper[1572]: Downloading sha256:974fe91d03e 0 B / 604 KB
Mar 03 19:18:27 kub-master flannel-wrapper[1572]: Downloading sha256:835ac7243dc 0 B / 4.71 MB
Mar 03 19:18:27 kub-master flannel-wrapper[1572]: Downloading sha256:d83fa8fedce 0 B / 1.04 KB
Mar 03 19:18:27 kub-master flannel-wrapper[1572]: Downloading sha256:9495713bb01 104 KB / 2.2 MB
Mar 03 19:18:27 kub-master flannel-wrapper[1572]: Downloading sha256:a3ed95caeb0 32 B / 32 B
Mar 03 19:18:27 kub-master flannel-wrapper[1572]: Downloading sha256:cfaba0bcdd4 17 KB / 53.3 KB
Mar 03 19:18:27 kub-master flannel-wrapper[1572]: Downloading sha256:a3ed95caeb0 32 B / 32 B
Mar 03 19:18:27 kub-master flannel-wrapper[1572]: Downloading sha256:a3ed95caeb0 32 B / 32 B
Mar 03 19:18:27 kub-master flannel-wrapper[1572]: Downloading sha256:974fe91d03e 0 B / 604 KB
Mar 03 19:18:27 kub-master flannel-wrapper[1572]: Downloading sha256:835ac7243dc 0 B / 4.71 MB
Mar 03 19:18:27 kub-master flannel-wrapper[1572]: Downloading sha256:d83fa8fedce 0 B / 1.04 KB
Mar 03 19:18:27 kub-master flannel-wrapper[1572]: Downloading sha256:9495713bb01 104 KB / 2.2 MB
Mar 03 19:18:27 kub-master flannel-wrapper[1572]: Downloading sha256:a3ed95caeb0 32 B / 32 B
Mar 03 19:18:27 kub-master flannel-wrapper[1572]: Downloading sha256:cfaba0bcdd4 17 KB / 53.3 KB
Mar 03 19:18:27 kub-master flannel-wrapper[1572]: Downloading sha256:a3ed95caeb0 32 B / 32 B
Mar 03 19:18:27 kub-master flannel-wrapper[1572]: Downloading sha256:a3ed95caeb0 32 B / 32 B
Mar 03 19:18:27 kub-master flannel-wrapper[1572]: Downloading sha256:974fe91d03e 0 B / 604 KB
Mar 03 19:18:27 kub-master flannel-wrapper[1572]: Downloading sha256:835ac7243dc 0 B / 4.71 MB
Mar 03 19:18:27 kub-master flannel-wrapper[1572]: Downloading sha256:d83fa8fedce 0 B / 1.04 KB
Mar 03 19:18:27 kub-master flannel-wrapper[1572]: Downloading sha256:9495713bb01 104 KB / 2.2 MB
Mar 03 19:18:27 kub-master flannel-wrapper[1572]: Downloading sha256:a3ed95caeb0 32 B / 32 B
Mar 03 19:18:27 kub-master flannel-wrapper[1572]: Downloading sha256:cfaba0bcdd4 17 KB / 53.3 KB
Mar 03 19:18:27 kub-master flannel-wrapper[1572]: Downloading sha256:a3ed95caeb0 32 B / 32 B
Mar 03 19:18:27 kub-master flannel-wrapper[1572]: Downloading sha256:a3ed95caeb0 32 B / 32 B
Mar 03 19:18:27 kub-master flannel-wrapper[1572]: Downloading sha256:974fe91d03e 0 B / 604 KB
Mar 03 19:18:27 kub-master flannel-wrapper[1572]: Downloading sha256:835ac7243dc 0 B / 4.71 MB
Mar 03 19:18:27 kub-master flannel-wrapper[1572]: Downloading sha256:d83fa8fedce 1.04 KB / 1.04 KB
Mar 03 19:18:27 kub-master flannel-wrapper[1572]: Downloading sha256:9495713bb01 156 KB / 2.2 MB
Mar 03 19:18:27 kub-master flannel-wrapper[1572]: Downloading sha256:a3ed95caeb0 32 B / 32 B
Mar 03 19:18:27 kub-master flannel-wrapper[1572]: Downloading sha256:cfaba0bcdd4 53.3 KB / 53.3 KB
Mar 03 19:18:27 kub-master flannel-wrapper[1572]: Downloading sha256:a3ed95caeb0 32 B / 32 B
Mar 03 19:18:27 kub-master flannel-wrapper[1572]: Downloading sha256:a3ed95caeb0 32 B / 32 B
Mar 03 19:18:27 kub-master flannel-wrapper[1572]: Downloading sha256:974fe91d03e 0 B / 604 KB
Mar 03 19:18:27 kub-master flannel-wrapper[1572]: Downloading sha256:835ac7243dc 0 B / 4.71 MB
Mar 03 19:18:27 kub-master flannel-wrapper[1572]: Downloading sha256:d83fa8fedce 1.04 KB / 1.04 KB
Mar 03 19:18:28 kub-master flannel-wrapper[1572]: Downloading sha256:9495713bb01 417 KB / 2.2 MB
Mar 03 19:18:28 kub-master flannel-wrapper[1572]: Downloading sha256:a3ed95caeb0 32 B / 32 B
Mar 03 19:18:28 kub-master flannel-wrapper[1572]: Downloading sha256:cfaba0bcdd4 53.3 KB / 53.3 KB
Mar 03 19:18:28 kub-master flannel-wrapper[1572]: Downloading sha256:a3ed95caeb0 32 B / 32 B
Mar 03 19:18:28 kub-master flannel-wrapper[1572]: Downloading sha256:a3ed95caeb0 32 B / 32 B
Mar 03 19:18:28 kub-master flannel-wrapper[1572]: Downloading sha256:974fe91d03e 226 KB / 604 KB
Mar 03 19:18:28 kub-master flannel-wrapper[1572]: Downloading sha256:835ac7243dc 85.6 KB / 4.71 MB
Mar 03 19:18:28 kub-master flannel-wrapper[1572]: Downloading sha256:d83fa8fedce 1.04 KB / 1.04 KB
Mar 03 19:18:28 kub-master flannel-wrapper[1572]: Downloading sha256:9495713bb01 626 KB / 2.2 MB
Mar 03 19:18:28 kub-master flannel-wrapper[1572]: Downloading sha256:a3ed95caeb0 32 B / 32 B
Mar 03 19:18:28 kub-master flannel-wrapper[1572]: Downloading sha256:cfaba0bcdd4 53.3 KB / 53.3 KB
Mar 03 19:18:28 kub-master flannel-wrapper[1572]: Downloading sha256:a3ed95caeb0 32 B / 32 B
Mar 03 19:18:28 kub-master flannel-wrapper[1572]: Downloading sha256:a3ed95caeb0 32 B / 32 B
Mar 03 19:18:28 kub-master flannel-wrapper[1572]: Downloading sha256:974fe91d03e 539 KB / 604 KB
Mar 03 19:18:28 kub-master flannel-wrapper[1572]: Downloading sha256:835ac7243dc 174 KB / 4.71 MB
Mar 03 19:18:28 kub-master flannel-wrapper[1572]: Downloading sha256:d83fa8fedce 1.04 KB / 1.04 KB
Mar 03 19:18:28 kub-master flannel-wrapper[1572]: Downloading sha256:9495713bb01 626 KB / 2.2 MB
Mar 03 19:18:28 kub-master flannel-wrapper[1572]: Downloading sha256:a3ed95caeb0 32 B / 32 B
Mar 03 19:18:28 kub-master flannel-wrapper[1572]: Downloading sha256:cfaba0bcdd4 53.3 KB / 53.3 KB
Mar 03 19:18:28 kub-master flannel-wrapper[1572]: Downloading sha256:a3ed95caeb0 32 B / 32 B
Mar 03 19:18:28 kub-master flannel-wrapper[1572]: Downloading sha256:a3ed95caeb0 32 B / 32 B
Mar 03 19:18:28 kub-master flannel-wrapper[1572]: Downloading sha256:974fe91d03e 604 KB / 604 KB
Mar 03 19:18:28 kub-master flannel-wrapper[1572]: Downloading sha256:835ac7243dc 174 KB / 4.71 MB
Mar 03 19:18:28 kub-master flannel-wrapper[1572]: Downloading sha256:d83fa8fedce 1.04 KB / 1.04 KB
Mar 03 19:18:29 kub-master flannel-wrapper[1572]: Downloading sha256:9495713bb01 748 KB / 2.2 MB
Mar 03 19:18:29 kub-master flannel-wrapper[1572]: Downloading sha256:a3ed95caeb0 32 B / 32 B
Mar 03 19:18:29 kub-master flannel-wrapper[1572]: Downloading sha256:cfaba0bcdd4 53.3 KB / 53.3 KB
Mar 03 19:18:29 kub-master flannel-wrapper[1572]: Downloading sha256:a3ed95caeb0 32 B / 32 B
Mar 03 19:18:29 kub-master flannel-wrapper[1572]: Downloading sha256:a3ed95caeb0 32 B / 32 B
Mar 03 19:18:29 kub-master flannel-wrapper[1572]: Downloading sha256:974fe91d03e 604 KB / 604 KB
Mar 03 19:18:29 kub-master flannel-wrapper[1572]: Downloading sha256:835ac7243dc 383 KB / 4.71 MB
Mar 03 19:18:29 kub-master flannel-wrapper[1572]: Downloading sha256:d83fa8fedce 1.04 KB / 1.04 KB
Mar 03 19:18:29 kub-master flannel-wrapper[1572]: Downloading sha256:9495713bb01 887 KB / 2.2 MB
Mar 03 19:18:29 kub-master flannel-wrapper[1572]: Downloading sha256:a3ed95caeb0 32 B / 32 B
Mar 03 19:18:29 kub-master flannel-wrapper[1572]: Downloading sha256:cfaba0bcdd4 53.3 KB / 53.3 KB
Mar 03 19:18:29 kub-master flannel-wrapper[1572]: Downloading sha256:a3ed95caeb0 32 B / 32 B
Mar 03 19:18:29 kub-master flannel-wrapper[1572]: Downloading sha256:a3ed95caeb0 32 B / 32 B
Mar 03 19:18:29 kub-master flannel-wrapper[1572]: Downloading sha256:974fe91d03e 604 KB / 604 KB
Mar 03 19:18:29 kub-master flannel-wrapper[1572]: Downloading sha256:835ac7243dc 644 KB / 4.71 MB
Mar 03 19:18:29 kub-master flannel-wrapper[1572]: Downloading sha256:d83fa8fedce 1.04 KB / 1.04 KB
Mar 03 19:18:30 kub-master flannel-wrapper[1572]: Downloading sha256:9495713bb01 1.04 MB / 2.2 MB
Mar 03 19:18:30 kub-master flannel-wrapper[1572]: Downloading sha256:a3ed95caeb0 32 B / 32 B
Mar 03 19:18:30 kub-master flannel-wrapper[1572]: Downloading sha256:cfaba0bcdd4 53.3 KB / 53.3 KB
Mar 03 19:18:30 kub-master flannel-wrapper[1572]: Downloading sha256:a3ed95caeb0 32 B / 32 B
Mar 03 19:18:30 kub-master flannel-wrapper[1572]: Downloading sha256:a3ed95caeb0 32 B / 32 B
Mar 03 19:18:30 kub-master flannel-wrapper[1572]: Downloading sha256:974fe91d03e 604 KB / 604 KB
Mar 03 19:18:30 kub-master flannel-wrapper[1572]: Downloading sha256:835ac7243dc 905 KB / 4.71 MB
Mar 03 19:18:30 kub-master flannel-wrapper[1572]: Downloading sha256:d83fa8fedce 1.04 KB / 1.04 KB
Mar 03 19:18:30 kub-master flannel-wrapper[1572]: Downloading sha256:9495713bb01 1.18 MB / 2.2 MB
Mar 03 19:18:30 kub-master flannel-wrapper[1572]: Downloading sha256:a3ed95caeb0 32 B / 32 B
Mar 03 19:18:30 kub-master flannel-wrapper[1572]: Downloading sha256:cfaba0bcdd4 53.3 KB / 53.3 KB
Mar 03 19:18:30 kub-master flannel-wrapper[1572]: Downloading sha256:a3ed95caeb0 32 B / 32 B
Mar 03 19:18:30 kub-master flannel-wrapper[1572]: Downloading sha256:a3ed95caeb0 32 B / 32 B
Mar 03 19:18:30 kub-master flannel-wrapper[1572]: Downloading sha256:974fe91d03e 604 KB / 604 KB
Mar 03 19:18:30 kub-master flannel-wrapper[1572]: Downloading sha256:835ac7243dc 1.2 MB / 4.71 MB
Mar 03 19:18:30 kub-master flannel-wrapper[1572]: Downloading sha256:d83fa8fedce 1.04 KB / 1.04 KB
Mar 03 19:18:31 kub-master flannel-wrapper[1572]: Downloading sha256:9495713bb01 1.34 MB / 2.2 MB
Mar 03 19:18:31 kub-master flannel-wrapper[1572]: Downloading sha256:a3ed95caeb0 32 B / 32 B
Mar 03 19:18:31 kub-master flannel-wrapper[1572]: Downloading sha256:cfaba0bcdd4 53.3 KB / 53.3 KB
Mar 03 19:18:31 kub-master flannel-wrapper[1572]: Downloading sha256:a3ed95caeb0 32 B / 32 B
Mar 03 19:18:31 kub-master flannel-wrapper[1572]: Downloading sha256:a3ed95caeb0 32 B / 32 B
Mar 03 19:18:31 kub-master flannel-wrapper[1572]: Downloading sha256:974fe91d03e 604 KB / 604 KB
Mar 03 19:18:31 kub-master flannel-wrapper[1572]: Downloading sha256:835ac7243dc 1.44 MB / 4.71 MB
Mar 03 19:18:31 kub-master flannel-wrapper[1572]: Downloading sha256:d83fa8fedce 1.04 KB / 1.04 KB
Mar 03 19:18:31 kub-master flannel-wrapper[1572]: Downloading sha256:9495713bb01 1.53 MB / 2.2 MB
Mar 03 19:18:31 kub-master flannel-wrapper[1572]: Downloading sha256:a3ed95caeb0 32 B / 32 B
Mar 03 19:18:31 kub-master flannel-wrapper[1572]: Downloading sha256:cfaba0bcdd4 53.3 KB / 53.3 KB
Mar 03 19:18:31 kub-master flannel-wrapper[1572]: Downloading sha256:a3ed95caeb0 32 B / 32 B
Mar 03 19:18:31 kub-master flannel-wrapper[1572]: Downloading sha256:a3ed95caeb0 32 B / 32 B
Mar 03 19:18:31 kub-master flannel-wrapper[1572]: Downloading sha256:974fe91d03e 604 KB / 604 KB
Mar 03 19:18:31 kub-master flannel-wrapper[1572]: Downloading sha256:835ac7243dc 1.71 MB / 4.71 MB
Mar 03 19:18:31 kub-master flannel-wrapper[1572]: Downloading sha256:d83fa8fedce 1.04 KB / 1.04 KB
Mar 03 19:18:32 kub-master flannel-wrapper[1572]: Downloading sha256:9495713bb01 1.74 MB / 2.2 MB
Mar 03 19:18:32 kub-master flannel-wrapper[1572]: Downloading sha256:a3ed95caeb0 32 B / 32 B
Mar 03 19:18:32 kub-master flannel-wrapper[1572]: Downloading sha256:cfaba0bcdd4 53.3 KB / 53.3 KB
Mar 03 19:18:32 kub-master flannel-wrapper[1572]: Downloading sha256:a3ed95caeb0 32 B / 32 B
Mar 03 19:18:32 kub-master flannel-wrapper[1572]: Downloading sha256:a3ed95caeb0 32 B / 32 B
Mar 03 19:18:32 kub-master flannel-wrapper[1572]: Downloading sha256:974fe91d03e 604 KB / 604 KB
Mar 03 19:18:32 kub-master flannel-wrapper[1572]: Downloading sha256:835ac7243dc 2.04 MB / 4.71 MB
Mar 03 19:18:32 kub-master flannel-wrapper[1572]: Downloading sha256:d83fa8fedce 1.04 KB / 1.04 KB
Mar 03 19:18:32 kub-master flannel-wrapper[1572]: Downloading sha256:9495713bb01 1.9 MB / 2.2 MB
Mar 03 19:18:32 kub-master flannel-wrapper[1572]: Downloading sha256:a3ed95caeb0 32 B / 32 B
Mar 03 19:18:32 kub-master flannel-wrapper[1572]: Downloading sha256:cfaba0bcdd4 53.3 KB / 53.3 KB
Mar 03 19:18:32 kub-master flannel-wrapper[1572]: Downloading sha256:a3ed95caeb0 32 B / 32 B
Mar 03 19:18:32 kub-master flannel-wrapper[1572]: Downloading sha256:a3ed95caeb0 32 B / 32 B
Mar 03 19:18:32 kub-master flannel-wrapper[1572]: Downloading sha256:974fe91d03e 604 KB / 604 KB
Mar 03 19:18:32 kub-master flannel-wrapper[1572]: Downloading sha256:835ac7243dc 2.33 MB / 4.71 MB
Mar 03 19:18:32 kub-master flannel-wrapper[1572]: Downloading sha256:d83fa8fedce 1.04 KB / 1.04 KB
Mar 03 19:18:33 kub-master flannel-wrapper[1572]: Downloading sha256:9495713bb01 2.11 MB / 2.2 MB
Mar 03 19:18:33 kub-master flannel-wrapper[1572]: Downloading sha256:a3ed95caeb0 32 B / 32 B
Mar 03 19:18:33 kub-master flannel-wrapper[1572]: Downloading sha256:cfaba0bcdd4 53.3 KB / 53.3 KB
Mar 03 19:18:33 kub-master flannel-wrapper[1572]: Downloading sha256:a3ed95caeb0 32 B / 32 B
Mar 03 19:18:33 kub-master flannel-wrapper[1572]: Downloading sha256:a3ed95caeb0 32 B / 32 B
Mar 03 19:18:33 kub-master flannel-wrapper[1572]: Downloading sha256:974fe91d03e 604 KB / 604 KB
Mar 03 19:18:33 kub-master flannel-wrapper[1572]: Downloading sha256:835ac7243dc 2.63 MB / 4.71 MB
Mar 03 19:18:33 kub-master flannel-wrapper[1572]: Downloading sha256:d83fa8fedce 1.04 KB / 1.04 KB
Mar 03 19:18:33 kub-master flannel-wrapper[1572]: Downloading sha256:9495713bb01 2.2 MB / 2.2 MB
Mar 03 19:18:33 kub-master flannel-wrapper[1572]: Downloading sha256:a3ed95caeb0 32 B / 32 B
Mar 03 19:18:33 kub-master flannel-wrapper[1572]: Downloading sha256:cfaba0bcdd4 53.3 KB / 53.3 KB
Mar 03 19:18:33 kub-master flannel-wrapper[1572]: Downloading sha256:a3ed95caeb0 32 B / 32 B
Mar 03 19:18:33 kub-master flannel-wrapper[1572]: Downloading sha256:a3ed95caeb0 32 B / 32 B
Mar 03 19:18:33 kub-master flannel-wrapper[1572]: Downloading sha256:974fe91d03e 604 KB / 604 KB
Mar 03 19:18:33 kub-master flannel-wrapper[1572]: Downloading sha256:835ac7243dc 2.8 MB / 4.71 MB
Mar 03 19:18:33 kub-master flannel-wrapper[1572]: Downloading sha256:d83fa8fedce 1.04 KB / 1.04 KB
Mar 03 19:18:33 kub-master flannel-wrapper[1572]: Downloading sha256:9495713bb01 2.2 MB / 2.2 MB
Mar 03 19:18:33 kub-master flannel-wrapper[1572]: Downloading sha256:a3ed95caeb0 32 B / 32 B
Mar 03 19:18:33 kub-master flannel-wrapper[1572]: Downloading sha256:cfaba0bcdd4 53.3 KB / 53.3 KB
Mar 03 19:18:33 kub-master flannel-wrapper[1572]: Downloading sha256:a3ed95caeb0 32 B / 32 B
Mar 03 19:18:33 kub-master flannel-wrapper[1572]: Downloading sha256:a3ed95caeb0 32 B / 32 B
Mar 03 19:18:33 kub-master flannel-wrapper[1572]: Downloading sha256:974fe91d03e 604 KB / 604 KB
Mar 03 19:18:33 kub-master flannel-wrapper[1572]: Downloading sha256:835ac7243dc 2.96 MB / 4.71 MB
Mar 03 19:18:33 kub-master flannel-wrapper[1572]: Downloading sha256:d83fa8fedce 1.04 KB / 1.04 KB
Mar 03 19:18:34 kub-master flannel-wrapper[1572]: Downloading sha256:9495713bb01 2.2 MB / 2.2 MB
Mar 03 19:18:34 kub-master flannel-wrapper[1572]: Downloading sha256:a3ed95caeb0 32 B / 32 B
Mar 03 19:18:34 kub-master flannel-wrapper[1572]: Downloading sha256:cfaba0bcdd4 53.3 KB / 53.3 KB
Mar 03 19:18:34 kub-master flannel-wrapper[1572]: Downloading sha256:a3ed95caeb0 32 B / 32 B
Mar 03 19:18:34 kub-master flannel-wrapper[1572]: Downloading sha256:a3ed95caeb0 32 B / 32 B
Mar 03 19:18:34 kub-master flannel-wrapper[1572]: Downloading sha256:974fe91d03e 604 KB / 604 KB
Mar 03 19:18:34 kub-master flannel-wrapper[1572]: Downloading sha256:835ac7243dc 3.06 MB / 4.71 MB
Mar 03 19:18:34 kub-master flannel-wrapper[1572]: Downloading sha256:d83fa8fedce 1.04 KB / 1.04 KB
Mar 03 19:18:34 kub-master flannel-wrapper[1572]: Downloading sha256:9495713bb01 2.2 MB / 2.2 MB
Mar 03 19:18:34 kub-master flannel-wrapper[1572]: Downloading sha256:a3ed95caeb0 32 B / 32 B
Mar 03 19:18:34 kub-master flannel-wrapper[1572]: Downloading sha256:cfaba0bcdd4 53.3 KB / 53.3 KB
Mar 03 19:18:34 kub-master flannel-wrapper[1572]: Downloading sha256:a3ed95caeb0 32 B / 32 B
Mar 03 19:18:34 kub-master flannel-wrapper[1572]: Downloading sha256:a3ed95caeb0 32 B / 32 B
Mar 03 19:18:34 kub-master flannel-wrapper[1572]: Downloading sha256:974fe91d03e 604 KB / 604 KB
Mar 03 19:18:34 kub-master flannel-wrapper[1572]: Downloading sha256:835ac7243dc 3.27 MB / 4.71 MB
Mar 03 19:18:34 kub-master flannel-wrapper[1572]: Downloading sha256:d83fa8fedce 1.04 KB / 1.04 KB
Mar 03 19:18:35 kub-master flannel-wrapper[1572]: Downloading sha256:9495713bb01 2.2 MB / 2.2 MB
Mar 03 19:18:35 kub-master flannel-wrapper[1572]: Downloading sha256:a3ed95caeb0 32 B / 32 B
Mar 03 19:18:35 kub-master flannel-wrapper[1572]: Downloading sha256:cfaba0bcdd4 53.3 KB / 53.3 KB
Mar 03 19:18:35 kub-master flannel-wrapper[1572]: Downloading sha256:a3ed95caeb0 32 B / 32 B
Mar 03 19:18:35 kub-master flannel-wrapper[1572]: Downloading sha256:a3ed95caeb0 32 B / 32 B
Mar 03 19:18:35 kub-master flannel-wrapper[1572]: Downloading sha256:974fe91d03e 604 KB / 604 KB
Mar 03 19:18:35 kub-master flannel-wrapper[1572]: Downloading sha256:835ac7243dc 3.55 MB / 4.71 MB
Mar 03 19:18:35 kub-master flannel-wrapper[1572]: Downloading sha256:d83fa8fedce 1.04 KB / 1.04 KB
Mar 03 19:18:35 kub-master flannel-wrapper[1572]: Downloading sha256:9495713bb01 2.2 MB / 2.2 MB
Mar 03 19:18:35 kub-master flannel-wrapper[1572]: Downloading sha256:a3ed95caeb0 32 B / 32 B
Mar 03 19:18:35 kub-master flannel-wrapper[1572]: Downloading sha256:cfaba0bcdd4 53.3 KB / 53.3 KB
Mar 03 19:18:35 kub-master flannel-wrapper[1572]: Downloading sha256:a3ed95caeb0 32 B / 32 B
Mar 03 19:18:35 kub-master flannel-wrapper[1572]: Downloading sha256:a3ed95caeb0 32 B / 32 B
Mar 03 19:18:35 kub-master flannel-wrapper[1572]: Downloading sha256:974fe91d03e 604 KB / 604 KB
Mar 03 19:18:35 kub-master flannel-wrapper[1572]: Downloading sha256:835ac7243dc 3.81 MB / 4.71 MB
Mar 03 19:18:35 kub-master flannel-wrapper[1572]: Downloading sha256:d83fa8fedce 1.04 KB / 1.04 KB
Mar 03 19:18:36 kub-master flannel-wrapper[1572]: Downloading sha256:9495713bb01 2.2 MB / 2.2 MB
Mar 03 19:18:36 kub-master flannel-wrapper[1572]: Downloading sha256:a3ed95caeb0 32 B / 32 B
Mar 03 19:18:36 kub-master flannel-wrapper[1572]: Downloading sha256:cfaba0bcdd4 53.3 KB / 53.3 KB
Mar 03 19:18:36 kub-master flannel-wrapper[1572]: Downloading sha256:a3ed95caeb0 32 B / 32 B
Mar 03 19:18:36 kub-master flannel-wrapper[1572]: Downloading sha256:a3ed95caeb0 32 B / 32 B
Mar 03 19:18:36 kub-master flannel-wrapper[1572]: Downloading sha256:974fe91d03e 604 KB / 604 KB
Mar 03 19:18:36 kub-master flannel-wrapper[1572]: Downloading sha256:835ac7243dc 4.06 MB / 4.71 MB
Mar 03 19:18:36 kub-master flannel-wrapper[1572]: Downloading sha256:d83fa8fedce 1.04 KB / 1.04 KB
Mar 03 19:18:36 kub-master flannel-wrapper[1572]: Downloading sha256:9495713bb01 2.2 MB / 2.2 MB
Mar 03 19:18:36 kub-master flannel-wrapper[1572]: Downloading sha256:a3ed95caeb0 32 B / 32 B
Mar 03 19:18:36 kub-master flannel-wrapper[1572]: Downloading sha256:cfaba0bcdd4 53.3 KB / 53.3 KB
Mar 03 19:18:36 kub-master flannel-wrapper[1572]: Downloading sha256:a3ed95caeb0 32 B / 32 B
Mar 03 19:18:36 kub-master flannel-wrapper[1572]: Downloading sha256:a3ed95caeb0 32 B / 32 B
Mar 03 19:18:36 kub-master flannel-wrapper[1572]: Downloading sha256:974fe91d03e 604 KB / 604 KB
Mar 03 19:18:36 kub-master flannel-wrapper[1572]: Downloading sha256:835ac7243dc 4.35 MB / 4.71 MB
Mar 03 19:18:36 kub-master flannel-wrapper[1572]: Downloading sha256:d83fa8fedce 1.04 KB / 1.04 KB
Mar 03 19:18:37 kub-master flannel-wrapper[1572]: Downloading sha256:9495713bb01 2.2 MB / 2.2 MB
Mar 03 19:18:37 kub-master flannel-wrapper[1572]: Downloading sha256:a3ed95caeb0 32 B / 32 B
Mar 03 19:18:37 kub-master flannel-wrapper[1572]: Downloading sha256:cfaba0bcdd4 53.3 KB / 53.3 KB
Mar 03 19:18:37 kub-master flannel-wrapper[1572]: Downloading sha256:a3ed95caeb0 32 B / 32 B
Mar 03 19:18:37 kub-master flannel-wrapper[1572]: Downloading sha256:a3ed95caeb0 32 B / 32 B
Mar 03 19:18:37 kub-master flannel-wrapper[1572]: Downloading sha256:974fe91d03e 604 KB / 604 KB
Mar 03 19:18:37 kub-master flannel-wrapper[1572]: Downloading sha256:835ac7243dc 4.6 MB / 4.71 MB
Mar 03 19:18:37 kub-master flannel-wrapper[1572]: Downloading sha256:d83fa8fedce 1.04 KB / 1.04 KB
Mar 03 19:18:37 kub-master flannel-wrapper[1572]: Downloading sha256:9495713bb01 2.2 MB / 2.2 MB
Mar 03 19:18:37 kub-master flannel-wrapper[1572]: Downloading sha256:a3ed95caeb0 32 B / 32 B
Mar 03 19:18:37 kub-master flannel-wrapper[1572]: Downloading sha256:cfaba0bcdd4 53.3 KB / 53.3 KB
Mar 03 19:18:37 kub-master flannel-wrapper[1572]: Downloading sha256:a3ed95caeb0 32 B / 32 B
Mar 03 19:18:37 kub-master flannel-wrapper[1572]: Downloading sha256:a3ed95caeb0 32 B / 32 B
Mar 03 19:18:37 kub-master flannel-wrapper[1572]: Downloading sha256:974fe91d03e 604 KB / 604 KB
Mar 03 19:18:37 kub-master flannel-wrapper[1572]: Downloading sha256:835ac7243dc 4.71 MB / 4.71 MB
Mar 03 19:18:37 kub-master flannel-wrapper[1572]: Downloading sha256:d83fa8fedce 1.04 KB / 1.04 KB
Mar 03 19:18:39 kub-master flannel-wrapper[1572]: I0303 19:18:39.124653 01572 main.go:126] Installing signal handlers
Mar 03 19:18:39 kub-master flannel-wrapper[1572]: I0303 19:18:39.126446 01572 manager.go:163] Using 192.168.65.101 as external interface
Mar 03 19:18:39 kub-master flannel-wrapper[1572]: I0303 19:18:39.126648 01572 manager.go:164] Using 192.168.65.101 as external endpoint
Mar 03 19:18:39 kub-master flannel-wrapper[1572]: I0303 19:18:39.132827 01572 local_manager.go:179] Picking subnet in range 10.2.1.0 ... 10.2.255.0
Mar 03 19:18:39 kub-master flannel-wrapper[1572]: I0303 19:18:39.190708 01572 ipmasq.go:47] Adding iptables rule: -s 10.2.0.0/16 -d 10.2.0.0/16 -j RETURN
Mar 03 19:18:39 kub-master flannel-wrapper[1572]: I0303 19:18:39.240441 01572 ipmasq.go:47] Adding iptables rule: -s 10.2.0.0/16 ! -d 224.0.0.0/4 -j MASQUERADE
Mar 03 19:18:39 kub-master flannel-wrapper[1572]: I0303 19:18:39.256186 01572 ipmasq.go:47] Adding iptables rule: ! -s 10.2.0.0/16 -d 10.2.0.0/16 -j MASQUERADE
Mar 03 19:18:39 kub-master flannel-wrapper[1572]: I0303 19:18:39.257711 01572 manager.go:246] Lease acquired: 10.2.26.0/24
Mar 03 19:18:39 kub-master systemd[1]: Started flannel - Network fabric for containers (System Application Container).
Mar 03 19:18:39 kub-master flannel-wrapper[1572]: I0303 19:18:39.260911 01572 network.go:98] Watching for new subnet leases
Mar 03 19:18:56 kub-master flannel-wrapper[1572]: I0303 19:18:56.433746 01572 network.go:191] Subnet added: 10.2.101.0/24
Mar 03 19:19:23 kub-master flannel-wrapper[1572]: I0303 19:19:23.270964 01572 network.go:191] Subnet added: 10.2.32.0/24

journalctl --no-pager -u flannel-docker-opts
-- Logs begin at Fri 2017-03-03 19:18:01 UTC, end at Fri 2017-03-03 19:38:59 UTC. --
Mar 03 19:18:13 kub-master systemd[1]: Starting flannel docker export service - Network fabric for containers (System Application Container)...
Mar 03 19:18:13 kub-master rkt[1467]: rm: unable to resolve UUID from file: open /var/lib/coreos/flannel-wrapper2.uuid: no such file or directory
Mar 03 19:18:13 kub-master rkt[1467]: rm: failed to remove one or more pods
Mar 03 19:18:13 kub-master flannel-wrapper[1487]: + exec /usr/bin/rkt run --insecure-options=image --net=host --volume run-flannel,kind=host,source=/run/flannel,readOnly=false --volume etc-ssl-certs,kind=host,source=/usr/share/ca-certificates,readOnly=true --volume usr-share-certs,kind=host,source=/usr/share/ca-certificates,readOnly=true --volume etc-hosts,kind=host,source=/etc/hosts,readOnly=true --volume etc-resolv,kind=host,source=/etc/resolv.conf,readOnly=true --mount volume=run-flannel,target=/run/flannel --mount volume=etc-ssl-certs,target=/etc/ssl/certs --mount volume=usr-share-certs,target=/usr/share/ca-certificates --mount volume=etc-hosts,target=/etc/hosts --mount volume=etc-resolv,target=/etc/resolv.conf --inherit-env --stage1-from-dir=stage1-fly.aci docker://quay.io/coreos/flannel:v0.6.2 --exec=/opt/bin/mk-docker-opts.sh -- -d /run/flannel/flannel_docker_opts.env -i
Mar 03 19:18:16 kub-master flannel-wrapper[1487]: Downloading sha256:9495713bb01 156 KB / 2.2 MB
Mar 03 19:18:16 kub-master flannel-wrapper[1487]: Downloading sha256:a3ed95caeb0 32 B / 32 B
Mar 03 19:18:16 kub-master flannel-wrapper[1487]: Downloading sha256:cfaba0bcdd4 17 KB / 53.3 KB
Mar 03 19:18:16 kub-master flannel-wrapper[1487]: Downloading sha256:974fe91d03e 17 KB / 604 KB
Mar 03 19:18:16 kub-master flannel-wrapper[1487]: Downloading sha256:d83fa8fedce 1.04 KB / 1.04 KB
Mar 03 19:18:16 kub-master flannel-wrapper[1487]: Downloading sha256:a3ed95caeb0 32 B / 32 B
Mar 03 19:18:16 kub-master flannel-wrapper[1487]: Downloading sha256:a3ed95caeb0 32 B / 32 B
Mar 03 19:18:16 kub-master flannel-wrapper[1487]: Downloading sha256:835ac7243dc 0 B / 4.71 MB
Mar 03 19:18:16 kub-master flannel-wrapper[1487]: Downloading sha256:9495713bb01 156 KB / 2.2 MB
Mar 03 19:18:16 kub-master flannel-wrapper[1487]: Downloading sha256:a3ed95caeb0 32 B / 32 B
Mar 03 19:18:16 kub-master flannel-wrapper[1487]: Downloading sha256:cfaba0bcdd4 17 KB / 53.3 KB
Mar 03 19:18:16 kub-master flannel-wrapper[1487]: Downloading sha256:974fe91d03e 17 KB / 604 KB
Mar 03 19:18:16 kub-master flannel-wrapper[1487]: Downloading sha256:d83fa8fedce 1.04 KB / 1.04 KB
Mar 03 19:18:16 kub-master flannel-wrapper[1487]: Downloading sha256:a3ed95caeb0 32 B / 32 B
Mar 03 19:18:16 kub-master flannel-wrapper[1487]: Downloading sha256:a3ed95caeb0 32 B / 32 B
Mar 03 19:18:16 kub-master flannel-wrapper[1487]: Downloading sha256:835ac7243dc 0 B / 4.71 MB
Mar 03 19:18:16 kub-master flannel-wrapper[1487]: Downloading sha256:9495713bb01 156 KB / 2.2 MB
Mar 03 19:18:16 kub-master flannel-wrapper[1487]: Downloading sha256:a3ed95caeb0 32 B / 32 B
Mar 03 19:18:16 kub-master flannel-wrapper[1487]: Downloading sha256:cfaba0bcdd4 17 KB / 53.3 KB
Mar 03 19:18:16 kub-master flannel-wrapper[1487]: Downloading sha256:974fe91d03e 17 KB / 604 KB
Mar 03 19:18:16 kub-master flannel-wrapper[1487]: Downloading sha256:d83fa8fedce 1.04 KB / 1.04 KB
Mar 03 19:18:16 kub-master flannel-wrapper[1487]: Downloading sha256:a3ed95caeb0 32 B / 32 B
Mar 03 19:18:16 kub-master flannel-wrapper[1487]: Downloading sha256:a3ed95caeb0 32 B / 32 B
Mar 03 19:18:16 kub-master flannel-wrapper[1487]: Downloading sha256:835ac7243dc 0 B / 4.71 MB
Mar 03 19:18:16 kub-master flannel-wrapper[1487]: Downloading sha256:9495713bb01 156 KB / 2.2 MB
Mar 03 19:18:16 kub-master flannel-wrapper[1487]: Downloading sha256:a3ed95caeb0 32 B / 32 B
Mar 03 19:18:16 kub-master flannel-wrapper[1487]: Downloading sha256:cfaba0bcdd4 17 KB / 53.3 KB
Mar 03 19:18:16 kub-master flannel-wrapper[1487]: Downloading sha256:974fe91d03e 17 KB / 604 KB
Mar 03 19:18:16 kub-master flannel-wrapper[1487]: Downloading sha256:d83fa8fedce 1.04 KB / 1.04 KB
Mar 03 19:18:16 kub-master flannel-wrapper[1487]: Downloading sha256:a3ed95caeb0 32 B / 32 B
Mar 03 19:18:16 kub-master flannel-wrapper[1487]: Downloading sha256:a3ed95caeb0 32 B / 32 B
Mar 03 19:18:16 kub-master flannel-wrapper[1487]: Downloading sha256:835ac7243dc 0 B / 4.71 MB
Mar 03 19:18:16 kub-master flannel-wrapper[1487]: Downloading sha256:9495713bb01 156 KB / 2.2 MB
Mar 03 19:18:16 kub-master flannel-wrapper[1487]: Downloading sha256:a3ed95caeb0 32 B / 32 B
Mar 03 19:18:16 kub-master flannel-wrapper[1487]: Downloading sha256:cfaba0bcdd4 53.3 KB / 53.3 KB
Mar 03 19:18:16 kub-master flannel-wrapper[1487]: Downloading sha256:974fe91d03e 17 KB / 604 KB
Mar 03 19:18:16 kub-master flannel-wrapper[1487]: Downloading sha256:d83fa8fedce 1.04 KB / 1.04 KB
Mar 03 19:18:16 kub-master flannel-wrapper[1487]: Downloading sha256:a3ed95caeb0 32 B / 32 B
Mar 03 19:18:16 kub-master flannel-wrapper[1487]: Downloading sha256:a3ed95caeb0 32 B / 32 B
Mar 03 19:18:16 kub-master flannel-wrapper[1487]: Downloading sha256:835ac7243dc 0 B / 4.71 MB
Mar 03 19:18:17 kub-master flannel-wrapper[1487]: Downloading sha256:9495713bb01 487 KB / 2.2 MB
Mar 03 19:18:17 kub-master flannel-wrapper[1487]: Downloading sha256:a3ed95caeb0 32 B / 32 B
Mar 03 19:18:17 kub-master flannel-wrapper[1487]: Downloading sha256:cfaba0bcdd4 53.3 KB / 53.3 KB
Mar 03 19:18:17 kub-master flannel-wrapper[1487]: Downloading sha256:974fe91d03e 296 KB / 604 KB
Mar 03 19:18:17 kub-master flannel-wrapper[1487]: Downloading sha256:d83fa8fedce 1.04 KB / 1.04 KB
Mar 03 19:18:17 kub-master flannel-wrapper[1487]: Downloading sha256:a3ed95caeb0 32 B / 32 B
Mar 03 19:18:17 kub-master flannel-wrapper[1487]: Downloading sha256:a3ed95caeb0 32 B / 32 B
Mar 03 19:18:17 kub-master flannel-wrapper[1487]: Downloading sha256:835ac7243dc 226 KB / 4.71 MB
Mar 03 19:18:17 kub-master flannel-wrapper[1487]: Downloading sha256:9495713bb01 731 KB / 2.2 MB
Mar 03 19:18:17 kub-master flannel-wrapper[1487]: Downloading sha256:a3ed95caeb0 32 B / 32 B
Mar 03 19:18:17 kub-master flannel-wrapper[1487]: Downloading sha256:cfaba0bcdd4 53.3 KB / 53.3 KB
Mar 03 19:18:17 kub-master flannel-wrapper[1487]: Downloading sha256:974fe91d03e 604 KB / 604 KB
Mar 03 19:18:17 kub-master flannel-wrapper[1487]: Downloading sha256:d83fa8fedce 1.04 KB / 1.04 KB
Mar 03 19:18:17 kub-master flannel-wrapper[1487]: Downloading sha256:a3ed95caeb0 32 B / 32 B
Mar 03 19:18:17 kub-master flannel-wrapper[1487]: Downloading sha256:a3ed95caeb0 32 B / 32 B
Mar 03 19:18:17 kub-master flannel-wrapper[1487]: Downloading sha256:835ac7243dc 487 KB / 4.71 MB
Mar 03 19:18:17 kub-master flannel-wrapper[1487]: Downloading sha256:9495713bb01 800 KB / 2.2 MB
Mar 03 19:18:17 kub-master flannel-wrapper[1487]: Downloading sha256:a3ed95caeb0 32 B / 32 B
Mar 03 19:18:17 kub-master flannel-wrapper[1487]: Downloading sha256:cfaba0bcdd4 53.3 KB / 53.3 KB
Mar 03 19:18:17 kub-master flannel-wrapper[1487]: Downloading sha256:974fe91d03e 604 KB / 604 KB
Mar 03 19:18:17 kub-master flannel-wrapper[1487]: Downloading sha256:d83fa8fedce 1.04 KB / 1.04 KB
Mar 03 19:18:17 kub-master flannel-wrapper[1487]: Downloading sha256:a3ed95caeb0 32 B / 32 B
Mar 03 19:18:17 kub-master flannel-wrapper[1487]: Downloading sha256:a3ed95caeb0 32 B / 32 B
Mar 03 19:18:17 kub-master flannel-wrapper[1487]: Downloading sha256:835ac7243dc 539 KB / 4.71 MB
Mar 03 19:18:18 kub-master flannel-wrapper[1487]: Downloading sha256:9495713bb01 1.17 MB / 2.2 MB
Mar 03 19:18:18 kub-master flannel-wrapper[1487]: Downloading sha256:a3ed95caeb0 32 B / 32 B
Mar 03 19:18:18 kub-master flannel-wrapper[1487]: Downloading sha256:cfaba0bcdd4 53.3 KB / 53.3 KB
Mar 03 19:18:18 kub-master flannel-wrapper[1487]: Downloading sha256:974fe91d03e 604 KB / 604 KB
Mar 03 19:18:18 kub-master flannel-wrapper[1487]: Downloading sha256:d83fa8fedce 1.04 KB / 1.04 KB
Mar 03 19:18:18 kub-master flannel-wrapper[1487]: Downloading sha256:a3ed95caeb0 32 B / 32 B
Mar 03 19:18:18 kub-master flannel-wrapper[1487]: Downloading sha256:a3ed95caeb0 32 B / 32 B
Mar 03 19:18:18 kub-master flannel-wrapper[1487]: Downloading sha256:835ac7243dc 870 KB / 4.71 MB
Mar 03 19:18:18 kub-master flannel-wrapper[1487]: Downloading sha256:9495713bb01 1.46 MB / 2.2 MB
Mar 03 19:18:18 kub-master flannel-wrapper[1487]: Downloading sha256:a3ed95caeb0 32 B / 32 B
Mar 03 19:18:18 kub-master flannel-wrapper[1487]: Downloading sha256:cfaba0bcdd4 53.3 KB / 53.3 KB
Mar 03 19:18:18 kub-master flannel-wrapper[1487]: Downloading sha256:974fe91d03e 604 KB / 604 KB
Mar 03 19:18:18 kub-master flannel-wrapper[1487]: Downloading sha256:d83fa8fedce 1.04 KB / 1.04 KB
Mar 03 19:18:18 kub-master flannel-wrapper[1487]: Downloading sha256:a3ed95caeb0 32 B / 32 B
Mar 03 19:18:18 kub-master flannel-wrapper[1487]: Downloading sha256:a3ed95caeb0 32 B / 32 B
Mar 03 19:18:18 kub-master flannel-wrapper[1487]: Downloading sha256:835ac7243dc 1.24 MB / 4.71 MB
Mar 03 19:18:19 kub-master flannel-wrapper[1487]: Downloading sha256:9495713bb01 1.64 MB / 2.2 MB
Mar 03 19:18:19 kub-master flannel-wrapper[1487]: Downloading sha256:a3ed95caeb0 32 B / 32 B
Mar 03 19:18:19 kub-master flannel-wrapper[1487]: Downloading sha256:cfaba0bcdd4 53.3 KB / 53.3 KB
Mar 03 19:18:19 kub-master flannel-wrapper[1487]: Downloading sha256:974fe91d03e 604 KB / 604 KB
Mar 03 19:18:19 kub-master flannel-wrapper[1487]: Downloading sha256:d83fa8fedce 1.04 KB / 1.04 KB
Mar 03 19:18:19 kub-master flannel-wrapper[1487]: Downloading sha256:a3ed95caeb0 32 B / 32 B
Mar 03 19:18:19 kub-master flannel-wrapper[1487]: Downloading sha256:a3ed95caeb0 32 B / 32 B
Mar 03 19:18:19 kub-master flannel-wrapper[1487]: Downloading sha256:835ac7243dc 1.43 MB / 4.71 MB
Mar 03 19:18:19 kub-master flannel-wrapper[1487]: Downloading sha256:9495713bb01 1.9 MB / 2.2 MB
Mar 03 19:18:19 kub-master flannel-wrapper[1487]: Downloading sha256:a3ed95caeb0 32 B / 32 B
Mar 03 19:18:19 kub-master flannel-wrapper[1487]: Downloading sha256:cfaba0bcdd4 53.3 KB / 53.3 KB
Mar 03 19:18:19 kub-master flannel-wrapper[1487]: Downloading sha256:974fe91d03e 604 KB / 604 KB
Mar 03 19:18:19 kub-master flannel-wrapper[1487]: Downloading sha256:d83fa8fedce 1.04 KB / 1.04 KB
Mar 03 19:18:19 kub-master flannel-wrapper[1487]: Downloading sha256:a3ed95caeb0 32 B / 32 B
Mar 03 19:18:19 kub-master flannel-wrapper[1487]: Downloading sha256:a3ed95caeb0 32 B / 32 B
Mar 03 19:18:19 kub-master flannel-wrapper[1487]: Downloading sha256:835ac7243dc 1.72 MB / 4.71 MB
Mar 03 19:18:20 kub-master flannel-wrapper[1487]: Downloading sha256:9495713bb01 2.14 MB / 2.2 MB
Mar 03 19:18:20 kub-master flannel-wrapper[1487]: Downloading sha256:a3ed95caeb0 32 B / 32 B
Mar 03 19:18:20 kub-master flannel-wrapper[1487]: Downloading sha256:cfaba0bcdd4 53.3 KB / 53.3 KB
Mar 03 19:18:20 kub-master flannel-wrapper[1487]: Downloading sha256:974fe91d03e 604 KB / 604 KB
Mar 03 19:18:20 kub-master flannel-wrapper[1487]: Downloading sha256:d83fa8fedce 1.04 KB / 1.04 KB
Mar 03 19:18:20 kub-master flannel-wrapper[1487]: Downloading sha256:a3ed95caeb0 32 B / 32 B
Mar 03 19:18:20 kub-master flannel-wrapper[1487]: Downloading sha256:a3ed95caeb0 32 B / 32 B
Mar 03 19:18:20 kub-master flannel-wrapper[1487]: Downloading sha256:835ac7243dc 2.09 MB / 4.71 MB
Mar 03 19:18:20 kub-master flannel-wrapper[1487]: Downloading sha256:9495713bb01 2.2 MB / 2.2 MB
Mar 03 19:18:20 kub-master flannel-wrapper[1487]: Downloading sha256:a3ed95caeb0 32 B / 32 B
Mar 03 19:18:20 kub-master flannel-wrapper[1487]: Downloading sha256:cfaba0bcdd4 53.3 KB / 53.3 KB
Mar 03 19:18:20 kub-master flannel-wrapper[1487]: Downloading sha256:974fe91d03e 604 KB / 604 KB
Mar 03 19:18:20 kub-master flannel-wrapper[1487]: Downloading sha256:d83fa8fedce 1.04 KB / 1.04 KB
Mar 03 19:18:20 kub-master flannel-wrapper[1487]: Downloading sha256:a3ed95caeb0 32 B / 32 B
Mar 03 19:18:20 kub-master flannel-wrapper[1487]: Downloading sha256:a3ed95caeb0 32 B / 32 B
Mar 03 19:18:20 kub-master flannel-wrapper[1487]: Downloading sha256:835ac7243dc 2.16 MB / 4.71 MB
Mar 03 19:18:20 kub-master flannel-wrapper[1487]: Downloading sha256:9495713bb01 2.2 MB / 2.2 MB
Mar 03 19:18:20 kub-master flannel-wrapper[1487]: Downloading sha256:a3ed95caeb0 32 B / 32 B
Mar 03 19:18:20 kub-master flannel-wrapper[1487]: Downloading sha256:cfaba0bcdd4 53.3 KB / 53.3 KB
Mar 03 19:18:20 kub-master flannel-wrapper[1487]: Downloading sha256:974fe91d03e 604 KB / 604 KB
Mar 03 19:18:20 kub-master flannel-wrapper[1487]: Downloading sha256:d83fa8fedce 1.04 KB / 1.04 KB
Mar 03 19:18:20 kub-master flannel-wrapper[1487]: Downloading sha256:a3ed95caeb0 32 B / 32 B
Mar 03 19:18:20 kub-master flannel-wrapper[1487]: Downloading sha256:a3ed95caeb0 32 B / 32 B
Mar 03 19:18:20 kub-master flannel-wrapper[1487]: Downloading sha256:835ac7243dc 2.4 MB / 4.71 MB
Mar 03 19:18:21 kub-master flannel-wrapper[1487]: Downloading sha256:9495713bb01 2.2 MB / 2.2 MB
Mar 03 19:18:21 kub-master flannel-wrapper[1487]: Downloading sha256:a3ed95caeb0 32 B / 32 B
Mar 03 19:18:21 kub-master flannel-wrapper[1487]: Downloading sha256:cfaba0bcdd4 53.3 KB / 53.3 KB
Mar 03 19:18:21 kub-master flannel-wrapper[1487]: Downloading sha256:974fe91d03e 604 KB / 604 KB
Mar 03 19:18:21 kub-master flannel-wrapper[1487]: Downloading sha256:d83fa8fedce 1.04 KB / 1.04 KB
Mar 03 19:18:21 kub-master flannel-wrapper[1487]: Downloading sha256:a3ed95caeb0 32 B / 32 B
Mar 03 19:18:21 kub-master flannel-wrapper[1487]: Downloading sha256:a3ed95caeb0 32 B / 32 B
Mar 03 19:18:21 kub-master flannel-wrapper[1487]: Downloading sha256:835ac7243dc 2.7 MB / 4.71 MB
Mar 03 19:18:21 kub-master flannel-wrapper[1487]: Downloading sha256:9495713bb01 2.2 MB / 2.2 MB
Mar 03 19:18:21 kub-master flannel-wrapper[1487]: Downloading sha256:a3ed95caeb0 32 B / 32 B
Mar 03 19:18:21 kub-master flannel-wrapper[1487]: Downloading sha256:cfaba0bcdd4 53.3 KB / 53.3 KB
Mar 03 19:18:21 kub-master flannel-wrapper[1487]: Downloading sha256:974fe91d03e 604 KB / 604 KB
Mar 03 19:18:21 kub-master flannel-wrapper[1487]: Downloading sha256:d83fa8fedce 1.04 KB / 1.04 KB
Mar 03 19:18:21 kub-master flannel-wrapper[1487]: Downloading sha256:a3ed95caeb0 32 B / 32 B
Mar 03 19:18:21 kub-master flannel-wrapper[1487]: Downloading sha256:a3ed95caeb0 32 B / 32 B
Mar 03 19:18:21 kub-master flannel-wrapper[1487]: Downloading sha256:835ac7243dc 3.01 MB / 4.71 MB
Mar 03 19:18:22 kub-master flannel-wrapper[1487]: Downloading sha256:9495713bb01 2.2 MB / 2.2 MB
Mar 03 19:18:22 kub-master flannel-wrapper[1487]: Downloading sha256:a3ed95caeb0 32 B / 32 B
Mar 03 19:18:22 kub-master flannel-wrapper[1487]: Downloading sha256:cfaba0bcdd4 53.3 KB / 53.3 KB
Mar 03 19:18:22 kub-master flannel-wrapper[1487]: Downloading sha256:974fe91d03e 604 KB / 604 KB
Mar 03 19:18:22 kub-master flannel-wrapper[1487]: Downloading sha256:d83fa8fedce 1.04 KB / 1.04 KB
Mar 03 19:18:22 kub-master flannel-wrapper[1487]: Downloading sha256:a3ed95caeb0 32 B / 32 B
Mar 03 19:18:22 kub-master flannel-wrapper[1487]: Downloading sha256:a3ed95caeb0 32 B / 32 B
Mar 03 19:18:22 kub-master flannel-wrapper[1487]: Downloading sha256:835ac7243dc 3.39 MB / 4.71 MB
Mar 03 19:18:22 kub-master flannel-wrapper[1487]: Downloading sha256:9495713bb01 2.2 MB / 2.2 MB
Mar 03 19:18:22 kub-master flannel-wrapper[1487]: Downloading sha256:a3ed95caeb0 32 B / 32 B
Mar 03 19:18:22 kub-master flannel-wrapper[1487]: Downloading sha256:cfaba0bcdd4 53.3 KB / 53.3 KB
Mar 03 19:18:22 kub-master flannel-wrapper[1487]: Downloading sha256:974fe91d03e 604 KB / 604 KB
Mar 03 19:18:22 kub-master flannel-wrapper[1487]: Downloading sha256:d83fa8fedce 1.04 KB / 1.04 KB
Mar 03 19:18:22 kub-master flannel-wrapper[1487]: Downloading sha256:a3ed95caeb0 32 B / 32 B
Mar 03 19:18:22 kub-master flannel-wrapper[1487]: Downloading sha256:a3ed95caeb0 32 B / 32 B
Mar 03 19:18:22 kub-master flannel-wrapper[1487]: Downloading sha256:835ac7243dc 3.69 MB / 4.71 MB
Mar 03 19:18:23 kub-master flannel-wrapper[1487]: Downloading sha256:9495713bb01 2.2 MB / 2.2 MB
Mar 03 19:18:23 kub-master flannel-wrapper[1487]: Downloading sha256:a3ed95caeb0 32 B / 32 B
Mar 03 19:18:23 kub-master flannel-wrapper[1487]: Downloading sha256:cfaba0bcdd4 53.3 KB / 53.3 KB
Mar 03 19:18:23 kub-master flannel-wrapper[1487]: Downloading sha256:974fe91d03e 604 KB / 604 KB
Mar 03 19:18:23 kub-master flannel-wrapper[1487]: Downloading sha256:d83fa8fedce 1.04 KB / 1.04 KB
Mar 03 19:18:23 kub-master flannel-wrapper[1487]: Downloading sha256:a3ed95caeb0 32 B / 32 B
Mar 03 19:18:23 kub-master flannel-wrapper[1487]: Downloading sha256:a3ed95caeb0 32 B / 32 B
Mar 03 19:18:23 kub-master flannel-wrapper[1487]: Downloading sha256:835ac7243dc 4 MB / 4.71 MB
Mar 03 19:18:23 kub-master flannel-wrapper[1487]: Downloading sha256:9495713bb01 2.2 MB / 2.2 MB
Mar 03 19:18:23 kub-master flannel-wrapper[1487]: Downloading sha256:a3ed95caeb0 32 B / 32 B
Mar 03 19:18:23 kub-master flannel-wrapper[1487]: Downloading sha256:cfaba0bcdd4 53.3 KB / 53.3 KB
Mar 03 19:18:23 kub-master flannel-wrapper[1487]: Downloading sha256:974fe91d03e 604 KB / 604 KB
Mar 03 19:18:23 kub-master flannel-wrapper[1487]: Downloading sha256:d83fa8fedce 1.04 KB / 1.04 KB
Mar 03 19:18:23 kub-master flannel-wrapper[1487]: Downloading sha256:a3ed95caeb0 32 B / 32 B
Mar 03 19:18:23 kub-master flannel-wrapper[1487]: Downloading sha256:a3ed95caeb0 32 B / 32 B
Mar 03 19:18:23 kub-master flannel-wrapper[1487]: Downloading sha256:835ac7243dc 4.37 MB / 4.71 MB
Mar 03 19:18:24 kub-master flannel-wrapper[1487]: Downloading sha256:9495713bb01 2.2 MB / 2.2 MB
Mar 03 19:18:24 kub-master flannel-wrapper[1487]: Downloading sha256:a3ed95caeb0 32 B / 32 B
Mar 03 19:18:24 kub-master flannel-wrapper[1487]: Downloading sha256:cfaba0bcdd4 53.3 KB / 53.3 KB
Mar 03 19:18:24 kub-master flannel-wrapper[1487]: Downloading sha256:974fe91d03e 604 KB / 604 KB
Mar 03 19:18:24 kub-master flannel-wrapper[1487]: Downloading sha256:d83fa8fedce 1.04 KB / 1.04 KB
Mar 03 19:18:24 kub-master flannel-wrapper[1487]: Downloading sha256:a3ed95caeb0 32 B / 32 B
Mar 03 19:18:24 kub-master flannel-wrapper[1487]: Downloading sha256:a3ed95caeb0 32 B / 32 B
Mar 03 19:18:24 kub-master flannel-wrapper[1487]: Downloading sha256:835ac7243dc 4.68 MB / 4.71 MB
Mar 03 19:18:24 kub-master flannel-wrapper[1487]: Downloading sha256:9495713bb01 2.2 MB / 2.2 MB
Mar 03 19:18:24 kub-master flannel-wrapper[1487]: Downloading sha256:a3ed95caeb0 32 B / 32 B
Mar 03 19:18:24 kub-master flannel-wrapper[1487]: Downloading sha256:cfaba0bcdd4 53.3 KB / 53.3 KB
Mar 03 19:18:24 kub-master flannel-wrapper[1487]: Downloading sha256:974fe91d03e 604 KB / 604 KB
Mar 03 19:18:24 kub-master flannel-wrapper[1487]: Downloading sha256:d83fa8fedce 1.04 KB / 1.04 KB
Mar 03 19:18:24 kub-master flannel-wrapper[1487]: Downloading sha256:a3ed95caeb0 32 B / 32 B
Mar 03 19:18:24 kub-master flannel-wrapper[1487]: Downloading sha256:a3ed95caeb0 32 B / 32 B
Mar 03 19:18:24 kub-master flannel-wrapper[1487]: Downloading sha256:835ac7243dc 4.71 MB / 4.71 MB
Mar 03 19:18:27 kub-master rkt[1607]: stop: unable to resolve UUID from file: open /var/lib/coreos/flannel-wrapper2.uuid: no such file or directory
Mar 03 19:18:27 kub-master systemd[1]: Started flannel docker export service - Network fabric for containers (System Application Container).

and you can see that flannel-docker-opts run before flanneld ready

Mar 03 19:18:39 kub-master systemd[1]: Started flannel - Network fabric for containers (System Application Container).

Mar 03 19:18:27 kub-master systemd[1]: Started flannel docker export service - Network fabric for containers (System Application Container).

@dm0-
Copy link

dm0- commented Mar 3, 2017

I can't reproduce that behavior. Did you copy any service files into /etc/systemd/system or add drop-ins?

@eghobo
Copy link

eghobo commented Mar 3, 2017

@dm0- yes we have

units:
    - name: flannel-docker-opts.service
      drop-ins:
        - name: 10-config.conf
          content: |
            [Service]
            EnvironmentFile=/run/flannel/options.env
            TimeoutStartSec=120

write_files:
  - path: /run/flannel/options.env
    content: |
      RKT_RUN_ARGS='--insecure-options=image'
      LANNEL_IMAGE_TAG='v0.6.2'
      FLANNEL_IMAGE_URL='docker://quay.ac.uda.io/coreos/flannel'
      FLANNELD_ETCD_ENDPOINTS=${etcd_endpoints}
      FLANNELD_IFACE=$private_ipv4

@dm0-
Copy link

dm0- commented Mar 3, 2017

What does systemctl cat flanneld say? Or ls -AlR /etc/systemd/system?

@eghobo
Copy link

eghobo commented Mar 3, 2017 

systemctl cat flanneld
# /usr/lib/systemd/system/flanneld.service
[Unit]
Description=flannel - Network fabric for containers (System Application Container)
Documentation=https://github.com/coreos/flannel
After=etcd.service etcd2.service etcd-member.service
Before=docker.service flannel-docker-opts.service
Requires=flannel-docker-opts.service

[Service]
Type=notify
Restart=always
RestartSec=10s
LimitNOFILE=40000
LimitNPROC=1048576

Environment="FLANNEL_IMAGE_TAG=v0.6.2"
Environment="FLANNEL_OPTS=--ip-masq=true"
Environment="RKT_RUN_ARGS=--uuid-file-save=/var/lib/coreos/flannel-wrapper.uuid"
EnvironmentFile=-/run/flannel/options.env

ExecStartPre=/sbin/modprobe ip_tables
ExecStartPre=/usr/bin/mkdir --parents /var/lib/coreos /run/flannel
ExecStartPre=-/usr/bin/rkt rm --uuid-file=/var/lib/coreos/flannel-wrapper.uuid
ExecStart=/usr/lib/coreos/flannel-wrapper $FLANNEL_OPTS
ExecStop=-/usr/bin/rkt stop --uuid-file=/var/lib/coreos/flannel-wrapper.uuid

[Install]
WantedBy=multi-user.target

# /etc/systemd/system/flanneld.service.d/10-config.conf
[Unit]
After=etcd2.service
Wants=etcd2.service
[Service]
EnvironmentFile=/run/flannel/options.env
ExecStartPre=/usr/bin/etcdctl set /coreos.com/network/config '{"Network": "10.2.0.0/16", "Backend": {"Type": "udp"}}'
TimeoutStartSec=120

@dm0-
Copy link

dm0- commented Mar 3, 2017

Okay, can you open a different bug? Those units have the correct ordering, and I couldn't reproduce the problem with them.

@eghobo
Copy link

eghobo commented Mar 6, 2017

@dm0- #1847 created

@alogoc
Copy link

alogoc commented Apr 15, 2017

1298.7.0 and this is still an issue.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
area/usability component/flannel kind/bug team/os
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@crawford @eghobo @dm0- @alogoc