New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

systemd v229 container networking is broken #1205

Closed
mischief opened this Issue Apr 4, 2016 · 8 comments

Comments

Projects
None yet
3 participants
@mischief

mischief commented Apr 4, 2016

$ ssh coretest
CoreOS developer (1000.0.0+2016-04-04-1135)
Failed Units: 1
  usr.mount
core@coreos_production_qemu-1000-0-0- ~ $ docker run --rm -ti busybox /bin/sh -c "nslookup coreos.com"
Unable to find image 'busybox:latest' locally
latest: Pulling from library/busybox
385e281300cc: Pull complete 
a3ed95caeb02: Pull complete 
Digest: sha256:4a887a2326ec9e0fa90cce7b4764b0e627b5d6afcb81a3f73c85dc29cea00048
Status: Downloaded newer image for busybox:latest
Server:    10.0.2.3
Address 1: 10.0.2.3

nslookup: can't resolve 'coreos.com'
core@coreos_production_qemu-1000-0-0- ~ $ 

on a related note, upstream neutered IPForward=kernel, but in this case all forwarding options appear on. not sure what the problem is yet.

core@coreos_production_qemu-1000-0-0- ~ $ cat /proc/sys/net/ipv4/ip_forward
1
core@coreos_production_qemu-1000-0-0- ~ $ for f in /proc/sys/net/ipv4/conf/*/forwarding; do echo -n $f ": "; cat $f; done
/proc/sys/net/ipv4/conf/all/forwarding : 1
/proc/sys/net/ipv4/conf/default/forwarding : 1
/proc/sys/net/ipv4/conf/docker0/forwarding : 1
/proc/sys/net/ipv4/conf/eth0/forwarding : 1
/proc/sys/net/ipv4/conf/lo/forwarding : 1
@mischief

This comment has been minimized.

Show comment
Hide comment
@mischief

mischief Apr 5, 2016

it would appear that docker will bring up docker0 and assign an IP to it, and networkd will get a netlink message about it. when it does, it reads /usr/lib64/systemd/network/50-docker.network and due to there being an empty address assignment, it seems networkd is removing the bridge IP.

mischief commented Apr 5, 2016

it would appear that docker will bring up docker0 and assign an IP to it, and networkd will get a netlink message about it. when it does, it reads /usr/lib64/systemd/network/50-docker.network and due to there being an empty address assignment, it seems networkd is removing the bridge IP.

@mischief

This comment has been minimized.

Show comment
Hide comment
@mischief

mischief Apr 5, 2016

the following change to 50-docker.network appears to fix it, but this solution seems untenable.

[DHCP]
CriticalConnection=true

mischief commented Apr 5, 2016

the following change to 50-docker.network appears to fix it, but this solution seems untenable.

[DHCP]
CriticalConnection=true
@mischief

This comment has been minimized.

Show comment
Hide comment
@mischief

mischief Apr 5, 2016

it appears systemd/systemd@5e5b137 changed this behavior.

mischief commented Apr 5, 2016

it appears systemd/systemd@5e5b137 changed this behavior.

@crawford

This comment has been minimized.

Show comment
Hide comment
@crawford

crawford Apr 5, 2016

Member

Interesting. That would seem to imply that we can stop downing interfaces before we restart networkd in certain OEM configs.

Member

crawford commented Apr 5, 2016

Interesting. That would seem to imply that we can stop downing interfaces before we restart networkd in certain OEM configs.

@marineam

This comment has been minimized.

Show comment
Hide comment
@marineam

marineam Apr 5, 2016

My chat commentary:

00:53 < marineam> for documentation's sake, here is the commit that that breaks networking: systemd/systemd@5e5b137
00:54 < marineam> this is a big fundimental change to how networkd behaves
00:54 < marineam> also, CriticalConnection is quite clearyly in the wrong section, it is applicale to all devices, not just DHCP configured ones.
00:56 < marineam> systemd/systemd@e5d44b3 and
systemd/systemd@02e2862 further fixed up the behavior
00:57 < marineam> the ciritical connection option predates this, which explains why it is in the dhcp section:
systemd/systemd@eb27aec

marineam commented Apr 5, 2016

My chat commentary:

00:53 < marineam> for documentation's sake, here is the commit that that breaks networking: systemd/systemd@5e5b137
00:54 < marineam> this is a big fundimental change to how networkd behaves
00:54 < marineam> also, CriticalConnection is quite clearyly in the wrong section, it is applicale to all devices, not just DHCP configured ones.
00:56 < marineam> systemd/systemd@e5d44b3 and
systemd/systemd@02e2862 further fixed up the behavior
00:57 < marineam> the ciritical connection option predates this, which explains why it is in the dhcp section:
systemd/systemd@eb27aec

@marineam

This comment has been minimized.

Show comment
Hide comment
@marineam

marineam Apr 5, 2016

For a short term fix we should add a patch that just drops the call to link_drop_foreign_config but we need to figure out what to do long term.

marineam commented Apr 5, 2016

For a short term fix we should add a patch that just drops the call to link_drop_foreign_config but we need to figure out what to do long term.

@mischief

This comment has been minimized.

Show comment
Hide comment
@mischief

mischief Apr 6, 2016

fixed by coreos/systemd#45 as per @marineam's suggestion, but we might want to keep this issue open to discuss addressing this in a proper manner.

mischief commented Apr 6, 2016

fixed by coreos/systemd#45 as per @marineam's suggestion, but we might want to keep this issue open to discuss addressing this in a proper manner.

@crawford

This comment has been minimized.

Show comment
Hide comment
@crawford

crawford Jul 27, 2016

Member

Let's discuss this in #1485.

Member

crawford commented Jul 27, 2016

Let's discuss this in #1485.

@crawford crawford closed this Jul 27, 2016

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment