Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

networkd: Allow networkd to work with keepalived / high availability #12511

Open
wants to merge 1 commit into
base: master
from

Conversation

5 participants
@ssahani
Copy link
Contributor

commented May 8, 2019

This looks pretty legit. networkd drops the foreign addresses upon
restart. This becomes a problem in cases where the additional address
is used as a virtual or floating IP in higih-available environments
(where a master IP is swapped to a different service upon failovers,
e.g. by services like Pacemaker, keepalived and alike, usually using
protocols like Corosync, VRRP or CARP). Those addresses are not handled
by systemd-networkd and are therefore purged upon DHCP renewals or
when an update triggers a restart of systemd-networkd, therefore
potentially resulting in downtimes or at least unecessary failovers.

closes #12050

@johannbg

This comment has been minimized.

Copy link
Contributor

commented May 8, 2019

@keszybz can you backport this fix for F29+ ( other downstream distribution maintainers should probably do so too ) once this lands, since this is somewhat of an urgent matter for systemd networkd only deployments as well as any network management stack that uses systemd-networkd ( nm/netplan etc ).

Downstreams/upstream BZ are probably full of misfiled bugs against keepalive, pacemaker etc related to this underlying issues ( and probably bz.rh will be filled with this against RHEL8 once the bussword summit is over )

@ssahani thanks for looking into this :)

@ssahani ssahani changed the title networkd: Allow networkd to work with keepalived / highavialbility networkd: Allow networkd to work with keepalived / high availability May 8, 2019

@yuwata yuwata added the network label May 8, 2019

@Jailbye

Jailbye approved these changes May 8, 2019

@yuwata

This comment has been minimized.

Copy link
Member

commented May 9, 2019

I think this also fixes #11575.

@yuwata

This comment has been minimized.

Copy link
Member

commented May 9, 2019

So, the setting is useful not only for HA environment. How about moving CriticalConnection= from [DHCP] section to [Network] section, and introduce CriticalConnection=static (though the name static may be misleading, but I have no idea)?

  • CriticalConnection=yes: do not drop configs on start, and DHCP renew.
  • CriticalConnection=static: do not drop configs on start, but DHCP addresses are dropped on renew.
  • CriticalConnection=no: drop configs on start.
@ssahani

This comment has been minimized.

Copy link
Contributor Author

commented May 9, 2019

We can't move CriticalConnection= to [Network] section because of backward compatibility. We actually do not allow any other servers to configure same interface with networkd because they may cause lots of misconfiguration. But this case makes a tons of sense and describes the purpose HA. We might want to rename but not sure.

@yuwata

This comment has been minimized.

Copy link
Member

commented May 9, 2019

We can't move CriticalConnection= to [Network] section because of backward compatibility.

Of course, CriticalConnection= in [DHCP] still needs to be supported.

We actually do not allow any other servers to configure same interface with networkd because they may cause lots of misconfiguration. But this case makes a tons of sense and describes the purpose HA. We might want to rename but not sure.

I think the name CriticalConnection= already suggests it is something special. Of course, the man page needs to warn about that.

Anyway, apart from the name of the setting, I like this change.

@johannbg You do not need to wait the patch to be backported. Setting CriticalConnection=yes in [DHCP] section (even if you do not use DHCP) should work.

@yuwata

This comment has been minimized.

Copy link
Member

commented May 9, 2019

One reason I do not like the name HighAvailability= is that the setting does not provide HA, but just for HA systems.

@ssahani

This comment has been minimized.

Copy link
Contributor Author

commented May 9, 2019

AllowHighAvailability= ?

@johannbg

This comment has been minimized.

Copy link
Contributor

commented May 9, 2019

@yuwata I'm in midst of setting up test ipv4/ipv6 environment on F30 networkd only setup to test the classic HAProxy + Keepalived setup so I'll try it out but this will need to be backported regardless.

@ssahani Presumably in the future HA application stacks would want to have networkd manage the network settings ( and associated sysctl settings ) for them? ( I would not be surprised if there is very little things missing from networkd to do that )

If so it would need to be something that goes nicely with the rest of the [Network] section so something like below instead of HighAvailability= ( I personally think HighAvailability= is quite self explanatory thou )

Redundant=
Primary=/Secondary=
VIP=
FIP=
VirtualIP=
FloatingIP=
VirtualAddresss=
FloatingAddress=

@ssahani

This comment has been minimized.

Copy link
Contributor Author

commented May 9, 2019

If so it would need to be something that goes nicely with the rest of the [Network] section so something like below instead of HighAvailability= ( I personally think HighAvailability= is quite self explanatory thou )
Redundant=
Primary=/Secondary=
VIP=
FIP=
VirtualIP=
FloatingIP=
VirtualAddresss=
FloatingAddress=

Let's just make it generic and something comes in future also we need not add / remove stuff. Other application which have same requirements and we can't just keep on adding them no ?

@johannbg

This comment has been minimized.

Copy link
Contributor

commented May 9, 2019

Redundant=True is probably the most generic terminology which would cover HA amongst other things.
networkd probably requires an capability-based IPC support (dbus/kdbus/bus1 whatever ) before applications and application stacks can make full use of what networkd is capable of and make such future become a reality.

@ssahani

This comment has been minimized.

Copy link
Contributor Author

commented May 10, 2019

Redundant=True is probably the most generic terminology which would cover HA amongst other things.

Well I am not sure @yuwata WDUT?

networkd probably requires an capability-based IPC support (dbus/kdbus/bus1 whatever ) before applications and application stacks can make full use of what networkd is capable of and make such future become a reality.

yeah varlink just went in and we have dbus stuff to do.

pqarmitage added a commit to pqarmitage/keepalived that referenced this pull request May 13, 2019

Add support for use_ipvlan (use an ipvlan i/f similar to use_vmac)
Issue acassen#1170 identified that use_vmac didn't work with systemd-networkd
since systemd-networkd was removing IP addresses created by keepalived
(and any other application). It was discovered that systemd-networkd
did not remove IP addresses from ipvlans.

This commit adds support for ipvlans, but to work around the problem,
and because it might have other uses.

Systemd commit - systemd/systemd#12511 has added
configuration options to stop systemd-networkd removing IP addresses
added by other applications, but it is not merged yet, and it will be a
while before all the distros merge it.

Signed-off-by: Quentin Armitage <quentin@armitage.org.uk>
@chr4

This comment has been minimized.

Copy link

commented May 13, 2019

In case it helps:

In my tests (although I'm using netplan), setting critical: true (which sets CriticalConnection=True according to the source) did not prevent purging the addresses upon a systemctl restart systemd-networkd.

network:
    version: 2
    ethernets:
        ens3:
            dhcp4: true
            critical: true
ip a a 1.2.3.4/32 dev eth0
systemctl restart systemd-networkd
ip a |grep 1.2.3.4

(Sorry, had no time to reproduce this using plain systemd-networkd.)

networkd: Allow networkd to work with keepalived / highavialbility
This looks pretty legit. networkd drops the foreign addresses upon
restart. This becomes a problem in cases where the additional address
is used as a virtual or floating IP in higih-available environments
(where a master IP is swapped to a different service upon failovers,
e.g. by services like Pacemaker, keepalived and alike, usually using
protocols like Corosync, VRRP or CARP). Those addresses are not handled
by systemd-networkd and are therefore purged upon DHCP renewals or
when an update triggers a restart of systemd-networkd, therefore
potentially resulting in downtimes or at least unecessary failovers.

closes #12050

@ssahani ssahani force-pushed the ssahani:high-avilability-12050 branch from be834e4 to b0fa0b4 May 14, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.