Two IP addresses set for vagrant libvirt centos/7 box

[font]

When testing with a vagrant libvirt centos/7 box, the instances receive their static IP addresses as defined in the Vagrantfile, and also receive another dynamic IP address. That is, when using the ip addr command you can see that each static IP private-network interface contains 2 addresses - the static one, and the dynamic one provided via DHCP.

Here is an example of a mon host:

[root@ceph-mon0 ~]# ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 52:54:00:48:08:cf brd ff:ff:ff:ff:ff:ff
    inet 192.168.121.66/24 brd 192.168.121.255 scope global dynamic eth0
       valid_lft 3445sec preferred_lft 3445sec
    inet6 fe80::5054:ff:fe48:8cf/64 scope link 
       valid_lft forever preferred_lft forever
3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 52:54:00:d2:ff:f6 brd ff:ff:ff:ff:ff:ff
    inet 192.168.125.198/24 brd 192.168.125.255 scope global dynamic eth1
       valid_lft 3442sec preferred_lft 3442sec
    inet 192.168.125.10/24 brd 192.168.125.255 scope global secondary eth1
       valid_lft forever preferred_lft forever
    inet6 fe80::5054:ff:fed2:fff6/64 scope link 
       valid_lft forever preferred_lft forever
[root@ceph-mon0 ~]# pgrep -a dhclient
664 /sbin/dhclient -d -q -sf /usr/libexec/nm-dhcp-helper -pf /var/run/dhclient-eth1.pid -lf /var/lib/NetworkManager/dhclient-53c5e689-3150-45a1-930d-14f4b7df7bc2-eth1.lease -cf /var/lib/NetworkManager/dhclient-eth1.conf eth1
3720 /sbin/dhclient -d -q -sf /usr/libexec/nm-dhcp-helper -pf /var/run/dhclient-eth0.pid -lf /var/lib/NetworkManager/dhclient-5fb06bd0-0bb0-7ffb-45f1-d6edd65f3e03-eth0.lease -cf /var/lib/NetworkManager/dhclient-eth0.conf eth0

You can see that eth1 for the mon host contains 2 IPv4 addresses, the static one defined in the Vagrantfile and the dynamic address assigned by the dhclient process running for eth1.

And here is an example of an osd host:

[root@ceph-osd0 ~]# ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 52:54:00:a0:fd:e6 brd ff:ff:ff:ff:ff:ff
    inet 192.168.121.172/24 brd 192.168.121.255 scope global dynamic eth0
       valid_lft 3423sec preferred_lft 3423sec
    inet6 fe80::5054:ff:fea0:fde6/64 scope link 
       valid_lft forever preferred_lft forever
3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 52:54:00:51:36:b3 brd ff:ff:ff:ff:ff:ff
    inet 192.168.125.197/24 brd 192.168.125.255 scope global dynamic eth1
       valid_lft 3420sec preferred_lft 3420sec
    inet 192.168.125.100/24 brd 192.168.125.255 scope global secondary eth1
       valid_lft forever preferred_lft forever
    inet6 fe80::5054:ff:fe51:36b3/64 scope link 
       valid_lft forever preferred_lft forever
4: eth2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 52:54:00:6f:a9:5f brd ff:ff:ff:ff:ff:ff
    inet 192.168.126.22/24 brd 192.168.126.255 scope global dynamic eth2
       valid_lft 3417sec preferred_lft 3417sec
    inet 192.168.126.200/24 brd 192.168.126.255 scope global secondary eth2
       valid_lft forever preferred_lft forever
    inet6 fe80::5054:ff:fe6f:a95f/64 scope link 
       valid_lft forever preferred_lft forever
[root@ceph-osd0 ~]# pgrep -a dhclient
659 /sbin/dhclient -d -q -sf /usr/libexec/nm-dhcp-helper -pf /var/run/dhclient-eth2.pid -lf /var/lib/NetworkManager/dhclient-5208d703-b9b9-4574-bc97-8d1cb73444e8-eth2.lease -cf /var/lib/NetworkManager/dhclient-eth2.conf eth2
662 /sbin/dhclient -d -q -sf /usr/libexec/nm-dhcp-helper -pf /var/run/dhclient-eth1.pid -lf /var/lib/NetworkManager/dhclient-e4da965c-a2ac-4ef8-8656-226d5872713a-eth1.lease -cf /var/lib/NetworkManager/dhclient-eth1.conf eth1
3910 /sbin/dhclient -d -q -sf /usr/libexec/nm-dhcp-helper -pf /var/run/dhclient-eth0.pid -lf /var/lib/NetworkManager/dhclient-5fb06bd0-0bb0-7ffb-45f1-d6edd65f3e03-eth0.lease -cf /var/lib/NetworkManager/dhclient-eth0.conf eth0

You can see that eth1 and eth2 for the osd host contains 2 IPv4 addresses, the static one defined in the Vagrantfile and the dynamic address assigned by the dhclient processes running for eth1 and eth2.

You can see that the actual ifcfg-ethX script is properly setup for the static IP assignment, but the dhclient process is still launched:

[root@ceph-mon0 ~]# cat /etc/sysconfig/network-scripts/ifcfg-eth1 
#VAGRANT-BEGIN
# The contents below are automatically generated by Vagrant. Do not modify.
NM_CONTROLLED=no
BOOTPROTO=none
ONBOOT=yes
IPADDR=192.168.125.10
NETMASK=255.255.255.0
DEVICE=eth1
HWADDR=52:54:00:d2:ff:f6
PEERDNS=no
#VAGRANT-END

A possible workaround for this is to vagrant halt and then vagrant up. The subsequent vagrant up will not launch the dhclient process and instead use the proper configuration that's been stored in /etc/sysconfig/network-scripts/ifcfg-ethX. This results in only the static IP assignment.

In any case, this doesn't appear to impact anything as both static and dynamic IP addresses can still be used to communicate. But I wanted to file an issue in case anybody encounters this.

This is a known issue and is discussed in more detail here:
vagrant-libvirt/vagrant-libvirt#312