Route configuration cleanup by nmeyerhans · Pull Request #76 · amazonlinux/amazon-ec2-net-utils

nmeyerhans · 2022-10-07T02:24:12Z

Issue #, if available: n/a

Description of changes:

This change contains two minor fixes to our routing configuration:

Leave a gap in the route metrics for each interface. This gives the option to override our configuration with custom routing, if that should be desirable for some reason.
Don't install unnecessary route configuration for the main table. Because we're no longer creating a device-specific table for the "primary" interface, we don't need to define custom routing; the systemd-networkd defaults will use the main table.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

After a8580fa ("Skip routing rules for default mac interface"), we no longer create a secondary routing table for the primary interface, so we no longer need to emit configuration to install routes into that table. Following this change, the primary ENI (that with device-number=0) will get an eni.conf similar to: [Match] MACAddress=06:4b:8d:43:93:59 [Network] DHCP=yes [DHCPv4] RouteMetric=512 [DHCPv6] RouteMetric=512 While a secondary ENI would continue to get additional stanzas creating routes in the secondary routing table, as in: [Match] MACAddress=06:8a:d8:09:9c:bf [Network] DHCP=yes [DHCPv4] RouteMetric=513 [DHCPv6] RouteMetric=513 [Route] Table=10001 Gateway=_ipv6ra [DHCPv4] RouteTable=10001 [IPv6AcceptRA] RouteTable=10001 [Route] Gateway=_dhcp4 Table=10001

Instead of incrementing metric values by 1 per interface, increment by 10. This leaves room between interfaces for custom configuration that may want to override our routing configuration.

bsmojver · 2022-10-07T03:30:05Z

A c6g.large instance, 3 ENIs, two with IPv6 enabled:

[ec2-user@ip-172-31-11-54 ~]$ ip addr list
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: ens5: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9001 qdisc mq state UP group default qlen 1000
    link/ether 02:60:cb:cf:19:3e brd ff:ff:ff:ff:ff:ff
    altname enp0s5
    altname eni-081b08c5b7f7520bb
    altname device-number-0
    inet 172.31.11.54/20 metric 1024 brd 172.31.15.255 scope global dynamic ens5
       valid_lft 3304sec preferred_lft 3304sec
    inet6 2406:da1c:ac4:600:5548:6d5d:efb5:4288/128 scope global dynamic noprefixroute 
       valid_lft 428sec preferred_lft 118sec
    inet6 fe80::60:cbff:fecf:193e/64 scope link 
       valid_lft forever preferred_lft forever
3: ens6: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9001 qdisc mq state UP group default qlen 1000
    link/ether 02:65:39:07:6e:66 brd ff:ff:ff:ff:ff:ff
    altname enp0s6
    altname eni-0cfc5325a9d5d121e
    altname device-number-1
    inet 172.31.64.97/24 metric 1024 brd 172.31.64.255 scope global dynamic ens6
       valid_lft 3413sec preferred_lft 3413sec
    inet6 fe80::65:39ff:fe07:6e66/64 scope link 
       valid_lft forever preferred_lft forever
4: ens7: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9001 qdisc mq state UP group default qlen 1000
    link/ether 02:69:db:87:0c:9a brd ff:ff:ff:ff:ff:ff
    altname enp0s7
    altname eni-00b5982e3bb2098e2
    altname device-number-2
    inet 172.31.5.127/20 metric 1024 brd 172.31.15.255 scope global dynamic ens7
       valid_lft 3418sec preferred_lft 3418sec
    inet6 2406:da1c:ac4:600:ce09:8a54:bb25:b597/128 scope global dynamic noprefixroute 
       valid_lft 408sec preferred_lft 98sec
    inet6 fe80::69:dbff:fe87:c9a/64 scope link 
       valid_lft forever preferred_lft forever

[ec2-user@ip-172-31-11-54 ~]$ ip route list
default via 172.31.0.1 dev ens5 proto dhcp src 172.31.11.54 metric 512 
172.31.0.0/20 dev ens5 proto kernel scope link src 172.31.11.54 metric 1024 
172.31.0.0/20 dev ens7 proto kernel scope link src 172.31.5.127 metric 1024 
172.31.0.1 dev ens5 proto dhcp scope link src 172.31.11.54 metric 512 
172.31.0.2 dev ens5 proto dhcp scope link src 172.31.11.54 metric 512 
172.31.64.0/24 dev ens6 proto kernel scope link src 172.31.64.97 metric 1024 

[ec2-user@ip-172-31-11-54 ~]$ ip rule list
0:	from all lookup local
10001:	from 172.31.64.97 lookup 10001 proto static
10002:	from 172.31.5.127 lookup 10002 proto static
32766:	from all lookup main
32767:	from all lookup default

[ec2-user@ip-172-31-11-54 ~]$ ip route list table 10001
default via 172.31.64.1 dev ens6 proto dhcp src 172.31.64.97 metric 522 
default via 172.31.64.1 dev ens6 proto dhcp metric 522 
172.31.0.2 via 172.31.64.1 dev ens6 proto dhcp src 172.31.64.97 metric 522 
172.31.64.0/24 dev ens6 proto dhcp scope link src 172.31.64.97 metric 522 
172.31.64.1 dev ens6 proto dhcp scope link src 172.31.64.97 metric 522 

[ec2-user@ip-172-31-11-54 ~]$ ip route list table 10002
default via 172.31.0.1 dev ens7 proto dhcp src 172.31.5.127 metric 532 
default via 172.31.0.1 dev ens7 proto dhcp metric 532 
172.31.0.0/20 dev ens7 proto dhcp scope link src 172.31.5.127 metric 532 
172.31.0.1 dev ens7 proto dhcp scope link src 172.31.5.127 metric 532 
172.31.0.2 dev ens7 proto dhcp scope link src 172.31.5.127 metric 532

Ping tests to public IPs, two of them EIPs:

[smojb@dev-dsk-smojb-2b-38748cd8 ~]$ ping -c 1 54.66.9.91
PING 54.66.9.91 (54.66.9.91) 56(84) bytes of data.
64 bytes from 54.66.9.91: icmp_seq=1 ttl=106 time=140 ms

--- 54.66.9.91 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 140.822/140.822/140.822/0.000 ms

$ ping -c 1 13.237.118.79
PING 13.237.118.79 (13.237.118.79) 56(84) bytes of data.
64 bytes from 13.237.118.79: icmp_seq=1 ttl=107 time=140 ms

--- 13.237.118.79 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 140.562/140.562/140.562/0.000 ms

$ ping -c 1 52.63.56.228
PING 52.63.56.228 (52.63.56.228) 56(84) bytes of data.
64 bytes from 52.63.56.228: icmp_seq=1 ttl=106 time=139 ms

--- 52.63.56.228 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 139.924/139.924/139.924/0.000 ms

Same against two interfaces with IPv6 enabled, using online tools:

PING 2406:da1c:ac4:600:5548:6d5d:efb5:4288(2406:da1c:ac4:600:5548:6d5d:efb5:4288) 56 data bytes
64 bytes from 2406:da1c:ac4:600:5548:6d5d:efb5:4288: icmp_seq=1 ttl=48 time=137 ms
64 bytes from 2406:da1c:ac4:600:5548:6d5d:efb5:4288: icmp_seq=2 ttl=48 time=137 ms
64 bytes from 2406:da1c:ac4:600:5548:6d5d:efb5:4288: icmp_seq=3 ttl=48 time=137 ms
64 bytes from 2406:da1c:ac4:600:5548:6d5d:efb5:4288: icmp_seq=4 ttl=48 time=137 ms
64 bytes from 2406:da1c:ac4:600:5548:6d5d:efb5:4288: icmp_seq=5 ttl=48 time=137 ms

--- 2406:da1c:ac4:600:5548:6d5d:efb5:4288 ping statistics ---
5 packets transmitted, 5 received, 0% packet loss, time 4005ms
rtt min/avg/max/mdev = 137.090/137.186/137.335/0.475 ms

PING 2406:da1c:ac4:600:ce09:8a54:bb25:b597(2406:da1c:ac4:600:ce09:8a54:bb25:b597) 56 data bytes
64 bytes from 2406:da1c:ac4:600:ce09:8a54:bb25:b597: icmp_seq=1 ttl=48 time=137 ms
64 bytes from 2406:da1c:ac4:600:ce09:8a54:bb25:b597: icmp_seq=2 ttl=48 time=137 ms
64 bytes from 2406:da1c:ac4:600:ce09:8a54:bb25:b597: icmp_seq=3 ttl=48 time=137 ms
64 bytes from 2406:da1c:ac4:600:ce09:8a54:bb25:b597: icmp_seq=4 ttl=48 time=137 ms
64 bytes from 2406:da1c:ac4:600:ce09:8a54:bb25:b597: icmp_seq=5 ttl=48 time=137 ms

--- 2406:da1c:ac4:600:ce09:8a54:bb25:b597 ping statistics ---
5 packets transmitted, 5 received, 0% packet loss, time 4006ms
rtt min/avg/max/mdev = 137.479/137.536/137.576/0.236 ms

The actual configuration files:

[root@ip-172-31-11-54 ~]# cat /var/run/systemd/network/70-ens5.network.d/eni.conf
# Configuration for ens5 generated by policy-routes@ens5.service
[Match]
MACAddress=02:60:cb:cf:19:3e
[Network]
DHCP=yes
[DHCPv4]
RouteMetric=512
[DHCPv6]
RouteMetric=512

[root@ip-172-31-11-54 ~]# cat /var/run/systemd/network/70-ens6.network.d/eni.conf
# Configuration for ens6 generated by policy-routes@ens6.service
[Match]
MACAddress=02:65:39:07:6e:66
[Network]
DHCP=yes
[DHCPv4]
RouteMetric=522
[DHCPv6]
RouteMetric=522
[Route]
Table=10001
Gateway=_ipv6ra
[DHCPv4]
RouteTable=10001
[IPv6AcceptRA]
RouteTable=10001
[Route]
Gateway=_dhcp4
Table=10001

[root@ip-172-31-11-54 ~]# cat /var/run/systemd/network/70-ens7.network.d/eni.conf
# Configuration for ens7 generated by policy-routes@ens7.service
[Match]
MACAddress=02:69:db:87:0c:9a
[Network]
DHCP=yes
[DHCPv4]
RouteMetric=532
[DHCPv6]
RouteMetric=532
[Route]
Table=10002
Gateway=_ipv6ra
[DHCPv4]
RouteTable=10002
[IPv6AcceptRA]
RouteTable=10002
[Route]
Gateway=_dhcp4
Table=10002

It all seems be looking good.

nmeyerhans added 2 commits October 6, 2022 14:02

Leave a gap in route metric values

b270e68

Instead of incrementing metric values by 1 per interface, increment by 10. This leaves room between interfaces for custom configuration that may want to override our routing configuration.

nmeyerhans requested a review from keithaws October 7, 2022 02:24

halfdime-code approved these changes Oct 7, 2022

View reviewed changes

nmeyerhans merged commit 477640f into amazonlinux:main Oct 7, 2022

nmeyerhans deleted the route-cleanup branch October 7, 2022 20:59

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Route configuration cleanup#76

Route configuration cleanup#76
nmeyerhans merged 2 commits into
amazonlinux:mainfrom
nmeyerhans:route-cleanup

nmeyerhans commented Oct 7, 2022

Uh oh!

bsmojver commented Oct 7, 2022 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

nmeyerhans commented Oct 7, 2022

Uh oh!

bsmojver commented Oct 7, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

bsmojver commented Oct 7, 2022 •

edited

Loading