Skip to content

Production Network

dhellstern edited this page Feb 20, 2022 · 24 revisions

Introduction

For the final DN42 separate, there are two main networks, the Management and the DN42 networks. The management network provides supporting services (mainly ROA) to the router, but does not expose them to the DN42 network. The management network also does not make use of DN42 address space.

Management

The management network uses the address space 172.19.255.240/28, which precedes the address space used by DN42, 172.20.0.0/14. The servers on this network have internet connectivity.

Hostname Description IP Address
- Router 172.19.255.241
roa-1 Primary RTR cache server 172.19.255.242
roa-2 Fallback RTR cache server 172.19.255.243
mgmt Management VM 172.19.255.254

For security reasons, no servers on the management network are able to reach any Champlain addresses, these have been specifically blocked in the firewall config. Management is performed via the mgmt VM, accessed through the Proxmox console.

DN42

The DN42 network is actually comprised of two separate networks, one providing services, and one used to access the network without using NAT. These networks uses the address space from AS4242421878: 172.23.119.64/27 and fde6:a552:874f::/48.

Services

The services network uses 172.23.119.64/28 (the first half of the IPv4 allocation) and fde6:a552:874f:1c2d::/64. EUI-64 is used for predictable address generation based on router advertisements.

Hostname Description IPv4 Address
- Router 172.23.119.65
ns-1 Primary DNS server 172.23.119.66
ns-2 Fallback DNS server 172.23.119.67
lg BGP Looking Glass 172.23.119.78

For security reasons, servers on this network do not have any internet access, only access to DN42. For maintenance, they may be temporarily connected to the management network so they have internet connectivity.

Access

The access network uses 172.23.119.80/28 (the second half of the IPv4 allocation) and fde6:a552:874f:9a3b::/64. It is accessible via Wireguard on standard port 51820/udp, providing access to DN42. Due to this network being directly connected to DN42, it is advised that all clients should be running a firewall on their Wireguard interface.

Firewalls

Access to each network is enforced using a VyOS zone-based firewall. Each of the above networks encompasses an entire zone, plus the WAN and LOCAL zones, for the WAN interface and router itself, respectively. The DN42 zone covers the local DN42 network as well as all peer interfaces, since there isn't a need to have firewalling between peers.

Zones

  • WAN
  • LOCAL
  • MGMT
  • DN42

Rulesets

Note that zone pairs not listed below are simply not connected and thereby cannot transit any traffic.

Name Zone Pairs Default Rules
ALLOW LOCAL > * Accept -
DENY WAN > MGMT, DN42 > MGMT Drop Accept established/related
MGMT_LOCAL MGMT > LOCAL Drop Accept established/related, Accept ICMP, Accept SSH
MGMT_WAN MGMT > WAN Accept Drop Champlain addresses
MGMT_DN42 MGMT > DN42 Drop Accept to AS4242421878
DN42_LOCAL DN42 > LOCAL Drop Accept established/related, Accept ICMP, Accept BGP
WAN_LOCAL WAN > LOCAL Drop Accept established/related, Accept ICMP, Accept primary Wireguard (51820/udp), Accept peer Wireguard (20000-23999/udp), Accept GRE

NAT is implemented for the pair MGMT > WAN.

Invalid States and rp_filter

Note that the firewall must not drop DN42 packets with invalid state under any circumstances, and should usually explicitly allow them. This is because packets transitting the router may be part of a connection taking separate paths in each direction (asymmetric routing), and so dropping invalid states will break routing through the firewall in some cases. See the Universal Network Rules for more info.

Another similar limitation applies from the network rules, which is that source validation should be disabled. This security measure rejects packets that return on a different interface than the outgoing connection departed from. It can be disabled following the instructions here.

Clone this wiki locally