Skip to content

InstallPMTUD

Jump to bottom
Jason Fesler edited this page Oct 27, 2016 · 23 revisions

mtu1280d

Forcing clients to validate connections through "tunneled" connections and verifying PMTUD works

Note that this is a required feature for all mirrors.

A new feature (Feb 2015) of falling-sky is to do a more aggressive "large packet" test. A new daemon (coupled with linux ip6tables rules) can create an artificial "Packet Too Big" message, to aggressive test any client who has anything larger than the minimum MTU size of 1280.

**This works on Linux, tested on Ubuntu 12 through 16, Centos 5 through 7. Later versions are expected to work as well, subject to crazy changes like "let's move everything to systemd" (I'm very much afraid of whatever comes next!).

To do this test, we need additional work to a base install of falling-sky:

  • Additional IPv6 address allocated (as "deprecated", preferred_lft 0)
  • Installed on your primary host
  • DNS name: mtu1280.test-ipv6.example.com (AAAA records only)
  • Firewalls, if any, permit port 80 to this new IP address
  • Host based firewall: ip6tables installed with custom NFQUEUE rule
  • New daemon listening (mtu1280d) to the NFQUEUE rule.

The benefit to all of this is not having to run an artificial IPv6 tunnel just to drop the MTU size; and not relying on the tunnel provider to send enough ICMPv6 "Packet Too Big" messages to clients.

Host Configuration

  • Allocate IPv6 address for your VM's subnet
  • Edit /etc/rc.local to configure the IPv6 address as a deprecated address. This will allow for the new address to be used for incoming connections; but stop from using it for outgoing connections.
ip -6 addr add 2001:db8:1:18::1280/64 dev eth0 preferred_lft 0
  • Make sure the name is similar to your existing falling-sky hostnames, only starting with "mtu1280". For example, "mtu1280.test-ipv6.example.com" for people running as a subdomain off of example.com.
  • Run the command above manually; or reboot to truly validate your /etc/rc.local changes
  • Make sure this new address is ping6-able from another host.

[Thanks to John Mann for https://github.com/falling-sky/mtu1280d/issues/1 ]

Apache configuration

  • First check to see if http://mtu1280.test-ipv6.example.com works. If this works, you can skip the Apache configuration. Otherwise..
  • Some people have their apache configuration Listen to explicit IP addresses. If this is the case for you, be sure to add the additional Listen statement.
  • You'll need to make sure that there is a new "Listen" statement that listens to your new IPv6 address.
  • You'll need to find your existing VirtualHost configuration; and add "ServerAlias 2001:db8:1:18::1280" (substitute your new IPv6 address)
  • Restart apache
  • Make sure that curl http://mtu1280.test-ipv6.example.com/ip/ works . If not, debug this before proceeding.

mtu120d Installation

rsync -av fsky@rsync.test-ipv6.com:stable/mtu1280d .
cd mtu1280d
less README.md   # Check for prerequisites
make mtu1280d
sudo make install upstart

Depending on your platform, you may need to use sudo make install init.d instead.

At this point you'll need to start the daemon. One of these should work:

  • sudo start mtu1280d (upstart)
  • `sudo /etc/init.d/mtu1280d start" (init.d)

Make sure the daemon is listening. Use "ps". init.d users may see an additional "-d" flag in the command line.

guest% ps auxww |grep mtu1280d
root     28932  0.0  0.0  10560   136 ?        Ss   13:28   0:00 /usr/sbin/mtu1280d -q 1280
jfesler  28940  0.0  0.0  11744   884 pts/1    S+   13:28   0:00 grep mtu1280d

Update ip6tables

This is something that is very site specific. You'll need to identify how your OS will persistently keep (and manage) iptables and ip6tables rules.

  • Debian/Ubuntu: consider the "iptables-persistent" package, which runs iptables-restore /etc/iptables/rules.v4 and ip6tables-restore /etc/iptables/rules.v6 on boot.
  • RedHat/Centos: consider enabling the "iptables" and "ip6tables" services; these will on boot run iptables-restore /etc/sysconfig/iptables and ip6tables-restore /etc/sysconfig/ip6tables

Example iptables rules

You're going to need to add 1 rule to the PREROUTING chain of your "mangle" table.
This will route all packets to your mtu1280 host IP to the mtu1280d daemon. This daemon will either ACCEPT or DROP packets based on the size; as well as generate the ICMPv6 Packet Too Big responses. ACCEPT at this point does not actually affect later filter rules, since this is still in PREROUTING.

ip6tables -t mangle -A PREROUTING -d 2001:db8:1:18::1280/128 -j NFQUEUE --queue-num 1280

Adjust the IPv6 address to match what you assigned to your host.

This is a complete (but overly simple) example:

# Generated by ip6tables-save v1.4.21 on Wed Feb 18 10:14:54 2015
*mangle
:PREROUTING ACCEPT [0:0]
:INPUT ACCEPT [0:0]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
:POSTROUTING ACCEPT [0:0]
-A PREROUTING -d 2001:470:1:18::1280/128 -j NFQUEUE --queue-num 1280
COMMIT
# Completed on Wed Feb 18 10:14:54 2015
# Generated by ip6tables-save v1.4.21 on Wed Feb 18 10:14:54 2015
*filter
:INPUT ACCEPT [0:0]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
:CHECK_ABUSE - [0:0]
COMMIT
# Completed on Wed Feb 18 10:14:54 2015

Confirmation

Confirmation requires a separate IPv6 enabled host, that can run "curl" and "tcpdump". Linux can work; I've also used my Mac.

linux ping6

The fastest way to test this is with a Linux version of ping6 that supports the "-M" option.

You need to do this from a **different Linux host **than your mirror.

% ping6 -c 3 -M dont -s 1400 mtu1280.test-ipv6.example.com
PING mtu1280.test-ipv6.example.com(mtu1280.test-ipv6.example.com) 1400 data bytes
From mtu1280.master.test-ipv6.com icmp_seq=1 Packet too big: mtu=1280
From mtu1280.master.test-ipv6.com icmp_seq=2 Packet too big: mtu=1280
From mtu1280.master.test-ipv6.com icmp_seq=3 Packet too big: mtu=1280

--- mtu1280.test-ipv6.example.com ping statistics ---
3 packets transmitted, 0 received, +3 errors, 100% packet loss, time 2009ms

MacOS ping6

Tested on OS X 10.11 - El Capitan. Must use -v to see the Packet Too Big messages. Must use -m to avoid fragmenting. Must use sudo.

Jasons-MacBook:home jfesler$ sudo ping6 -s 1400 -v  -m mtu1280.test-ipv6.com
PING6(1448=40+8+1400 bytes) 2601:647:4900:78ae:25c9:d601:c352:a669 --> 2001:470:1:18::1281
new path MTU (1280) is notified
1240 bytes from 2001:470:1:18::1281: Packet too big mtu = 1280
Vr TC  Flow Plen Nxt Hlim
 6 00 af090 0580  3a   36
2601:647:4900:78ae:25c9:d601:c352:a669->2001:470:1:18::1281
ICMP6: type = 128, code = 0

Online validator

You can use the online Validation tool to confirm all parts of your mirror operate the way they are expected to. This includes testing for a full MTU of 1500 on your primary address; and a lower MTU of 1280 on the PMTUD test address.

tcpdump

This is the most thorough way to do this. This will allow you to actually observe what happens when large packets come in; and how TCP responds.

Run this on your client host. This must be a different host than your mirror runs on. Adjust the IPv6 address to match your new mtu1280 IPv6 address.

sudo tcpdump -i eth0 -vvv 'host 2001:db8:1:18::1280 or (ip6 and icmp6 and ip6[40] = 2)'

On the same client host, run this curl command. Adjust the hostname to match your site.

curl
   'http://mtu1280.test-ipv6.example.com/ip/?size=1600&fill=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

You should see, the first time you run it,

 laptop:~ jfesler$ sudo tcpdump 'host 2001:db8:1:18::1280 or (ip6 and icmp6 and ip6[40] = 2)' 
 Password:
 tcpdump: data link type PKTAP
 tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
 listening on pktap, link-type PKTAP (Packet Tap), capture size 65535 bytes
 09:03:57.697254 IP6 2001:db8:effd::1234.53649 > 2001:db8:1:18::1280.http: Flags [S], seq 2919600708, win 65535, options [mss 1440,nop,wscale 5,nop,nop,TS val 326886165 ecr 0,sackOK,eol], length 0
 09:03:57.702436 IP6 2001:db8:1:18::1280.http > 2001:db8:effd::1234.53649: Flags [S.], seq 1207337852, ack 2919600709, win 28160, options [mss 1366,sackOK,TS val 325898144 ecr 326886165,nop,wscale 7], length 0
 09:03:57.702469 IP6 2001:db8:effd::1234.53649 > 2001:db8:1:18::1280.http: Flags [.], ack 1, win 4104, options [nop,nop,TS val 326886170 ecr 325898144], length 0
 09:03:57.702534 IP6 2001:db8:effd::1234.53649 > 2001:db8:1:18::1280.http: Flags [.], seq 1:1355, ack 1, win 4104, options [nop,nop,TS val 326886170 ecr 325898144], length 1354
 09:03:57.702540 IP6 2001:db8:effd::1234.53649 > 2001:db8:1:18::1280.http: Flags [P.], seq 1355:1805, ack 1, win 4104, options [nop,nop,TS val 326886170 ecr 325898144], length 450
 09:03:57.710307 IP6 2001:db8:1:18::1280 > 2001:db8:effd::1234: ICMP6, packet too big, mtu 1280, length 1240
 09:03:57.710433 IP6 2001:db8:effd::1234.53649 > 2001:db8:1:18::1280.http: Flags [.], seq 1:1209, ack 1, win 4104, options [nop,nop,TS val 326886170 ecr 325898144], length 1208
 09:03:57.710457 IP6 2001:db8:effd::1234.53649 > 2001:db8:1:18::1280.http: Flags [P.], seq 1209:1805, ack 1, win 4104, options [nop,nop,TS val 326886170 ecr 325898144], length 596
 09:03:57.714993 IP6 2001:db8:1:18::1280.http > 2001:db8:effd::1234.53649: Flags [.], ack 1209, win 239, options [nop,nop,TS val 325898147 ecr 326886170], length 0
 09:03:57.715904 IP6 2001:db8:1:18::1280.http > 2001:db8:effd::1234.53649: Flags [.], ack 1805, win 258, options [nop,nop,TS val 325898147 ecr 326886170], length 0
 09:03:57.719508 IP6 2001:db8:1:18::1280.http > 2001:db8:effd::1234.53649: Flags [.], seq 1:1355, ack 1805, win 258, options [nop,nop,TS val 325898148 ecr 326886170], length 1354
 09:03:57.720537 IP6 2001:db8:1:18::1280.http > 2001:db8:effd::1234.53649: Flags [P.], seq 1355:1871, ack 1805, win 258, options [nop,nop,TS val 325898148 ecr 326886170], length 516
 09:03:57.720605 IP6 2001:db8:effd::1234.53649 > 2001:db8:1:18::1280.http: Flags [.], ack 1871, win 4079, options [nop,nop,TS val 326886187 ecr 325898148], length 0
 09:03:57.721019 IP6 2001:db8:effd::1234.53649 > 2001:db8:1:18::1280.http: Flags [F.], seq 1805, ack 1871, win 4096, options [nop,nop,TS val 326886187 ecr 325898148], length 0
 09:03:57.725160 IP6 2001:db8:1:18::1280.http > 2001:db8:effd::1234.53649: Flags [F.], seq 1871, ack 1806, win 258, options [nop,nop,TS val 325898150 ecr 326886187], length 0
 09:03:57.725218 IP6 2001:db8:effd::1234.53649 > 2001:db8:1:18::1280.http: Flags [.], ack 1872, win 4096, options [nop,nop,TS val 326886191 ecr 325898150], length 0
 ^C
 16 packets captured
 423 packets received by filter
 0 packets dropped by kernel
 laptop:~ jfesler$

The "packet too big" will be in response to the first TCP packet to that host where the payload length is more than 1280.

The second time you run the curl command, you'll see no packet too big - but you'll also see no packets larger than 1280 bytes. This is due to your client host remembering the MTU. The length of time this is remembered varies from OS to OS.

Final Browser Test

  • Point your browser at your mirror site.
  • Shift-Reload the browser (to clear out any cached /site/config.js)
  • Go to "Tests Run"
  • Go to "Technical Info"
  • Verify that "Test IPv6 large packet" shows "mtu1280.test-ipv6.domain.com" and success
Clone this wiki locally