MP_FAIL support

[matttbe]

As defined in RFC8684:

When multiple subflows are in use, the data in flight on a subflow will likely involve data that is not contiguously part of the connection-level stream, since segments will be spread across the multiple subflows. Due to the problems identified above, it is not possible to determine what adjustments have been done to the data (notably, any changes to the subflow sequence numbering). Therefore, it is not possible to recover the subflow, and the affected subflow must be immediately closed with a RST that includes an MP_FAIL option (Figure 16), which defines the data sequence number at the start of the segment (defined by the Data Sequence Mapping) that had the checksum failure. Note that the MP_FAIL option requires the use of the full 64-bit sequence number, even if 32-bit sequence numbers are normally in use in the DSS signals on the path.

                       1                   2                   3
   0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
  +---------------+---------------+-------+----------------------+
  |     Kind      |   Length=12   |Subtype|      (reserved)      |
  +---------------+---------------+-------+----------------------+
  |                                                              |
  |                 Data Sequence Number (8 octets)              |
  |                                                              |
  +--------------------------------------------------------------+

Figure 16: Fallback (MP_FAIL) Option

The receiver of this option MUST discard all data following the data sequence number specified. Failed data MUST NOT be DATA_ACKed and so will be retransmitted on other subflows (Section 3.3.6).
A special case is when there is a single subflow and it fails with a checksum error. If it is known that all unacknowledged data in flight is contiguous (which will usually be the case with a single subflow), an infinite mapping can be applied to the subflow without the need to close it first, essentially turning off all further MPTCP signaling. In this case, if a receiver identifies a checksum failure when there is only one path, it will send back an MP_FAIL option on the subflow-level ACK, referring to the data-level sequence number of the start of the segment on which the checksum error was detected. The sender will receive this information and, if all unacknowledged data in flight is contiguous, will signal an infinite mapping. This infinite mapping will be a DSS option (Section 3.3) on the first new packet, containing a Data Sequence Mapping that acts retroactively, referring to the start of the subflow sequence number of the most recent segment that was known to be delivered intact (i.e., was successfully DATA_ACKed). From that point onward, data can be altered by a middlebox without affecting MPTCP, as the data stream is equivalent to a regular, legacy TCP session. While in theory paths may only be damaged in one direction -- and the MP_FAIL signal affects only one direction of traffic -- for simplicity of implementation, the receiver of an MP_FAIL MUST also respond with an MP_FAIL in the reverse direction and entirely revert to a regular TCP session.
In the rare case that the data is not contiguous (which could happen when there is only one subflow but it is retransmitting data from a subflow that has recently been uncleanly closed), the receiver MUST close the subflow with a RST with MP_FAIL. The receiver MUST discard all data that follows the data sequence number specified. The sender MAY attempt to create a new subflow belonging to the same connection and, if it chooses to do so, SHOULD immediately place the single subflow in single-path mode by setting an infinite Data Sequence Mapping. This mapping will begin from the data-level sequence number that was declared in the MP_FAIL.
After a sender signals an infinite mapping, it MUST only use subflow ACKs to clear its send buffer. This is because Data ACKs may become misaligned with the subflow ACKs when middleboxes insert or delete data. The receiver SHOULD stop generating Data ACKs after it receives an infinite mapping.

https://www.rfc-editor.org/rfc/rfc8684.html#name-fallback-mp_fail-option

If needed, new tickets can be created to track: receiver/sender side only and packetdrill support.

(Feature from the initial roadmap)

[geliangtang]

Note that if checksum usage has not been negotiated, this fallback mechanism cannot be used unless there is some higher-layer or lower‑layer signal to inform the MPTCP implementation that the payload has been tampered with.

[mjmartineau]

mp_join-01-ns1-0-D2LlXf.pcap.gz

Attached pcap (running v6 patch set with @geliangtang test code) shows:

1. The checksum failure is detected, a packet is sent with MP_FAIL.
   However, the packet also has data payload and no DSS option.

2. The peer receives MP_FAIL, and echoes back. But it sends two TCP RST +
   MP_FAIL packets back-to-back.

Cross reference to mailing list: https://lore.kernel.org/mptcp/4dfb9a58-6cdf-c432-ed9-4245a1c57bbd@linux.intel.com/

[matttbe]

Can be closed thanks to @geliangtang 's patches now in our tree:

• 6bf0897: mptcp: MP_FAIL suboption sending
• 913f5ee: mptcp: MP_FAIL suboption receiving
• cce924a: mptcp: send out MP_FAIL when data checksum fails
• e143565: mptcp: add the mibs for MP_FAIL
• 7740516: selftests: mptcp: add MP_FAIL mibs check
• Results: 7706daf..d2d5ed4