Skip to content

rds-conn-kref-base-v1

tagged this 18 Jul 16:09
rds_recv_incoming() drops a message with an old sequence number only
when it carries RDS_FLAG_RETRANSMITTED; any other message is accepted
and cp_next_rx_seq is unconditionally reset to its sequence number
plus one.  Accepting the unflagged-old case is deliberate: it lets a
receiver follow a restarted peer, whose new instance numbers its
stream from 1 again, below whatever the old stream reached.  TCP can
detect a restart through the generation number extension header, but
IB has no equivalent, so the rewind is the only restart recovery the
protocol has there.

Accepting an unflagged old sequence number at *any* time, however,
makes the check self-disarming: one duplicated or misdirected packet
is not only delivered to the application a second time, it also drags
cp_next_rx_seq backwards - after which the sender's flagged
retransmission window arrives with sequence numbers at or above the
rewound cp_next_rx_seq, evades the check entirely, and is re-delivered
wholesale.  One stray packet turns into thousands of duplicate
deliveries.  This was observed in practice with the rds
connection-reset selftests on rxe, where a soft-RoCE responder bug
(fixed separately) re-executed stale packets into a recycled QPN under
reset churn.

A peer restart is always preceded by a transport reconnect on the
receiving path, so restrict the rewind to that window: arm a one-shot
RDS_RX_REWIND_ALLOWED bit when a path is torn down - in
rds_conn_path_reset(), and in rds_tcp_reset_callbacks() for rds_tcp's
socket-swap reconnect which bypasses it - and honor an unflagged old
sequence number only while the bit is set.  The first message accepted
on the path consumes the bit.  A restarted peer still gets its rewind
on the first message of the fresh stream, while a mid-stream duplicate
is now dropped (counted in s_recv_drop_old_seq, like the flagged case)
and can no longer move cp_next_rx_seq backwards.

Flagged retransmits are dropped exactly as before.  With a
well-behaved fabric the unflagged-old case never occurs mid-stream, so
this is hardening against misbehaving hardware/drivers rather than a
fix for a protocol-level bug; hence no Fixes tag.

Assisted-by: Claude-Code:claude-fable-5
Signed-off-by: Allison Henderson <achender@kernel.org>
Assets 2
Loading