[syzkaller] WARNING in subflow_data_ready

[cpaasch]

HEAD is at:

4fb948e08615 ("Cleanup") (HEAD) (3 minutes ago)
55a9c834a69e ("net: mptcp: improve fallback to TCP") (3 minutes ago)
431bc5f80631 ("mptcp: add receive buffer auto-tuning") (3 minutes ago)
e15b65dd24f2 ("bpf: fix unused-var without NETDEVICES") (3 minutes ago)
bc4f114 ("[DO-NOT-MERGE] mptcp: enabled by default") (tag: export/20200605T181020, mptcp_net-next/export) (57 minutes ago)
420e02a ("[DO-NOT-MERGE] mptcp: use kmalloc on kasan build") (57 minutes ago)
36b7954 ("mptcp: don't leak msk in token container") (57 minutes ago)
3c886ec ("mptcp: introduce token KUNIT self-tests") (58 minutes ago)
fa2e5ed ("mptcp: move crypto test to KUNIT") (58 minutes ago)
d830aaf ("mptcp: refactor token container.") (58 minutes ago)
1cbe672 ("mptcp: add __init annotation on setup functions") (58 minutes ago)
213fe1d ("mptcp: fix races between shutdown and recvmsg") (58 minutes ago)
5b67054 ("inet_connection_sock: clear inet_num out of destroy helper") (58 minutes ago)
5e296dd ("bpf: fix unused-var without NETDEVICES") (58 minutes ago)
cb8e59c ("Merge git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net-next") (netnext/master, mptcp_net-next/net-next) (2 days ago)

------------[ cut here ]------------
WARNING: CPU: 1 PID: 1951 at net/mptcp/subflow.c:920 subflow_data_ready+0x16c/0x1d0 net/mptcp/subflow.c:920
Kernel panic - not syncing: panic_on_warn set ...
CPU: 1 PID: 1951 Comm: syz-executor357 Not tainted 5.7.0-rc7 #90
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.12.1-0-ga5cab58e9a3f-prebuilt.qemu.org 04/01/2014
Call Trace:
 __dump_stack lib/dump_stack.c:77 [inline]
 dump_stack+0xb7/0xfe lib/dump_stack.c:118
 panic+0x22d/0x5b2 kernel/panic.c:221
 __warn.cold+0x2f/0x3b kernel/panic.c:582
 report_bug+0x1d1/0x200 lib/bug.c:195
 fixup_bug arch/x86/kernel/traps.c:175 [inline]
 fixup_bug arch/x86/kernel/traps.c:170 [inline]
 do_error_trap+0xcf/0x100 arch/x86/kernel/traps.c:267
 do_invalid_op+0x32/0x40 arch/x86/kernel/traps.c:286
 invalid_op+0x1e/0x30 arch/x86/entry/entry_64.S:1027
RIP: 0010:subflow_data_ready+0x16c/0x1d0 net/mptcp/subflow.c:920
Code: 5f c3 e8 f7 fc af fe 49 8d 7e 48 e8 4e c7 cf fe 41 0f b6 5e 48 31 ff 83 e3 18 89 de e8 0d fe af fe 84 db 75 87 e8 d4 fc af fe <0f> 0b e9 7b ff ff ff e8 c8 fc af fe 48 89 ee 4c 89 ef e8 ed b5 ff
RSP: 0018:ffff8881116579f0 EFLAGS: 00010293
RAX: ffff888113dd4600 RBX: 0000000000000000 RCX: ffffffff827485f3
RDX: 0000000000000000 RSI: ffffffff827485fc RDI: 0000000000000001
RBP: ffff88810e7d8940 R08: ffff888113dd4600 R09: ffffed10222320b4
R10: ffff88811119059f R11: ffffed10222320b3 R12: 1ffff110222caf3e
R13: ffff888111190000 R14: ffff888117c34800 R15: ffff888111190598
 tcp_data_ready+0x72/0x110 net/ipv4/tcp_input.c:4776
 tcp_data_queue+0x9a8/0x2200 net/ipv4/tcp_input.c:4842
 tcp_rcv_state_process+0x7d4/0x25aa net/ipv4/tcp_input.c:6392
 tcp_v4_do_rcv+0x1ed/0x480 net/ipv4/tcp_ipv4.c:1651
 sk_backlog_rcv include/net/sock.h:996 [inline]
 __release_sock+0x12b/0x1d0 net/core/sock.c:2546
 release_sock+0x40/0x100 net/core/sock.c:3062
 mptcp_subflow_shutdown net/mptcp/protocol.c:1403 [inline]
 mptcp_shutdown+0x15f/0x320 net/mptcp/protocol.c:2115
 __sys_shutdown+0xce/0x150 net/socket.c:2203
 __do_sys_shutdown net/socket.c:2211 [inline]
 __se_sys_shutdown net/socket.c:2209 [inline]
 __x64_sys_shutdown+0x2b/0x30 net/socket.c:2209
 do_syscall_64+0x8a/0x290 arch/x86/entry/common.c:295
 entry_SYSCALL_64_after_hwframe+0x44/0xa9
RIP: 0033:0x7f7bb01c2469
Code: 00 f3 c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 40 00 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d ff 49 2b 00 f7 d8 64 89 01 48
RSP: 002b:00007ffcf584ac58 EFLAGS: 00000246 ORIG_RAX: 0000000000000030
RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007f7bb01c2469
RDX: 00007f7bb01c2469 RSI: 0000000000000001 RDI: 0000000000000003
RBP: 0000000000400680 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000246 R12: 000000000040059f
R13: 00007ffcf584ad40 R14: 0000000000000000 R15: 0000000000000000
Dumping ftrace buffer:
   (ftrace buffer empty)
Kernel Offset: disabled
Rebooting in 1 seconds..

syz-repro:

# {Threaded:false Collide:false Repeat:false RepeatTimes:0 Procs:1 Sandbox: Fault:false FaultCall:-1 FaultNth:0 Leak:false NetInjection:false NetDevices:false NetReset:false Cgroups:false BinfmtMisc:false CloseFDs:false KCSAN:false DevlinkPCI:false UseTmpDir:false HandleSegv:false Repro:false Trace:false}
r0 = socket$inet_mptcp(0x2, 0x1, 0x106)
bind$inet(r0, &(0x7f00000013c0)={0x2, 0x4e20, @multicast2}, 0x10)
connect$inet(r0, &(0x7f0000000040)={0x2, 0x4e20, @loopback}, 0x4d)
shutdown(r0, 0x1)

C-repro:

// autogenerated by syzkaller (https://github.com/google/syzkaller)

#define _GNU_SOURCE

#include <endian.h>
#include <stdint.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <sys/syscall.h>
#include <sys/types.h>
#include <unistd.h>

uint64_t r[1] = {0xffffffffffffffff};

int main(void)
{
  syscall(__NR_mmap, 0x20000000ul, 0x1000000ul, 3ul, 0x32ul, -1, 0ul);
  intptr_t res = 0;
  res = syscall(__NR_socket, 2ul, 1ul, 0x106);
  if (res != -1)
    r[0] = res;
  *(uint16_t*)0x200013c0 = 2;
  *(uint16_t*)0x200013c2 = htobe16(0x4e20);
  *(uint32_t*)0x200013c4 = htobe32(0xe0000002);
  syscall(__NR_bind, r[0], 0x200013c0ul, 0x10ul);
  *(uint16_t*)0x20000040 = 2;
  *(uint16_t*)0x20000042 = htobe16(0x4e20);
  *(uint32_t*)0x20000044 = htobe32(0x7f000001);
  syscall(__NR_connect, r[0], 0x20000040ul, 0x4dul);
  syscall(__NR_shutdown, r[0], 1ul);
  return 0;
}

[EDIT 06/05: Updated HEAD]

[cpaasch]

This here is triggering a new warn in Davide's patch. However, the issue seems to be a bit deeper.

This socket is trying to connect to itself. When this happens it goes into the "simultaneous-open" code-path in tcp_rcv_synsent_state_process (for the SYN) and into tcp_rcv_state_process, case TCP_SYN_RECEIVED (for the SYN/ACK). Thus, we never call tcp_finish_connect which would end up calling subflow_finish_connect to set subflow->mp_capable to 1.

Thus, we are ending up here with a connection that has neither subflow->mp_capable set to 1, nor has it the flag MPTCP_FALLBACK_DONE set.

[dcaratti]

even with the WARN() line removed (or equivalently, removing the fallback rework patch), a write() operation subsequent to the connect() in the above scenario would hang forever, probably because of the value of sk_state:

PID: 838    TASK: ffff8881430de200  CPU: 2   COMMAND: "gh35"
 #0 [ffff888132797680] __schedule at ffffffffbbe30994
 #1 [ffff888132797778] schedule at ffffffffbbe31f0f
 #2 [ffff8881327977a0] schedule_timeout at ffffffffbbe3a760
 #3 [ffff8881327978a8] wait_woken at ffffffffba4aa1ca
 #4 [ffff888132797940] sk_stream_wait_connect at ffffffffbb822b19
 #5 [ffff888132797a28] mptcp_sendmsg at ffffffffbbdfde35
 #6 [ffff888132797bc0] sock_sendmsg at ffffffffbb7d720b
 #7 [ffff888132797be8] sock_write_iter at ffffffffbb7d745a
 #8 [ffff888132797ce8] new_sync_write at ffffffffbaa534a2
 #9 [ffff888132797e50] vfs_write at ffffffffbaa5bdcf
#10 [ffff888132797e90] ksys_write at ffffffffbaa5c676
#11 [ffff888132797f28] do_syscall_64 at ffffffffba2084aa
#12 [ffff888132797f50] entry_SYSCALL_64_after_hwframe at ffffffffbc00008c
    RIP: 00007f7a540d66ed  RSP: 00007fffdfa84388  RFLAGS: 00000206
    RAX: ffffffffffffffda  RBX: 0000000020000044  RCX: 00007f7a540d66ed
    RDX: 0000000000000005  RSI: 0000000000601038  RDI: 0000000000000003
    RBP: 00007fffdfa843b0   R8: 0000000000400770   R9: 0000000000400770
    R10: 0000000000400770  R11: 0000000000000206  R12: 00000000004004f0
    R13: 00007fffdfa84490  R14: 0000000000000000  R15: 0000000000000000
    ORIG_RAX: 0000000000000001  CS: 0033  SS: 002b

[cpaasch]

Seems to be taken care of with Davide's latest patch.