[syzkaller] WARNING in dst_release

[cpaasch]

------------[ cut here ]------------
dst_release underflow
WARNING: CPU: 0 PID: 1319 at net/core/dst.c:175 dst_release+0xc1/0xd0 net/core/dst.c:175
Modules linked in:
CPU: 0 PID: 1319 Comm: syz-executor217 Not tainted 5.11.0-rc6af8e85128b4d0d24083c5cac646e891227052e0c #70
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.13.0-0-gf21b5a4aeb02-prebuilt.qemu.org 04/01/2014
RIP: 0010:dst_release+0xc1/0xd0 net/core/dst.c:175
Code: 89 c3 89 c6 e8 a0 f0 79 ff 85 db 74 a5 e9 ed 84 25 00 e8 12 eb 79 ff 48 c7 c7 33 7a 4c 82 c6 05 ca 57 de 00 01 e8 fe 93 20 00 <0f> 0b eb c6 90 66 2e 0f 1f 84 00 00 00 00 00 41 54 55 48 89 fd 53
RSP: 0018:ffffc90000547bb0 EFLAGS: 00010282
RAX: 0000000000000000 RBX: 0000000000000000 RCX: 0000000000000000
RDX: ffff888100278e00 RSI: ffffffff810e4db3 RDI: 0000000000000003
RBP: ffff88813bc2a080 R08: 0000000000000000 R09: 0000000000000001
R10: ffffffff811bcf33 R11: 0000000000000000 R12: 00000000ffffffff
R13: ffff8881010ce240 R14: 0000000000000001 R15: ffff888100293f80
FS:  00007fe62f0e4700(0000) GS:ffff88813bc00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000020000040 CR3: 0000000101d8a005 CR4: 0000000000170ef0
Call Trace:
 rt_cache_route+0x12e/0x140 net/ipv4/route.c:1503
 rt_set_nexthop.constprop.0+0x1fc/0x590 net/ipv4/route.c:1612
 __mkroute_output net/ipv4/route.c:2484 [inline]
 ip_route_output_key_hash_rcu+0x4e2/0xd80 net/ipv4/route.c:2684
 ip_route_output_key_hash+0x7c/0xb0 net/ipv4/route.c:2512
 __ip_route_output_key include/net/route.h:126 [inline]
 ip_route_connect include/net/route.h:319 [inline]
 tcp_v4_connect+0x407/0x6f0 net/ipv4/tcp_ipv4.c:229
 __inet_stream_connect+0x14a/0x5a0 net/ipv4/af_inet.c:661
 inet_stream_connect+0x37/0x50 net/ipv4/af_inet.c:725
 mptcp_stream_connect+0x78/0x270 net/mptcp/protocol.c:3197
 __sys_connect_file net/socket.c:1835 [inline]
 __sys_connect+0x148/0x170 net/socket.c:1852
 __do_sys_connect net/socket.c:1862 [inline]
 __se_sys_connect net/socket.c:1859 [inline]
 __x64_sys_connect+0x18/0x20 net/socket.c:1859
 do_syscall_64+0x33/0x40 arch/x86/entry/common.c:46
 entry_SYSCALL_64_after_hwframe+0x44/0xa9
RIP: 0033:0x7fe62e9f1469
Code: 00 f3 c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 40 00 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d ff 49 2b 00 f7 d8 64 89 01 48
RSP: 002b:00007fffe82abd58 EFLAGS: 00000246 ORIG_RAX: 000000000000002a
RAX: ffffffffffffffda RBX: 000000000000811f RCX: 00007fe62e9f1469
RDX: 000000000000004d RSI: 0000000020000040 RDI: 0000000000000003
RBP: 0000000000000000 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000246 R12: 431bde82d7b634db
R13: 00007fffe82abe80 R14: 0000000000000000 R15: 0000000000000000

HEAD:
af8e85128b4d ("Revert "DO-NOT-MERGE: mptcp: use kmalloc on kasan build"") (25 hours ago)
4a71543af897 ("mptcp: init mptcp request socket earlier") (25 hours ago)
87ccece4131d ("mptcp: fix spurious retransmissions") (25 hours ago)
a15e285 ("DO-NOT-MERGE: mptcp: enabled by default") (tag: export/20210207T062839) (2 days ago)
119c5bb ("DO-NOT-MERGE: mptcp: add GitHub Actions") (2 days ago)
8e5ec86 ("DO-NOT-MERGE: mptcp: use kmalloc on kasan build") (2 days ago)
238d6b5 ("mptcp: add local addr info in mptcp_info") (2 days ago)
928f65d ("mptcp: fix poll after shutdown") (2 days ago)
e0d489b ("mptcp: deliver ssk errors to msk") (2 days ago)
4e78a50 ("mptcp: add netlink event support") (2 days ago)
c267230 ("genetlink: add CAP_NET_ADMIN test for multicast bind") (2 days ago)
1afc248 ("mptcp: avoid lock_fast usage in accept path") (2 days ago)
3a90f9b ("mptcp: pass subflow socket to a few helpers") (2 days ago)
ac2f529 ("mptcp: move subflow close loop after sk close check") (2 days ago)
8099dad ("mptcp: schedule worker when subflow is closed") (2 days ago)
57feb11 ("mptcp: split __mptcp_close_ssk helper") (2 days ago)
a2043de ("mptcp: move pm netlink work into pm_netlink") (2 days ago)
02f0a51 ("bpf:selftests: add bpf_mptcp_sock() verifier tests") (2 days ago)
6512eb1 ("bpf:selftests: add MPTCP test base") (2 days ago)
1e8a1ee ("bpf: add 'bpf_mptcp_sock' structure and helper") (2 days ago)
d439d5d ("bpf: expose is_mptcp flag to bpf_tcp_sock") (2 days ago)
ae3cb86 ("linux: handle MPTCP consistently with TCP") (2 days ago)
badc6ac ("Merge branch '100GbE' of git://git.kernel.org/pub/scm/linux/kernel/git/tnguy/next-queue") (3 days ago)

reproducer:

# {Threaded:false Collide:false Repeat:true RepeatTimes:0 Procs:1 Slowdown:1 Sandbox: Fault:false FaultCall:-1 FaultNth:0 Leak:false NetInjection:false NetDevices:false NetReset:false Cgroups:false BinfmtMisc:false CloseFDs:false KCSAN:false DevlinkPCI:false USB:false VhciInjection:false Wifi:false Sysctl:false UseTmpDir:false HandleSegv:false Repro:false Trace:false}
r0 = socket$inet_mptcp(0x2, 0x1, 0x106)
bind$inet(r0, &(0x7f00000013c0)={0x2, 0x4e20, @multicast2}, 0x10)
connect$inet(r0, &(0x7f0000000040)={0x2, 0x4e20, @loopback}, 0x4d)
shutdown(r0, 0x1)
bind$inet(r0, &(0x7f00000002c0)={0x2, 0x4e21, @multicast2}, 0x10)

config-file: Simple config without kasan, lockdep,...

[fw-strlen]

Looks like this happens when mptcp worker closes the ssk that is also msk->subflow (and msk->first).
2nd mptcp_bind re-uses this subflow sk, but its already been zapped.

Hack (add in __mptcp_close_ssk):
if (mptcp_sk(sk)->subflow) {
struct socket *sock = mptcp_sk(sk)->subflow;

            if (sock->sk == ssk) {
                    iput(SOCK_INODE(mptcp_sk(sk)->subflow));
                    mptcp_sk(sk)->subflow = NULL;
            }
    }

... and no dst underflow anymore. I wonder what the correct behaviour should be.
Should the worker refuse to close the first/subflow socket ssk instead?

[pabeni]

... and no dst underflow anymore. I wonder what the correct behaviour should be.
Should the worker refuse to close the first/subflow socket ssk instead?

Good catch! As a fix, I think we should just zap msk->subflow in __mptcp_close_ssk(): either the peer or the local PM should be able to close the first subflow.

Side note: I think there could be similar problems while accessing msk->first, but there the fix would be a little more complex due to mptcp_stream_accept() unconditionally dereferencing msk->first.