Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

BUG: soft lockup in ip_rcv #377

Closed
cpaasch opened this issue Mar 20, 2023 · 6 comments
Closed

BUG: soft lockup in ip_rcv #377

cpaasch opened this issue Mar 20, 2023 · 6 comments
Assignees
@dcaratti
Labels
bug reproducer Has a simple program to reproduce the bug syzkaller

Comments

@cpaasch
Copy link
Member

cpaasch commented Mar 20, 2023

syzkaller-id: a9873c720cb23a52c717882417dc16e1493a2cbe

HEAD: de5e8fd

Trace:

watchdog: BUG: soft lockup - CPU#0 stuck for 26s! [syz-executor219:883]
Modules linked in:
CPU: 0 PID: 883 Comm: syz-executor219 Not tainted 6.3.0-rc1-gde5e8fd0123c #7
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.11.0-2.el7 04/01/2014
RIP: 0010:fq_dequeue+0x4a1/0x1f40 net/sched/sch_fq.c:549
Code: c1 e8 03 48 01 d8 48 89 44 24 20 49 8d 86 90 01 00 00 48 89 44 24 48 48 c1 e8 03 48 89 44 24 30 e8 54 df 53 fe 48 8b 44 24 20 <80> 38 00 0f 85 8b 15 00 00 4d 8b a6 80 01 00 00 4c 8b 7c 24 10 4d
RSP: 0018:ffffc90000006f10 EFLAGS: 00000246
RAX: ffffed1020ff2b30 RBX: dffffc0000000000 RCX: 0000000000000100
RDX: ffff88810435dc00 RSI: ffffffff82eb07bc RDI: ffff8881082c9108
RBP: ffff888107f95998 R08: 0000000000000005 R09: 0000000000000000
R10: 00000000ae562051 R11: 0000000000038000 R12: ffff8881082c90c0
R13: ffff8881082c9000 R14: ffff888107f95800 R15: ffff888107f95990
FS:  00000000020f3880(0000) GS:ffff88811b000000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000558f1ee10000 CR3: 0000000107d7d002 CR4: 0000000000770ef0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
PKRU: 55555554
Call Trace:
 <IRQ>
 dequeue_skb net/sched/sch_generic.c:292 [inline]
 qdisc_restart net/sched/sch_generic.c:397 [inline]
 __qdisc_run+0x1b6/0x1760 net/sched/sch_generic.c:415
 __dev_xmit_skb net/core/dev.c:3868 [inline]
 __dev_queue_xmit+0x2221/0x3520 net/core/dev.c:4210
 dev_queue_xmit include/linux/netdevice.h:3052 [inline]
 neigh_hh_output include/net/neighbour.h:528 [inline]
 neigh_output include/net/neighbour.h:542 [inline]
 ip_finish_output2+0x100a/0x1960 net/ipv4/ip_output.c:228
 __ip_finish_output.part.0+0x1b4/0x340 net/ipv4/ip_output.c:306
 __ip_finish_output net/ipv4/ip_output.c:434 [inline]
 ip_finish_output net/ipv4/ip_output.c:316 [inline]
 NF_HOOK_COND include/linux/netfilter.h:291 [inline]
 ip_output+0x327/0x520 net/ipv4/ip_output.c:430
 dst_output include/net/dst.h:444 [inline]
 ip_local_out net/ipv4/ip_output.c:126 [inline]
 ip_send_skb net/ipv4/ip_output.c:1586 [inline]
 ip_push_pending_frames+0x1d0/0x230 net/ipv4/ip_output.c:1606
 ip_send_unicast_reply+0xbae/0x1040 net/ipv4/ip_output.c:1747
 tcp_v4_send_reset+0xf49/0x22b0 net/ipv4/tcp_ipv4.c:833
 tcp_v4_rcv+0x2569/0x39c0 net/ipv4/tcp_ipv4.c:2171
 ip_protocol_deliver_rcu+0x6c/0xc30 net/ipv4/ip_input.c:205
 ip_local_deliver_finish+0x29f/0x3b0 net/ipv4/ip_input.c:233
 NF_HOOK include/linux/netfilter.h:302 [inline]
 NF_HOOK include/linux/netfilter.h:296 [inline]
 ip_local_deliver+0x1ba/0x320 net/ipv4/ip_input.c:254
 dst_input include/net/dst.h:454 [inline]
 ip_rcv_finish net/ipv4/ip_input.c:449 [inline]
 NF_HOOK include/linux/netfilter.h:302 [inline]
 NF_HOOK include/linux/netfilter.h:296 [inline]
 ip_rcv+0x39b/0x410 net/ipv4/ip_input.c:569
 __netif_receive_skb_one_core+0x199/0x1e0 net/core/dev.c:5477
 __netif_receive_skb+0x1f/0x1c0 net/core/dev.c:5591
 process_backlog+0x1af/0x5a0 net/core/dev.c:5919
 __napi_poll+0xb7/0x640 net/core/dev.c:6480
 napi_poll net/core/dev.c:6547 [inline]
 net_rx_action+0x97f/0xd50 net/core/dev.c:6657
 __do_softirq+0x1a5/0x5a0 kernel/softirq.c:571
 do_softirq kernel/softirq.c:472 [inline]
 do_softirq+0xcf/0x100 kernel/softirq.c:459
 </IRQ>
 <TASK>
 __local_bh_enable_ip+0x6e/0x70 kernel/softirq.c:396
 local_bh_enable include/linux/bottom_half.h:33 [inline]
 rcu_read_unlock_bh include/linux/rcupdate.h:843 [inline]
 ip_finish_output2+0x67b/0x1960 net/ipv4/ip_output.c:229
 __ip_finish_output.part.0+0x1b4/0x340 net/ipv4/ip_output.c:306
 __ip_finish_output net/ipv4/ip_output.c:434 [inline]
 ip_finish_output net/ipv4/ip_output.c:316 [inline]
 NF_HOOK_COND include/linux/netfilter.h:291 [inline]
 ip_output+0x327/0x520 net/ipv4/ip_output.c:430
 dst_output include/net/dst.h:444 [inline]
 ip_local_out+0xd1/0x100 net/ipv4/ip_output.c:126
 __ip_queue_xmit+0x902/0x1620 net/ipv4/ip_output.c:532
 __tcp_transmit_skb+0x2a6d/0x3520 net/ipv4/tcp_output.c:1399
 tcp_transmit_skb net/ipv4/tcp_output.c:1417 [inline]
 tcp_send_syn_data net/ipv4/tcp_output.c:3825 [inline]
 tcp_connect+0x2177/0x3b50 net/ipv4/tcp_output.c:3864
 tcp_v4_connect+0x146d/0x1bc0 net/ipv4/tcp_ipv4.c:321
 __inet_stream_connect+0x88f/0xde0 net/ipv4/af_inet.c:666
 mptcp_connect+0x3a3/0x860 net/mptcp/protocol.c:3659
 __inet_stream_connect+0x88f/0xde0 net/ipv4/af_inet.c:666
 tcp_sendmsg_fastopen+0x3d4/0x710 net/ipv4/tcp.c:1201
 mptcp_sendmsg_fastopen net/mptcp/protocol.c:1739 [inline]
 mptcp_sendmsg+0x106b/0x1af0 net/mptcp/protocol.c:1778
 inet_sendmsg+0x115/0x140 net/ipv4/af_inet.c:830
 sock_sendmsg_nosec net/socket.c:724 [inline]
 sock_sendmsg net/socket.c:747 [inline]
 ____sys_sendmsg+0x78f/0x970 net/socket.c:2501
 ___sys_sendmsg+0x11d/0x1c0 net/socket.c:2555
 __sys_sendmsg+0xf6/0x1d0 net/socket.c:2584
 do_syscall_x64 arch/x86/entry/common.c:50 [inline]
 do_syscall_64+0x37/0x90 arch/x86/entry/common.c:80
 entry_SYSCALL_64_after_hwframe+0x72/0xdc
RIP: 0033:0x4322b9
Code: fd ff 48 81 c4 80 00 00 00 e9 f1 fe ff ff 0f 1f 00 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 0f 83 1b ad fd ff c3 66 2e 0f 1f 84 00 00 00 00
RSP: 002b:00007fff092c8438 EFLAGS: 00000246 ORIG_RAX: 000000000000002e
RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00000000004322b9
RDX: 0000000030000004 RSI: 0000000020000440 RDI: 0000000000000005
RBP: 00000000006e2018 R08: 0000000000000001 R09: 0000000000000001
R10: 0000000000000001 R11: 0000000000000246 R12: 431bde82d7b634db
R13: 000000000040d320 R14: 000000000040d3b0 R15: 0000000000000055
 </TASK>

Reproducer:

# {Threaded:false Repeat:true RepeatTimes:0 Procs:1 Slowdown:1 Sandbox: SandboxArg:0 Leak:false NetInjection:false NetDevices:false NetReset:false Cgroups:false BinfmtMisc:false CloseFDs:false KCSAN:false DevlinkPCI:false NicVF:false USB:false VhciInjection:false Wifi:false IEEE802154:false Sysctl:false UseTmpDir:false HandleSegv:false Repro:false Trace:false LegacyOptions:{Collide:false Fault:false FaultCall:0 FaultNth:0}}
r0 = socket$nl_route(0x10, 0x3, 0x0)
r1 = socket(0x2a, 0x2, 0x0)
getsockname$packet(r1, &(0x7f0000000080)={0x11, 0x0, <r2=>0x0}, &(0x7f0000000040)=0x14)
sendmsg$nl_route_sched(r0, &(0x7f0000000200)={0x0, 0x0, &(0x7f00000001c0)={&(0x7f00000000c0)=@newqdisc={0x70, 0x24, 0xd01, 0x0, 0x0, {0x0, 0x0, 0x0, r2, {}, {0xffff, 0xffff}}, [@qdisc_kind_options=@q_fq={{0x7}, {0x44, 0x2, [@TCA_FQ_QUANTUM={0x8, 0x3, 0x9}, @TCA_FQ_BUCKETS_LOG={0x8, 0x8, 0x1c}, @TCA_FQ_BUCKETS_LOG={0x8, 0x8, 0x3}, @TCA_FQ_RATE_ENABLE={0x8, 0x5, 0x1}, @TCA_FQ_QUANTUM={0x8, 0x3, 0x7}, @TCA_FQ_FLOW_MAX_RATE={0x8, 0x7, 0xfffffb90}, @TCA_FQ_BUCKETS_LOG={0x8, 0x8, 0x4}, @TCA_FQ_INITIAL_QUANTUM={0x8, 0x4, 0x80000000}]}}]}, 0x70}}, 0x0)
r3 = socket$inet_mptcp(0x2, 0x1, 0x106)
sendmsg$inet(r3, &(0x7f0000000440)={&(0x7f00000000c0)={0x2, 0x4e24, @loopback}, 0x10, 0x0, 0x0, &(0x7f0000000480)=ANY=[], 0xc0}, 0x30000004)

C-repro:
repro.c.txt

Kconfig:
Kconfig_k9_kasan.txt
@matttbe matttbe added the reproducer Has a simple program to reproduce the bug label Mar 21, 2023
@matttbe
Copy link
Member

matttbe commented Mar 21, 2023

Note that @cpaasch tried to reproduce it with the reproducer but it didn't work.

@matttbe
Copy link
Member

matttbe commented Mar 21, 2023

Davide is looking at it, maybe linked to an issue with qdisc

@pabeni pabeni mentioned this issue Mar 27, 2023
Closed
@dcaratti
Copy link
Contributor

dcaratti commented Apr 4, 2023 •
edited
Loading

The problem also happen with IPPROTO_TCP socket, and seems related to the fq scheduler. The script tries many connect() attempts with no listener socket on the server side. The TFO SYN followed by a RST is classified by sch_fq as a new flow, and fq_dequeue() takes a lot of time when dequeueing the RST packet.

@dcaratti
Copy link
Contributor

The problem also happen with IPPROTO_TCP socket, and seems related to the fq scheduler. The script tries many connect() attempts with no listener socket on the server side. The TFO SYN followed by a RST is classified by sch_fq as a new flow, and the sch_fq garbage collector is slow at cleaning it up. As a consequence, fq_dequeue() is slow in doing the lookup when dequeueing the RST packet.

fq_dequeue() is slow because initial_quantum is bigger than INT_MAX. Because of this, credit becomes very negative and it takes a lot of time to go out of the goto begin; loop. Capping initial_quantum to INT_MAX proved to fix the reported problem; however the problem is the integer overflow of credit. Also quantum had similar problems in the past and was fixed with

https://git.kernel.org/pub/scm/linux/kernel/git/netdev/net-next.git/commit/?id=d9e15a2733067c9

I will draft a similar patch and send it upstream soon.

intel-lab-lkp pushed a commit to intel-lab-lkp/linux that referenced this issue Apr 17, 2023
@dcaratti @intel-lab-lkp
net/sched: sch_fq: fix integer overflow of "credit"
4e0e67f 
if sch_fq is configured with "initial quantum" having values greater than
INT_MAX, the first assignment of "credit" does signed integer overflow to
a very negative value.
In this situation, the syzkaller script provided by Cristoph triggers the
CPU soft-lockup warning even with few sockets. It's not an infinite loop,
but "credit" wasn't probably meant to be minus 2Gb for each new flow.
Capping "initial quantum" to INT_MAX proved to fix the issue.

Reported-by: Christoph Paasch <cpaasch@apple.com>
Link: multipath-tcp/mptcp_net-next#377
Fixes: afe4fd0 ("pkt_sched: fq: Fair Queue packet scheduler")
Signed-off-by: Davide Caratti <dcaratti@redhat.com>
@matttbe
Copy link
Member

matttbe commented Apr 20, 2023

@dcaratti: can you close this issue when you no longer need it please? :)

intel-lab-lkp pushed a commit to intel-lab-lkp/linux that referenced this issue Apr 20, 2023
@dcaratti @intel-lab-lkp
net/sched: sch_fq: fix integer overflow of "credit"
5668435 
if sch_fq is configured with "initial quantum" having values greater than
INT_MAX, the first assignment of "credit" does signed integer overflow to
a very negative value.
In this situation, the syzkaller script provided by Cristoph triggers the
CPU soft-lockup warning even with few sockets. It's not an infinite loop,
but "credit" wasn't probably meant to be minus 2Gb for each new flow.
Capping "initial quantum" to INT_MAX proved to fix the issue.

v2: validation of "initial quantum" is done in fq_policy, instead of open
    coding in fq_change() _ suggested by Jakub Kicinski

Reported-by: Christoph Paasch <cpaasch@apple.com>
Link: multipath-tcp/mptcp_net-next#377
Fixes: afe4fd0 ("pkt_sched: fq: Fair Queue packet scheduler")
Reviewed-by: Eric Dumazet <edumazet@google.com>
Signed-off-by: Davide Caratti <dcaratti@redhat.com>
@dcaratti
Copy link
Contributor

https://lore.kernel.org/netdev/7b3a3c7e36d03068707a021760a194a8eb5ad41a.1682002300.git.dcaratti@redhat.com/

ammarfaizi2 pushed a commit to ammarfaizi2/linux-block that referenced this issue Apr 22, 2023
@dcaratti @kuba-moo
net/sched: sch_fq: fix integer overflow of "credit"
7041101 
if sch_fq is configured with "initial quantum" having values greater than
INT_MAX, the first assignment of "credit" does signed integer overflow to
a very negative value.
In this situation, the syzkaller script provided by Cristoph triggers the
CPU soft-lockup warning even with few sockets. It's not an infinite loop,
but "credit" wasn't probably meant to be minus 2Gb for each new flow.
Capping "initial quantum" to INT_MAX proved to fix the issue.

v2: validation of "initial quantum" is done in fq_policy, instead of open
    coding in fq_change() _ suggested by Jakub Kicinski

Reported-by: Christoph Paasch <cpaasch@apple.com>
Link: multipath-tcp/mptcp_net-next#377
Fixes: afe4fd0 ("pkt_sched: fq: Fair Queue packet scheduler")
Reviewed-by: Eric Dumazet <edumazet@google.com>
Signed-off-by: Davide Caratti <dcaratti@redhat.com>
Link: https://lore.kernel.org/r/7b3a3c7e36d03068707a021760a194a8eb5ad41a.1682002300.git.dcaratti@redhat.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
mj22226 pushed a commit to mj22226/linux that referenced this issue May 7, 2023
@dcaratti @gregkh
net/sched: sch_fq: fix integer overflow of "credit"
d1a446f 
[ Upstream commit 7041101 ]

if sch_fq is configured with "initial quantum" having values greater than
INT_MAX, the first assignment of "credit" does signed integer overflow to
a very negative value.
In this situation, the syzkaller script provided by Cristoph triggers the
CPU soft-lockup warning even with few sockets. It's not an infinite loop,
but "credit" wasn't probably meant to be minus 2Gb for each new flow.
Capping "initial quantum" to INT_MAX proved to fix the issue.

v2: validation of "initial quantum" is done in fq_policy, instead of open
    coding in fq_change() _ suggested by Jakub Kicinski

Reported-by: Christoph Paasch <cpaasch@apple.com>
Link: multipath-tcp/mptcp_net-next#377
Fixes: afe4fd0 ("pkt_sched: fq: Fair Queue packet scheduler")
Reviewed-by: Eric Dumazet <edumazet@google.com>
Signed-off-by: Davide Caratti <dcaratti@redhat.com>
Link: https://lore.kernel.org/r/7b3a3c7e36d03068707a021760a194a8eb5ad41a.1682002300.git.dcaratti@redhat.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Signed-off-by: Sasha Levin <sashal@kernel.org>
Whissi pushed a commit to Whissi/linux-stable that referenced this issue May 11, 2023
@dcaratti @gregkh
net/sched: sch_fq: fix integer overflow of "credit"
4fbefea 
[ Upstream commit 7041101 ]

if sch_fq is configured with "initial quantum" having values greater than
INT_MAX, the first assignment of "credit" does signed integer overflow to
a very negative value.
In this situation, the syzkaller script provided by Cristoph triggers the
CPU soft-lockup warning even with few sockets. It's not an infinite loop,
but "credit" wasn't probably meant to be minus 2Gb for each new flow.
Capping "initial quantum" to INT_MAX proved to fix the issue.

v2: validation of "initial quantum" is done in fq_policy, instead of open
    coding in fq_change() _ suggested by Jakub Kicinski

Reported-by: Christoph Paasch <cpaasch@apple.com>
Link: multipath-tcp/mptcp_net-next#377
Fixes: afe4fd0 ("pkt_sched: fq: Fair Queue packet scheduler")
Reviewed-by: Eric Dumazet <edumazet@google.com>
Signed-off-by: Davide Caratti <dcaratti@redhat.com>
Link: https://lore.kernel.org/r/7b3a3c7e36d03068707a021760a194a8eb5ad41a.1682002300.git.dcaratti@redhat.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Signed-off-by: Sasha Levin <sashal@kernel.org>
Whissi pushed a commit to Whissi/linux-stable that referenced this issue May 11, 2023
@dcaratti @gregkh
net/sched: sch_fq: fix integer overflow of "credit"
d0b4312 
[ Upstream commit 7041101 ]

if sch_fq is configured with "initial quantum" having values greater than
INT_MAX, the first assignment of "credit" does signed integer overflow to
a very negative value.
In this situation, the syzkaller script provided by Cristoph triggers the
CPU soft-lockup warning even with few sockets. It's not an infinite loop,
but "credit" wasn't probably meant to be minus 2Gb for each new flow.
Capping "initial quantum" to INT_MAX proved to fix the issue.

v2: validation of "initial quantum" is done in fq_policy, instead of open
    coding in fq_change() _ suggested by Jakub Kicinski

Reported-by: Christoph Paasch <cpaasch@apple.com>
Link: multipath-tcp/mptcp_net-next#377
Fixes: afe4fd0 ("pkt_sched: fq: Fair Queue packet scheduler")
Reviewed-by: Eric Dumazet <edumazet@google.com>
Signed-off-by: Davide Caratti <dcaratti@redhat.com>
Link: https://lore.kernel.org/r/7b3a3c7e36d03068707a021760a194a8eb5ad41a.1682002300.git.dcaratti@redhat.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Signed-off-by: Sasha Levin <sashal@kernel.org>
Whissi pushed a commit to Whissi/linux-stable that referenced this issue May 11, 2023
@dcaratti @gregkh
net/sched: sch_fq: fix integer overflow of "credit"
85f24cb 
[ Upstream commit 7041101 ]

if sch_fq is configured with "initial quantum" having values greater than
INT_MAX, the first assignment of "credit" does signed integer overflow to
a very negative value.
In this situation, the syzkaller script provided by Cristoph triggers the
CPU soft-lockup warning even with few sockets. It's not an infinite loop,
but "credit" wasn't probably meant to be minus 2Gb for each new flow.
Capping "initial quantum" to INT_MAX proved to fix the issue.

v2: validation of "initial quantum" is done in fq_policy, instead of open
    coding in fq_change() _ suggested by Jakub Kicinski

Reported-by: Christoph Paasch <cpaasch@apple.com>
Link: multipath-tcp/mptcp_net-next#377
Fixes: afe4fd0 ("pkt_sched: fq: Fair Queue packet scheduler")
Reviewed-by: Eric Dumazet <edumazet@google.com>
Signed-off-by: Davide Caratti <dcaratti@redhat.com>
Link: https://lore.kernel.org/r/7b3a3c7e36d03068707a021760a194a8eb5ad41a.1682002300.git.dcaratti@redhat.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Signed-off-by: Sasha Levin <sashal@kernel.org>
Whissi pushed a commit to Whissi/linux-stable that referenced this issue May 11, 2023
@dcaratti @gregkh
net/sched: sch_fq: fix integer overflow of "credit"
2322462 
[ Upstream commit 7041101 ]

if sch_fq is configured with "initial quantum" having values greater than
INT_MAX, the first assignment of "credit" does signed integer overflow to
a very negative value.
In this situation, the syzkaller script provided by Cristoph triggers the
CPU soft-lockup warning even with few sockets. It's not an infinite loop,
but "credit" wasn't probably meant to be minus 2Gb for each new flow.
Capping "initial quantum" to INT_MAX proved to fix the issue.

v2: validation of "initial quantum" is done in fq_policy, instead of open
    coding in fq_change() _ suggested by Jakub Kicinski

Reported-by: Christoph Paasch <cpaasch@apple.com>
Link: multipath-tcp/mptcp_net-next#377
Fixes: afe4fd0 ("pkt_sched: fq: Fair Queue packet scheduler")
Reviewed-by: Eric Dumazet <edumazet@google.com>
Signed-off-by: Davide Caratti <dcaratti@redhat.com>
Link: https://lore.kernel.org/r/7b3a3c7e36d03068707a021760a194a8eb5ad41a.1682002300.git.dcaratti@redhat.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Signed-off-by: Sasha Levin <sashal@kernel.org>
charwliu pushed a commit to charwliu/linux that referenced this issue May 12, 2023
@dcaratti @charwliu
net/sched: sch_fq: fix integer overflow of "credit"
9abbca1 
[ Upstream commit 7041101 ]

if sch_fq is configured with "initial quantum" having values greater than
INT_MAX, the first assignment of "credit" does signed integer overflow to
a very negative value.
In this situation, the syzkaller script provided by Cristoph triggers the
CPU soft-lockup warning even with few sockets. It's not an infinite loop,
but "credit" wasn't probably meant to be minus 2Gb for each new flow.
Capping "initial quantum" to INT_MAX proved to fix the issue.

v2: validation of "initial quantum" is done in fq_policy, instead of open
    coding in fq_change() _ suggested by Jakub Kicinski

Reported-by: Christoph Paasch <cpaasch@apple.com>
Link: multipath-tcp/mptcp_net-next#377
Fixes: afe4fd0 ("pkt_sched: fq: Fair Queue packet scheduler")
Reviewed-by: Eric Dumazet <edumazet@google.com>
Signed-off-by: Davide Caratti <dcaratti@redhat.com>
Link: https://lore.kernel.org/r/7b3a3c7e36d03068707a021760a194a8eb5ad41a.1682002300.git.dcaratti@redhat.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Signed-off-by: Sasha Levin <sashal@kernel.org>
Whissi pushed a commit to Whissi/linux-stable that referenced this issue May 17, 2023
@dcaratti @gregkh
net/sched: sch_fq: fix integer overflow of "credit"
4b8a05e 
[ Upstream commit 7041101 ]

if sch_fq is configured with "initial quantum" having values greater than
INT_MAX, the first assignment of "credit" does signed integer overflow to
a very negative value.
In this situation, the syzkaller script provided by Cristoph triggers the
CPU soft-lockup warning even with few sockets. It's not an infinite loop,
but "credit" wasn't probably meant to be minus 2Gb for each new flow.
Capping "initial quantum" to INT_MAX proved to fix the issue.

v2: validation of "initial quantum" is done in fq_policy, instead of open
    coding in fq_change() _ suggested by Jakub Kicinski

Reported-by: Christoph Paasch <cpaasch@apple.com>
Link: multipath-tcp/mptcp_net-next#377
Fixes: afe4fd0 ("pkt_sched: fq: Fair Queue packet scheduler")
Reviewed-by: Eric Dumazet <edumazet@google.com>
Signed-off-by: Davide Caratti <dcaratti@redhat.com>
Link: https://lore.kernel.org/r/7b3a3c7e36d03068707a021760a194a8eb5ad41a.1682002300.git.dcaratti@redhat.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Signed-off-by: Sasha Levin <sashal@kernel.org>
ahsanhussain pushed a commit to ahsanhussain/linux-flex-imx that referenced this issue May 19, 2023
@dcaratti @ahsanhussain
net/sched: sch_fq: fix integer overflow of "credit"
4496c33 
[ Upstream commit 7041101ff6c3073fd8f2e99920f535b111c929cb ]

if sch_fq is configured with "initial quantum" having values greater than
INT_MAX, the first assignment of "credit" does signed integer overflow to
a very negative value.
In this situation, the syzkaller script provided by Cristoph triggers the
CPU soft-lockup warning even with few sockets. It's not an infinite loop,
but "credit" wasn't probably meant to be minus 2Gb for each new flow.
Capping "initial quantum" to INT_MAX proved to fix the issue.

v2: validation of "initial quantum" is done in fq_policy, instead of open
    coding in fq_change() _ suggested by Jakub Kicinski

Reported-by: Christoph Paasch <cpaasch@apple.com>
Link: multipath-tcp/mptcp_net-next#377
Fixes: afe4fd0 ("pkt_sched: fq: Fair Queue packet scheduler")
Reviewed-by: Eric Dumazet <edumazet@google.com>
Signed-off-by: Davide Caratti <dcaratti@redhat.com>
Link: https://lore.kernel.org/r/7b3a3c7e36d03068707a021760a194a8eb5ad41a.1682002300.git.dcaratti@redhat.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Signed-off-by: Sasha Levin <sashal@kernel.org>
jpuhlman pushed a commit to MontaVista-OpenSourceTechnology/linux-mvista that referenced this issue May 31, 2023
@dcaratti @mvakuster
net/sched: sch_fq: fix integer overflow of "credit"
d57669d 
Source: Kernel.org
MR: 126295
Type: Integration
Disposition: Backport from git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable linux-5.10.y
ChangeID: 4b8a05e3801661a0438fcd0cdef181030d966a5a
Description:

[ Upstream commit 7041101 ]

if sch_fq is configured with "initial quantum" having values greater than
INT_MAX, the first assignment of "credit" does signed integer overflow to
a very negative value.
In this situation, the syzkaller script provided by Cristoph triggers the
CPU soft-lockup warning even with few sockets. It's not an infinite loop,
but "credit" wasn't probably meant to be minus 2Gb for each new flow.
Capping "initial quantum" to INT_MAX proved to fix the issue.

v2: validation of "initial quantum" is done in fq_policy, instead of open
    coding in fq_change() _ suggested by Jakub Kicinski

Reported-by: Christoph Paasch <cpaasch@apple.com>
Link: multipath-tcp/mptcp_net-next#377
Fixes: afe4fd0 ("pkt_sched: fq: Fair Queue packet scheduler")
Reviewed-by: Eric Dumazet <edumazet@google.com>
Signed-off-by: Davide Caratti <dcaratti@redhat.com>
Link: https://lore.kernel.org/r/7b3a3c7e36d03068707a021760a194a8eb5ad41a.1682002300.git.dcaratti@redhat.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Signed-off-by: Sasha Levin <sashal@kernel.org>
Signed-off-by: Armin Kuster <akuster@mvista.com>
jpuhlman pushed a commit to MontaVista-OpenSourceTechnology/linux-mvista that referenced this issue May 31, 2023
@dcaratti @mvakuster
net/sched: sch_fq: fix integer overflow of "credit"
b2b6aa4 
Source: Kernel.org
MR: 126295
Type: Integration
Disposition: Backport from git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable linux-5.10.y
ChangeID: 4b8a05e3801661a0438fcd0cdef181030d966a5a
Description:

[ Upstream commit 7041101 ]

if sch_fq is configured with "initial quantum" having values greater than
INT_MAX, the first assignment of "credit" does signed integer overflow to
a very negative value.
In this situation, the syzkaller script provided by Cristoph triggers the
CPU soft-lockup warning even with few sockets. It's not an infinite loop,
but "credit" wasn't probably meant to be minus 2Gb for each new flow.
Capping "initial quantum" to INT_MAX proved to fix the issue.

v2: validation of "initial quantum" is done in fq_policy, instead of open
    coding in fq_change() _ suggested by Jakub Kicinski

Reported-by: Christoph Paasch <cpaasch@apple.com>
Link: multipath-tcp/mptcp_net-next#377
Fixes: afe4fd0 ("pkt_sched: fq: Fair Queue packet scheduler")
Reviewed-by: Eric Dumazet <edumazet@google.com>
Signed-off-by: Davide Caratti <dcaratti@redhat.com>
Link: https://lore.kernel.org/r/7b3a3c7e36d03068707a021760a194a8eb5ad41a.1682002300.git.dcaratti@redhat.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Signed-off-by: Sasha Levin <sashal@kernel.org>
Signed-off-by: Armin Kuster <akuster@mvista.com>
jpuhlman pushed a commit to MontaVista-OpenSourceTechnology/linux-mvista that referenced this issue May 31, 2023
@dcaratti @mvakuster
net/sched: sch_fq: fix integer overflow of "credit"
f7bcb5f 
Source: Kernel.org
MR: 126295
Type: Integration
Disposition: Backport from git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable linux-5.10.y
ChangeID: 4b8a05e3801661a0438fcd0cdef181030d966a5a
Description:

[ Upstream commit 7041101 ]

if sch_fq is configured with "initial quantum" having values greater than
INT_MAX, the first assignment of "credit" does signed integer overflow to
a very negative value.
In this situation, the syzkaller script provided by Cristoph triggers the
CPU soft-lockup warning even with few sockets. It's not an infinite loop,
but "credit" wasn't probably meant to be minus 2Gb for each new flow.
Capping "initial quantum" to INT_MAX proved to fix the issue.

v2: validation of "initial quantum" is done in fq_policy, instead of open
    coding in fq_change() _ suggested by Jakub Kicinski

Reported-by: Christoph Paasch <cpaasch@apple.com>
Link: multipath-tcp/mptcp_net-next#377
Fixes: afe4fd0 ("pkt_sched: fq: Fair Queue packet scheduler")
Reviewed-by: Eric Dumazet <edumazet@google.com>
Signed-off-by: Davide Caratti <dcaratti@redhat.com>
Link: https://lore.kernel.org/r/7b3a3c7e36d03068707a021760a194a8eb5ad41a.1682002300.git.dcaratti@redhat.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Signed-off-by: Sasha Levin <sashal@kernel.org>
Signed-off-by: Armin Kuster <akuster@mvista.com>
Fishwaldo pushed a commit to Fishwaldo/Star64_linux that referenced this issue Jun 7, 2023
@dcaratti @Fishwaldo
net/sched: sch_fq: fix integer overflow of "credit"
b1b17ce 
[ Upstream commit 7041101 ]

if sch_fq is configured with "initial quantum" having values greater than
INT_MAX, the first assignment of "credit" does signed integer overflow to
a very negative value.
In this situation, the syzkaller script provided by Cristoph triggers the
CPU soft-lockup warning even with few sockets. It's not an infinite loop,
but "credit" wasn't probably meant to be minus 2Gb for each new flow.
Capping "initial quantum" to INT_MAX proved to fix the issue.

v2: validation of "initial quantum" is done in fq_policy, instead of open
    coding in fq_change() _ suggested by Jakub Kicinski

Reported-by: Christoph Paasch <cpaasch@apple.com>
Link: multipath-tcp/mptcp_net-next#377
Fixes: afe4fd0 ("pkt_sched: fq: Fair Queue packet scheduler")
Reviewed-by: Eric Dumazet <edumazet@google.com>
Signed-off-by: Davide Caratti <dcaratti@redhat.com>
Link: https://lore.kernel.org/r/7b3a3c7e36d03068707a021760a194a8eb5ad41a.1682002300.git.dcaratti@redhat.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Signed-off-by: Sasha Levin <sashal@kernel.org>
oraclelinuxkernel pushed a commit to oracle/linux-uek that referenced this issue Jun 16, 2023
@dcaratti @jfvogel
net/sched: sch_fq: fix integer overflow of "credit"
0075f91 
[ Upstream commit 7041101 ]

if sch_fq is configured with "initial quantum" having values greater than
INT_MAX, the first assignment of "credit" does signed integer overflow to
a very negative value.
In this situation, the syzkaller script provided by Cristoph triggers the
CPU soft-lockup warning even with few sockets. It's not an infinite loop,
but "credit" wasn't probably meant to be minus 2Gb for each new flow.
Capping "initial quantum" to INT_MAX proved to fix the issue.

v2: validation of "initial quantum" is done in fq_policy, instead of open
    coding in fq_change() _ suggested by Jakub Kicinski

Reported-by: Christoph Paasch <cpaasch@apple.com>
Link: multipath-tcp/mptcp_net-next#377
Fixes: afe4fd0 ("pkt_sched: fq: Fair Queue packet scheduler")
Reviewed-by: Eric Dumazet <edumazet@google.com>
Signed-off-by: Davide Caratti <dcaratti@redhat.com>
Link: https://lore.kernel.org/r/7b3a3c7e36d03068707a021760a194a8eb5ad41a.1682002300.git.dcaratti@redhat.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Signed-off-by: Sasha Levin <sashal@kernel.org>
(cherry picked from commit d0b43125ec892aeb1b03e5df5aab595097da225a)
Signed-off-by: Jack Vogel <jack.vogel@oracle.com>
sileshn pushed a commit to sileshn/ubuntu-kernel-lunar that referenced this issue Jul 9, 2023
@dcaratti @smb49
net/sched: sch_fq: fix integer overflow of "credit"
91e4354 
BugLink: https://bugs.launchpad.net/bugs/2025067

[ Upstream commit 7041101ff6c3073fd8f2e99920f535b111c929cb ]

if sch_fq is configured with "initial quantum" having values greater than
INT_MAX, the first assignment of "credit" does signed integer overflow to
a very negative value.
In this situation, the syzkaller script provided by Cristoph triggers the
CPU soft-lockup warning even with few sockets. It's not an infinite loop,
but "credit" wasn't probably meant to be minus 2Gb for each new flow.
Capping "initial quantum" to INT_MAX proved to fix the issue.

v2: validation of "initial quantum" is done in fq_policy, instead of open
    coding in fq_change() _ suggested by Jakub Kicinski

Reported-by: Christoph Paasch <cpaasch@apple.com>
Link: multipath-tcp/mptcp_net-next#377
Fixes: afe4fd0 ("pkt_sched: fq: Fair Queue packet scheduler")
Reviewed-by: Eric Dumazet <edumazet@google.com>
Signed-off-by: Davide Caratti <dcaratti@redhat.com>
Link: https://lore.kernel.org/r/7b3a3c7e36d03068707a021760a194a8eb5ad41a.1682002300.git.dcaratti@redhat.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Signed-off-by: Sasha Levin <sashal@kernel.org>
Signed-off-by: Kamal Mostafa <kamal@canonical.com>
Signed-off-by: Stefan Bader <stefan.bader@canonical.com>
delphix-devops-bot pushed a commit to delphix/linux-kernel-aws that referenced this issue Aug 16, 2023
@dcaratti @smb49
net/sched: sch_fq: fix integer overflow of "credit"
0538922 
BugLink: https://bugs.launchpad.net/bugs/2025095

[ Upstream commit 7041101ff6c3073fd8f2e99920f535b111c929cb ]

if sch_fq is configured with "initial quantum" having values greater than
INT_MAX, the first assignment of "credit" does signed integer overflow to
a very negative value.
In this situation, the syzkaller script provided by Cristoph triggers the
CPU soft-lockup warning even with few sockets. It's not an infinite loop,
but "credit" wasn't probably meant to be minus 2Gb for each new flow.
Capping "initial quantum" to INT_MAX proved to fix the issue.

v2: validation of "initial quantum" is done in fq_policy, instead of open
    coding in fq_change() _ suggested by Jakub Kicinski

Reported-by: Christoph Paasch <cpaasch@apple.com>
Link: multipath-tcp/mptcp_net-next#377
Fixes: afe4fd0 ("pkt_sched: fq: Fair Queue packet scheduler")
Reviewed-by: Eric Dumazet <edumazet@google.com>
Signed-off-by: Davide Caratti <dcaratti@redhat.com>
Link: https://lore.kernel.org/r/7b3a3c7e36d03068707a021760a194a8eb5ad41a.1682002300.git.dcaratti@redhat.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Signed-off-by: Sasha Levin <sashal@kernel.org>
Signed-off-by: Kamal Mostafa <kamal@canonical.com>
Signed-off-by: Roxana Nicolescu <roxana.nicolescu@canonical.com>
wanghao75 pushed a commit to openeuler-mirror/kernel that referenced this issue Nov 10, 2023
@dcaratti @sanglipeng1
net/sched: sch_fq: fix integer overflow of "credit"
0932e59 
stable inclusion
from stable-v5.10.180
commit 4b8a05e3801661a0438fcd0cdef181030d966a5a
category: bugfix
bugzilla: https://gitee.com/openeuler/kernel/issues/I8DDFN

Reference: https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit/?id=4b8a05e3801661a0438fcd0cdef181030d966a5a

--------------------------------

[ Upstream commit 7041101 ]

if sch_fq is configured with "initial quantum" having values greater than
INT_MAX, the first assignment of "credit" does signed integer overflow to
a very negative value.
In this situation, the syzkaller script provided by Cristoph triggers the
CPU soft-lockup warning even with few sockets. It's not an infinite loop,
but "credit" wasn't probably meant to be minus 2Gb for each new flow.
Capping "initial quantum" to INT_MAX proved to fix the issue.

v2: validation of "initial quantum" is done in fq_policy, instead of open
    coding in fq_change() _ suggested by Jakub Kicinski

Reported-by: Christoph Paasch <cpaasch@apple.com>
Link: multipath-tcp/mptcp_net-next#377
Fixes: afe4fd0 ("pkt_sched: fq: Fair Queue packet scheduler")
Reviewed-by: Eric Dumazet <edumazet@google.com>
Signed-off-by: Davide Caratti <dcaratti@redhat.com>
Link: https://lore.kernel.org/r/7b3a3c7e36d03068707a021760a194a8eb5ad41a.1682002300.git.dcaratti@redhat.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Signed-off-by: Sasha Levin <sashal@kernel.org>
Signed-off-by: sanglipeng <sanglipeng1@jd.com>
wanghao75 pushed a commit to openeuler-mirror/kernel that referenced this issue Nov 10, 2023
@dcaratti @sanglipeng1
net/sched: sch_fq: fix integer overflow of "credit"
e8e3c8f 
stable inclusion
from stable-v5.10.180
commit 4b8a05e3801661a0438fcd0cdef181030d966a5a
category: bugfix
bugzilla: https://gitee.com/openeuler/kernel/issues/I8FC2O

Reference: https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit/?id=4b8a05e3801661a0438fcd0cdef181030d966a5a

--------------------------------

[ Upstream commit 7041101 ]

if sch_fq is configured with "initial quantum" having values greater than
INT_MAX, the first assignment of "credit" does signed integer overflow to
a very negative value.
In this situation, the syzkaller script provided by Cristoph triggers the
CPU soft-lockup warning even with few sockets. It's not an infinite loop,
but "credit" wasn't probably meant to be minus 2Gb for each new flow.
Capping "initial quantum" to INT_MAX proved to fix the issue.

v2: validation of "initial quantum" is done in fq_policy, instead of open
    coding in fq_change() _ suggested by Jakub Kicinski

Reported-by: Christoph Paasch <cpaasch@apple.com>
Link: multipath-tcp/mptcp_net-next#377
Fixes: afe4fd0 ("pkt_sched: fq: Fair Queue packet scheduler")
Reviewed-by: Eric Dumazet <edumazet@google.com>
Signed-off-by: Davide Caratti <dcaratti@redhat.com>
Link: https://lore.kernel.org/r/7b3a3c7e36d03068707a021760a194a8eb5ad41a.1682002300.git.dcaratti@redhat.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Signed-off-by: Sasha Levin <sashal@kernel.org>
Signed-off-by: sanglipeng <sanglipeng1@jd.com>
eclipse-oniro-oh-bot pushed a commit to eclipse-oniro-mirrors/kernel_linux_5.10 that referenced this issue Feb 10, 2024
@dcaratti
net/sched: sch_fq: fix integer overflow of "credit"
affb7e8 
stable inclusion
from stable-5.10.180
commit 4b8a05e3801661a0438fcd0cdef181030d966a5a
category: bugfix
issue: #I8TAOD
CVE: NA

Signed-off-by: Ywenrui44091 <yaowenrui2@huawei.com>
---------------------------------------

[ Upstream commit 7041101ff6c3073fd8f2e99920f535b111c929cb ]

if sch_fq is configured with "initial quantum" having values greater than
INT_MAX, the first assignment of "credit" does signed integer overflow to
a very negative value.
In this situation, the syzkaller script provided by Cristoph triggers the
CPU soft-lockup warning even with few sockets. It's not an infinite loop,
but "credit" wasn't probably meant to be minus 2Gb for each new flow.
Capping "initial quantum" to INT_MAX proved to fix the issue.

v2: validation of "initial quantum" is done in fq_policy, instead of open
    coding in fq_change() _ suggested by Jakub Kicinski

Reported-by: Christoph Paasch <cpaasch@apple.com>
Link: multipath-tcp/mptcp_net-next#377
Fixes: afe4fd0 ("pkt_sched: fq: Fair Queue packet scheduler")
Reviewed-by: Eric Dumazet <edumazet@google.com>
Signed-off-by: Davide Caratti <dcaratti@redhat.com>
Link: https://lore.kernel.org/r/7b3a3c7e36d03068707a021760a194a8eb5ad41a.1682002300.git.dcaratti@redhat.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Signed-off-by: Sasha Levin <sashal@kernel.org>
Signed-off-by: Ywenrui44091 <yaowenrui2@huawei.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@dcaratti dcaratti

Labels
bug reproducer Has a simple program to reproduce the bug syzkaller
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@cpaasch @matttbe @dcaratti