Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

selftests: join: "backup flow used, multi links fail" is unstable #221

Closed
matttbe opened this issue Aug 4, 2021 · 8 comments
Closed

selftests: join: "backup flow used, multi links fail" is unstable #221

matttbe opened this issue Aug 4, 2021 · 8 comments
Assignees
Projects

Comments

@matttbe
Copy link
Member

matttbe commented Aug 4, 2021

Here is what my CI just reported with the latest export branch and a debug kernel:

ack, echo and stale have failed:

00:27:06.928 # 18 backup flow used, multi links fail   syn[ ok ] - synack[ ok ] - ack[fail] got 1 JOIN[s] ack expected 2
00:27:18.415 # Server ns stats
00:27:18.415 # MPTcpExtMPCapableSYNRX          1                  0.0
00:27:18.496 # MPTcpExtMPCapableACKRX          1                  0.0
00:27:18.497 # MPTcpExtMPJoinSynRx             2                  0.0
00:27:18.499 # MPTcpExtMPJoinAckRx             1                  0.0
00:27:18.501 # MPTcpExtMPJoinAckHMacFailure    1                  0.0
00:27:18.501 # Client ns stats
00:27:18.504 # MPTcpExtMPCapableSYNTX          1                  0.0
00:27:18.589 # MPTcpExtMPCapableSYNACKRX       1                  0.0
00:27:18.590 # MPTcpExtMPTCPRetrans            3                  0.0
00:27:18.592 # MPTcpExtMPJoinSynAckRx          2                  0.0
00:27:18.593 # MPTcpExtAddAddr                 1                  0.0
00:27:18.595 # MPTcpExtSubflowStale            1                  0.0
00:27:18.596 #                                         add[ ok ] - echo  [fail] got 0 ADD_ADDR echo[s] expected 1
00:27:18.809 # 
00:27:18.809 # Server ns stats
00:27:18.810 # MPTcpExtMPCapableSYNRX          1                  0.0
00:27:18.887 # MPTcpExtMPCapableACKRX          1                  0.0
00:27:18.888 # MPTcpExtMPJoinSynRx             2                  0.0
00:27:18.890 # MPTcpExtMPJoinAckRx             1                  0.0
00:27:18.891 # MPTcpExtMPJoinAckHMacFailure    1                  0.0
00:27:18.892 # Client ns stats
00:27:18.895 # MPTcpExtMPCapableSYNTX          1                  0.0
00:27:18.982 # MPTcpExtMPCapableSYNACKRX       1                  0.0
00:27:18.984 # MPTcpExtMPTCPRetrans            3                  0.0
00:27:18.987 # MPTcpExtMPJoinSynAckRx          2                  0.0
00:27:18.988 # MPTcpExtAddAddr                 1                  0.0
00:27:18.988 # MPTcpExtSubflowStale            1                  0.0
00:27:18.990 #                                         stale             [fail] got 1 stale[s] 0 recover[s],   expected stale in range [2..4],  stale-recover delta 2 
00:27:19.183 # ns2-0-rNwRm1 stats
00:27:19.184 # 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000
00:27:19.307 #     link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
00:27:19.308 #     RX: bytes  packets  errors  dropped missed  mcast   
00:27:19.308 #     1260       3        0       0       0       0       
00:27:19.309 #     TX: bytes  packets  errors  dropped carrier collsns 
00:27:19.310 #     1260       3        0       0       0       0       
00:27:19.311 # 2: sit0@NONE: <NOARP> mtu 1480 qdisc noop state DOWN mode DEFAULT group default qlen 1000
00:27:19.313 #     link/sit 0.0.0.0 brd 0.0.0.0
00:27:19.314 #     RX: bytes  packets  errors  dropped missed  mcast   
00:27:19.316 #     0          0        0       0       0       0       
00:27:19.317 #     TX: bytes  packets  errors  dropped carrier collsns 
00:27:19.319 #     0          0        0       0       0       0       
00:27:19.320 # 3: ns2eth1@if3: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc netem state DOWN mode DEFAULT group default qlen 1000
00:27:19.323 #     link/ether 52:52:c7:a1:b6:9e brd ff:ff:ff:ff:ff:ff link-netns ns1-0-rNwRm1
00:27:19.325 #     RX: bytes  packets  errors  dropped missed  mcast   
00:27:19.329 #     172778     2199     0       0       0       0       
00:27:19.330 #     TX: bytes  packets  errors  dropped carrier collsns 
00:27:19.330 #     6505198    2275     0       0       0       0       
00:27:19.331 # 4: ns2eth2@if4: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc netem state DOWN mode DEFAULT group default qlen 1000
00:27:19.332 #     link/ether 32:97:c9:2c:fc:df brd ff:ff:ff:ff:ff:ff link-netns ns1-0-rNwRm1
00:27:19.333 #     RX: bytes  packets  errors  dropped missed  mcast   
00:27:19.334 #     856        10       0       0       0       0       
00:27:19.335 #     TX: bytes  packets  errors  dropped carrier collsns 
00:27:19.336 #     1048       12       0       0       0       0       
00:27:19.338 # 5: ns2eth3@if5: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc netem state UP mode DEFAULT group default qlen 1000
00:27:19.342 #     link/ether 66:d3:e2:c0:64:8f brd ff:ff:ff:ff:ff:ff link-netns ns1-0-rNwRm1
00:27:19.346 #     RX: bytes  packets  errors  dropped missed  mcast   
00:27:19.348 #     145952     1870     0       0       0       0       
00:27:19.349 #     TX: bytes  packets  errors  dropped carrier collsns 
00:27:19.350 #     6473958    1943     0       0       0       0       
00:27:19.352 # 6: ns2eth4@if6: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc netem state UP mode DEFAULT group default qlen 1000
00:27:19.355 #     link/ether fe:4f:a7:73:0a:9a brd ff:ff:ff:ff:ff:ff link-netns ns1-0-rNwRm1
00:27:19.356 #     RX: bytes  packets  errors  dropped missed  mcast   
00:27:19.356 #     666        7        0       0       0       0       
00:27:19.358 #     TX: bytes  packets  errors  dropped carrier collsns 
00:27:19.359 #     776        8        0       0       0       0       
00:27:19.363 # MPTcpExtMPCapableSYNTX          1                  0.0
00:27:19.449 # MPTcpExtMPCapableSYNACKRX       1                  0.0
00:27:19.451 # MPTcpExtMPTCPRetrans            3                  0.0
00:27:19.453 # MPTcpExtMPJoinSynAckRx          2                  0.0
00:27:19.456 # MPTcpExtAddAddr                 1                  0.0
00:27:19.458 # MPTcpExtSubflowStale            1                  0.0
00:27:19.461 #                                         link usage        [ ok ]
00:27:19.640 [  364.597105] IPv6: ADDRCONF(NETDEV_CHANGE): ns1eth1: link becomes ready
00:27:21.016 [  365.083052] IPv6: ADDRCONF(NETDEV_CHANGE): ns1eth2: link becomes ready
00:27:21.502 [  365.268868] IPv6: ADDRCONF(NETDEV_CHANGE): ns2eth1: link becomes ready
00:27:21.688 [  365.588002] IPv6: ADDRCONF(NETDEV_CHANGE): ns1eth3: link becomes ready
00:27:22.007 [  366.043673] IPv6: ADDRCONF(NETDEV_CHANGE): ns1eth4: link becomes ready
00:27:22.463 [  366.276816] IPv6: ADDRCONF(NETDEV_CHANGE): ns2eth3: link becomes ready
00:27:22.696 # copyfd_io_poll: poll timed out (events: POLLIN 1, POLLOUT 0)
00:28:01.663 # copyfd_io_poll: poll timed out (events: POLLIN 0, POLLOUT 4)
00:28:02.386 #  client exit code 2, server 2
00:28:02.574 # 
00:28:02.574 # netns ns1-0-B1yWSF socket stat for 10018:
00:28:02.575 # Netid State    Recv-Q Send-Q Local Address:Port  Peer Address:Port Process                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                         
00:28:03.073 # tcp   ESTAB    0      26896       10.0.2.1:10018     10.0.2.2:37987 timer:(persist,27sec,7) ino:0 sk:1 <->
00:28:03.075 # 	 ts sack cubic wscale:7,7 rto:243 backoff:7 rtt:42.765/23.3 ato:40 mss:1448 pmtu:1500 rcvmss:1420 advmss:1448 cwnd:10 ssthresh:138 bytes_sent:4535040 bytes_retrans:756 bytes_acked:4534284 bytes_received:2382271 segs_out:3390 segs_in:2544 data_segs_out:3240 data_segs_in:1710 send 2708757bps lastsnd:34603 lastrcv:36410 lastack:34603 pacing_rate 22753288bps delivery_rate 431776bps delivered:3241 busy:36837ms rwnd_limited:34683ms(94.2%) sndbuf_limited:145ms(0.4%) retrans:0/1 dsack_dups:1 rcv_rtt:91.771 rcv_space:14600 rcv_ssthresh:894376 notsent:26896 minrtt:1 tcp-ulp-mptcp flags:Jec token:0000(id:0)/918d4646(id:1) seq:ef63bd635cfcd762 sfseq:244d38 ssnoff:4aca1a07 maplen:c88
00:28:03.091 # tcp   ESTAB    0      0           10.0.1.1:10018     10.0.3.2:37873 ino:0 sk:2 <->
00:28:03.093 # 	 ts sack cubic wscale:7,7 rto:203 rtt:2.887/2.703 ato:40 mss:1448 pmtu:1500 rcvmss:1420 advmss:1448 cwnd:27 bytes_sent:25968 bytes_acked:25968 bytes_received:6291484 segs_out:1872 segs_in:4439 data_segs_out:19 data_segs_in:4432 send 108336682bps lastsnd:34053 lastrcv:30546 lastack:30546 pacing_rate 216617088bps delivery_rate 16335944bps delivered:20 busy:11ms rcv_rtt:319.652 rcv_space:14600 rcv_ssthresh:1857682 minrtt:1 tcp-ulp-mptcp flags:JBec token:0000(id:1)/918d4646(id:0) seq:ef63bd635d5e3596 sfseq:5fb641 ssnoff:4484a2cb maplen:49dc                                                                                                                                                                 
00:28:03.109 # tcp   ESTAB    0      22872       10.0.1.1:10018     10.0.1.2:36800 timer:(persist,32sec,6) ino:0 sk:3 <->
00:28:03.112 # 	 ts sack cubic wscale:7,7 rto:259 backoff:7 rtt:58.699/22.793 ato:40 mss:1448 pmtu:1500 rcvmss:1420 advmss:1448 cwnd:146 ssthresh:16 bytes_sent:4131212 bytes_acked:4131212 bytes_received:3915536 segs_out:3946 segs_in:3778 data_segs_out:2962 data_segs_in:2789 send 28812484bps lastsnd:35155 lastrcv:36374 lastack:34891 pacing_rate 34574536bps delivery_rate 34579104bps delivered:2963 busy:37643ms rwnd_limited:35515ms(94.3%) sndbuf_limited:152ms(0.4%) rcv_rtt:89.61 rcv_space:14600 rcv_ssthresh:894438 notsent:22872 minrtt:0.303 tcp-ulp-mptcp flags:Mec token:0000(id:0)/918d4646(id:0) seq:ef63bd635cfddd86 sfseq:3b1d41 ssnoff:74c2d1e maplen:a1d0                                   
00:28:03.127 # mptcp LAST-ACK 0      0           10.0.1.1:10018     10.0.1.2:36800 timer:(keepalive,59sec,0) ino:0 sk:4 ---
00:28:03.129 # 	 subflows:2 add_addr_signal:1 subflows_max:2 add_addr_signal_max:1 remote_key token:918d4646 write_seq:9370caac62e26e snd_una:9370caac528c3d rcv_nxt:ef63bd635d5e7f73                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                
00:28:03.145 # TcpPassiveOpens                 3                  0.0
00:28:03.146 # TcpInSegs                       6177               0.0
00:28:03.148 # TcpOutSegs                      9210               0.0
00:28:03.149 # TcpRetransSegs                  1                  0.0
00:28:03.150 # TcpExtTCPPureAcks               1827               0.0
00:28:03.151 # TcpExtTCPLossProbes             3                  0.0
00:28:03.153 # TcpExtTCPDSACKRecv              1                  0.0
00:28:03.154 # TcpExtTCPDSACKIgnoredNoUndo     1                  0.0
00:28:03.155 # TcpExtTCPOFOQueue               1                  0.0
00:28:03.156 # TcpExtTCPOrigDataSent           6220               0.0
00:28:03.158 # TcpExtTCPHystartTrainDetect     2                  0.0
00:28:03.159 # TcpExtTCPHystartTrainCwnd       47                 0.0
00:28:03.160 # TcpExtTCPWinProbe               14                 0.0
00:28:03.162 # TcpExtTCPDelivered              6221               0.0
00:28:03.163 # TcpExtTCPDSACKRecvSegs          1                  0.0
00:28:03.164 # MPTcpExtMPCapableSYNRX          1                  0.0
00:28:03.166 # MPTcpExtMPCapableACKRX          1                  0.0
00:28:03.167 # MPTcpExtMPTCPRetrans            8                  0.0
00:28:03.168 # MPTcpExtMPJoinSynRx             2                  0.0
00:28:03.170 # MPTcpExtMPJoinAckRx             2                  0.0
00:28:03.171 # MPTcpExtOFOQueueTail            951                0.0
00:28:03.172 # MPTcpExtOFOQueue                976                0.0
00:28:03.173 # MPTcpExtOFOMerge                82                 0.0
00:28:03.175 # MPTcpExtDuplicateData           1                  0.0
00:28:03.175 # MPTcpExtEchoAdd                 1                  0.0
00:28:03.177 # MPTcpExtSubflowStale            1                  0.0
00:28:03.178 # 
00:28:03.178 # netns ns2-0-B1yWSF socket stat for 10018:
00:28:03.179 # Netid State      Recv-Q Send-Q     Local Address:Port  Peer Address:Port Process                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                           
00:28:03.245 # tcp   ESTAB      0      319648          10.0.2.2:37987     10.0.2.1:10018 timer:(on,1.881ms,14) ino:0 sk:5 <->
00:28:03.248 # 	 ts sack cubic wscale:7,7 rto:4752 backoff:4 rtt:96.669/1.492 ato:40 mss:1448 pmtu:1500 rcvmss:1420 advmss:1448 cwnd:1 ssthresh:7 bytes_sent:2398223 bytes_retrans:1920 bytes_acked:2382272 bytes_received:4534284 segs_out:2569 segs_in:3383 data_segs_out:1733 data_segs_in:3240 send 119832bps lastsnd:2871 lastrcv:34990 lastack:34911 pacing_rate 14954904bps delivery_rate 5537944bps delivered:1711 busy:36503ms sndbuf_limited:2ms(0.0%) unacked:11 retrans:1/12 lost:11 rcv_rtt:158.037 rcv_space:14480 rcv_ssthresh:1079500 notsent:305616 minrtt:0.233 tcp-ulp-mptcp flags:Jjecv token:918d4646(id:1)/77fc2329(id:0) seq:9370caac51333d sfseq:45278d ssnoff:6641cf87 maplen:6570
00:28:03.264 # tcp   ESTAB      0      0       10.0.3.2%ns2eth3:37873     10.0.1.1:10018 ino:0 sk:6 <->
00:28:03.266 # 	 ts sack cubic wscale:7,7 rto:520 rtt:319.901/0.396 ato:40 mss:1448 pmtu:1500 rcvmss:1420 advmss:1448 cwnd:556 ssthresh:353 bytes_sent:6291484 bytes_acked:6291485 bytes_received:25968 segs_out:4439 segs_in:1873 data_segs_out:4432 data_segs_in:19 send 20133429bps lastsnd:31175 lastrcv:34356 lastack:30854 pacing_rate 24160056bps delivery_rate 19151464bps delivered:4433 busy:2685ms rcv_rtt:7.625 rcv_space:14480 rcv_ssthresh:116024 minrtt:0.509 tcp-ulp-mptcp flags:JjBbec token:918d4646(id:0)/77fc2329(id:1) seq:9370caac51333d sfseq:1 ssnoff:3d3de6ae maplen:6570                                                                                                                               
00:28:03.281 # tcp   ESTAB      0      1163104         10.0.1.2:36800     10.0.1.1:10018 timer:(on,3.641ms,15) ino:0 sk:7 <->
00:28:03.282 # 	 ts sack cubic wscale:7,7 rto:4592 backoff:4 rtt:86.557/2.044 ato:160 mss:1448 pmtu:1500 rcvmss:1420 advmss:1448 cwnd:1 ssthresh:7 bytes_sent:3936836 bytes_retrans:15620 bytes_acked:3915537 bytes_received:4131212 segs_out:3795 segs_in:3940 data_segs_out:2804 data_segs_in:2962 send 133831bps lastsnd:951 lastrcv:35463 lastack:35463 pacing_rate 24892544bps delivery_rate 9753352bps delivered:2790 busy:36496ms unacked:4 retrans:1/11 lost:4 rcv_rtt:87.51 rcv_space:14480 rcv_ssthresh:1079500 notsent:1157424 minrtt:0.267 tcp-ulp-mptcp flags:Mmecv token:0000(id:0)/77fc2329(id:0) seq:9370caac51333d sfseq:3efd75 ssnoff:78d35bf6 maplen:6570                               
00:28:03.297 # mptcp FIN-WAIT-2 0      0               10.0.1.2:36800     10.0.1.1:10018 timer:(keepalive,58sec,0) ino:0 sk:8 ---
00:28:03.300 # 	 subflows:2 add_addr_accepted:1 subflows_max:3 add_addr_accepted_max:1 remote_key token:77fc2329 write_seq:ef63bd635d5e7f73 snd_una:ef63bd635d5e7f73 rcv_nxt:9370caac528c3d                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            
00:28:03.324 # TcpActiveOpens                  3                  0.0
00:28:03.326 # TcpInSegs                       6128               0.0
00:28:03.327 # TcpOutSegs                      10780              0.0
00:28:03.329 # TcpRetransSegs                  23                 0.0
00:28:03.332 # TcpExtDelayedACKs               10                 0.0
00:28:03.332 # TcpExtDelayedACKLost            1                  0.0
00:28:03.333 # TcpExtTCPPureAcks               2971               0.0
00:28:03.333 # TcpExtTCPLostRetransmit         21                 0.0
00:28:03.334 # TcpExtTCPTimeouts               29                 0.0
00:28:03.336 # TcpExtTCPLossProbes             3                  0.0
00:28:03.340 # TcpExtTCPDSACKOldSent           1                  0.0
00:28:03.344 # TcpExtTCPRcvCoalesce            2                  0.0
00:28:03.346 # TcpExtTCPSpuriousRtxHostQueues  6                  0.0
00:28:03.347 # TcpExtTCPFromZeroWindowAdv      2                  0.0
00:28:03.348 # TcpExtTCPToZeroWindowAdv        2                  0.0
00:28:03.350 # TcpExtTCPWantZeroWindowAdv      36                 0.0
00:28:03.353 # TcpExtTCPOrigDataSent           8946               0.0
00:28:03.353 # TcpExtTCPHystartTrainDetect     2                  0.0
00:28:03.354 # TcpExtTCPHystartTrainCwnd       37                 0.0
00:28:03.355 # TcpExtTCPHystartDelayDetect     1                  0.0
00:28:03.358 # TcpExtTCPHystartDelayCwnd       353                0.0
00:28:03.361 # TcpExtTCPDelivered              8934               0.0
00:28:03.364 # TcpExtTcpTimeoutRehash          29                 0.0
00:28:03.366 # MPTcpExtMPCapableSYNTX          1                  0.0
00:28:03.368 # MPTcpExtMPCapableSYNACKRX       1                  0.0
00:28:03.370 # MPTcpExtMPTCPRetrans            4                  0.0
00:28:03.371 # MPTcpExtMPJoinSynAckRx          2                  0.0
00:28:03.374 # MPTcpExtOFOQueueTail            1392               0.0
00:28:03.378 # MPTcpExtOFOQueue                1424               0.0
00:28:03.379 # MPTcpExtOFOMerge                219                0.0
00:28:03.381 # MPTcpExtDuplicateData           49                 0.0
00:28:03.383 # MPTcpExtAddAddr                 1                  0.0
00:28:03.384 # MPTcpExtRcvPruned               58                 0.0
00:28:03.387 # MPTcpExtSubflowStale            2                  0.0

Could it be a timing issue?

@matttbe matttbe added this to Needs triage in MPTCP Bugs via automation Aug 4, 2021
@matttbe
Copy link
Member Author

matttbe commented Aug 4, 2021

I got a similar issue with the previous build with the export branch from yesterday evening (same MPTCP code as today):

00:21:51.169 # 15 multiple flows, signal, link failure syn[ ok ] - synack[ ok ] - ack[fail] got 2 JOIN[s] ack expected 3
00:21:58.045 # Server ns stats
00:21:58.045 # MPTcpExtMPCapableSYNRX          1                  0.0
00:21:58.123 # MPTcpExtMPCapableACKRX          1                  0.0
00:21:58.124 # MPTcpExtMPJoinSynRx             3                  0.0
00:21:58.126 # MPTcpExtMPJoinAckRx             2                  0.0
00:21:58.127 # MPTcpExtMPJoinAckHMacFailure    1                  0.0
00:21:58.129 # MPTcpExtOFOQueueTail            1095               0.0
00:21:58.131 # MPTcpExtOFOQueue                1855               0.0
00:21:58.132 # MPTcpExtOFOMerge                1657               0.0
00:21:58.133 # MPTcpExtDuplicateData           7                  0.0
00:21:58.135 # MPTcpExtRcvPruned               5                  0.0
00:21:58.135 # Client ns stats
00:21:58.136 # MPTcpExtMPCapableSYNTX          1                  0.0
00:21:58.221 # MPTcpExtMPCapableSYNACKRX       1                  0.0
00:21:58.222 # MPTcpExtMPTCPRetrans            3                  0.0
00:21:58.224 # MPTcpExtMPJoinSynAckRx          3                  0.0
00:21:58.226 # MPTcpExtAddAddr                 1                  0.0
00:21:58.227 #                                         add[ ok ] - echo  [fail] got 0 ADD_ADDR echo[s] expected 1
00:21:58.478 # 
00:21:58.478 # Server ns stats
00:21:58.478 # MPTcpExtMPCapableSYNRX          1                  0.0
00:21:58.560 # MPTcpExtMPCapableACKRX          1                  0.0
00:21:58.561 # MPTcpExtMPJoinSynRx             3                  0.0
00:21:58.563 # MPTcpExtMPJoinAckRx             2                  0.0
00:21:58.564 # MPTcpExtMPJoinAckHMacFailure    1                  0.0
00:21:58.566 # MPTcpExtOFOQueueTail            1095               0.0
00:21:58.569 # MPTcpExtOFOQueue                1855               0.0
00:21:58.570 # MPTcpExtOFOMerge                1657               0.0
00:21:58.572 # MPTcpExtDuplicateData           7                  0.0
00:21:58.574 # MPTcpExtRcvPruned               5                  0.0
00:21:58.575 # Client ns stats
00:21:58.582 # MPTcpExtMPCapableSYNTX          1                  0.0
00:21:58.666 # MPTcpExtMPCapableSYNACKRX       1                  0.0
00:21:58.667 # MPTcpExtMPTCPRetrans            3                  0.0
00:21:58.668 # MPTcpExtMPJoinSynAckRx          3                  0.0
00:21:58.670 # MPTcpExtAddAddr                 1                  0.0
00:21:58.671 #                                         stale             [fail] got 0 stale[s] 0 recover[s],   expected stale in range [1..5],  stale-recover delta 1 
00:21:58.893 # ns2-0-zsnm16 stats
00:21:58.893 # 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000
00:21:59.047 #     link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
00:21:59.049 #     RX: bytes  packets  errors  dropped missed  mcast   
00:21:59.050 #     0          0        0       0       0       0       
00:21:59.052 #     TX: bytes  packets  errors  dropped carrier collsns 
00:21:59.053 #     0          0        0       0       0       0       
00:21:59.055 # 2: sit0@NONE: <NOARP> mtu 1480 qdisc noop state DOWN mode DEFAULT group default qlen 1000
00:21:59.058 #     link/sit 0.0.0.0 brd 0.0.0.0
00:21:59.058 #     RX: bytes  packets  errors  dropped missed  mcast   
00:21:59.059 #     0          0        0       0       0       0       
00:21:59.061 #     TX: bytes  packets  errors  dropped carrier collsns 
00:21:59.062 #     0          0        0       0       0       0       
00:21:59.063 # 3: ns2eth1@if3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc netem state UP mode DEFAULT group default qlen 1000
00:21:59.065 #     link/ether 32:3d:af:ca:fa:91 brd ff:ff:ff:ff:ff:ff link-netns ns1-0-zsnm16
00:21:59.067 #     RX: bytes  packets  errors  dropped missed  mcast   
00:21:59.068 #     80360      1014     0       0       0       0       
00:21:59.070 #     TX: bytes  packets  errors  dropped carrier collsns 
00:21:59.074 #     3079382    1085     0       0       0       0       
00:21:59.075 # 4: ns2eth2@if4: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc netem state DOWN mode DEFAULT group default qlen 1000
00:21:59.076 #     link/ether 4a:90:d1:f2:a6:ff brd ff:ff:ff:ff:ff:ff link-netns ns1-0-zsnm16
00:21:59.076 #     RX: bytes  packets  errors  dropped missed  mcast   
00:21:59.077 #     856        10       0       0       0       0       
00:21:59.077 #     TX: bytes  packets  errors  dropped carrier collsns 
00:21:59.079 #     1158       13       0       0       0       0       
00:21:59.080 # 5: ns2eth3@if5: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc netem state UP mode DEFAULT group default qlen 1000
00:21:59.083 #     link/ether 56:e3:87:1e:16:9b brd ff:ff:ff:ff:ff:ff link-netns ns1-0-zsnm16
00:21:59.085 #     RX: bytes  packets  errors  dropped missed  mcast   
00:21:59.086 #     75254      963      0       0       0       0       
00:21:59.088 #     TX: bytes  packets  errors  dropped carrier collsns 
00:21:59.089 #     2879224    1026     0       0       0       0       
00:21:59.090 # 6: ns2eth4@if6: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc netem state UP mode DEFAULT group default qlen 1000
00:21:59.093 #     link/ether 0a:61:23:3e:02:17 brd ff:ff:ff:ff:ff:ff link-netns ns1-0-zsnm16
00:21:59.095 #     RX: bytes  packets  errors  dropped missed  mcast   
00:21:59.098 #     71026      909      0       0       0       0       
00:21:59.099 #     TX: bytes  packets  errors  dropped carrier collsns 
00:21:59.100 #     2753843    981      0       0       0       0       
00:21:59.100 # MPTcpExtMPCapableSYNTX          1                  0.0
00:21:59.173 # MPTcpExtMPCapableSYNACKRX       1                  0.0
00:21:59.175 # MPTcpExtMPTCPRetrans            3                  0.0
00:21:59.176 # MPTcpExtMPJoinSynAckRx          3                  0.0
00:21:59.178 # MPTcpExtAddAddr                 1                  0.0
00:21:59.179 [  314.461828] IPv6: ADDRCONF(NETDEV_CHANGE): ns1eth1: link becomes ready
00:22:00.545 [  314.925182] IPv6: ADDRCONF(NETDEV_CHANGE): ns1eth2: link becomes ready
00:22:01.008 [  315.222192] IPv6: ADDRCONF(NETDEV_CHANGE): ns2eth1: link becomes ready
00:22:01.305 [  315.378525] IPv6: ADDRCONF(NETDEV_CHANGE): ns1eth3: link becomes ready
00:22:01.462 [  315.821746] IPv6: ADDRCONF(NETDEV_CHANGE): ns1eth4: link becomes ready
00:22:01.905 [  316.230196] IPv6: ADDRCONF(NETDEV_CHANGE): ns2eth3: link becomes ready
00:22:02.314 # Created /tmp/tmp.j8kyIZvQDQ (size 6144 KB) containing data sent by server
00:22:03.106 # 16 multi flows, signal, bidi, link fail syn[ ok ] - synack[ ok ] - ack[fail] got 2 JOIN[s] ack expected 3
00:22:10.481 # Server ns stats
00:22:10.482 # MPTcpExtMPCapableSYNRX          1                  0.0
00:22:10.568 # MPTcpExtMPCapableACKRX          1                  0.0
00:22:10.569 # MPTcpExtMPTCPRetrans            10                 0.0
00:22:10.569 # MPTcpExtMPJoinSynRx             3                  0.0
00:22:10.570 # MPTcpExtMPJoinAckRx             2                  0.0
00:22:10.571 # MPTcpExtMPJoinAckHMacFailure    1                  0.0
00:22:10.572 # MPTcpExtOFOQueueTail            1067               0.0
00:22:10.573 # MPTcpExtOFOQueue                1623               0.0
00:22:10.574 # MPTcpExtOFOMerge                1288               0.0
00:22:10.575 # MPTcpExtDuplicateData           469                0.0
00:22:10.575 # MPTcpExtSubflowStale            1                  0.0
00:22:10.576 # Client ns stats
00:22:10.580 # MPTcpExtMPCapableSYNTX          1                  0.0
00:22:10.664 # MPTcpExtMPCapableSYNACKRX       1                  0.0
00:22:10.665 # MPTcpExtMPTCPRetrans            3                  0.0
00:22:10.666 # MPTcpExtMPJoinSynAckRx          3                  0.0
00:22:10.666 # MPTcpExtOFOQueueTail            1080               0.0
00:22:10.667 # MPTcpExtOFOQueue                1459               0.0
00:22:10.668 # MPTcpExtOFOMerge                950                0.0
00:22:10.668 # MPTcpExtDuplicateData           76                 0.0
00:22:10.670 # MPTcpExtAddAddr                 1                  0.0
00:22:10.671 # MPTcpExtRcvPruned               119                0.0
00:22:10.672 # MPTcpExtSubflowStale            1                  0.0
00:22:10.674 #                                         add[ ok ] - echo  [fail] got 0 ADD_ADDR echo[s] expected 1
00:22:10.873 # 
00:22:10.873 # Server ns stats
00:22:10.873 # MPTcpExtMPCapableSYNRX          1                  0.0
00:22:10.955 # MPTcpExtMPCapableACKRX          1                  0.0
00:22:10.956 # MPTcpExtMPTCPRetrans            10                 0.0
00:22:10.956 # MPTcpExtMPJoinSynRx             3                  0.0
00:22:10.957 # MPTcpExtMPJoinAckRx             2                  0.0
00:22:10.958 # MPTcpExtMPJoinAckHMacFailure    1                  0.0
00:22:10.958 # MPTcpExtOFOQueueTail            1067               0.0
00:22:10.959 # MPTcpExtOFOQueue                1623               0.0
00:22:10.960 # MPTcpExtOFOMerge                1288               0.0
00:22:10.962 # MPTcpExtDuplicateData           469                0.0
00:22:10.964 # MPTcpExtSubflowStale            1                  0.0
00:22:10.965 # Client ns stats
00:22:10.969 # MPTcpExtMPCapableSYNTX          1                  0.0
00:22:11.063 # MPTcpExtMPCapableSYNACKRX       1                  0.0
00:22:11.064 # MPTcpExtMPTCPRetrans            3                  0.0
00:22:11.064 # MPTcpExtMPJoinSynAckRx          3                  0.0
00:22:11.068 # MPTcpExtOFOQueueTail            1080               0.0
00:22:11.068 # MPTcpExtOFOQueue                1459               0.0
00:22:11.068 # MPTcpExtOFOMerge                950                0.0
00:22:11.068 # MPTcpExtDuplicateData           76                 0.0
00:22:11.068 # MPTcpExtAddAddr                 1                  0.0
00:22:11.069 # MPTcpExtRcvPruned               119                0.0
00:22:11.069 # MPTcpExtSubflowStale            1                  0.0
00:22:11.069 #                                         stale             [ ok ]
00:22:11.280 [  326.494313] IPv6: ADDRCONF(NETDEV_CHANGE): ns1eth1: link becomes ready
00:22:12.578 [  326.941318] IPv6: ADDRCONF(NETDEV_CHANGE): ns1eth2: link becomes ready
00:22:13.025 [  327.166201] IPv6: ADDRCONF(NETDEV_CHANGE): ns2eth1: link becomes ready
00:22:13.250 [  327.367515] IPv6: ADDRCONF(NETDEV_CHANGE): ns1eth3: link becomes ready
00:22:13.451 [  327.801542] IPv6: ADDRCONF(NETDEV_CHANGE): ns1eth4: link becomes ready
00:22:13.885 [  328.174258] IPv6: ADDRCONF(NETDEV_CHANGE): ns2eth3: link becomes ready

@matttbe
Copy link
Member Author

matttbe commented Aug 4, 2021

Yet another one:

00:27:35.535 # 17 backup subflow unused, link failure  syn[ ok ] - synack[ ok ] - ack[fail] got 1 JOIN[s] ack expected 2
00:27:55.706 # Server ns stats
00:27:55.707 # MPTcpExtMPCapableSYNRX          1                  0.0
00:27:55.796 # MPTcpExtMPCapableACKRX          1                  0.0
00:27:55.799 # MPTcpExtMPJoinSynRx             2                  0.0
00:27:55.800 # MPTcpExtMPJoinAckRx             1                  0.0
00:27:55.800 # MPTcpExtMPJoinAckHMacFailure    1                  0.0
00:27:55.802 # Client ns stats
00:27:55.802 # MPTcpExtMPCapableSYNTX          1                  0.0
00:27:55.882 # MPTcpExtMPCapableSYNACKRX       1                  0.0
00:27:55.883 # MPTcpExtMPTCPRetrans            3                  0.0
00:27:55.884 # MPTcpExtMPJoinSynAckRx          2                  0.0
00:27:55.886 # MPTcpExtAddAddr                 1                  0.0
00:27:55.887 # MPTcpExtSubflowStale            1                  0.0
00:27:55.888 #                                         add[ ok ] - echo  [fail] got 0 ADD_ADDR echo[s] expected 1
00:27:56.115 # 
00:27:56.115 # Server ns stats
00:27:56.115 # MPTcpExtMPCapableSYNRX          1                  0.0
00:27:56.198 # MPTcpExtMPCapableACKRX          1                  0.0
00:27:56.199 # MPTcpExtMPJoinSynRx             2                  0.0
00:27:56.201 # MPTcpExtMPJoinAckRx             1                  0.0
00:27:56.202 # MPTcpExtMPJoinAckHMacFailure    1                  0.0
00:27:56.206 # Client ns stats
00:27:56.212 # MPTcpExtMPCapableSYNTX          1                  0.0
00:27:56.313 # MPTcpExtMPCapableSYNACKRX       1                  0.0
00:27:56.315 # MPTcpExtMPTCPRetrans            3                  0.0
00:27:56.317 # MPTcpExtMPJoinSynAckRx          2                  0.0
00:27:56.318 # MPTcpExtAddAddr                 1                  0.0
00:27:56.320 # MPTcpExtSubflowStale            1                  0.0
00:27:56.321 #                                         link usage        [fail] got 51% usage, expected 0%

@matttbe
Copy link
Member Author

matttbe commented Aug 4, 2021

and another one:

00:21:57.040 # 13 signal invalid addresses             syn[ ok ] - synack[ ok ] - ack[ ok ]
00:21:59.581 #                                         add[ ok ] - echo  [ ok ]
00:21:59.799 [  332.272490] IPv6: ADDRCONF(NETDEV_CHANGE): ns1eth1: link becomes ready
00:22:01.111 [  332.714271] IPv6: ADDRCONF(NETDEV_CHANGE): ns1eth2: link becomes ready
00:22:01.553 [  333.097262] IPv6: ADDRCONF(NETDEV_CHANGE): ns2eth1: link becomes ready
00:22:01.936 [  333.152870] IPv6: ADDRCONF(NETDEV_CHANGE): ns1eth3: link becomes ready
00:22:01.991 [  333.596765] IPv6: ADDRCONF(NETDEV_CHANGE): ns1eth4: link becomes ready
00:22:02.435 [  334.105442] IPv6: ADDRCONF(NETDEV_CHANGE): ns2eth3: link becomes ready
00:22:02.944 #                                         add[ ok ] - echo  [ ok ]
00:22:05.968 [  337.843115] ip (5380) used greatest stack depth: 22904 bytes left
00:22:06.681 [  338.497543] IPv6: ADDRCONF(NETDEV_CHANGE): ns1eth1: link becomes ready
00:22:07.336 [  338.969483] IPv6: ADDRCONF(NETDEV_CHANGE): ns1eth2: link becomes ready
00:22:07.808 [  339.265243] IPv6: ADDRCONF(NETDEV_CHANGE): ns2eth1: link becomes ready
00:22:08.104 [  339.460364] IPv6: ADDRCONF(NETDEV_CHANGE): ns1eth3: link becomes ready
00:22:08.299 [  339.916189] IPv6: ADDRCONF(NETDEV_CHANGE): ns1eth4: link becomes ready
00:22:08.755 [  340.273186] IPv6: ADDRCONF(NETDEV_CHANGE): ns2eth3: link becomes ready
00:22:09.112 # Created /tmp/tmp.qMyUfKSLmQ (size 8192 KB) containing data sent by client
00:22:10.145 # 15 multiple flows, signal, link failure syn[ ok ] - synack[ ok ] - ack[fail] got 2 JOIN[s] ack expected 3
00:22:17.704 # Server ns stats
00:22:17.704 # MPTcpExtMPCapableSYNRX          1                  0.0
00:22:17.790 # MPTcpExtMPCapableACKRX          1                  0.0
00:22:17.790 # MPTcpExtMPJoinSynRx             3                  0.0
00:22:17.791 # MPTcpExtMPJoinAckRx             2                  0.0
00:22:17.792 # MPTcpExtMPJoinAckHMacFailure    1                  0.0
00:22:17.793 # MPTcpExtOFOQueueTail            1996               0.0
00:22:17.793 # MPTcpExtOFOQueue                3854               0.0
00:22:17.794 # MPTcpExtOFOMerge                3423               0.0
00:22:17.795 # MPTcpExtDuplicateData           1                  0.0
00:22:17.795 # MPTcpExtRcvPruned               5                  0.0
00:22:17.796 # Client ns stats
00:22:17.799 # MPTcpExtMPCapableSYNTX          1                  0.0
00:22:17.880 # MPTcpExtMPCapableSYNACKRX       1                  0.0
00:22:17.881 # MPTcpExtMPTCPRetrans            1                  0.0
00:22:17.881 # MPTcpExtMPJoinSynAckRx          3                  0.0
00:22:17.882 # MPTcpExtAddAddr                 1                  0.0
00:22:17.883 #                                         add[ ok ] - echo  [fail] got 0 ADD_ADDR echo[s] expected 1
00:22:18.116 # 
00:22:18.116 # Server ns stats
00:22:18.117 # MPTcpExtMPCapableSYNRX          1                  0.0
00:22:18.203 # MPTcpExtMPCapableACKRX          1                  0.0
00:22:18.203 # MPTcpExtMPJoinSynRx             3                  0.0
00:22:18.204 # MPTcpExtMPJoinAckRx             2                  0.0
00:22:18.205 # MPTcpExtMPJoinAckHMacFailure    1                  0.0
00:22:18.205 # MPTcpExtOFOQueueTail            1996               0.0
00:22:18.206 # MPTcpExtOFOQueue                3854               0.0
00:22:18.207 # MPTcpExtOFOMerge                3423               0.0
00:22:18.208 # MPTcpExtDuplicateData           1                  0.0
00:22:18.211 # MPTcpExtRcvPruned               5                  0.0
00:22:18.211 # Client ns stats
00:22:18.214 # MPTcpExtMPCapableSYNTX          1                  0.0
00:22:18.297 # MPTcpExtMPCapableSYNACKRX       1                  0.0
00:22:18.298 # MPTcpExtMPTCPRetrans            1                  0.0
00:22:18.299 # MPTcpExtMPJoinSynAckRx          3                  0.0
00:22:18.300 # MPTcpExtAddAddr                 1                  0.0
00:22:18.301 #                                         stale             [fail] got 0 stale[s] 0 recover[s],   expected stale in range [1..5],  stale-recover delta 1 
00:22:18.510 # ns2-0-uennU9 stats
00:22:18.510 # 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000
00:22:18.605 #     link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
00:22:18.609 #     RX: bytes  packets  errors  dropped missed  mcast   
00:22:18.610 #     0          0        0       0       0       0       
00:22:18.610 #     TX: bytes  packets  errors  dropped carrier collsns 
00:22:18.611 #     0          0        0       0       0       0       
00:22:18.611 # 2: sit0@NONE: <NOARP> mtu 1480 qdisc noop state DOWN mode DEFAULT group default qlen 1000
00:22:18.613 #     link/sit 0.0.0.0 brd 0.0.0.0
00:22:18.616 #     RX: bytes  packets  errors  dropped missed  mcast   
00:22:18.616 #     0          0        0       0       0       0       
00:22:18.617 #     TX: bytes  packets  errors  dropped carrier collsns 
00:22:18.617 #     0          0        0       0       0       0       
00:22:18.618 # 3: ns2eth1@if3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc netem state UP mode DEFAULT group default qlen 1000
00:22:18.622 #     link/ether fa:10:a4:7c:d1:81 brd ff:ff:ff:ff:ff:ff link-netns ns1-0-uennU9
00:22:18.627 #     RX: bytes  packets  errors  dropped missed  mcast   
00:22:18.627 #     150076     1908     0       0       0       0       
00:22:18.628 #     TX: bytes  packets  errors  dropped carrier collsns 
00:22:18.628 #     5795118    2007     0       0       0       0       
00:22:18.629 # 4: ns2eth2@if4: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc netem state DOWN mode DEFAULT group default qlen 1000
00:22:18.633 #     link/ether de:1a:21:1c:04:f5 brd ff:ff:ff:ff:ff:ff link-netns ns1-0-uennU9
00:22:18.636 #     RX: bytes  packets  errors  dropped missed  mcast   
00:22:18.637 #     856        10       0       0       0       0       
00:22:18.640 #     TX: bytes  packets  errors  dropped carrier collsns 
00:22:18.643 #     1048       12       0       0       0       0       
00:22:18.643 # 5: ns2eth3@if5: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc netem state UP mode DEFAULT group default qlen 1000
00:22:18.649 #     link/ether be:7a:26:c6:57:94 brd ff:ff:ff:ff:ff:ff link-netns ns1-0-uennU9
00:22:18.649 #     RX: bytes  packets  errors  dropped missed  mcast   
00:22:18.650 #     147700     1892     0       0       0       0       
00:22:18.653 #     TX: bytes  packets  errors  dropped carrier collsns 
00:22:18.653 #     5786867    1975     0       0       0       0       
00:22:18.655 # 6: ns2eth4@if6: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc netem state UP mode DEFAULT group default qlen 1000
00:22:18.659 #     link/ether 9a:72:d9:aa:0c:9a brd ff:ff:ff:ff:ff:ff link-netns ns1-0-uennU9
00:22:18.660 #     RX: bytes  packets  errors  dropped missed  mcast   
00:22:18.662 #     149712     1918     0       0       0       0       
00:22:18.664 #     TX: bytes  packets  errors  dropped carrier collsns 
00:22:18.664 #     5759959    2006     0       0       0       0       
00:22:18.665 # MPTcpExtMPCapableSYNTX          1                  0.0
00:22:18.745 # MPTcpExtMPCapableSYNACKRX       1                  0.0
00:22:18.746 # MPTcpExtMPTCPRetrans            1                  0.0
00:22:18.747 # MPTcpExtMPJoinSynAckRx          3                  0.0
00:22:18.747 # MPTcpExtAddAddr                 1                  0.0
00:22:18.748 [  351.217912] IPv6: ADDRCONF(NETDEV_CHANGE): ns1eth1: link becomes ready
00:22:20.056 [  351.648464] IPv6: ADDRCONF(NETDEV_CHANGE): ns1eth2: link becomes ready
00:22:20.487 [  352.041151] IPv6: ADDRCONF(NETDEV_CHANGE): ns2eth1: link becomes ready
00:22:20.879 [  352.090639] IPv6: ADDRCONF(NETDEV_CHANGE): ns1eth3: link becomes ready
00:22:20.929 [  352.549917] IPv6: ADDRCONF(NETDEV_CHANGE): ns1eth4: link becomes ready
00:22:21.388 [  353.049182] IPv6: ADDRCONF(NETDEV_CHANGE): ns2eth3: link becomes ready
00:22:21.887 # Created /tmp/tmp.gqA8qI3JkU (size 12288 KB) containing data sent by server
00:22:22.776 [  356.274108] hrtimer: interrupt took 21318 ns
00:22:25.112 # 16 multi flows, signal, bidi, link fail syn[ ok ] - synack[ ok ] - ack[ ok ]

@pabeni pabeni self-assigned this Aug 10, 2021
@pabeni
Copy link

pabeni commented Aug 10, 2021

there are 2 problems:

  • if the file sent by the client is not big enough - relevantly bigger than the max cwin value, the link usage will be "random" because a full cwin could be rejnected at link failure time
  • since we always clear the mptcp option for add_addr, on established connection, we can clear MPJ ACK opt and break the MPJ handshake

@matttbe
Copy link
Member Author

matttbe commented Aug 11, 2021

Thank you for your analysis! (and your patches)

When trying to reproduce this error locally, I hit another issue, see #225.

@matttbe
Copy link
Member Author

matttbe commented Aug 12, 2021

Reported issues are now fixed thanks to Paolo's patches:

  • 3b6b8c9: "squashed" patch 1/2 in "selftests: mptcp: add testcase for active-back"

  • Results: e41bc27..c636d84

  • 32c3e4d: "squashed" patch 2/2 in "mptcp: move drop_other_suboptions check under pm lock"

  • Results: c636d84..3e7443d

Note, I can still reproduce some self-test failures, on other
link_failure test case, but with lower rate and different
cause - to be investigated.

@pabeni Do you prefer to keep the issue #221 open or should we open a new one?

@matttbe
Copy link
Member Author

matttbe commented Aug 12, 2021

the patch 32c3e4d seems causing a regression in packetdrill tests (add_addr4_port_client.pkt & v6):

  • an ADD_ADDR is sent by the server after the 3WHS
  • the client now replies by a dup ACK: the 3rd-ACK is retransmitted.

See: issue-221-no-echo.zip

It is unclear what should be the reaction of the client if we are not in fully-established:

If A does not immediately have data to send, it MUST include the MP_CAPABLE on the first ACK, but without the additional data parameters. When A does have data to send, it must repeat the sending of the MP_CAPABLE option from the first ACK, with additional data parameters.

It might be subject to interpretations but it looks like we should first send the 4th ACK, then send the echo bit. But probably OK to send the echo bit and later on the 4th ACK.

@matttbe
Copy link
Member Author

matttbe commented Aug 18, 2021

Reported issues are now fixed thanks to Paolo's patches:

* [3b6b8c9](https://github.com/multipath-tcp/mptcp_net-next/commit/3b6b8c96b67c7ca09f9481517e0fd92d3929d2cb): "squashed" patch 1/2 in "selftests: mptcp: add testcase for active-back"

* Results: [e41bc27](https://github.com/multipath-tcp/mptcp_net-next/commit/e41bc27ecfa957f4a1a002afecfab773da34e408)..[c636d84](https://github.com/multipath-tcp/mptcp_net-next/commit/c636d849945e90efa9eb0c11f2892a8a1fca6d6f)

* [32c3e4d](https://github.com/multipath-tcp/mptcp_net-next/commit/32c3e4d5080615fbef046e060042abea9c2c13b4): "squashed" patch 2/2 in "mptcp: move drop_other_suboptions check under pm lock"

* Results: [c636d84](https://github.com/multipath-tcp/mptcp_net-next/commit/c636d849945e90efa9eb0c11f2892a8a1fca6d6f)..[3e7443d](https://github.com/multipath-tcp/mptcp_net-next/commit/3e7443d827465e04c93dbd760be2b8f13bc045a6)

Note, I can still reproduce some self-test failures, on other
link_failure test case, but with lower rate and different
cause - to be investigated.

@pabeni Do you prefer to keep the issue #221 open or should we open a new one?

Now that an extra patch has been added:

  • 3c39f1c: mptcp: full fully established support after ADD_ADDR

We can close this ticket.

@matttbe matttbe closed this as completed Aug 18, 2021
MPTCP Bugs automation moved this from Needs triage to Closed Aug 18, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
No open projects
MPTCP Bugs
  
Closed
Development

No branches or pull requests

2 participants