Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Handling 'interface down' events #21

Open
nemethf opened this issue May 6, 2021 · 1 comment
Open

Handling 'interface down' events #21

nemethf opened this issue May 6, 2021 · 1 comment

Comments

@nemethf
Copy link

nemethf commented May 6, 2021

It seems mp-quic doesn't handle changes in interface status. Case 1 and 2 (below) are almost identical, but one of the client interfaces is turned down at 2s in Case 2. Additionally, the available link capacity is always higher in Case 2 than Case 1. Case 1 finishes in 11s, but Case 2 throws an error at 14.3s.

For sake of reproducibility, the end of the bug report contains a small modification of minitopo.

Thanks!

case 1
topo 
topoType:MultiIf
leftSubnet:10.0.
rightSubnet:10.1.
path_c2r_0:40,40,2
path_c2r_1:40,40,2
xp 
xpType:pquic
clientPcap:yes
serverPcap:yes
snaplenPcap:100
pquicPlugins:~/pquic/plugins/multipath/multipath_rr_cond.plugin
pquicSize:5120000
pquic_ifdown:0
logs 
$ mprun -t topo -x ./xp_norm; tail pquic_client.log 
2021-05-06 13:11:33,634 [INFO] __init__: Extract parameters from file topo
2021-05-06 13:11:33,635 [WARNING] load_parameter_file: Got error 'need more than 1 value to unpack' for line '
'; ignore it
2021-05-06 13:11:33,638 [INFO] __init__: {'path_c2r_0': '40,40,2', 'path_c2r_1': '40,40,2', 'rightSubnet': '10.1.', 'topoType': 'MultiIf', 'leftSubnet': '10.0.'}
Link type: c2r
Link id: 0
  Delay: 40.0
  Queue Size: 40
  Bandwidth: 2.0
  Loss: 0.0
  Backup: 0
      
Link type: c2r
Link id: 1
  Delay: 40.0
  Queue Size: 40
  Bandwidth: 2.0
  Loss: 0.0
  Backup: 0
      
2021-05-06 13:11:33,643 [INFO] __init__: Initializing MultiInterfaceTopo...
2021-05-06 13:11:33,646 [INFO] apply_topo: Using topo Simple multiple interface topology 
/-sw---bl---sw-\ 
c                r                s
\-sw---bl---sw-/ \-sw---bl---sw-/

2021-05-06 13:11:33,649 [INFO] apply_topo_config: Using topo config <topos.multi_interface.MultiInterfaceConfig object at 0x7f8ea9af0710>
2021-05-06 13:11:34,234 [INFO] disable_tso: Disable TSO, GSO and GRO on all interfaces of all nodes
2021-05-06 13:11:34,374 [INFO] configure_interfaces: Configure interfaces using MultiInterfaceConfig...
2021-05-06 13:11:34,375 [INFO] configure_bottleneck: tc qdisc del dev bs_c2r_0_1-eth1 root; tc qdisc del dev bs_c2r_0_1-eth1 ingress 
2021-05-06 13:11:34,396 [INFO] configure_bottleneck: tc qdisc del dev bs_c2r_0_1-eth2 root; tc qdisc del dev bs_c2r_0_1-eth2 ingress 
2021-05-06 13:11:34,410 [INFO] configure_bottleneck: tc qdisc del dev bs_c2r_0_2-eth1 root; tc qdisc del dev bs_c2r_0_2-eth1 ingress 
2021-05-06 13:11:34,423 [INFO] configure_bottleneck: tc qdisc del dev bs_c2r_0_2-eth2 root; tc qdisc del dev bs_c2r_0_2-eth2 ingress 
2021-05-06 13:11:34,444 [INFO] configure_bottleneck: tc qdisc add dev bs_c2r_0_1-eth2 root handle 10: netem  delay 40.0ms limit 50000
2021-05-06 13:11:34,453 [INFO] configure_bottleneck: tc qdisc add dev bs_c2r_0_2-eth2 root handle 1:0 tbf rate 2.0mbit burst 15000 limit 78000.0
2021-05-06 13:11:34,466 [INFO] configure_bottleneck: tc qdisc add dev bs_c2r_0_2-eth1 root handle 10: netem  delay 40.0ms limit 50000
2021-05-06 13:11:34,475 [INFO] configure_bottleneck: tc qdisc add dev bs_c2r_0_1-eth1 root handle 1:0 tbf rate 2.0mbit burst 15000 limit 78000.0
2021-05-06 13:11:34,495 [INFO] configure_bottleneck: tc qdisc del dev bs_c2r_1_1-eth1 root; tc qdisc del dev bs_c2r_1_1-eth1 ingress 
2021-05-06 13:11:34,510 [INFO] configure_bottleneck: tc qdisc del dev bs_c2r_1_1-eth2 root; tc qdisc del dev bs_c2r_1_1-eth2 ingress 
2021-05-06 13:11:34,525 [INFO] configure_bottleneck: tc qdisc del dev bs_c2r_1_2-eth1 root; tc qdisc del dev bs_c2r_1_2-eth1 ingress 
2021-05-06 13:11:34,542 [INFO] configure_bottleneck: tc qdisc del dev bs_c2r_1_2-eth2 root; tc qdisc del dev bs_c2r_1_2-eth2 ingress 
2021-05-06 13:11:34,558 [INFO] configure_bottleneck: tc qdisc add dev bs_c2r_1_1-eth2 root handle 10: netem  delay 40.0ms limit 50000
2021-05-06 13:11:34,574 [INFO] configure_bottleneck: tc qdisc add dev bs_c2r_1_2-eth2 root handle 1:0 tbf rate 2.0mbit burst 15000 limit 78000.0
2021-05-06 13:11:34,582 [INFO] configure_bottleneck: tc qdisc add dev bs_c2r_1_2-eth1 root handle 10: netem  delay 40.0ms limit 50000
2021-05-06 13:11:34,596 [INFO] configure_bottleneck: tc qdisc add dev bs_c2r_1_1-eth1 root handle 1:0 tbf rate 2.0mbit burst 15000 limit 78000.0
2021-05-06 13:11:34,664 [INFO] __init__: Extract parameters from file ./xp_norm
2021-05-06 13:11:34,670 [INFO] __init__: Extract parameters from file ./xp_norm
2021-05-06 13:11:34,683 [INFO] ping: ping -c 5 -I 10.0.0.1 10.1.0.1 >> ping.log
2021-05-06 13:11:38,799 [INFO] ping: ping -c 5 -I 10.0.1.1 10.1.0.1 >> ping.log
2021-05-06 13:11:42,959 [INFO] run_tcpdump: Activating tcpdump, waiting for it to run
2021-05-06 13:11:47,963 [INFO] run_netem_at: No need to change netem
2021-05-06 13:11:47,975 [INFO] get_pquic_server_cmd: ~/pquic/picoquicdemo  -P ~/pquic/plugins/multipath/multipath_rr_cond.plugin  -c ~/pquic/certs/cert.pem -k ~/pquic/certs/key.pem &> pquic_server.log &
2021-05-06 13:11:49,983 [INFO] get_pquic_client_cmd: ~/pquic/picoquicdemo  -P ~/pquic/plugins/multipath/multipath_rr_cond.plugin  -4 -G 5120000 10.1.0.1 4443 &> pquic_client.log
2021-05-06 13:12:03,797 [INFO] <module>: cleanup mininet
bb3da03cb78ee84e: Receiving packet type: 6 (1rtt protected phi0), S1,
bb3da03cb78ee84e:     <ccdb1e8a6bfc3bec>, Seq: 5 (5)
bb3da03cb78ee84e:     Decrypted 4 bytes
bb3da03cb78ee84e:     connection_close, Error 0x0, Reason length 0

Processed 33 bytes, state = 18 (disconnected), return 0

5120000 bytes received
11022.279 ms
Client exit with code = 0
case 2
topo 
topoType:MultiIf
leftSubnet:10.0.
rightSubnet:10.1.
path_c2r_0:40,40,4
path_c2r_1:40,40,4
xp 
xpType:pquic
clientPcap:yes
serverPcap:yes
snaplenPcap:100
pquicPlugins:~/pquic/plugins/multipath/multipath_rr_cond.plugin
pquicSize:5120000
pquic_ifdown:2
logs 
$ mprun -t topo2 -x ./xp_down; tail pquic_client.log 
2021-05-06 13:13:41,012 [INFO] __init__: Extract parameters from file topo2
2021-05-06 13:13:41,013 [INFO] __init__: {'path_c2r_0': '40,40,4', 'path_c2r_1': '40,40,4', 'rightSubnet': '10.1.', 'topoType': 'MultiIf', 'leftSubnet': '10.0.'}
Link type: c2r
Link id: 0
  Delay: 40.0
  Queue Size: 40
  Bandwidth: 4.0
  Loss: 0.0
  Backup: 0
      
Link type: c2r
Link id: 1
  Delay: 40.0
  Queue Size: 40
  Bandwidth: 4.0
  Loss: 0.0
  Backup: 0
      
2021-05-06 13:13:41,017 [INFO] __init__: Initializing MultiInterfaceTopo...
2021-05-06 13:13:41,021 [INFO] apply_topo: Using topo Simple multiple interface topology 
/-sw---bl---sw-\ 
c                r                s
\-sw---bl---sw-/ \-sw---bl---sw-/

2021-05-06 13:13:41,021 [INFO] apply_topo_config: Using topo config <topos.multi_interface.MultiInterfaceConfig object at 0x7fad6ef68710>
2021-05-06 13:13:41,616 [INFO] disable_tso: Disable TSO, GSO and GRO on all interfaces of all nodes
2021-05-06 13:13:41,758 [INFO] configure_interfaces: Configure interfaces using MultiInterfaceConfig...
2021-05-06 13:13:41,759 [INFO] configure_bottleneck: tc qdisc del dev bs_c2r_0_1-eth1 root; tc qdisc del dev bs_c2r_0_1-eth1 ingress 
2021-05-06 13:13:41,777 [INFO] configure_bottleneck: tc qdisc del dev bs_c2r_0_1-eth2 root; tc qdisc del dev bs_c2r_0_1-eth2 ingress 
2021-05-06 13:13:41,793 [INFO] configure_bottleneck: tc qdisc del dev bs_c2r_0_2-eth1 root; tc qdisc del dev bs_c2r_0_2-eth1 ingress 
2021-05-06 13:13:41,814 [INFO] configure_bottleneck: tc qdisc del dev bs_c2r_0_2-eth2 root; tc qdisc del dev bs_c2r_0_2-eth2 ingress 
2021-05-06 13:13:41,831 [INFO] configure_bottleneck: tc qdisc add dev bs_c2r_0_1-eth2 root handle 10: netem  delay 40.0ms limit 50000
2021-05-06 13:13:41,849 [INFO] configure_bottleneck: tc qdisc add dev bs_c2r_0_2-eth2 root handle 1:0 tbf rate 4.0mbit burst 15000 limit 97500.0
2021-05-06 13:13:41,857 [INFO] configure_bottleneck: tc qdisc add dev bs_c2r_0_2-eth1 root handle 10: netem  delay 40.0ms limit 50000
2021-05-06 13:13:41,870 [INFO] configure_bottleneck: tc qdisc add dev bs_c2r_0_1-eth1 root handle 1:0 tbf rate 4.0mbit burst 15000 limit 97500.0
2021-05-06 13:13:41,886 [INFO] configure_bottleneck: tc qdisc del dev bs_c2r_1_1-eth1 root; tc qdisc del dev bs_c2r_1_1-eth1 ingress 
2021-05-06 13:13:41,932 [INFO] configure_bottleneck: tc qdisc del dev bs_c2r_1_1-eth2 root; tc qdisc del dev bs_c2r_1_1-eth2 ingress 
2021-05-06 13:13:41,945 [INFO] configure_bottleneck: tc qdisc del dev bs_c2r_1_2-eth1 root; tc qdisc del dev bs_c2r_1_2-eth1 ingress 
2021-05-06 13:13:41,968 [INFO] configure_bottleneck: tc qdisc del dev bs_c2r_1_2-eth2 root; tc qdisc del dev bs_c2r_1_2-eth2 ingress 
2021-05-06 13:13:41,986 [INFO] configure_bottleneck: tc qdisc add dev bs_c2r_1_1-eth2 root handle 10: netem  delay 40.0ms limit 50000
2021-05-06 13:13:41,996 [INFO] configure_bottleneck: tc qdisc add dev bs_c2r_1_2-eth2 root handle 1:0 tbf rate 4.0mbit burst 15000 limit 97500.0
2021-05-06 13:13:42,015 [INFO] configure_bottleneck: tc qdisc add dev bs_c2r_1_2-eth1 root handle 10: netem  delay 40.0ms limit 50000
2021-05-06 13:13:42,030 [INFO] configure_bottleneck: tc qdisc add dev bs_c2r_1_1-eth1 root handle 1:0 tbf rate 4.0mbit burst 15000 limit 97500.0
2021-05-06 13:13:42,102 [INFO] __init__: Extract parameters from file ./xp_down
2021-05-06 13:13:42,104 [INFO] __init__: Extract parameters from file ./xp_down
2021-05-06 13:13:42,119 [INFO] ping: ping -c 5 -I 10.0.0.1 10.1.0.1 >> ping.log
2021-05-06 13:13:46,232 [INFO] ping: ping -c 5 -I 10.0.1.1 10.1.0.1 >> ping.log
2021-05-06 13:13:50,383 [INFO] run_tcpdump: Activating tcpdump, waiting for it to run
2021-05-06 13:13:55,387 [INFO] run_netem_at: No need to change netem
2021-05-06 13:13:55,399 [INFO] get_pquic_server_cmd: ~/pquic/picoquicdemo  -P ~/pquic/plugins/multipath/multipath_rr_cond.plugin  -c ~/pquic/certs/cert.pem -k ~/pquic/certs/key.pem &> pquic_server.log &
2021-05-06 13:13:57,409 [INFO] get_pquic_client_cmd: ((sleep 2; ip link set dev Client_0-eth1 down)&);~/pquic/picoquicdemo  -P ~/pquic/plugins/multipath/multipath_rr_cond.plugin  -4 -G 5120000 10.1.0.1 4443 &> pquic_client.log
2021-05-06 13:14:14,587 [INFO] <module>: cleanup mininet
6e586e5ba058d0cc: Receiving 33 bytes from 10.1.0.1:4443 at T=14.314553 (5c1a916532b20)
6e586e5ba058d0cc: Receiving packet type: 6 (1rtt protected phi0), S1,
6e586e5ba058d0cc:     <2533849e9bd76492>, Seq: 8 (8)
6e586e5ba058d0cc:     Decrypted 4 bytes
6e586e5ba058d0cc:     connection_close, Error 0x0, Reason length 0

Processed 33 bytes, state = 18 (disconnected), return 0

-1.0
Client exit with code = 0
modifications of minitopo 
diff --git a/experiments/pquic.py b/experiments/pquic.py
index 3ebe491..53acee0 100644
--- a/experiments/pquic.py
+++ b/experiments/pquic.py
@@ -8,6 +8,7 @@ class PQUICParameter(ExperimentParameter):
     CLIENT_PLUGINS = "pquicClientPlugins"
     SERVER_PLUGINS = "pquicServerPlugins"
     SIZE = "pquicSize"
+    IF_DOWN = "pquic_ifdown"
 
     def __init__(self, experiment_parameter_filename):
         super(PQUICParameter, self).__init__(experiment_parameter_filename)
@@ -16,6 +17,7 @@ class PQUICParameter(ExperimentParameter):
             PQUICParameter.CLIENT_PLUGINS: "",
             PQUICParameter.SERVER_PLUGINS: "",
             PQUICParameter.SIZE: 10240000,
+            PQUICParameter.IF_DOWN: 0,
         })
 
 
@@ -40,6 +42,7 @@ class PQUIC(Experiment):
         self.client_plugins = self.experiment_parameter.get(PQUICParameter.CLIENT_PLUGINS)
         self.server_plugins = self.experiment_parameter.get(PQUICParameter.SERVER_PLUGINS)
         self.size = int(self.experiment_parameter.get(PQUICParameter.SIZE))
+        self.if_down = int(self.experiment_parameter.get(PQUICParameter.IF_DOWN))
 
     def prepare(self):
         super(PQUIC, self).prepare()
@@ -62,7 +65,10 @@ class PQUIC(Experiment):
         return s
 
     def get_pquic_client_cmd(self):
-        s = "{} {} -4 -G {} {} 4443 &> {}".format(PQUIC.BIN, self.get_plugin_cmd(client=True), self.size,
+        s = ""
+        if self.if_down:
+            s += "((sleep {}; ip link set dev Client_0-eth1 down)&);".format(self.if_down)
+        s += "{} {} -4 -G {} {} 4443 &> {}".format(PQUIC.BIN, self.get_plugin_cmd(client=True), self.size,
             self.topo_config.get_server_ip(), PQUIC.CLIENT_LOG)
         logging.info(s)
         return s
@qdeconinck
Copy link
Member

Indeed, the current version of the multipath plugin does not handle address mobility and may stuck with a lost path. This is something we may want to fix someday.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@nemethf @qdeconinck