Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

scheduler: frequently probe "stale" subflow with reinjected data #348

Open
Tracked by #350
matttbe opened this issue Feb 1, 2023 · 2 comments
Open
Tracked by #350

scheduler: frequently probe "stale" subflow with reinjected data #348

matttbe opened this issue Feb 1, 2023 · 2 comments
Labels
enhancement sched packets scheduler

Comments

@matttbe
Copy link
Member

matttbe commented Feb 1, 2023

Subflows marked as "stale" should be probed from time to time, e.g. once per RTT if there are data to send, at the end of a batch.

Currently, these subflows will be somehow probed if there are data in the subflow send queue thanks to TCP RTO. But once the send queue has been emptied and if the packet scheduler wants to keep the subflow as "stale", it is no longer probed.

If there are data to send at MPTCP level and nothing in the send queue of the subflow marked as "stale", the subflow should be probed by reinjecting already and still in-flight data (one full packet or one byte). It is important here to duplicate data not to block the transfer by using a "bad" subflow.

@matttbe matttbe added enhancement sched packets scheduler labels Feb 1, 2023
@matttbe matttbe added this to Needs triage in MPTCP Future via automation Feb 1, 2023
@sferlin
Copy link

sferlin commented May 2, 2023

One of the challenges here is how to design the probing.

  • If you send probes, it is to verify that the subflow connection is still alive but the RTT value resulting of the probe might be unreliable to consider it for a scheduling decision;
  • If you probe too often, depending on the environment you are e.g. cellualr networks, there is a cost per byte sent and the probe_t1 might not reflecht the RTT of probe_t2 depending on the behavior of the user (mobility);

I agree in principle to mark subflows as stale, but the question is how to best "reactiviate" it without making a suboptimal scheduling decision e.g. if you look at RTT and CWND as metrics. Maybe using already inflight or duplicates is a way, however, will it need some additional logic to sort this out for the sender?

@matttbe
Copy link
Member Author

matttbe commented Jun 2, 2023

Hi @sferlin,

Thank you for your feedback (and sorry for the delay, I was off and a huge backlog :( )

Good point, when we use this, we need to be careful about the consequences.

I think the use-case we had in mind is something like: we were using a subflow but it got bad, we marked it as "stale" but we want to know if we can re-use it. So it should be fine to re-use it and if we would like to (e.g. because we need more capacity) we just need to probe it min once every 1 RTT to be able to unmark it.

matttbe pushed a commit that referenced this issue May 28, 2024
Add a test case to assert that the skb->pkt_type which was set from the BPF
program is retained from the netkit xmit side to the peer's device at tcx
ingress location.

  # ./vmtest.sh -- ./test_progs -t netkit
  [...]
  ./test_progs -t netkit
  [    1.140780] bpf_testmod: loading out-of-tree module taints kernel.
  [    1.141127] bpf_testmod: module verification failed: signature and/or required key missing - tainting kernel
  [    1.284601] tsc: Refined TSC clocksource calibration: 3408.006 MHz
  [    1.286672] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x311fd9b189d, max_idle_ns: 440795225691 ns
  [    1.290384] clocksource: Switched to clocksource tsc
  #345     tc_netkit_basic:OK
  #346     tc_netkit_device:OK
  #347     tc_netkit_multi_links:OK
  #348     tc_netkit_multi_opts:OK
  #349     tc_netkit_neigh_links:OK
  #350     tc_netkit_pkt_type:OK
  Summary: 6/0 PASSED, 0 SKIPPED, 0 FAILED

Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Link: https://lore.kernel.org/r/20240524163619.26001-4-daniel@iogearbox.net
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement sched packets scheduler
Projects
Status: Needs triage
MPTCP Future
  
Needs triage
Development

No branches or pull requests

2 participants