add BBR congestion algorithm from quiche

[fbzhong]

Hi,

I am trying to implement bbr algorithm support from chromium to quic-go. Currently I migrated the current bbr src (bbr_sender.cc) to quic-go.

0. Which version of BBR?
   https://quiche.googlesource.com/quiche.git/+/66dea072431f94095dfc3dd2743cb94ef365f7ef/quic/core/congestion_control/bbr_sender.cc

1. How to enable BBR?
   set GODEBUG=bbr=1 to enable, otherwise cubic will be the default congestion algorithm.

2. Sorry, so far there is no unit tests. However I already used the BBR version in my own private project. It works well, significantly faster than cubic.
   For an instance, I try send data from AWS US server to AWS JP server, both servers are t2.small. The rate of Cubic is 75Mbps, and the rate of BBR is 85Mbps.

3. According to the src from Chromium, those developers are experiencing different configs of BBR. I'd like to try those different configs. However right now it looks hard to pass config values to congestion algorithm. I hope we may find a way to do that.

Any comments, please let me know.

[googlebot]

Thanks for your pull request. It looks like this may be your first contribution to a Google open source project (if not, look below for help). Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

📝 Please visit https://cla.developers.google.com/ to sign.

Once you've signed (or fixed any issues), please reply here (e.g. I signed it!) and we'll verify it.

---

What to do if you already signed the CLA

Individual signers

• It's possible we don't have your GitHub username or you're using a different email address on your commit. Check your existing CLA data and verify that your email is set on your git commits.

Corporate signers

• Your company has a Point of Contact who decides which employees are authorized to participate. Ask your POC to be added to the group of authorized contributors. If you don't know who your Point of Contact is, direct the Google project maintainer to go/cla#troubleshoot (Public version).
• The email used to register you as an authorized contributor must be the email used for the Git commit. Check your existing CLA data and verify that your email is set on your git commits.
• The email used to register you as an authorized contributor must also be attached to your GitHub account.

ℹ️ Googlers: Go here for more info.

[codecov-io]

Codecov Report

Merging #1888 into master will decrease coverage by 6.17%.
The diff coverage is 4.4%.

@@            Coverage Diff             @@
##           master    #1888      +/-   ##
==========================================
- Coverage   86.43%   80.27%   -6.17%     
==========================================
  Files          98      102       +4     
  Lines        6701     7231     +530     
==========================================
+ Hits         5792     5804      +12     
- Misses        651     1164     +513     
- Partials      258      263       +5

Impacted Files	Coverage Δ
internal/congestion/bbr_sender.go	0% <0%> (ø)
internal/congestion/bandwidth_sampler.go	0% <0%> (ø)
internal/congestion/windowed_filter.go	0% <0%> (ø)
internal/ackhandler/packet.go	0% <0%> (ø)
internal/ackhandler/sent_packet_handler.go	73.37% <48.98%> (-4.06%)	⬇️
session.go	76.36% <0%> (+0.27%)	⬆️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 8afed81...a7430fa. Read the comment docs.

[googlebot]

CLAs look good, thanks!

ℹ️ Googlers: Go here for more info.

[fbzhong]

Hi @marten-seemann

Is that possible to merge this PR? Do you have any comments on that?

[jared2501]

Wow this is exciting to see! I'm going to try this on one of our LTE links tomorrow.

[jared2501]

Did some quic(k) testing on an LTE link:

host:~
$ GODEBUG=bbr=1 ./testcmd -addr 0.0.0.0:6001 -server
2019/05/21 14:21:22 quic-go: bbr is enabled.
2019/05/21 14:21:22 Accepted!
2019/05/21 14:21:23 10.57 Mbit/s 1.32 MB/s
2019/05/21 14:21:24 13.96 Mbit/s 1.75 MB/s
2019/05/21 14:21:25 8.32 Mbit/s 1.04 MB/s
2019/05/21 14:21:26 17.08 Mbit/s 2.13 MB/s
2019/05/21 14:21:27 2.17 Mbit/s 0.27 MB/s
2019/05/21 14:21:28 6.17 Mbit/s 0.77 MB/s
2019/05/21 14:21:29 33.62 Mbit/s 4.20 MB/s
2019/05/21 14:21:30 1.11 Mbit/s 0.14 MB/s
2019/05/21 14:21:31 8.74 Mbit/s 1.09 MB/s
2019/05/21 14:21:32 31.29 Mbit/s 3.91 MB/s
2019/05/21 14:21:33 13.14 Mbit/s 1.64 MB/s
2019/05/21 14:21:34 8.14 Mbit/s 1.02 MB/s
2019/05/21 14:21:35 3.55 Mbit/s 0.44 MB/s
^C
host:~
$ GODEBUG=bbr=0 ./testcmd -addr 0.0.0.0:6001 -server
2019/05/21 14:21:44 Accepted!
2019/05/21 14:21:45 9.28 Mbit/s 1.16 MB/s
2019/05/21 14:21:46 8.47 Mbit/s 1.06 MB/s
2019/05/21 14:21:47 10.93 Mbit/s 1.37 MB/s
2019/05/21 14:21:48 0.62 Mbit/s 0.08 MB/s
2019/05/21 14:21:49 18.10 Mbit/s 2.26 MB/s
2019/05/21 14:21:50 9.22 Mbit/s 1.15 MB/s
2019/05/21 14:21:51 11.50 Mbit/s 1.44 MB/s
2019/05/21 14:21:52 12.10 Mbit/s 1.51 MB/s
2019/05/21 14:21:53 13.50 Mbit/s 1.69 MB/s
2019/05/21 14:21:54 13.43 Mbit/s 1.68 MB/s
2019/05/21 14:21:55 13.37 Mbit/s 1.67 MB/s
2019/05/21 14:21:56 13.15 Mbit/s 1.64 MB/s
2019/05/21 14:21:57 13.85 Mbit/s 1.73 MB/s
2019/05/21 14:21:58 13.16 Mbit/s 1.64 MB/s
2019/05/21 14:21:59 13.79 Mbit/s 1.72 MB/s

Looks a little choppy... I'm running TCP BBR and it's achieving roughly the same throughput as the cubic flow from quic-go.

[fbzhong]

Hi @jared2501 ,

Thanks for testing. I did same test before, it has better performance when transfer data from two different region, from Tokyo to US West.

Any comments or suggestion, pls let me know. I want more people helping me to make this better.

[marten-seemann]

Hello @fbzhong, thank you for this PR! BBR has been on our TODO list for a long time already. However, at this moment, we're in the process of getting Reno right first, which takes a lot of effort. I started working on a benchmarking suite for quic-go, but it doesn't include a lot of scenarios yet, and will need A LOT more work. Congestion control is complicated and far from straightforward to test. There are a lot of corner cases which are difficult to simulate, but which have to be included in any fair comparison between different congestion control algorithms.

I'd therefore like to focus on getting one congestion control algorithm up to speed. Adding additional congestion controllers at this point would just increase our technical debt, so I hope you can understand that we're not going to merge your PR.

[fbzhong]

Hi @marten-seemann,

Good to know. Let me know if I could help.

And I will keep sync my BBR repo, I am using it now :)

[Zeymo]

Hi @marten-seemann,

Good to know. Let me know if I could help.

And I will keep sync my BBR repo, I am using it now :)

@fbzhong Looks a little choppy +1 , any sync from ur BBR repo so far ? it's make sense of data transfer under high latency/ high loss

may be upgrade to BBR2 alpha?

[fbzhong]

@Zeymo So far no plan to keep sync with latest quic-go.

[Zeymo]

https://github.com/fbzhong/quic-go/blob/a7430fa9d53a11c3156dc50625fbdf23f6173f33/internal/ackhandler/sent_packet_handler.go#L282

@fbzhong duplicate variable of lostPacketsForEvent drop lost packet event for congestionEventHandler.OnCongestionEvent and effect recovery-state