Skip to content

perf: Refactor onion_client.c do_friends() to reduce network traffic#2078

Merged
toktok-releaser merged 2 commits into
TokTok:masterfrom
JFreegman:onion_lookups
Mar 3, 2022
Merged

perf: Refactor onion_client.c do_friends() to reduce network traffic#2078
toktok-releaser merged 2 commits into
TokTok:masterfrom
JFreegman:onion_lookups

Conversation

@JFreegman

@JFreegman JFreegman commented Feb 19, 2022

Copy link
Copy Markdown
Member

This PR simplifies the logic determining how often we
send announce request packets to offline peers and how often
we attempt to re-populate our nodes lists. It also removes
some randomness from packet sending that was intended to
reduce network traffic but greatly increased the code complexity
and didn't really do all that much to help.

We now aggressively search the DHT for offline friends when we first
bootstrap, add a new friend, or when a friend goes offline, and
exponentially increase the lookup interval based on how many
successful lookup requests we've already made, with a max
interval of 60 minutes per node, per friend (with 8 max nodes).
Requests are also now spaced evenly apart for each node
which guarantees at least one lookup per friend, per n seconds,
where n depends on the current timeout.

We also no longer spam node requests whenever our list of living
nodes drops below the maximum. Instead, we repopulate the nodes
list only when it drops to or below half the maximum, or after
a 10 minute timeout.

I ran two identical tox instances simultaneously for 10+ hours using
the netprof branch, and found that these changes reduced total tox
network traffic by between 30% and 50% (there is some randomness
based on the amount of in-traffic you get). Results are below (click
links for full results):

Current master with UDP enabled:

Run time: 40341 seconds
Average kilobytes per second: 7.69
Average packets per second: 38
Average packet size: 201 bytes

Total bytes:            310048540
Total bytes sent:       186386371 (60.12%)
Total bytes recv:       123662169 (39.88%)

This branch with UDP enabled:

--- Tox network profile log dump ---
Run time: 40401 seconds
Average kilobytes per second: 3.90
Average packets per second: 27
Average packet size: 142 bytes

Total bytes:            157547766
Total bytes sent:       104530935 (66.35%)
Total bytes recv:       53016831 (33.65%)

Current master with UDP disabled:

Run time: 48829 seconds
Average kilobytes per second: 0.99
Average packets per second: 5
Average packet size: 184 bytes

Total bytes:            48251725
Total bytes sent:       37546683 (77.81%)
Total bytes recv:       10705042 (22.19%)

This branch with UDP disabled:

Run time: 48843 seconds
Average kilobytes per second: 0.47
Average packets per second: 3
Average packet size: 124 bytes

Total bytes:            23143619
Total bytes sent:       17819044 (76.99%)
Total bytes recv:       5324575 (23.01%)

This change is Reviewable

@JFreegman JFreegman added the performance A code change that improves performance label Feb 19, 2022
@JFreegman JFreegman added this to the v0.2.x milestone Feb 19, 2022
@JFreegman JFreegman changed the title Refactor onion_client.c do_friends() to reduce network bandwidth perf: Refactor onion_client.c do_friends() to reduce network bandwidth Feb 19, 2022
@codecov

codecov Bot commented Feb 19, 2022

Copy link
Copy Markdown

Codecov Report

Merging #2078 (0dca481) into master (6c2014d) will increase coverage by 0.06%.
The diff coverage is 96.51%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #2078      +/-   ##
==========================================
+ Coverage   78.15%   78.22%   +0.06%     
==========================================
  Files         117      117              
  Lines       21516    21527      +11     
==========================================
+ Hits        16816    16839      +23     
+ Misses       4700     4688      -12     
Impacted Files Coverage Δ
toxcore/onion_client.c 91.25% <96.51%> (-0.63%) ⬇️
toxcore/onion_announce.c 90.41% <0.00%> (-0.92%) ⬇️
toxcore/net_crypto.c 89.43% <0.00%> (-0.75%) ⬇️
toxcore/friend_connection.c 83.06% <0.00%> (-0.70%) ⬇️
toxcore/onion.c 87.18% <0.00%> (-0.28%) ⬇️
toxcore/DHT.c 83.77% <0.00%> (ø)
toxcore/Messenger.c 83.83% <0.00%> (+0.14%) ⬆️
toxav/toxav.c 69.37% <0.00%> (+0.29%) ⬆️
toxav/groupav.c 75.91% <0.00%> (+1.33%) ⬆️
... and 2 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6c2014d...0dca481. Read the comment docs.

@JFreegman JFreegman changed the title perf: Refactor onion_client.c do_friends() to reduce network bandwidth WIP: Refactor onion_client.c do_friends() to reduce network bandwidth Feb 19, 2022
@auto-add-label auto-add-label Bot added WIP and removed performance A code change that improves performance labels Feb 19, 2022
@JFreegman JFreegman modified the milestones: v0.2.x, v0.2.17 Feb 19, 2022
@JFreegman JFreegman changed the title WIP: Refactor onion_client.c do_friends() to reduce network bandwidth WIP: Refactor onion_client.c do_friends() to reduce network traffic Feb 19, 2022
@JFreegman JFreegman force-pushed the onion_lookups branch 2 times, most recently from 27de9a5 to 8a7a9d9 Compare February 19, 2022 07:08
@iphydf iphydf marked this pull request as draft February 19, 2022 13:35
@JFreegman JFreegman force-pushed the onion_lookups branch 7 times, most recently from 9a05ed5 to aca376d Compare February 20, 2022 17:35
@JFreegman JFreegman marked this pull request as ready for review February 20, 2022 17:38
@JFreegman JFreegman changed the title WIP: Refactor onion_client.c do_friends() to reduce network traffic perf: Refactor onion_client.c do_friends() to reduce network traffic Feb 20, 2022
@auto-add-label auto-add-label Bot added performance A code change that improves performance and removed WIP labels Feb 20, 2022
@JFreegman JFreegman changed the title perf: Refactor onion_client.c do_friends() to reduce network traffic wip: Refactor onion_client.c do_friends() to reduce network traffic Feb 20, 2022
@auto-add-label auto-add-label Bot added WIP and removed performance A code change that improves performance labels Feb 20, 2022
@JFreegman JFreegman force-pushed the onion_lookups branch 2 times, most recently from 41e0d2c to 8dc9cf4 Compare February 20, 2022 18:35
@JFreegman JFreegman force-pushed the onion_lookups branch 8 times, most recently from 83a191e to 5120e59 Compare February 24, 2022 02:55
@JFreegman JFreegman changed the title wip: Refactor onion_client.c do_friends() to reduce network traffic perf: Refactor onion_client.c do_friends() to reduce network traffic Feb 24, 2022
@auto-add-label auto-add-label Bot added the performance A code change that improves performance label Feb 24, 2022
Comment thread toxcore/onion_client.c Outdated
@JFreegman JFreegman force-pushed the onion_lookups branch 4 times, most recently from e294a9b to c1d1a23 Compare February 25, 2022 18:28
@JFreegman JFreegman removed the WIP label Feb 25, 2022
@JFreegman JFreegman force-pushed the onion_lookups branch 2 times, most recently from 8d5e25f to 5a75979 Compare February 28, 2022 22:29
@iphydf iphydf modified the milestones: v0.2.17, v0.2.19 Feb 28, 2022
Comment thread toxcore/onion_client.c Outdated
This commit simplifies the logic determining how often we
send announce request packets to offline peers and how often
we attempt to re-populate our nodes lists. It also removes
some randomness from packet sending that was intended to
reduce network traffic but greatly increased the code complexity
and didn't really do all that much to help.

We now aggressively search the DHT for offline friends when we first
bootstrap, add a new friend, or when a friend goes offline, for a
short duration until we've sent a certain number of successful
lookup requests. Thereafter, we send one lookup request every 3
minutes for each node associated with the friend (typically
between 4 and 8), for each friend.

We also no longer spam node requests whenever our list of living
nodes drops below the maximum. Instead, we repopulate the nodes
list only when it drops to or below half the maximum, or after
a 10 minute timeout.

The effects of these changes is a decrease in overall tox network
traffic by about 30-40% with UDP enabled, and a bit less than
that with UDP disabled. The network may also be more reliable
overall because everything is significantly more deterministic
and less random
We now exponentially reduce the frequency of friend lookups based on how many
lookups we've already made for an offline friend
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

performance A code change that improves performance

Development

Successfully merging this pull request may close these issues.

3 participants