Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix high CPU usage #187

Merged
merged 1 commit into from Nov 30, 2020
Merged

Fix high CPU usage #187

merged 1 commit into from Nov 30, 2020

Conversation

c0re100
Copy link
Owner

@c0re100 c0re100 commented Nov 30, 2020

I discovered new implementation(ban bad peers plugin) can consume lots of CPU resources.
Since each connection has been exchange Peer ID and libtorrent will parse the client name on handshake stage.
So we can remove on_interested/on_have....checking handlers.

high CPU usage (when session have many connections, ~800 on my test):
photo_2020-11-30_05-54-50

After remove (~600 connections):
{6D9D049C-865B-4A12-9E03-690FB320F88A}

Please feel free to test and post your test result here.

@c0re100 c0re100 changed the base branch from master to v4_3_x November 30, 2020 03:12
@c0re100 c0re100 changed the title Fix high cpu usage Fix high CPU usage Nov 30, 2020
@Kolcha
Copy link

Kolcha commented Nov 30, 2020

looks like reasonable solution, and it should be enough. but when I was implementing this filter, I noticed that on_handshake() is called not for each peer (very likely depending on the client on another side, I didn't see it for uTorrent in most cases), so I added other handlers which are called in any case.

I think it is worth to keep at least on_bitfield() handler, this is a rare call, and it happens before a series of on_have(). it looks like the first call before anything else. and in this call client name field contains data reported by peer client rather than just resolved by libtorrent. but this also is not always called...

maybe I'm wrong, something was missed during testing, and everything if fine... but I have concerns... I'll disable any used filters on my server and collect some statistics of these calls (which call is called first) and back with it here.
also I can share such "debug filter" which can collect statistics, and you will be able to try it by yourself. it is looks like you have much more torrents rather than me, or they are just more popular. because even with ~2000 seeding torrents I never have 600-800 connections, only 100-150, and this is very rare cases.

@c0re100
Copy link
Owner Author

c0re100 commented Nov 30, 2020

It look like bitfield is optional or send after handshake only.
But it is a peer messages, so client can send this peer messages at anytime.
Hope I understand BEP0003 correctly.
I will test it tonight.

ref: https://www.bittorrent.org/beps/bep_0003.html

About the connection test:
I am using a dedicated server and using nyaa(subekei) RSS for testing.
After 20-30 torrents is added, it is easy to connect 600 peers(or more) if you have a good network.

@Kolcha
Copy link

Kolcha commented Nov 30, 2020

so, I'm back. I added ~25 torrents, wait until few of them are downloaded and seed for few hours.
during this time I got ~8000 unique IPs in my database, and analyzed what handler was called first for each of them first. the result seems to be good - on_handshake() was called for all of them! so, something was wrong when I developed that filter plugin, and filtering peers only in on_handshake() is completely fine and enough.

@c0re100 c0re100 merged commit 99c5dff into v4_3_x Nov 30, 2020
@c0re100 c0re100 deleted the fix_high_cpu_usage branch November 30, 2020 17:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants