Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Default value for Asynchronous I/O threads should be changed #11461

Open
FranciscoPombal opened this issue Nov 8, 2019 · 4 comments

Comments

@FranciscoPombal
Copy link
Contributor

@FranciscoPombal FranciscoPombal commented Nov 8, 2019

Please provide the following information

qBittorrent version and Operating System

Any version on any OS.

If on linux, libtorrent-rasterbar and Qt version

Not applicable

What is the problem

qBittorrent currently ships with the Asynchronous I/O threads advanced libtorrent config set to 4. As per my own testing based on this comment, SHA-1 hashing happens on n/4 threads, where n is the value of this setting. So, having this set to 4 leaves a lot of performance on the table especially in rechecks for most modern machines which usually have at least 4 hardware threads (e.g 2c/4t CPUs on notebooks).

What is the expected behavior

qBittorrent should either ship with this set to at least 16 or have some way of auto-setting it to 4*n_hardware_threads.

Steps to reproduce

Download moderately large torrent that takes some time to recheck.
Force recheck the torrent.
Observe how only one thread does the work
Change Asynchronous I/O threads to n*4 where n is the number of hardware threads in your machine.
Force recheck again, observe how many threads do the work and it completes much faster.

Extra info(if any)

I edited the wiki page with these findings - https://github.com/qbittorrent/qBittorrent/wiki/Explanation-of-Options-in-qBittorrent/957bb3f50d0cff41f33b58d90b581409a648b089

Setting values greater than n*4 for this setting will most likely not help and at worst reduce performance as well due to extra thread overhead.

Increasing the value of this setting might not help (at least very much) if the storage medium is a severe bottleneck for the system already, but in that case it should not hurt much if at all either.

@arvidn
I'd like your input on this, namely regarding the correctness of my analysis and whether or not all of this is expected behaviour from qBIttorrentlibtorrent.

@xavier2k6

This comment has been minimized.

Copy link

@xavier2k6 xavier2k6 commented Nov 19, 2019

From Libtorrent

If disk IO is a bottleneck (which I would expect it to be given a fast internet connection), you probably want to set aio_threads to something much greater than 2. probably 16 or 32.

If i understand that setting correctly, the value corresponds to engaged cpu threads. If you have a hyperthreaded 2 core cpu, then setting the value to 4 would engage all 4 threads. But i read somewhere that utilizing both threads on 1 core could mean it's fighting it self over resources, so the value should be physical cores. But may be that is not correct considering the values you put.

Those are specifically disk I/O threads, so they will mostly either be idle (waiting for work) or suspended in a blocking disk I/O system call. Either way, they won't do a lot of actual computation.
However, one in every 4 disk I/O thread is dedicated to perform the SHA-1 hashing of incoming data, so those can actually end up using CPU while downloading.

@FranciscoPombal

This comment has been minimized.

Copy link
Contributor Author

@FranciscoPombal FranciscoPombal commented Nov 20, 2019

From Libtorrent

If disk IO is a bottleneck (which I would expect it to be given a fast internet connection), you probably want to set aio_threads to something much greater than 2. probably 16 or 32.

If i understand that setting correctly, the value corresponds to engaged cpu threads. If you have a hyperthreaded 2 core cpu, then setting the value to 4 would engage all 4 threads. But i read somewhere that utilizing both threads on 1 core could mean it's fighting it self over resources, so the value should be physical cores. But may be that is not correct considering the values you put.
Those are specifically disk I/O threads, so they will mostly either be idle (waiting for work) or suspended in a blocking disk I/O system call. Either way, they won't do a lot of actual computation.
However, one in every 4 disk I/O thread is dedicated to perform the SHA-1 hashing of incoming data, so those can actually end up using CPU while downloading.

Nice, this pretty much confirms my initial analysis/prediction.

@a-raccoon

This comment has been minimized.

Copy link

@a-raccoon a-raccoon commented Nov 20, 2019

We just went through disabling qBittorrent from being able to perform multiple force-rechecks at the same time. In the majority of machine configurations, you do not want to have two torrents being checked / rechecked, otherwise it causes undue disk thrashing and random seeks. Almost all drives perform best with sequential seeks, reading one-file-at-a-time.

@FranciscoPombal

This comment has been minimized.

Copy link
Contributor Author

@FranciscoPombal FranciscoPombal commented Nov 20, 2019

We just went through disabling qBittorrent from being able to perform multiple force-rechecks at the same time. In the majority of machine configurations, you do not want to have two torrents being checked / rechecked, otherwise it causes undue disk thrashing and random seeks. Almost all drives perform best with sequential seeks, reading one-file-at-a-time.

Sure, but this discussion is not about force-rechecks, it's about the SHA-1 calculations of pieces that are downloading.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants
You can’t perform that action at this time.