Skip to content

Conversation

@Fizn-Ahmd
Copy link

When a transfer is paused via curl_easy_pause(), the progress callback gets invoked excessively (hundreds of thousands of times per second), causing 100% CPU usage.

The issue occurs because Curl_rlimit_wait_ms() returns 0 when blocked, preventing the multi-handle from entering MSTATE_RATELIMITING state. This causes the event loop to spin without any delay.

When blocked (paused), return 1000ms wait time to ensure the multi-handle properly waits between iterations while still allowing periodic progress callback updates.

Reproduced with: ~260,000 callbacks/sec, 100% CPU usage
After fix: ~4-5 callbacks/sec, 0.2% CPU usage

Follow-up to 24b36fd

When a transfer is paused via curl_easy_pause(), the progress callback
gets invoked excessively (hundreds of thousands of times per second),
causing 100% CPU usage.

The issue occurs because Curl_rlimit_wait_ms() returns 0 when blocked,
preventing the multi-handle from entering MSTATE_RATELIMITING state.
This causes the event loop to spin without any delay.

When blocked (paused), return 1000ms wait time to ensure the multi-handle
properly waits between iterations while still allowing periodic progress
callback updates for UI responsiveness.

Reproduced with: ~260,000 callbacks/sec, 100% CPU usage
After fix: ~4-5 callbacks/sec, 0.2% CPU usage

Follow-up to 24b36fd
@testclutch
Copy link

Analysis of PR #20091 at bcc135f6:

Test ../../tests/http/test_07_upload.py::TestUpload::test_07_42c_upload_disconnect[h2] failed, which has NOT been flaky recently, so there could be a real issue in this PR. Note that this test has failed in 2 different CI jobs (the link just goes to one of them).

Test ../../tests/http/test_07_upload.py::TestUpload::test_07_42a_upload_disconnect[http/1.1] failed, which has NOT been flaky recently, so there could be a real issue in this PR.

Test ../../tests/http/test_07_upload.py::TestUpload::test_07_42a_upload_disconnect[h2] failed, which has NOT been flaky recently, so there could be a real issue in this PR. Note that this test has failed in 2 different CI jobs (the link just goes to one of them).

Test ../../tests/http/test_07_upload.py::TestUpload::test_07_42b_upload_disconnect[h2] failed, which has NOT been flaky recently, so there could be a real issue in this PR. Note that this test has failed in 2 different CI jobs (the link just goes to one of them).

Generated by Testclutch

@bagder
Copy link
Member

bagder commented Dec 25, 2025

Clearly, this breaks tests.

@bagder
Copy link
Member

bagder commented Dec 26, 2025

@icing do you have any alternative take on this?

icing added a commit to icing/curl that referenced this pull request Dec 29, 2025
Fix the pollset in perform state to not add sockets for directions
that are blocked. This otherwise will lead to busy loops for a
transfer that cannot be progressed.

refs curl#20091
@icing
Copy link
Contributor

icing commented Dec 29, 2025

@Fizn-Ahmd thanks for your report and finding that problem. The bug is that the transfer socket is still being monitored for events, even though the send/recv is blocked and never handles the sockets. This raised the socket event again and this continues over and over.

While your proposed fix mediates this problem in your case by an imposed idle wait, curl needs to handle this differently. The test failures are scenarios when an upload is paused, but server messages still need to be received. A forced block time prevents that.

I propose an alternate fix in #20109, where sockets are no longer monitored for a blocked direction. It would be great if you could verify that this works for you as well. Thanks!

@icing icing self-assigned this Dec 29, 2025
@Fizn-Ahmd
Copy link
Author

Tested #20109 - works perfectly. CPU usage back to normal, no more callback spam. Thanks for the proper fix!

Closing in favor of #20109

@Fizn-Ahmd Fizn-Ahmd closed this Dec 29, 2025
bagder pushed a commit that referenced this pull request Dec 29, 2025
Fix the pollset in perform state to not add sockets for directions
that are blocked. This otherwise will lead to busy loops for a
transfer that cannot be progressed.

Reported-by: Fizn-Ahmd on github
Fixes #20091
Closes #20109
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

4 participants