Skip to content

Commit

Permalink
Switch agent from curl-provided select to polling (#243)
Browse files Browse the repository at this point in the history
Replace the built-in `select(2)`-based I/O driver with [polling](https://github.com/stjepang/polling), which should deliver noticeably better throughput under high activity, _especially_ on Windows.

This implementation took several attempts before I fully understood some of the curl quirks on how it handles sockets; for example, curl will often close sockets before asking them to be de-registered, register and deregister the same file descriptor number between polls (because the OS reused the number), and request sockets to be registered before they are initialized. To handle these quirks this provides a wrapper layer around the polling crate that translates this behavior into something polling can handle.

In addition to verifying all the tests pass, I've also run a sort of "soak test" on Linux, Windows, and macOS for 12 hours straight that makes many requests repeatedly in order to weed out potential random errors (I learned of the possible Win32 error code `ERROR_NOT_FOUND (0x490)` this way!).

Fixes #17.
  • Loading branch information
sagebind committed Mar 10, 2021
1 parent 8444aba commit d2830a2
Show file tree
Hide file tree
Showing 7 changed files with 536 additions and 198 deletions.
1 change: 1 addition & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,7 @@ futures-lite = "1.11"
http = "0.2.1"
log = "0.4"
once_cell = "1"
polling = "2.0"
slab = "0.4"
sluice = "0.5.4"
url = "2.2"
Expand Down

0 comments on commit d2830a2

Please sign in to comment.