Skip to content

upload: adaptive payload buffer and packet chunk size for fast connections#444

Open
got3nks wants to merge 2 commits intoamule-project:masterfrom
got3nks:adaptive-upload-buffer
Open

upload: adaptive payload buffer and packet chunk size for fast connections#444
got3nks wants to merge 2 commits intoamule-project:masterfrom
got3nks:adaptive-upload-buffer

Conversation

@got3nks
Copy link
Copy Markdown
Contributor

@got3nks got3nks commented Apr 13, 2026

Summary

Two minimal changes to UploadClient.cpp that significantly improve upload throughput on fast connections by scaling two previously hardcoded constants with the actual slot data rate.

Adaptive payload buffer

Replaces the hardcoded 100 KiB per-client payload buffer with a rate-adaptive value:

const uint32 payloadBufferLimit = std::min(std::max(GetUploadDatarate() * 10u, 180u*1024u), 16u*1024u*1024u);
  • Floor: 180 KiB — matches eMule's slow-connection baseline (EMBLOCKSIZE × 1)
  • Scales to ~10 seconds worth of data at the current slot rate
  • Ceiling: 16 MiB — prevents unbounded memory use

The 100 KiB stock buffer drains in ~10ms at 10 MB/s, leaving the socket queue empty while CreateNextBlockPackage() refills synchronously from disk. A larger buffer gives the main thread more headroom between disk reads.

Adaptive packet chunk size

Replaces the hardcoded 10 KiB packet split threshold with a rate-adaptive value:

const uint32 chunkSize = std::min(std::max(GetUploadDatarate() / 8u, 10240u), 131072u);

Applied in both CreateStandardPackets() and CreatePackedPackets().

  • Floor: 10 KiB — preserves stock behaviour on slow connections
  • Ceiling: 128 KiB — well within MAX_PACKET_SIZE (2 MB), compatible with all peers
  • Larger packets reduce per-packet header overhead and throttler loop iterations at high speed

Backward compatibility

  • Connections below ~80 KiB/s use exactly the same constants as stock aMule
  • No wire protocol changes — compatible with all eMule/aMule clients
  • Both changes are in UploadClient.cpp only, no throttler or socket layer touched

Test results

Tested on a 1 Gbps dedicated server running aMule in Docker:

  • Before (stock aMule): ~450 KB/s per upload slot
  • After: 7–8 MB/s per upload slot sustained

Note: peak results were achieved together with #436 (uint16 → uint32 speed limits), which removes the 524 Mbps configuration cap. This PR alone still provides significant improvement on connections below that limit.

Recommended configuration for fast connections

For best results, set an explicit MaxUpload value rather than relying on unlimited mode (MaxUpload=0). In unlimited mode aMule ramps up slots slowly based on observed throughput; an explicit value ensures the full slot count is available immediately.

Setting MaxUpload to your actual uplink capacity also gives the slot allocation algorithm accurate data to work with.

got3nks added 2 commits April 13, 2026 22:16
Buffer ~10 seconds worth of data per upload slot, clamped between
180 KiB (eMule minimum) and 16 MiB (upper bound):

    payloadBufferLimit = clamp(GetUploadDatarate() * 10, 180 KiB, 16 MiB)

Compensates for aMule's synchronous disk I/O: the throttler can drain
the buffer between CreateNextBlockPackage() calls, so a larger lookahead
prevents socket starvation on fast slots. Intended as a temporary measure
until an async disk I/O thread (eMule-style) is implemented.
Scale chunk size with actual slot throughput, floor 10 KiB (stock aMule),
ceil 128 KiB:

    chunkSize = clamp(GetUploadDatarate() / 8, 10 KiB, 128 KiB)

Slow slots keep small chunks for smooth flow; fast slots (>1 MB/s) get
128 KiB chunks to reduce per-packet header overhead and socket queue
churn.
@got3nks got3nks force-pushed the adaptive-upload-buffer branch from ab6af17 to de3ac51 Compare April 13, 2026 20:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant