Skip to content

Perf: use io copy with buffer#6389

Merged
xhofe merged 2 commits intoAlistGo:mainfrom
Mmx233:improve/io-perf
Apr 25, 2024
Merged

Perf: use io copy with buffer#6389
xhofe merged 2 commits intoAlistGo:mainfrom
Mmx233:improve/io-perf

Conversation

@Mmx233
Copy link
Contributor

@Mmx233 Mmx233 commented Apr 24, 2024

Using io.CopyBuffer can reduce the resource consumption caused by multiple memory allocations and appropriately increase the buffer size to optimize the performance of Copy.

This may mitigate the issue described in #6325

@Mmx233 Mmx233 marked this pull request as draft April 24, 2024 14:53
@pull-request-size pull-request-size bot added size/M and removed size/S labels Apr 24, 2024
@Mmx233
Copy link
Contributor Author

Mmx233 commented Apr 24, 2024

Test image available at mmx233/alist:v3.33.0-rc2, I will test it at tomorrow morning for my test environment is down now.

@Mmx233 Mmx233 marked this pull request as ready for review April 24, 2024 17:56
@Mmx233
Copy link
Contributor Author

Mmx233 commented Apr 25, 2024

My test environment is keeping unavailable. I tested it on my local computer. Using a 20GB single file, Windows amd64 SSD, with signature enabled, HTTP service, Chrome single-threaded download, and Free Download Manager 15-threaded download.

Only the occupation of alist is being measured. Iowait is an important performance metric but cannot be calculated in the current environment. I only ran it once and manually retrieved the data from the task manager, and there may be many other unstable factors in between. So the resulting outcome should be considered for reference only.

original io copy

Threads CPU Usage Memorary Usage Speed
1 10% 19.2Mb 400Mb/s
15 12% 20.9Mb 500Mb/s

io copy buffer

Threads CPU Usage Memorary Usage Speed
1 4.5% 16.5Mb 400Mb/s
15 8% 19Mb 485Mb/s

@xhofe xhofe merged commit b95df1d into AlistGo:main Apr 25, 2024
@Mmx233 Mmx233 deleted the improve/io-perf branch May 7, 2024 08:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants