Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for System.IO.Pipelines #146

Closed
smatsson opened this issue Aug 17, 2021 · 3 comments
Closed

Add support for System.IO.Pipelines #146

smatsson opened this issue Aug 17, 2021 · 3 comments
Assignees
Milestone

Comments

@smatsson
Copy link
Collaborator

This issue has been split from #116

Original PR: #123

Dev branch: pipelines-dev

@smatsson smatsson added this to the 2.4+1 milestone Aug 17, 2021
@smatsson smatsson self-assigned this Aug 17, 2021
@smatsson
Copy link
Collaborator Author

Initial testing looks good. Roughly half the memory usage :) Downside: A bit longer write times (marginal) and higher CPU usage. Still need to add unit tests and do some more benchmarking before releasing.

Might also add a build target for .NET6 as there are multiple huge performance improvements when it comes to file IO. Will wait for the final release of .NET6 before releasing that on nuget.

@smatsson
Copy link
Collaborator Author

Update: Stress test seems good. Currently investigating whether or not to enable pipelines by default.

@smatsson
Copy link
Collaborator Author

smatsson commented Aug 27, 2021

Stress test output. Test was 3 test runs with 50 simulations clients, each uploading 20 files (à 10 MB) without any file metadata. See source file: https://github.com/tusdotnet/tusdotnet/blob/pipelines-dev/Source/tusdotnet.performance/Program.cs#L30 (modified to not include metadata to only measure disk write performance). Test server was using a 11 GB RAM disk for writing files.

Pipelines

Test name Avg memory used (MB) Avg CPU usage (%) Time taken (ms)
Pipelines 1 379 28 90 499
Pipelines 2 435 32 73 322
Pipelines 3 405 29 76 407

10 minutes after Pipelines 3: 0% CPU usage, 48 MB RAM

Streams

Test name Avg memory used (MB) Avg CPU usage (%) Time taken (ms)
Streams 1 718 28 60 161
Streams 2 500 30 59 952
Streams 3 601 31 53 296

10 minutes after Streams 3: 0% CPU usage, 253 MB RAM

@smatsson smatsson mentioned this issue Sep 22, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant