Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor: use key/value cursor for serialization, retire ring buffer #230

Merged
merged 4 commits into from
Dec 15, 2023

Conversation

MrCroxx
Copy link
Collaborator

@MrCroxx MrCroxx commented Dec 13, 2023

What's changed and what's your intention?

Please explain IN DETAIL what the changes are in this PR and why they are needed. :D

As titled.

Checklist

  • I have written necessary rustdoc comments
  • I have added necessary unit tests and integration tests
  • I have passed make check and make test or make all in my local envirorment.

Related issues or PRs (optional)

#216
close #226

Signed-off-by: MrCroxx <mrcroxx@outlook.com>
@MrCroxx MrCroxx added the feature New feature or request label Dec 13, 2023
@MrCroxx MrCroxx self-assigned this Dec 13, 2023
@MrCroxx
Copy link
Collaborator Author

MrCroxx commented Dec 13, 2023

Benchmark

PR

Total:
disk total iops: 16239.5
disk total throughput: 614.8 MiB/s
disk read iops: 8700.9
disk read throughput: 115.6 MiB/s
disk write iops: 7538.5
disk write throughput: 499.2 MiB/s
insert iops: 7514.1/s
insert throughput: 469.6 MiB/s
insert lat p50: 58us
insert lat p90: 149us
insert lat p99: 1447us
insert lat p999: 7391us
insert lat p9999: 19327us
insert lat p99999: 35583us
insert lat pmax: 40191us
get iops: 289454.7/s
get miss: 54.77%
get throughput: 8.0 GiB/s
get hit lat p50: 1247us
get hit lat p90: 2543us
get hit lat p99: 7231us
get hit lat p999: 32511us
get hit lat p9999: 44799us
get hit lat p99999: 52991us
get hit lat pmax: 68095us
get miss lat p50: 179us
get miss lat p90: 679us
get miss lat p99: 3215us
get miss lat p999: 7615us
get miss lat p9999: 14207us
get miss lat p99999: 25855us
get miss lat pmax: 40959us

100s mem

image

main

Total:
disk total iops: 21664.7
disk total throughput: 685.2 MiB/s
disk read iops: 14151.4
disk read throughput: 187.9 MiB/s
disk write iops: 7513.2
disk write throughput: 497.3 MiB/s
insert iops: 7484.6/s
insert throughput: 467.8 MiB/s
insert lat p50: 83us
insert lat p90: 547us
insert lat p99: 5439us
insert lat p999: 20607us
insert lat p9999: 59391us
insert lat p99999: 73215us
insert lat pmax: 80383us
get iops: 288570.4/s
get miss: 54.65%
get throughput: 8.0 GiB/s
get hit lat p50: 1807us
get hit lat p90: 6143us
get hit lat p99: 25343us
get hit lat p999: 47359us
get hit lat p9999: 62975us
get hit lat p99999: 114687us
get hit lat pmax: 196607us
get miss lat p50: 313us
get miss lat p90: 1935us
get miss lat p99: 5919us
get miss lat p999: 13695us
get miss lat p9999: 29951us
get miss lat p99999: 43519us
get miss lat pmax: 65023us

100s mem

image

Copy link

codecov bot commented Dec 13, 2023

Codecov Report

Attention: 3 lines in your changes are missing coverage. Please review.

Comparison is base (96c6caf) 75.45% compared to head (0f25661) 75.65%.

Files Patch % Lines
foyer-common/src/code.rs 50.00% 2 Missing ⚠️
foyer-storage/src/flusher.rs 50.00% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #230      +/-   ##
==========================================
+ Coverage   75.45%   75.65%   +0.19%     
==========================================
  Files          47       47              
  Lines        5847     5775      -72     
==========================================
- Hits         4412     4369      -43     
+ Misses       1435     1406      -29     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@MrCroxx
Copy link
Collaborator Author

MrCroxx commented Dec 13, 2023

@hzxa21 @Li0k

@Li0k
Copy link
Collaborator

Li0k commented Dec 13, 2023

I will review ASAP

Signed-off-by: MrCroxx <mrcroxx@outlook.com>
@Li0k Li0k self-requested a review December 13, 2023 08:26
Signed-off-by: MrCroxx <mrcroxx@outlook.com>
Signed-off-by: MrCroxx <mrcroxx@outlook.com>
@MrCroxx
Copy link
Collaborator Author

MrCroxx commented Dec 13, 2023

image

Close #226

@@ -433,7 +395,7 @@ impl std::io::Read for ArcVecU8Cursor {
let slice = self.inner.as_ref().as_slice();
let len = std::cmp::min(slice.len() - self.pos, buf.len());
buf.put_slice(&slice[self.pos..self.pos + len]);
self.pos -= len;
self.pos += len;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it a bug introduced from previous PR ?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep, this PR is part of a series to make each PR small. The previous one is not fully tested.

@Li0k
Copy link
Collaborator

Li0k commented Dec 14, 2023

Could you add some background info about this PR?

@MrCroxx
Copy link
Collaborator Author

MrCroxx commented Dec 14, 2023

Could you add some background info about this PR?

The related issues informed some.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature New feature or request
Projects
Status: No status
Development

Successfully merging this pull request may close these issues.

bug: abnormally high latency with gp3
2 participants