New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: enable concurrent write #3214
feat: enable concurrent write #3214
Conversation
18578c0
to
f97c0eb
Compare
Codecov ReportAttention:
Additional details and impacted files@@ Coverage Diff @@
## main #3214 +/- ##
==========================================
- Coverage 85.82% 85.44% -0.38%
==========================================
Files 840 844 +4
Lines 137704 138639 +935
==========================================
+ Hits 118183 118463 +280
- Misses 19521 20176 +655 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good job! Could we provide some benchmark results for this feature? And I wonder if the default value of 8 is the best practice for most common usage.
I will create a benchmark with throughput and latency in the different write concurrency settings. BTW, Based on some simple experiments I did before, a 10Gpbs bandwidth EC2 instance, setting the write concurrency to 25, the program could reach throughput: 500MiB/s with p50 ~250ms. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
I hereby agree to the terms of the GreptimeDB CLA
What's changed and what's your intention?
DEFAULT_WRITE_CONCURRENT
to 8.Copy To
andParquetWriter
.Checklist
Refer to a related PR or issue link (optional)