Join GitHub today
GitHub is home to over 40 million developers working together to host and review code, manage projects, and build software together.Sign up
Reserve disk space before create db engine #6321
Signed-off-by: Little-Wallace email@example.com
What have you changed?
I create a big file in
How to recover with this feature.
If you found TiKV crash or failed to write data into RocksDB, you could update config file
What is the type of the changes?
How is the PR tested?
Does this PR affect documentation (docs) or should it be mentioned in the release notes?
Does this PR affect
brson left a comment •
This seems like a plausible solution for solving what I think the problem is.
It needs more comments about the nature of the problem it's solving and how. What kind of recovery triggers the problem? What is happening that causes the disk-space exhaustion? Why does this help? Why is 2GB a reasonable default?
From reading the patch I don't understand how it works. It looks like TiKV allocates a chunk of disk space on startup, but never truncates the file to free the diskspace. How does this patch work to free diskspace during recovery?
A static size of 2GB seems suspicious. Does recovery take a fixed amount of disk space so that a single number should be enough for most (or every) database? Does a human need to tune this number for their specific database, and change it as the size grows?
Having a single function that is overloaded for two purposes (one to allocate the space, one to free the space) makes for a confusing read. One of the purposes (delete) does the opposite of the name of the function (reserve). Two functions would be clearer. If it makes sense to do so, please split it into two functions.
@@ Benchmark Diff @@ ================================================================================ tidb: 81a6eb04a585ba38bc8e6e7c0591411a39a5a827 --- tikv: 8376bb6b8458d3ba0cc179f9158fb0e931abbb26 +++ tikv: cb5897f6e75fd6b3077cfe02fa718cfea6edef62 pd: b3612833fde586a77bb520dbe1eeb7caad7d64ea ================================================================================ oltp_update_index: * QPS: 8833.66 ± 0.31% (std=18.13) delta: 0.79% (p=0.049) * Latency p50: 114.59 ± 0.27% (std=0.20) delta: -1.80% * Latency p99: 225.37 ± 0.90% (std=2.03) delta: -0.89% oltp_insert: * QPS: 12102.43 ± 0.82% (std=65.57) delta: 0.50% (p=0.215) * Latency p50: 84.59 ± 0.83% (std=0.46) delta: -0.49% * Latency p99: 141.12 ± 0.90% (std=1.27) delta: -0.89% oltp_read_write: * QPS: 15782.61 ± 0.25% (std=26.22) delta: 0.35% (p=0.127) * Latency p50: 1317.28 ± 0.30% (std=2.47) delta: -0.43% * Latency p99: 2493.86 ± 0.00% (std=0.00) delta: -1.78% oltp_point_select: * QPS: 52041.12 ± 0.07% (std=27.79) delta: 0.67% (p=0.025) * Latency p50: 19.69 ± 0.37% (std=0.04) delta: -0.55% * Latency p99: 52.89 ± 0.00% (std=0.00) delta: 0.00% oltp_update_non_index: * QPS: 14284.31 ± 0.68% (std=66.37) delta: 0.54% (p=0.136) * Latency p50: 71.66 ± 0.68% (std=0.33) delta: -0.53% * Latency p99: 155.80 ± 0.00% (std=0.00) delta: -1.78%
Thanks for the updates @Little-Wallace. I don't think my review has been sufficiently addressed. The use of this features is now described in this PR, but the code is still inscrutable. The code is what should be documented.
I now see that the function doesn't serve two purposes as I guessed. It is not meant to automatically free space at all - the entire feature requires user intervention, which is not so user-friendly. I would expect this to "just work" - run out of space -> delete file -> do operation -> create file again. But this might be reasonable too, or at least a reasonable first step. I don't know.
Since this is a new feature, the OP should seemingly indicate that docs should be updated, and docs should be updated.
I'm removing my request for changes, but I think the code should at least be documented. As-is I expect future readers to be confused.