-
Notifications
You must be signed in to change notification settings - Fork 13.8k
[FLINK-30751] [docs] Remove references to disableDataSync in RocksDB documentation #21729
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the contribution!
I think this PR should be split into two commits:
- Remove references to disableDataSync in RocksDB documentation.
- Align pyflink rocksdb options with java side.
c3a904a
to
38f4da5
Compare
Hi @Myasuka, Thanks for reviewing this PR! I split the changes into separate commits so that each change is separated & easier to review. The first commit removes |
Hi @Myasuka Is there any chance we can merge this PR? |
@dchristle Thanks for the reminder, I will take a look recently. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM, @dchristle could you rebase it with latest master branch so that we can get a green CI.
…ftover from mentioning that setUseFsync is false. But this is true for all predefined options, as mentioned at the top of the doc. We should remove this for clarity, and can mention it elsewhere a single time.
… code for improved readability and maintainability.
…, irrespective of the PredefinedOptions setting. Add the explanation of why syncing is not necessary in Flink to the introduction, where it has better context, and remove the leftover reference to syncing from the DEFAULT.
38f4da5
to
f557045
Compare
@Myasuka Got it - I just rebased on the latest master branch. |
…with its Java equivalent. This closes #21729.
…with its Java equivalent. This closes #21729.
@Myasuka Thanks for your review & help here! |
…with its Java equivalent. This closes apache#21729.
…with its Java equivalent. This closes apache#21729.
…with its Java equivalent. This closes apache#21729.
What is the purpose of the change
This PR corrects and improves the readability of the documentation for RocksDB predefined option sets. References to
disableDataSync
, which is not set and has been removed from RocksDB for some time, are removed.While fixing this error, I also noticed other confusing aspects of this documentation. The options are not sorted, which makes it harder for a reader to find a particular one, and for maintainers to ensure the code & documentation match. I found an error of this kind in
SPINNING_DISK_OPTIMIZED_HIGH_MEM
, where Bloom filter usage is enabled but not documented. The unsorted documentation/code may have played a role in this accidental omission. I kept the Bloom filter setting, so that users' job performance does not change unexpectedly between versions, and I added the Bloom filter setting to the documentation along with a description of where its input parameters are set.References to Flink not needing to sync changes to the file system due to its checkpointing ability are also present, but it is confusing to a reader because this point isn't relevant to any of the listed options in the Java documentation.
setUseFsync(false)
is documented for each option set in the Python documentation, but isn't in the Java documentation.setUseFsync(false)
is not set in the predefined options, so we should not emphasize it. Instead, I added a passing reference to it at the top of both Java and Python introductions.Brief change log
disableDataSync
in RocksDB documentation. This is not set, and has been removed from RocksDB for some time.setUseFsync
references on each option.setIncreaseParallelism
in Python documentation tosetMaxBackgroundJobs
, which matches the Java usage.SPINNING_DISK_OPTIMIZED_HIGH_MEM
documentation, and describe where its parameters are set.Verifying this change
This change is a trivial rework / code cleanup without any test coverage.
Does this pull request potentially affect one of the following parts:
@Public(Evolving)
: noDocumentation