(fix) rocksdb-compression-on-windows #22
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
… any of the compression type
killme2008
reviewed
Mar 9, 2019
|
|
||
| if (Platform.isWindows()) { | ||
| // Seems like the rocksdb jni for Windows doesn't come linked with any of the compression type | ||
| options.setCompressionType(CompressionType.NO_COMPRESSION); |
killme2008
reviewed
Mar 9, 2019
| .setLevel0FileNumCompactionTrigger(LEVEL0_FILE_NUM_COMPACTION_TRIGGER) // | ||
| .setLevel0SlowdownWritesTrigger(LEVEL0_SLOWDOWN_WRITES_TRIGGER) // | ||
| .setLevel0StopWritesTrigger(LEVEL0_STOP_WRITES_TRIGGER) // | ||
| .setMaxBytesForLevelBase(MAX_BYTES_FOR_LEVEL_BASE) // | ||
| .setTargetFileSizeBase(TARGET_FILE_SIZE_BASE) // | ||
| .setMergeOperator(mergeOperator) // | ||
| .setMemtablePrefixBloomSizeRatio(0.125); | ||
| if (Platform.isWindows()) { |
There was a problem hiding this comment.
先不要 merge 吧，我把 export rocksdb options 一块改掉
|
@killme2008 再看下是否有其他意见？ |
added 8 commits
March 10, 2019 20:45
killme2008
reviewed
Mar 11, 2019
| Runtime.getRuntime().addShutdownHook(new Thread(() -> { | ||
| for (final DBOptions opts : rocksDBOptionsTable.values()) { | ||
| if (opts != null) { | ||
| opts.close(); |
There was a problem hiding this comment.
如果用户已经释放了，这里会 core dump 的。
There was a problem hiding this comment.
我记得这些选项会在 Storage 这些服务里 close，所以无论是用户，还是这些 shutdown hook，都没有必要去 close，选项统一交给使用的服务关闭。
There was a problem hiding this comment.
理论上只要允许用户传入 options，都阻止不了用户提前释放 options 产生的 core dump， 这个比较头疼；
StorageOptionsFactory 中其实是存储了一份 options 模板，getXXX 方法会返回一个 copy 副本，这些副本都是有使用者统一释放的；
shutdown hook 释放的是那一份模板 options
我做了以下改进：
- 增加 checkInvalid (opts) 方法，在 copy 之前先检查 opts 是否被释放， 如果已被用户释放则抛出异常，避免直接 copy 产生的 core dump，但先检查再拷贝不是原子操作，无法完全避免用户提前释放导致 core dump
- 去掉 shutdown hook， 增加 releaseAllOptions() , 用户可以在程序退出时选择主动调用该方法来释放所有模板options，但其实也可以不调用，因为 rocksdb 的options在进程退出时都会自动释放
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Seems like the rocksdb jni for Windows doesn't come linked with any of the compression type