HDDS-7851. Apply flush and compact on all column families#4219
HDDS-7851. Apply flush and compact on all column families#4219kerneltime merged 4 commits intoapache:masterfrom
Conversation
|
@ChenSammi Could you help to review the PR? |
|
Should the column family be flushed when flushing the wal as well? cc @errose28 @duongkame |
|
Thank you at @symious for fixing this issue. |
@kerneltime Thank you for the review. Are your refering to the function of "db.flushWal(bool)"? I think the wal is shared by all column families, should be no need of this operation. |
|
@GeorgeJahad can you please take a look? |
I think WAL is flushed (to OS cache) automatically after each writes to achieve durability (unless Ozone has a config to disable it). On the other hand, this PR deals with flushing column families' memtables to SST files to persist the compaction after a snapshot is taken. We already flush memtable of the default column family so it makes sense to flush all the column families as well. |
|
Some of these methods are already implemented in the snapshot branch (HDDS-6517). Please take a look. |
...mework/src/main/java/org/apache/hadoop/hdds/utils/db/managed/ManagedCompactRangeOptions.java
Outdated
Show resolved
Hide resolved
| <allowedImport>org.rocksdb.StatsLevel</allowedImport> | ||
| <allowedImport>org.rocksdb.TransactionLogIterator.BatchResult</allowedImport> | ||
| <allowedImport>org.rocksdb.TickerType</allowedImport> | ||
| <allowedImport>org.rocksdb.LiveFileMetaData</allowedImport> |
There was a problem hiding this comment.
nit: it doesn't look like this class is used anywhere.
There was a problem hiding this comment.
@duongkame Thank you for the review, removed the unused entry, please have a look.
…ls/db/managed/ManagedCompactRangeOptions.java Co-authored-by: Duong Nguyen <duongnt.is@gmail.com>
@sadanand48 Thank you for the review. I think it should be easy to resolve the conflicts, right? |
The snapshot branch got merged to master today, The conflicts need to be resolved from this PR now. |
4 participants
What changes were proposed in this pull request?
The operation of flush and compact only works for "default" column family.
This ticket is to also apply the operations on other column families.
What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-7851
How was this patch tested?
unit test.