Add compile option WITH_PMEM and runtime config enable_pmem #327
Conversation
Signed-off-by: Yu, Peng <peng.yu@intel.com>
iyupeng
force-pushed
the
2022083101.kvdk_on_general_memory
branch
from
0eeb1e1
to
4e33e91
Compare
Signed-off-by: Yu, Peng <peng.yu@intel.com>
engine/dram_allocator.hpp
Outdated
| std::string AllocatorName() override { return "System"; } | ||
|
|
||
| private: | ||
| std::atomic<int64_t> bytes_allocated_{0}; |
There was a problem hiding this comment.
Potential bottleneck.
XADD(fetch_add) usually has a maximum Ops at about 20M for multithreading.
Maybe add a TODO for future optimization here?
There was a problem hiding this comment.
we can record usage with thread cache, like in PMEMAllocator
There was a problem hiding this comment.
Allocator gets improved now, supporting multi-threads.
| @@ -67,21 +67,27 @@ struct SkiplistNode { | |||
| // 4 bytes for alignment, the actually allocated size may > 4 | |||
| char cached_key[4]; | |||
|
|
|||
| static void DeleteNode(SkiplistNode* node) { free(node->heap_space_start()); } | |||
| static void DeleteNode(SkiplistNode* node, Allocator* alloc) { | |||
There was a problem hiding this comment.
This is actually not a good design, as NewNode and DeleteNode should use the same Allocator and such constraint should be enforced by the class. Skiplist class should hold a pointer to the Allocator, although this may bring 8B DRAM cost.
There was a problem hiding this comment.
Since we have to call DeleteNode without a Skiplist instance, we may leave a TODO here. Some day we should have different SkiplistFactory to create Skiplist with nodes on DRAM/HBM/CXL mempool and each factory should destroy the skiplists created by it.
There was a problem hiding this comment.
The alloc argument is now removed from these 2 static methods: NewNode() and DeleteNode(). Will try to use an allocator in thread_manager, accessed by thread local data. Here, it also handles cases where the thread_manager is not set.
Signed-off-by: Yu, Peng <peng.yu@intel.com>
Signed-off-by: Yu, Peng <peng.yu@intel.com>
Signed-off-by: Yu, Peng <peng.yu@intel.com>
Signed-off-by: Yu, Peng <peng.yu@intel.com>
Signed-off-by: Yu, Peng <peng.yu@intel.com>
Signed-off-by: Yu, Peng <peng.yu@intel.com>
Signed-off-by: Yu, Peng <peng.yu@intel.com>
iyupeng
force-pushed
the
2022083101.kvdk_on_general_memory
branch
from
7bd6fc0
to
b8a93a4
Compare
Signed-off-by: Yu, Peng <peng.yu@intel.com>
Signed-off-by: Yu, Peng <peng.yu@intel.com>
Signed-off-by: Yu, Peng <peng.yu@intel.com>
Signed-off-by: Yu, Peng <peng.yu@intel.com>
|
Closing this pull request. It will be split into smaller ones. |
What problem does this PR solve?
Problem Summary: Introducing optional volatile KV storage on DRAM
What is changed and how it works?
What's Changed:
WITH_PMEM, which isONby defaultenable_pmeminKVDK::Configs, which istrueby default-DWITH_PMEM=OFFto remove dependency tolibpmem-DWITH_PMEM=ON, settingConfigs.enable_pmem=falsecan make KVDK store data on DRAMCheck List
Tests
Side effects