extend mempool #112
Merged
extend mempool #112
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
thesues
reviewed
Jan 28, 2025
src/config.h
Outdated
| std::string link_type; | ||
| int minimal_allocate_size; // unit: KB | ||
| int num_stream; // can only be 1,2,4, number of stream for each client | ||
| bool autoincrease; |
thesues
reviewed
Jan 28, 2025
| ibv_mtu active_mtu; | ||
|
|
||
| bool extend_in_flight = false; | ||
| std::atomic<unsigned int> opened_ipc{0}; |
Collaborator
There was a problem hiding this comment.
need a comment to indicate the define and usage of opened_ipc
thesues
reviewed
Jan 28, 2025
src/mempool.cpp
Outdated
|
|
||
| auto total_blocks = mempools_[i]->get_total_blocks(); | ||
| auto allocated_blocks = mempools_[i]->get_allocated_blocks(); | ||
| INFO( |
Collaborator
There was a problem hiding this comment.
might be DEBUG instead of INFO. if we have large number for mempool
rainj-me
reviewed
Jan 28, 2025
|
|
||
| #define BLOCK_USAGE_RATIO 0.5 | ||
| #define EXTEND_POOL_SIZE 10 << 30 | ||
| #define EXTEND_BLOCK_SIZE 64 << 10 |
Collaborator
There was a problem hiding this comment.
Is it a problem if the preallocated block size with 32k and the extend block_size as 64k ?
Collaborator
Author
There was a problem hiding this comment.
64K is the default minimal-allocate-size, if we consider this kind of details, it will take a lot of time... e.g., add more verification for the input parameters
infinistore/lib.py
Outdated
| prealloc_size (int): The preallocation size. Defaults to 16. | ||
| minimal_allocate_size (int): The minimal allocation size. Defaults to 64. | ||
| num_stream (int): The number of streams. Defaults to 1. | ||
| autoincrease (bool): indicate if infinistore will be automatically increased. 10GB each time. Default False. |
Collaborator
There was a problem hiding this comment.
I would recommend to set a cap for the dynamically allocated RAM.
Collaborator
Author
There was a problem hiding this comment.
this parameter is just a flag to enable auto increase or not,
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.