Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

thread-local arena: grow buf if there is no enough space #116

Merged
merged 13 commits into from Jun 7, 2022

Conversation

zhangjinpeng87
Copy link
Member

@zhangjinpeng87 zhangjinpeng87 commented Dec 31, 2021

Signed-off-by: zhangjinpeng1987 zhangjinpeng@pingcap.com

Changes

  • Arena: add UT for arena
  • Arena: grow buffer if there is no enough space, so Arena::alloc alway will success
  • Arena: Allocate 8-byte aligned memory
  • Arena: support larger arena whose size is large than 4GB

Signed-off-by: zhangjinpeng1987 <zhangjinpeng@pingcap.com>
@@ -44,13 +45,13 @@ impl Arena {
pub fn alloc(&self, align: usize, size: usize) -> u32 {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add a debug assertion align.is_power_of_two()

// 0 means there is not enough space to allocate.
return 0;
}
offset as u32
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought the returning offset is supposed to be aligned.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm considering what's the benefits of aligning to the number smaller than size of pointer. Should we simply announce that the align must >= sizeof(ptr), to make the code simple.

let offset = arena.offset::<u64>(ptr);
assert_eq!(offset, 8);
}

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Allocate a more aligned ptr on the same arena: alloc(16,16)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good catch

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Currently we simply alloc align to 8

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually some libraries and compilers expects to align with 16. Check jemalloc/jemalloc#1533. Perhaps we should add static assert on all types we want to store can be aligned with 8.

Signed-off-by: zhangjinpeng1987 <zhangjinpeng@pingcap.com>
@@ -30,7 +32,8 @@ impl Arena {
mem::forget(buf);
Arena {
core: Arc::new(ArenaCore {
len: AtomicU32::new(1),
// Reserve 8 bytes at the beginning
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what's the purpose of reservation?

@@ -30,7 +32,8 @@ impl Arena {
mem::forget(buf);
Arena {
core: Arc::new(ArenaCore {
len: AtomicU32::new(1),
// Reserve 8 bytes at the beginning, because func offset return 0 means invalid value
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why not use Option

Copy link
Member

@Connor1996 Connor1996 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@Connor1996
Copy link
Member

Please fix the lint @zhangjinpeng1987

@zhangjinpeng87 zhangjinpeng87 changed the title skiplist: adding ut for arena [DNM] skiplist: adding ut for arena Jun 4, 2022
Signed-off-by: zhangjinpeng1987 <zhangjinpeng@pingcap.com>
@zhangjinpeng87 zhangjinpeng87 changed the title [DNM] skiplist: adding ut for arena thread-local arena: grow buf if there is no enough space Jun 4, 2022
@zhangjinpeng87
Copy link
Member Author

@Connor1996 PTAL again. I have updated this PR for more changes, please refer to the description.

Signed-off-by: zhangjinpeng1987 <zhangjinpeng@pingcap.com>
Signed-off-by: zhangjinpeng1987 <zhangjinpeng@pingcap.com>
pub struct Arena {
core: Arc<ArenaCore>,
core: ArenaCore,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As Arena is just a direct wrapper of AreanaCore, seems no need to introduce ArenaCore anymore

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done @Connor1996 , PTAL again.

Signed-off-by: zhangjinpeng1987 <zhangjinpeng@pingcap.com>
Copy link
Member

@Connor1996 Connor1996 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants