Skip to content

ARROW-2855: [C++] Blog post that outlines the benefits of using jemalloc#2271

Closed
xhochy wants to merge 3 commits intoapache:masterfrom
xhochy:ARROW-2855
Closed

ARROW-2855: [C++] Blog post that outlines the benefits of using jemalloc#2271
xhochy wants to merge 3 commits intoapache:masterfrom
xhochy:ARROW-2855

Conversation

@xhochy
Copy link
Copy Markdown
Member

@xhochy xhochy commented Jul 15, 2018

I will update the date once this has passed review.

Copy link
Copy Markdown
Member

@kou kou left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1
(I found some typos.)

allocations that span several memory pages.

Outside of Apache Arrow, [jemalloc powers the infrastructure of Facebook][2]
(this is also where most of its development happens). t is also used as the
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"t" -> "It"?

For aligned allocations, the POSIX APIs only provide the
`aligned_alloc(void** ptr, size_t alignment, size_t size)` function to
allocate aligned memory. There is also
`posix_memalign( void **ptr, size_t alignment, size_t size)` to modify an
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A white space in ( void exists accidentally.

Copy link
Copy Markdown
Member

@wesm wesm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great, thank you for writing! I left a couple comments. Feel free to update date, merge, and publish to the Arrow website whenever you like; I can help promote on social media.

@@ -0,0 +1,141 @@
---
layout: post
title: "Ligthning fast memory allocation in Apache Arrow with jemalloc"
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Typo.

Maybe "Faster, more scalable memory allocation..."? Trying to think of there's a better way to describe the benefits than "Lightning fast"

-->

With the release of the 0.9 version of Apache Arrow, we have switched our
default allocator from the system allocator to jemalloc on OSX and Linux. This
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Say "default memory allocator" or if you want to be more specific "default memory allocator for array buffers" (though there are other kinds of buffers, these are the ones perhaps most of interest)

@pitrou
Copy link
Copy Markdown
Member

pitrou commented Jul 18, 2018

Very informative! +1 from me.

@xhochy
Copy link
Copy Markdown
Member Author

xhochy commented Jul 19, 2018

If there are no further comments, I will publish this tomorrow

@wesm
Copy link
Copy Markdown
Member

wesm commented Jul 19, 2018

+1 from me. Thanks @xhochy!

@xhochy xhochy closed this in 87c976a Jul 21, 2018
@xhochy xhochy deleted the ARROW-2855 branch July 21, 2018 16:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants