-
Notifications
You must be signed in to change notification settings - Fork 903
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
update default value of allocatorPoolingConcurrency #3001
Conversation
Set a more reasonable default value to avoid OutOfDirectMemoryError.
@eolivelli Please help to review it. |
@@ -1060,7 +1061,7 @@ public T setAllocatorPoolingPolicy(PoolingPolicy poolingPolicy) { | |||
* @return the configured pooling concurrency for the allocator. | |||
*/ | |||
public int getAllocatorPoolingConcurrency() { | |||
return this.getInteger(ALLOCATOR_POOLING_CONCURRENCY, 2 * Runtime.getRuntime().availableProcessors()); | |||
return this.getInteger(ALLOCATOR_POOLING_CONCURRENCY, PooledByteBufAllocator.defaultNumDirectArena()); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The default value of defaultNumDirectArena
is 2 * cpus
https://github.com/netty/netty/blob/4e439264df0523fb6efce5f8f6c7c7fa74addd07/buffer/src/main/java/io/netty/buffer/PooledByteBufAllocator.java#L413-L418
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The default value of
defaultNumDirectArena
is 2 * cpus https://github.com/netty/netty/blob/4e439264df0523fb6efce5f8f6c7c7fa74addd07/buffer/src/main/java/io/netty/buffer/PooledByteBufAllocator.java#L413-L418
Thanks for your reply. Netty default value use the min value between 2*cpus and max mem defaultChunkSize / 2 / 3 to avoid oom. And it helps me resolved bookeeper oom problem in my product environment. Can you help me to merge it?
DEFAULT_NUM_DIRECT_ARENA = Math.max(0,
SystemPropertyUtil.getInt(
"io.netty.allocator.numDirectArenas",
(int) Math.min(
defaultMinNumArena,
PlatformDependent.maxDirectMemory() / defaultChunkSize / 2 / 3)));
ping @merlimat @eolivelli @dlg99, Would you please help take a look? thanks. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nice catch! It happend at low maxDirectMemory
.
And there is a more suitable value for it. In we use When we calculate the concurrency, we should subtract the memory which writeCache and readCache total occupy. See: Lines 153 to 156 in 39a9c28
bookkeeper/bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/storage/ldb/WriteCache.java Line 116 in 39a9c28
bookkeeper/bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/storage/ldb/ReadCache.java Line 79 in 39a9c28
|
ping @sunshujie1990, would you please take a look at the last comment? |
sure |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
lgtm
if no problem, please merge it,thanks @eolivelli @dlg99 @hangc0276 |
Set a more reasonable default value to avoid OutOfDirectMemoryError. Descriptions of the changes in this PR: ### Motivation The default value of allocatorPoolingConcurrency is 2 * Runtime.getRuntime().availableProcessors(). It's used to specify the num of Arena in PooledByteBufAllocator. Assume: 40 processors 80 arena (processors*2) 2 chunk per arena 16MiB per chunk JVM's total direct mem should be larger then 80*2*16=2560MiB, otherwise OutOfDirectMemoryError occured. OutOfDirectMemoryError much more likely to occur in Docker, cause JDK versions earlier than Java SE 8U131 does not support Docker CPU limits. ### Changes Use Netty default arena num `PooledByteBufAllocator.defaultNumDirectArena()` as default allocatorPoolingConcurrency. (cherry picked from commit 01c8824)
Set a more reasonable default value to avoid OutOfDirectMemoryError. Descriptions of the changes in this PR: ### Motivation The default value of allocatorPoolingConcurrency is 2 * Runtime.getRuntime().availableProcessors(). It's used to specify the num of Arena in PooledByteBufAllocator. Assume: 40 processors 80 arena (processors*2) 2 chunk per arena 16MiB per chunk JVM's total direct mem should be larger then 80*2*16=2560MiB, otherwise OutOfDirectMemoryError occured. OutOfDirectMemoryError much more likely to occur in Docker, cause JDK versions earlier than Java SE 8U131 does not support Docker CPU limits. ### Changes Use Netty default arena num `PooledByteBufAllocator.defaultNumDirectArena()` as default allocatorPoolingConcurrency. (cherry picked from commit 01c8824)
Set a more reasonable default value to avoid OutOfDirectMemoryError. Descriptions of the changes in this PR: ### Motivation The default value of allocatorPoolingConcurrency is 2 * Runtime.getRuntime().availableProcessors(). It's used to specify the num of Arena in PooledByteBufAllocator. Assume: 40 processors 80 arena (processors*2) 2 chunk per arena 16MiB per chunk JVM's total direct mem should be larger then 80*2*16=2560MiB, otherwise OutOfDirectMemoryError occured. OutOfDirectMemoryError much more likely to occur in Docker, cause JDK versions earlier than Java SE 8U131 does not support Docker CPU limits. ### Changes Use Netty default arena num `PooledByteBufAllocator.defaultNumDirectArena()` as default allocatorPoolingConcurrency. (cherry picked from commit 01c8824)
Set a more reasonable default value to avoid OutOfDirectMemoryError. Descriptions of the changes in this PR: ### Motivation The default value of allocatorPoolingConcurrency is 2 * Runtime.getRuntime().availableProcessors(). It's used to specify the num of Arena in PooledByteBufAllocator. Assume: 40 processors 80 arena (processors*2) 2 chunk per arena 16MiB per chunk JVM's total direct mem should be larger then 80*2*16=2560MiB, otherwise OutOfDirectMemoryError occured. OutOfDirectMemoryError much more likely to occur in Docker, cause JDK versions earlier than Java SE 8U131 does not support Docker CPU limits. ### Changes Use Netty default arena num `PooledByteBufAllocator.defaultNumDirectArena()` as default allocatorPoolingConcurrency. (cherry picked from commit 01c8824) (cherry picked from commit 35d0ecf)
Set a more reasonable default value to avoid OutOfDirectMemoryError. Descriptions of the changes in this PR: ### Motivation The default value of allocatorPoolingConcurrency is 2 * Runtime.getRuntime().availableProcessors(). It's used to specify the num of Arena in PooledByteBufAllocator. Assume: 40 processors 80 arena (processors*2) 2 chunk per arena 16MiB per chunk JVM's total direct mem should be larger then 80*2*16=2560MiB, otherwise OutOfDirectMemoryError occured. OutOfDirectMemoryError much more likely to occur in Docker, cause JDK versions earlier than Java SE 8U131 does not support Docker CPU limits. ### Changes Use Netty default arena num `PooledByteBufAllocator.defaultNumDirectArena()` as default allocatorPoolingConcurrency.
Set a more reasonable default value to avoid OutOfDirectMemoryError.
Descriptions of the changes in this PR:
Motivation
The default value of allocatorPoolingConcurrency is 2 * Runtime.getRuntime().availableProcessors(). It's used to specify the num of Arena in PooledByteBufAllocator.
Assume:
40 processors
80 arena (processors2)
2 chunk per arena
16MiB per chunk
JVM's total direct mem should be larger then 802*16=2560MiB, otherwise OutOfDirectMemoryError occured.
OutOfDirectMemoryError much more likely to occur in Docker, cause JDK versions earlier than Java SE 8U131 does not support Docker CPU limits.
Changes
Use Netty default arena num
PooledByteBufAllocator.defaultNumDirectArena()
as default allocatorPoolingConcurrency.