Bump SharedArrayPool's max arrays per partition default from 8 to 32 #87905
Conversation
ghost
assigned 
|
Tagging subscribers to this area: @dotnet/area-system-buffers Issue DetailsThe 8 value was picked arbitrarily years ago, with a smaller value being picked because nothing was ever trimmed from the pool. Since then, we've seen a significant increase in use of the pool, putting pressure on its storage, and we also added trimming so that memory pressure causes arrays to be pitched. Longer term, we might want to remove this limit entirely and have more of a dynamic scheme for allowing the buckets to grow and shrink. For now, though, I'm bumping the limit up from 8 arrays per core to 32 arrays per core to provide some more wiggle room. 32 is also somewhat arbitrary, though recent examples on a few real services that were hitting the 8 limit (resulting in increased allocation and contention) were mollified by 32.
|
The 8 value was picked arbitrarily years ago, with a smaller value being picked because nothing was ever trimmed from the pool. Since then, we've seen a significant increase in use of the pool, putting pressure on its storage, and we also added trimming so that memory pressure causes arrays to be pitched. Longer term, we might want to remove this limit entirely and have more of a dynamic scheme for allowing the buckets to grow and shrink. For now, though, I'm bumping the limit up from 8 arrays per core to 32 arrays per core to provide some more wiggle room. 32 is also somewhat arbitrary, though recent examples on a few real services that were hitting the 8 limit (resulting in increased allocation and contention) were mollified by 32.
|
FYI I tracked a 10-15 % RPS bump in YARP HTTP2-HTTP2 benchmarks to this change.
|
Nice. Does it improve further if you set the |
|
Nevermind, I see that's what you did in those tables. Looks like 32 is indeed a sweet spot then for yarp? |
|
Looks like a higher limit can improve it a bit further, depending on machine. |
The 8 value was picked arbitrarily years ago, with a smaller value being picked because nothing was ever trimmed from the pool. Since then, we've seen a significant increase in use of the pool, putting pressure on its storage, and we also added trimming so that memory pressure causes arrays to be pitched. Longer term, we might want to remove this limit entirely and have more of a dynamic scheme for allowing the buckets to grow and shrink. For now, though, I'm bumping the limit up from 8 arrays per core to 32 arrays per core to provide some more wiggle room. 32 is also somewhat arbitrary, though recent examples of a few real services that were hitting the 8 limit (resulting in increased allocation and contention) were mollified by 32.