Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Avoid tracking small allocations by setting memThreshold #935

Merged
merged 1 commit into from
Sep 30, 2021

Conversation

ronawho
Copy link
Contributor

@ronawho ronawho commented Sep 29, 2021

In order to avoid running out of memory and crashing the server, Arkouda
uses memTrack to track memory allocations and raise a client error
when a large allocation would likely exceed the available memory.

Recently, we discovered some performance overheads from this tracking
while doing highly concurrent short-lived allocations. The memTrack
flag is implemented with a global lock to protect access to a hash
table, which is serializing and really hurting the performance of
concurrent allocs. To limit the impact of this, use memThreshold=1M to
only track larger allocations and skip skip tracking smaller ones.

Using memThreshold did not have quite the performance boost I was
expecting at first with Chapel 1.25 because we didn't have a way to do
threshold checks on free. That was added in chapel-lang/chapel#18465.

Here's an allocation micro-benchmark that shows the performance on a
single node with 128-Rome CPUs:

use Time;
config const trials = 1_000_000;

var t: Timer; t.start();
coforall 1..here.maxTaskPar do
  for i in 1..trials do
    var s = i:string;
writeln(t.elapsed());
config Time
w/o memTrack 0.19s
w/ memTrack 144.50s
w/ threshold 1.25.0 33.06s
w/ threshold main 0.22s

So this should benefit code that has highly concurrent allocations like
casting to strings and currently regex operations (though we're working
on optimizing the regex allocations out.)

Part of #675
Part of #929

In order to avoid running out of memory and crashing the server, Arkouda
uses `memTrack` to track memory allocations and raise a client error
when a large allocation would likely exceed the available memory.

Recently, we discovered some performance overheads from this tracking
while doing highly concurrent short-lived allocations. The `memTrack`
flag is implemented with a global lock to protect access to a hash
table, which is serializing and really hurting the performance of
concurrent allocs. To limit the impact of this, use `memThreshold=1M` to
only track larger allocations and skip skip tracking smaller ones.

Using `memThreshold` did not have quite the performance boost I was
expecting at first with Chapel 1.25 because we didn't have a way to do
threshold checks on free. That was added in chapel-lang/chapel 18465.

Here's an allocation micro-benchmark that shows the performance on a
single node with 128-Rome CPUs:

```chpl
use Time;
config const trials = 1_000_000;

var t: Timer; t.start();
coforall 1..here.maxTaskPar do
  for i in 1..trials do
    var s = i:string;
writeln(t.elapsed());
```

| config              | Time    |
| ------------------- | ------: |
| w/o memTrack        |   0.19s |
| w/ memTrack         | 144.50s |
| w/ threshold 1.25.0 |  33.06s |
| w/ threshold main   |   0.22s |

So this should benefit code that has highly concurrent allocations like
casting to strings and currently regex operations (though we're working
on optimizing the regex allocations out.)

Part of 675
Part of 929
@glitch glitch merged commit db8c105 into Bears-R-Us:master Sep 30, 2021
@ronawho ronawho deleted the set-memThreshold branch September 30, 2021 16:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants