-
Notifications
You must be signed in to change notification settings - Fork 416
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
spike: how to address performance issues in #11333 #11426
Comments
@vasslitvinov : The theory with the reductions was that the identity routine in the UDR interface creates a local variable of the element type and returns it, assigning it to the original variable. So if the eltType was That said, on reflection, this should only result in O(numTasks) overhead I think, so maybe there's something else going on here. For instance, compare the delay in the following code between printing "Before forall" and "in forall" with the reduce intent. Then rewrite it to use an var n_bodies = 10000;
const pDomain = {0..#n_bodies};
type vec3 = [0..#3] real;
var forces: [pDomain] vec3;
var velocities: [pDomain] vec3;
var positions: [pDomain] vec3;
var masses: [pDomain] real;
writeln("Before forall");
forall q in pDomain with (+ reduce forces) {
if (q == 0) then
writeln("In forall");
} |
@bradcray - how do you feel about investigating first the above code where |
There's not a problem with that case is there? What would we hope to learn from the investigation? |
If that case is not problematic, then indeed there is no reason to look at it. |
Avoid creating an array in chpl__sumType() chpl__sumType() computes the type of `x+x` given the type of `x`. To do so, it declares the variable `x`, computes `x+x` and takes its type. All this happens strictly at compile time, which is desired, as long as the type is not a "runtime type" i.e. an array or a domain. This is time-consuming in the case `x` is an array. Even though chpl__sumType() is smart to run `x+x` only on a variable of the array's element type, and if that itself is an array, then only on its element type recursively, allocating an array is already expensive, especially when: * an array is large, * it is an array of arrays, and/or * it is a distributed array. This is one of the causes of the slowdown discussed in #11333 / #11426. In most practical cases, the type of `x+x` is the same as the type of `x`. If so, we can save the hassle of allocating the array. This PR adds a check "is the type of `x+x` the same as the type of `x` ?". If so, chpl__sumType() skips declaring the variable `x`, instead returning its type immediately. This check is "conservative" aka has false negatives. I.e. it may say "no" in some cases where chpl__sumType() could return immediately. If so, chpl__sumType() will do the unnecessary work of allocating the `x`. This is largly only a performance concern - because chpl__sumType() still produces a correct result. In practice this situation should be rare. While there, emit user-friendly error when + reducing something that does not have a valid + implementation. r: @mppf
Here are my findings.
This leads me to believe that the underlying cause is lock contention. Which is due to all inner arrays being tied to the same domain -- the domain of the
|
The reduction of loop startup time is dramatic, thanks! W.r.t. domain/array overheads, note the relationship to issue #10911. @vasslitvinov, Do you attribute the remaining difference between this case and the reduce intents:
in intents:
|
In addition, there are two AoAs for the global Accumulation State. They are created before the loop and deleted afterwards. Re #10911, I wonder if a const domain still needs to account for arrays it is associated with. At the very least, it needs to know how many it got. That way when an array is deallocated, we know whether to deallocate the domain or not. |
Related to the performance issues chapel-lang#11133 / chapel-lang#11426. Since it is not doing anything correctness-check-worthy, I skipif it unless testing performance. FYI, the numbers I got on my Linux desktop without/with the fix in chapel-lang#11556 are: vec3 in-intent - startup: 0.011575 / 0.011829 vec3 in-intent - completion: 7.54034 / 7.61882 vec3 reduce intent - startup: 16.8032 / 0.634863 vec3 reduce intent - completion: 38.3702 / 19.1754 Also testing arrays fo tuples (not affected by chapel-lang#11556): tup3 in-intent - startup: 0.000096 tup3 in-intent - completion: 0.000286 tup3 reduce intent - startup: 0.000281 tup3 reduce intent - completion: 0.000371
…test Add a performance test for reduce intents with arrays-of-arrays. Related to the performance issues #11133 / #11426. Since it is not doing anything correctness-check-worthy, I skipif it unless testing performance. FYI, the numbers I got on my Linux desktop without/with the fix in #11556 are: vec3 in-intent - startup: 0.011575 / 0.011829 vec3 in-intent - completion: 7.54034 / 7.61882 vec3 reduce intent - startup: 16.8032 / 0.634863 vec3 reduce intent - completion: 38.3702 / 19.1754 Also testing arrays fo tuples (not affected by #11556): tup3 in-intent - startup: 0.000096 tup3 in-intent - completion: 0.000286 tup3 reduce intent - startup: 0.000281 tup3 reduce intent - completion: 0.000371
#11333 identifies a couple of performance issues with forall intents:
This task is to identify changes to improve performance.
The text was updated successfully, but these errors were encountered: