Control forall parameters using a context manager?

[vasslitvinov]

Currently I can control some parameters of how a forall over a domain is executed, for example:

forall i in D.these(tasksPerLocale=5) { ...... }

What about the situation where the forall loop is in a library function and the application code wants to set its tasksPerLocale parameter? Obviously, the library function can have a tasksPerLocale formal. However this does not look scalable because now this formal may need to be propagated through several levels of calls before reaching a forall. Also, as foralls start getting more control knobs -- think GPUs -- libraries will need to be updated to cater to these extensions and changes well.

Instead, I propose that the caller of the library function controls the settings via a context manager. For example:

manage forallControls(tasksPerLocale=5) {
  libraryFunction(....);
}

The proposal is that with this code the participating parallel iterators will take tasksPerLocale=5 into consideration.

[vasslitvinov]

Related: #14405 #18532 CHIP 17

[benharsh]

This idea is certainly interesting to me, and while it may make sense from the user side of things it's not clear to me how iterator authors could or should interact with this manager.

I think it would be interesting to consider some kind of "aliasing" locale with its own limits for tasking, and a context manager like this creating and moving execution to such a locale. Of course, I just sketched out something very vague and probably quite complicated, so take that with a grain of salt.

Should this manager contain a barrier for various tasks to use?

[mppf]

This idea is certainly interesting to me, and while it may make sense from the user side of things it's not clear to me how iterator authors could or should interact with this manager.

I would imagine that we add a record forallSettings that all of the good parallel iterator implementations would accept as an argument.

See also #16405 which has been talking about how to handle this. For GPU support, I think that there are 2 aspects:

1. How to communicate loop configuration details to the loop? E.g. setting tasksPerLocale or GPU block size.
2. How to access information related to the loop while inside the loop. That might be querying the set values, or it might have more to do with getting the vector lane / GPU thread number.

I can see how a context manager might make 1. easier to write. I am not seeing how it helps with 2 - or at least not in a satisfying way.

In contrast, the proposal that @e-kayrakli has been working towards looks would look like this this:

// 1. setting tasksPerLocale
forall  (i,j) in Dom with (config loopCfg = new LoopConfig(tasksPerLocale=5)) {
  // 2. accessing tasksPerLocale or other loop details
  ... loopCfg.tasksPerLocale ...
}

For 2. we discussed in a deep dive three different strategies

• a special variable/keyword kindof like here
• something returned by the iterator
• something that looks like a task private variable (like the above)

and the task-private-variable-looking strategy had by far more support.

I think we could still write something like this using a context manager, though. E.g.

// 1. setting tasksPerLocale
manage forallControls(tasksPerLocale=5) as loopCfg {
  forall  (i,j) in Dom {
    // 2. accessing tasksPerLocale or other loop details
    ... loopCfg.tasksPerLocale ...
  }
}

However I think that has some pretty serious drawbacks:

• if we can get "Current GPU thread number" from the loopCfg it should be associated with the loop, not with the manage block. This is particularly weird when working with nested foralls.
• we have to decide whether the manage block applies to forall statements created in functions called (i.e. impacts foralls dynamically called or merely the ones syntactically within it) and IMO neither option is obvious from the syntax.

[bradcray]

forall i in D.these(tasksPerLocale=5) { ...... }

Only vaguely related to this issue, but I've never been happy that the way one would pass in these arguments is to invoke/name .these() explicitly. Maybe I'll feel a little better about it once we complete the special method naming changes.

I agree that it could be attractive to have more of a scoped way of controlling such settings in any case.

[vasslitvinov]

To clarify, I was thinking of using a context manager for controlling forall settings "remotely", that is, around a call to a library function, where the forall loop is in that (or another) library function. This is a different setup than controlling forall settings in the code next to (or inside) the forall loop. Something in one of our group discussions made me think of such a scenario. I don't know how important it is in practice.

[vasslitvinov]

For the case where the settings go close to the forall loop:

The ideal way to me is to pass those settings directly to the iterator, rather than designing some special syntax to accomplish that. I am assuming that it is the iterator that "reads" those settings. However, there is no explicit iterator invocations in the above examples, which makes this solution not directly applicable.

Several proposals above take normal-looking syntax and give it magic meaning. This means that users will need to memorize exactly what to write to use that feature. My preference is to introduce dedicated syntax in such cases. I prefer dedicated syntax over normal syntax with magic meaning or "the user needs to get it just right for it to work".

Specifically, here are some ideas:

forall idx in Dom config(tasksPerLocale=5)  // 1. setting tasksPerLocale
  with (...)   // a normal with-clause, if needed
{
  ... here.forall.blockID ....   // 2. accessing loop details
  ... here.loop.blockID ...      // or, use 'loop' to make this not forall-specific
  ... config.blockID ...         // this is another option
}

In the above, I was very tempted to write forall ... with config(...) which flows as a sentence. However, with is unnecessary for parsing. Perhaps the following instead?

forall idx in Dom with (config(tasksPerLocale=5))
{.....}

OTOH if loop configuration is distinct from task intents and TPVs, we may as well keep it separate syntactically, as in my first snippet above.

[mppf]

@stonea - I know @e-kayrakli was planning to implement something along these lines following the deep dive - is that something you are expecting to take on? I think @vasslitvinov 's config() proposal above is a minor syntax variation but it is interesting to me.

[bradcray]

I know @e-kayrakli was planning to implement something along these lines following the deep dive

Specifically, I think he was going to implement a low-level way of doing this (e.g., via primitives or whatever it takes) to get the ball rolling and avoid getting hamstrung by syntax design questions (?).

[stonea]

Engin pitched some potential syntaxes for this in the deep dive:

https://github.hpe.com/hpe/hpc-chapel-docs/blob/main/notes/deepDiveMeetings/2022-05-10-forall-spmd.pptx

Slide 22 has a table of various options. According to the meeting notes (https://github.hpe.com/hpe/hpc-chapel-docs/blob/main/notes/deepDiveMeetings/2022-05-10-forall-spmd.txt) option 3 was the most popular at the time. Option 3 was something like this:

forall (i,j) in Dom with (var loopCfg = new LoopConfig())

or this:

forall (i,j) in Dom with (config loopCfg = new LoopConfig())

@stonea - I know @e-kayrakli was planning to implement something along these lines following the deep dive - is that something you are expecting to take on?

No it's not something I planning on taking on in the near term. If someone get's the ball rolling on this (even outside of a GPU context) I'd be thrilled. Just keep me in the loop so I can see how it develops.

---

One concern I have: I'd like to make sure that whatever syntax we choose we can capture the loop configuration "object" into a variable and passed to multiple loops.

In other words, if I have ten forall/foreach loops that are going to turn into GPU kernels and I want all of them to be configured the same (with the same block size, etc.) I don't want to have to type out the details of that configuration 10 times.

So if we went with:

forall idx in Dom config(tasksPerLocale=5)

would I be able to something like:

var myConfig = config(tasksPerLocale=5);
forall idx in Dom myConfig

[bradcray]

Engin pitched some potential syntaxes for this in the deep dive:
...
option 3 was the most popular at the time.

Sure, but I thought the main takeaway from that deep-dive (reiterated in his recent "state of the GPU" deep-dive) was that the implementation would not focus on the syntax for the time being and worry more about the mechanisms because the discussions about syntax were inconclusive and hard to feel very strongly about without more experience. E.g.,

• Engin: I am seeing an action item to just crank this out
  with primitives and then having some more
  discussion later with actual code that uses the
  primitives could be more useful.
  ...
  Next steps -> use some primitives to get something working
  and explore some use cases and then hopefully
  next level of discussion will have a working
  prototype.

None of which is to say that wrestling with syntax isn't valuable, Michael just gave me pause when he used "take on [the implementation]" and "syntax" in the same comment when I thought we'd agreed to start with the primitive-based approach and grow from there.

[mppf]

None of which is to say that wrestling with syntax isn't valuable, Michael just gave me pause when he used "take on [the implementation]" and "syntax" in the same comment when I thought we'd agreed to start with the primitive-based approach and grow from there.

In fact, I thought Engin was going to go and implement a particular syntax, and learn from that process to inform the language design choices. But anyway I think we are in agreement that wrestling with syntax is still valuable & we are also at a point where we can learn from an implementation effort.

[e-kayrakli]

The task that was on my hands, and still is, was to implement something that exhibits the behavior we want without focusing on the syntax of things. Not that I have prototyped anything yet, but I would probably start by adding bunch of primitives to get the behavior we want without having to deal with implementing any syntactic approaches. After we show that we can achieve what we want, then we can start sugaring things up. That's how I would characterize the next steps as I understand them.