R2 RFC for task_group dynamic dependencies

[kboyarinov]

Description

Add RFC describing the semantics for concrete task_group dynamic dependencies APIs

Fixes # - issue number(s) if exists

Type of change

Choose one or multiple, leave empty if none of the other choices apply

Add a respective label(s) to PR if you have permissions

• bug fix - change that fixes an issue
• new feature - change that adds functionality
• tests - change in tests
• infrastructure - change in infrastructure and CI
• documentation - documentation update

Tests

• added - required for new features and some bug fixes
• not needed

Documentation

• updated in # - add PR number
• needs to be updated
• not needed

Breaks backward compatibility

• Yes
• No
• Unknown

Notify the following users

List users with @ to send notifications

Other information

[aleksei-fedotov]

It seems from the description, depending on the value of n_division, which I assume could be any value from a natural number set, the n_grid variable should been having values equal to the power of two, i.e. 2, 4, 8, 16, 32, etc. However, the description says they are 4, 16, 64, 256, and so on.
So what are the values that n_grid variable can store?

[kboyarinov]

n_grid stores the number of elements in each dimension of the grid on the particular level of division.
So, the previous version of the implementation is not correct:) Thanks for catching that!

During the first division, we have 2x2 grid of subtasks (n_grid = 2). On the second - 4x4 (n_grid = 4).
I have rewritten this part and corrected the comment

[aleksei-fedotov]

From my point of view, introducing another entity for essentially the same thing from the user-perspective complicates the design. The user will need to keep them separate in mind and remember what they are good for and what they are not.

As far as I understand, implementation-wise task_tracker represents this kind of shared-ownership, as it still needs to have a pointer to representation of at least the task state and the list of successors shared among task_tracker instances pointing to the same task. In this sense, it differs from the task_handle only in impossibility to be submitted for execution.
To avoid possible confusion about acting on moved task_handle, can we just deprecate the move semantics and add only lvalue signatures? After all, the same function object can be submitted several times into task_group.

[akukanov]

Overall, the detailed proposal now looks good to me.

[aleksei-fedotov]

LGTM. Found a couple of possible discrepancies due to recent renamings.

[vossmjp]

Looks good to me.

That said, I think there should be a link to the sub-RFC from the main RFC. README.md is automatically previewed in github and so it may be easy to overlook the extended_semantics.md if a link does not appear in README.md. I'd suggest putting it immediately before the Open Questions section.

[kboyarinov]

Looks good to me.

That said, I think there should be a link to the sub-RFC from the main RFC. README.md is automatically previewed in github and so it may be easy to overlook the extended_semantics.md if a link does not appear in README.md. I'd suggest putting it immediately before the Open Questions section.

Thanks for catching that. Applied.

I think we will need to think on how to merge README.md and extended_semantics.md while moving to experimental since most of API from README.md were renamed, the semantic has changed and most of the open questions are resolved in extended_semantics.md or not relevant at all any more.

[aleksei-fedotov]

One more approve to be able to merge the PR.