matmul: modify split 10 case to avoid possible memory issues

[coquelin77]

Related
currently the 10 functionality of matmul creates a matrix which is the size of the result on each process. this can be very dangerous if there is not enough memory.

Feature functionality
An update is needed to apply blocking to the split 10 case so that the memory of the nodes is not stretched quite so much.

Additional context
the blocking sizes should be based on the result of the split sizes of the output not of the inputs, it should be clear why this is a special case and not the general rule

[ClaudiaComito]

Is this still relevant @coquelin77 ?

[ClaudiaComito]

I'm not sure we need to provide this functionality.

If just for convenience, then I'd say an in-place resplit of the input matrices (from 1-0 to 0-1) would be better than having a matrix the size of the result on each process.

Thoughts @coquelin77 @mrfh92 ?

---

Reviewed within #1109

[mrfh92]

I think there could be several options:

• Follow Claudias idea. To do so, I think we need to discuss the following:
  • If we stick to the current rule that the output split equals the split of the first factor of the product, we would have to introduce another resplit after multiplying.
  • Also, if we in-place resplit the first factor, this factor is either modified (which would be a strange side effect of matmul then) or a copy is taken which causes memory consumption again.
  • Another option could be, to do this resplit if and only if the size of the result matrix exceeds a certain limit (e.g. the size of the inputs or sth like that) and to keep the current version for "small" outputs.
• having a look on the 10 case and try to modify the algorithm (no guarantee that we find sth better than the current implementation - this is maybe one of the few limitations due to the slicing along a single dimension...)

[github-actions]

Branch 360-matmul_modify_split_10_case_to_avoid_possible_memory_issues created!