Implement consistent linear algebra for arrays with dimension > 2, in particular matmul

[mrfh92]

In Pytorch, most linear algebra routines can be applied to batches of matrices. In Heat it would be nice to have a similar rule how to "define" linear algebra (matmul,qr,cholesky,svd etc.) for DNDarray's with dimension exceeding 2.

To do so, follow the PyTorch convention (see, e.g., https://pytorch.org/docs/stable/generated/torch.linalg.solve.html#torch.linalg.solve or also the code in #893): for arrays with ndim>=3 we regard the last two dimensions (in the case of matrices) or the last dimension (in the case of vectors) of the array as "linear algebra-dimensions" and the indices of the first ndim-2 (matrices) or ndim-1 (vectors) dimensions as indices of a "batch of linear algebra objects"

Particular example: see matmul (#890 )

Suggestion:

• "minimal solution" (data/batch-parallel higher-dim matmul only):
  introduce batched matmul for ndim>=3 for the case that the split-dimension is a batch-dimension, i.e. no parallel matrix-matrix multiplication algorithm is required
• "full solution": go through the existing matmul-code and adapt it in such a way that you add batch-dimensions in order to handle the case of split equal to one of the two last dimensions that has been omitted in the "mimal solution"

Note: qr, svd, cholesky have been split off to #1320

[ClaudiaComito]

Related: #890

[mrfh92]

see #890 for the particular case of matmul on higher dim arrays

Reviewed within #1109 : still open and relevant

[github-actions]

Branch features/1104-Implement_consistent_linear_algebra_for_arrays_with_dimension_2_in_particular_matmul created!

[github-actions]

This issue is stale because it has been open for 60 days with no activity.

[github-actions]

This issue was closed because it has been inactive for 60 days since being marked as stale.