Add compile-time tracked tensor shapes

[sbrunk]

Compile-time tracked tensor shapes means that the shape of a tensor, or parts of it, is statically known at compile time.

Why?

Statically know shapes can be useful for shape-safe operations. I.e. only tensors of a certain size are allowed for some operation and the compiler will verify this.

Another reason is that as #56 has shown that in order to have correct datatype promotion, we also have to track the shape. At a minimum, we have to distinguish scalar tensors (empty shape, single number) from non-scalar tensors (any other shape). But we can also go further and track the full shape.

Like type promotion rules, we also need to track the changes of reshaping operations at compile-time where possible.

Adding compile-time shapes will probably make the Tensor type more complicated as we probably need a second parameter type for the shape. It's important that we don't complicate the API to much with this. It might also make sense to add a raw Tensor type as well as fallback.

Existing approaches

We already have lists of different approaches, so I'm just quoting them here:

#56 (comment)

... After building some more complex models, I realized the pain for named tensors/shape checking, so will take a look at that repo, and also some other resources I've found. Here is a compiled list of them:

• https://github.com/tribbloid/shapesafe (this is a potential candidate we could add as a dependency, as it seems to be their goal)
• http://nlp.seas.harvard.edu/NamedTensor
• https://github.com/ctongfei/nexus
• https://maximekjaer.github.io/tf-dotty/
• https://github.com/MrVPlusOne/TensorSafe

#35 (comment)

... I recall a presentation on the TDM library. I searched and got the following, possibly interesting, links:

1. https://tongfei.me/assets/18nescala-slides.pdf
   1. https://arxiv.org/pdf/1710.06892.pdf
   2. https://tongfei.me/nexus/
2. https://www.youtube.com/watch?v=d-VbiyLwHYQ
   1. https://github.com/AnnabelleGillet/TDM
   2. https://hal.science/hal-03073789/file/IDEAS_2020_TDM_final.pdf
3. https://github.com/dieproht/matr
4. https://github.com/emptyflash/shapeless-matrix
5. https://github.com/tribbloid/shapesafe

May also be of interest:

1. https://github.com/breandan/kotlingrad
2. http://platanios.org/tensorflow_scala/guides/tensors.html
3. https://github.com/pashashiz/scanet3
4. https://arxiv.org/pdf/1801.08771.pdf

I am sure their are more out there, but this may help in the initial analysis. Maybe Scala 3's new features may make this possible, but if not done correctly, will be a maintenance headache.