multiply_dimensionwise: mortars etc.

[ranocha]

Once JuliaLang/julia#37444 is merged into mainstream Julia, we should revisit the choice of MortarMatrix in DG methods. Currently, we use mortar_forward_upper::SMatrix etc. which basically disables LoopVectorization in @tullio there. Once the performance of getting pointers to views is increased, we should check whether it's better to use mortar_forward_upper::Matrix with LoopVectorization instead.

Xref https://discourse.julialang.org/t/tullio-loopvectorization-pointer-becomes-slow-for-views-because-of-base-memory-offset-for-ndims-5/46178

[sloede]

Sounds good to me! By the way, exceptional work on the performance of Trixi, @ranocha ! 👍

[chriselrod]

PaddedMatrices.jl is going to go through some serious breaking changes (including a probably name change) based on this ArrayInterface.jl PR, so it may be something to keep an eye on as well, especially given that it sounds like you do have static size information, which it will let you mix arbitrarily with dynamic size info.

[ranocha]

That's great to hear, @chriselrod! We'll definitely want to make more use of your nice work.

[ranocha]

@chriselrod To be a bit more specific: We will operate on large arrays, where we know the first few dimensions (say, three or four dimensions) but the last dimension is only known at runtime. Is that something for the new version of PaddedArrays.jl?

[mateuszbaran]

HybridArrays: https://github.com/mateuszbaran/HybridArrays.jl is also a good solution when you have arrays with both static and dynamic indices.

[ranocha]

Thanks for the hint, @mateuszbaran! Does HybridArrays interact well with Tullio.jl and LoopVectorization.jl?

[mateuszbaran]

No, not at the moment. You can however wrap any array type in HybridArray and use for example Tullio.jl on the wrapped array when it's faster and HybridArray for other operations. Generally HybridArray should have very good performance for operations on statically-sized parts of arrays.

[chriselrod]

The current version already handles that, but as the changes will be extremely breaking.

The ArrayInterface changes should make it easy for other libraries (like HybridArrays) to support LoopVecotrization.

One of the aims of the PaddedMatrices changes are to make it easier to experiment with different memory layouts and see the impact on the code's performance.

[mateuszbaran]

Sorry, I didn't notice the word "well" in your message. HybridArrays shouldn't obstruct any operations from other libraries, so in that sense is should interact well. If you have any problems, let me know. As I wrote, HybridArrays can wrap around any array and you can perfectly well use other libraries to work on the wrapped array and use HybridArray only for a part of your workflow.

[ranocha]

Okay, thanks for the info. We'll consider that. It would just be nice to not have to switch between too many representations of the same data for some general access routines (where HybridArrays.jl can give improvements) and other parts making use of Tullio/LoopVectorization.

The current version already handles that, but as the changes will be extremely breaking.

The ArrayInterface changes should make it easy for other libraries (like HybridArrays) to support LoopVecotrization.

In that case, I think we'll just wait a bit longer until everything has stabilized a bit. Thank you all for your nice suggestions and the interest!

[mateuszbaran]

Okay, thanks for the info. We'll consider that. It would just be nice to not have to switch between too many representations of the same data for some general access routines (where HybridArrays.jl can give improvements) and other parts making use of Tullio/LoopVectorization.

HybridArray isn't actually a different representation, it just generates more efficient code for operations on the wrapped array. That said, having to un-wrap and re-wrap is not great I will definitely look into updating HybridArrays to the new version of ArrayInterface.jl (once it's ready).

[chriselrod]

That said, having to un-wrap and re-wrap is not great I will definitely look into updating HybridArrays to the new version of ArrayInterface.jl (once it's ready).

This is the PR in question.
I'll add docs describing everything needed, but the most important bits would be to support pointer (assuming the wrapped array does), and also providing methods for sdsize, sdstrides (, and sdoffsets) that use ArrayInterface.Static for static sizes known at compile time, and regular integers otherwise.