resize! for multidimensional arrays

[ranocha]

It would be really nice to be able to resize! multidimensional Arrays, even if restricted to the last dimension (because of memory layout), similarly to ElasticArrays.jl.

Background

We want to make use of adaptive mesh refinement (AMR) for our hyperbolic PDE solver package Trixi.jl. For us, the most natural data layout is given by multidimensional arrays. To use AMR, we need to resize! the last dimension of these arrays.

Needing to use ElasticArray doesn't feel completely good to me, since we also want to experiment with other array types such as PaddedMatrices.jl, see trixi-framework/Trixi.jl#166. Then, we would need to wrap an Array inside something from PaddedMatrices.jl inside something from ElasticArrays.jl - or the other way round? Is that possible at all? If we go further, this looks like it could easily explode.

[JeffBezanson]

This is a very tall order, since the possibility of the array moving (and hence pointers changing) can inhibit a lot of compiler optimizations. So I think the best approach is to make Array as efficient as possible, and use wrappers like ElasticArray around it to add more features. If Array is slow we're stuck and can't get the lost performance back.

[ranocha]

Would it be possible to basically use jl_array_grow_at_end (or something similar) also for multidimensional arrays?

julia/src/array.c

Line 875 in 7d3dac4

STATIC_INLINE void jl_array_grow_at_end(jl_array_t *a, size_t idx,

I am willing to invest some work in this but would like to get your feedback whether that's a) possible at all and b) acceptable for Base Julia?

[timholy]

What Jeff is saying is that if you do this then we have to disable other optimizations and everyone else suffers performance-wise because of it.

[ranocha]

Okay, I see. Thanks a lot for your feedback. Feel free to close this issue in that case.

[jebej]

This is a very tall order, since the possibility of the array moving (and hence pointers changing) can inhibit a lot of compiler optimizations

Is this not the case for 1D arrays currently then?

It seems like if 1D arrays can be resized, then multidimensional arrays should be able to be resized in the last dimension too. If that's not true, would you mind quickly explaining the difference?