DiskArrayEngine crashes Julia with too much memory consumption

[felixcremer]

When I run the PyramidScheme fill_pyramids function with a 45000 x 45000 matrix the memory consumption of the julia process keeps slowely increasing until there is no RAM left on the computer and julia crashes.
This happens with threaded=true.
With threaded=false it seems as if the memory consumption stays roughly constant.
This is currently running and I will post an update as soon as this run through or crashed as well.

[meggart]

Can you quickly dump the chunking structure of your 45000 x 45000 matrix, i.e. the result of eachchunk(matrix).chunks?

[felixcremer]

I think this is fixed and at least I don't know which data I used while this happened. I am going to open a new issue if this happens again.