Finite volume operators

[ali-ramadhan]

Right now the element-wise operators are all written for the constant Δx, Δy, Δz case. As we transition to supporting variable grid spacings, the operators should be written as finite volume operators that take into account the variable areas and volumes.

We should be able to figure out a way to write the operators such that they work efficiently for all Cartesian grids.

Would be good to do this before #47 is implemented. This may require #59 to be resolved first.

[glwagner]

It does seem like solving #59 will make dispatch a lot easier.

What needs to be done to solve #59? Let me know if I can help.

[ali-ramadhan]

I was thinking of doing some prototyping and benchmarking in a sandbox by building off the example in my PR vchuravy/GPUifyLoops.jl#18.

The PR contains an example that can be extended to rely on a Grid struct, multiple FaceFields and CellField. So I'll prototype grids and fields that are isbitstype (you already helped by doing this for a grid in #59 (comment)) and test to see if they work on the GPU with GPUifyLoops.jl. If they do work and performance isn't degraded then I'll rewrite the operators to use grid and field structs.

You probably know how to do this better than me, but might be good if I rewrite the operators as they's still undocumented and do some slightly convoluted stuff to avoid having to store intermediate calculations.

Right now I'm focusing on system tests and benchmarks but once @christophernhill @jm-c and I get closer to implementing the variable Δz grid #47 I will work on this.

[glwagner]

Okay, sounds good!

Let's not focus too much on optimization prematurely, though if I understand you I think you are really talking about the obvious things, like avoiding memory allocation and transfers to/from the CPU to GPU.

[ali-ramadhan]

The title suggested that the finite volume operators in PR #283 are for Cartesian grids but they should actually work on all grids, even spherical or cubed sphere as long as the grid spacings are stored correctly and the halo communication is done right.