Use macros in nested functions

[simonbyrne]

I would like to be able to call macros (e.g. @localmem) from functions which are called (perhaps indirectly) from the @kernel function. e.g.
here:
https://github.com/CliMA/ClimaCore.jl/blob/14712189be7c4a2ea8bc329955b219c17c604ca3/src/Operators/spectralelement.jl#L564-L565

I guess we could do this by explicitly passing the secret __ctx__ arg through, but it's not clear that is part of the API.

[vchuravy]

It is not. The problem is what do you do with @synchronize and how do you gurantuee equivalency of execution between CPU and GPU

[simonbyrne]

Yeah, I don't see a way around that, unless you switched the CPU backend to use tasks instead of for loops.

[vchuravy]

We would need a task per lane and that would be too expensive.

I am open to have GPU only features. Let's take a half step back and ask the question: What do you actually want to achieve.

[vchuravy]

In conversation we decided to add a @kernel cpu=false function mode, that disables code generation for the CPU and allows access to __ctx__ in child funtions