Consider adopting a metaprogramming library

[bernhardmgruber]

During the accessor development #1249 I needed to implement a few meta functions on the side. Since alpaka is TMP heavy, we are going to need such metaprogramming facilities regularly so I think we should consider picking an appropriate library providing this functionality.

LLAMA uses boost::mp11 quite successfully and it provides a good feature set. Mind, that boost::mp11 is also available as a standalone library outside the usual boost distribution.

[sbastrakov]

FYI we actually tried boost::mp11 for PIConGPU 3 years ago, turned out there were issues with GPUs, and general approach not matching the needs very well back then. Doesn't mean these two factors are still relevant.

[bernhardmgruber]

That's good to know! I have not encountered GPU issues yet in LLAMA, but my GPU usage is also limited.

[bussmann]

Your last comment is extremely relevant. HPC is often bleeding edge and certain hardware 2-3 years behind in supporting standards, but these years make all the difference in real world applications. Thus, dependencies have to be carefully chosen. Without first hand experience this choice is difficult. One would be amazed which simple assumptions may be wrong for HPC applications.

[psychocoderHPC]

During the accessor development #1249 I needed to implement a few meta functions on the side. Since alpaka is TMP heavy, we are going to need such metaprogramming facilities regularly so I think we should consider picking an appropriate library providing this functionality.

LLAMA uses boost::mp11 quite successfully and it provides a good feature set. Mind, that boost::mp11 is also available as a standalone library outside the usual boost distribution.

Could you please link the stand-alone mp11 version? A quick search was always showing the boost version only.

[bernhardmgruber]

HPC is often bleeding edge and certain hardware 2-3 years behind in supporting standards, but these years make all the difference in real world applications.

Thankfully, mp11 is written in C++11 and thus uses a 10 year old standard.

Could you please link the stand-alone mp11 version? A quick search was always showing the boost version only.

https://github.com/boostorg/mp11
The README says:

Mp11 is part of Boost, starting with release 1.66.0. It however has no Boost dependencies and can be used standalone, as a Git submodule, for instance. For CMake users, add_subdirectory is supported, as is installation and find_package(boost_mp11).

[psychocoderHPC]

Thanks for the link: https://github.com/boostorg/mp11
The problem is that the CI is covering only compiler for the CPU, important HPC setups Intel, IBM XL, nvcc and hipcc is not covered. We should give it a try but need to test it on our own CI and all important HPC compilers.
Maybe you cover already nvcc with LLAMA.

[bernhardmgruber]

The problem is that the CI is covering only compiler for the CPU, important HPC setups Intel, IBM XL, nvcc and hipcc is not covered. We should give it a try but need to test it on our own CI and all important HPC compilers.

Very good point! Maybe you can test this implicitely if we test LLAMA on these compilers.

Maybe you cover already nvcc with LLAMA.

Yes nvcc is covered by LLAMA.

[psychocoderHPC]

Maybe you can test this implicitely if we test LLAMA on these compilers.

You would only get a real meaningful result if you test the stand-alone version of mp11.
If you use mp11 from boost, what LLAMA does, you could get compiler issues during the parsing of the boost config header.
Using boost mp11 is a good preview but to get the full picture you should test the stand-alone version.

[ax3l]

important HPC setups Intel, IBM XL, nvcc and hipcc is not covered

Just to complete the list, adding:

• essential GPU: Intel ICX/DPCPP (SYCL) as well as the old and trusty ICC
• new and hot: nvc++ (Nvidia GPU)
• new and hot: FujitsuClang (A64FX)

All but FujitsuClang/IBM/Cray are available as public apt repos for public CI. If you add dependencies, make sure the dependencies that you don't maintain also cover the compilers you care about, otherwise you will be constantly running behind.

[bernhardmgruber]

Thanks, @ax3l! Those are good points!

• essential GPU: Intel ICX/DPCPP (SYCL) as well as the old and trusty ICC

LLAMA has icpc (which is ICC IIRC) and icpx (DPCPP) in the CI and builds mp11 fine.

• new and hot: nvc++ (Nvidia GPU)

I want to test that one soonish! I guess since it's using a clang frontend, that it will work.

• new and hot: FujitsuClang (A64FX)

I guess this is also a clang frontend?

All but FujitsuClang/IBM/Cray are available as public apt repos for public CI.

Those are definitely my nemeses and we need to think about how we can deal with these. Like where we can get systems to test there etc. I think we could schedule a discussion in an alpaka VC at some point.

[ax3l]

I guess this is also a clang frontend?

Yes, Fujitsu comes with a "traditional" and a new Clang fronted (same story with IBM and Cray and Intel).

LLAMA has icpc (which is ICC IIRC) and icpx (DPCPP) in the CI and builds mp11 fine.

Although dpcpp is ideally only icx -fsycl, there are a few more details when compiling for GPU targets, e.g. some functions and types don't exist in certain scenarios, etc. Better test both by building an actual SYCL GPU target in CI, too.

[jkelling]

* new and hot: nvc++ (Nvidia GPU)

If am interpreting the NVHPC docs correctly, then this does not mean, that nvc++ supports cuda. nvc++ is pgc++ and supports NVIDA GPU via OpenACC (and OpenMP target). There is no mention of CUDA in the description of nvc++. The NVHPC SDK also ships nvcc for CUDA.

Does anyone have some different information on this?

[ax3l]

Yes, this changed as of GTC 2021 and will be released with the next HPC Toolkit (not the CTK).

…

On April 26, 2021 3:43:19 PM MDT, jkelling ***@***.***> wrote: > * new and hot: nvc++ (Nvidia GPU) If am interpreting the [NVHPC docs](https://docs.nvidia.com/hpc-sdk/index.html) correctly, then this does not mean, that nvc++ supports cuda. nvc++ is pgc++ and supports NVIDA GPU via OpenACC (and OpenMP target). There is no mention of CUDA in the description of nvc++. The NVHPC SDK also ships nvcc for CUDA. Does anyone have some different information on this? -- You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub: #1296 (comment)

-- Sent from my Android device with K-9 Mail. Please excuse my brevity.

[bernhardmgruber]

The linked documentation reads: Last updated April 08, 2021
That is before all the new announcements at GTC. I guess the documentation is not yet up-to-date. I remember Bryce tweeting about CUDA support in nvc++ (can't find it ATM).

[j-stephan]

Since we switched to C++17 last year: Which useful parts of (for example) Mp11 are we missing that can't be easily implemented through fold expressions etc.?

[bernhardmgruber]

I am sure almost 90% of the list on the right here: https://www.boost.org/doc/libs/master/libs/mp11/doc/html/mp11.html. It would be crazy to implement that ourselves.

[bernhardmgruber]

new and hot: nvc++ (Nvidia GPU)

Btw, LLAMA with mp11 runs fine on nvc++.

[bernhardmgruber]

Btw: PIConGPU adopted Boost.Mp11 in the meantime.

And here is a new contender: https://github.com/boost-ext/mp. Just saw it at this lightning talk: https://www.youtube.com/watch?v=-4MSlna4gKE. I am amazed with how much it can do a lot with just a few hundret LOCs.