Compile HIP related code only once in CI.

[tcojean]

Use a workaround for not having two HIP compilation in CI jobs.

The workaround is to not give CMake the chance to have a second build! This is the only way to do this AFAIK, since due to how HIP is built, the code will always be entirely compiled two successive times if given the chance, after which things settle down and only changed files are recompiled from the third time on.

Improvements in practice

Before: 92 minutes
After: 69 minutes

Explanation of the two compilations issue

The two compilations happen due to the way HIP manages its compilations and dependencies. The problem was the same when using FindCUDA.cmake through CMake at some point. FindHIP.cmake provides macros which automatically do the HIP code compilation, including separate host and device compilation.
As part of this process, for every file HIP needs to generate a dependency file such as ginkgo_hip_generated_par_ilu_kernels.hip.cpp.o.depend which contains every single header and other file that this file in question uses. The use of this is that when any dependency listed in this list changes, the targeted file has to be recompiled.

The problem is that with the way things are, there is no way to have this list at CMake configuration time, so this list is built at "build time" (first make). What happens then is that the second time the project is built, CMake sees this new dependency file which did not exist before (i.e. was considered empty I guess), and therefore CMake has to rebuild all the related files during the second make, since to it all the dependencies of the file changed. Every subsequent compilations (from the third make on) will work as expected, with rebuilds happening only for files which really changed.

[pratikvn]

LGTM!

[upsj]

LGTM!

[thoasm]

LGTM!