Long Build Times on Darwin

[william76]

@ndellingwood -- I think I was able to confirm my suspicions from our conversation today. Modivying cp to use the -u option on my OSX system got my build times back down to what I've been used to. We discussed putting in an issue on this so here it is:

I did some updates and went from a 2 (ish) minute build time for Kokkos Kernels up to a 20 minute build time on my mac laptop. I modified my KOKKOSKERNELS_SCALARS var in my compile config file to have only double and not double,"complex<double>" which reduced my compile time to 10 minutes but that's still significantly longer than the 2 minutes I had had before...

In doing some digging, I found that the kokkos makefile@ line 52 has a check on Darwin OS to set a copy_flag variable:

ifeq ($(KOKKOS_OS),Darwin)
  COPY_FLAG =
endif

but for Linux and CENTOS systems, we set COPY_FLAG = -u.

I imagine this was due to vanilla OSX's version of cp not having the -u option... however on my system, I have the coreutils package installed via homebrew which provides a gnu cp command that does provide the -u option.

$ cp --help
  <snip/>
  -u, --update                 copy only when the SOURCE file is newer
                                 than the destination file or when the
                                 destination file is missing
  <snip/>

Without this option, I think the kokkos makefile was fully copying all the files over each time I re-built my kokkos-kernels codes, triggering a full rebuild of kokkos each time I did a new make build-test in my perf tests.

On my system, I've modified kokkos/core/src/Makefile to check for the existence of the cp command that's installed by homebrew by modifying the above block of code to be:

ifeq ($(KOKKOS_OS),Darwin)
  COPY_FLAG = -u
  ifneq ("$(wildcard /usr/local/opt/coreutils/libexec/gnubin/cp)","")
    COPY_FLAG = -u
  endif
endif

This changed my build time from 10 minutes down to about 1.5 minutes. My guess is that the vanilla cp command smashes the timestamps on the files forcing a full rebuild of kokkos every time I recompiled my Kokkos-Kernels files.

In retrospect, I recall changing the cp commands in the kokkos many months ago but when I updated my local copy of Kokkos Kernels I think that modification got smashed away which jumped my build times back up to a full rebuild every time.

The -u option does not exist in the standard /bin/cp command.

Would it be possible to add a check in the Kokkos build system for Darwin based systems to see if the homebrew cp is available and the default and if so, use the -u option?

I'm happy to put in a PR with my modifications above, but thought there is probably a preferred place to put that kind of a check that would fit more properly in the current system.

[ibaned]

@william76 , could you please open that pull request as a first pass? Thanks!

[william76]

@ibaned sure, I put in the PR just now.

[ibaned]

I think this was fixed by that PR, which is #1724