Detect serial build of ESMF?

[tclune]

A recent GCHP issue might have been resolved more quickly if MAPL did a check to verify that ESMF has been built with MPI. Please check to see if there is some query we could do (and then throw an error).

[bena-nasa]

@tclune I'm not seeing anything in the API, either looking at the ESMF_VMGet or when you call ESM_Init, unless somehow if you don't build with MPI that maybe the MPIcommunicaor returned from ESMF_VMGet is always null?
I see this cryptic remark several places in the user manual
"Not supported in ESMF_COMM=mpiuni mode."
But there's not context on how one can detect this.

@mathomp4 do we have a no/ MPIESMF build of baselibs we could play with?
Also, how can MAPL even function without MPI, it is so ubiquitous?

[tclune]

Also, how can MAPL even function without MPI, it is so ubiquitous?

Well, you can't. But the error that you get early on is not very informative. And happens when doing analysis about cores-per-node or some such. One possibility is just to change that message to indicate that MPIUNI is a likely culprit.

[mathomp4]

Let's consider two ways: build-time detection and run-time detection.

At build-time, from my staring at FindESMF.cmake, I don't think ESMF_COMM is exposed mainly because I don't think it's exposed in esmf.mk. It's in a comment, but commented entries are skipped. Now, I could probably hack FindESMF.cmake to also skip all commented lines but then also look for the one that is # ESMF_COMM:...

So then perhaps at run-time? But for that I probably need to invoke the names of @theurich and @oehmke and see if they have thoughts? Maybe there is some ESMF_Initialize-like call that acts differently under mpiuni?

Or maybe we add a CMake style try_compile where if the ESMF folks know of a call that fails under mpiuni we could say "Hey, this simple ESMF code failed, your ESMF must have been built as mpiuni" or the like.

[tclune]

We can query the VM and get numPets. Not sure that works with MPIUNI, but by the interface we would expect a result of 1. The question then is what if we run say doubly-periodic on 1 core for testing purposes ....

Hopefully @theurich has a better solution. We could ask for some patch to ESMF for this but that seems heavy for one problem by one student using a non GMAO application ...

[oehmke]

ESMF_VMGet() <http://earthsystemmodeling.org/docs/nightly/develop/ESMF_refdoc/node6.html#SECTION0601361800000000000000> has the argument esmfComm which is a string that contains the value of the ESMF_COMM build environment variable. You could use that to check for: mpiuni

…

On Jan 25, 2024, at 2:47 PM, Tom Clune ***@***.***> wrote: We can query the VM and get numPets. Not sure that works with MPIUNI, but by the interface we would expect a result of 1. The question then is what if we run say doubly-periodic on 1 core for testing purposes .... Hopefully @theurich <https://github.com/theurich> has a better solution. We could ask for some patch to ESMF for this but that seems heavy for one problem by one student using a non GMAO application ... — Reply to this email directly, view it on GitHub <#2553 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AE6A7U5RT246HDEHBGGX2CTYQLHGZAVCNFSM6AAAAABCJHHGISVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSMJRGA2TAOJQHA>. You are receiving this because you were mentioned.

[mathomp4]

Ooh. Thanks @oehmke! I'll test it out with an mpiuni Baselibs I have around.

[oehmke]

You’re welcome! Let me know if it doesn’t do what you want.

…

On Jan 25, 2024, at 4:17 PM, Matthew Thompson ***@***.***> wrote: Ooh. Thanks @oehmke <https://github.com/oehmke>! I'll test it out with an mpiuni Baselibs I have around. — Reply to this email directly, view it on GitHub <#2553 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AE6A7U6XF37PZFQKPN2DL73YQLRWZAVCNFSM6AAAAABCJHHGISVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSMJRGE2DQNZZG4>. You are receiving this because you were mentioned.

[mathomp4]

You’re welcome! Let me know if it doesn’t do what you want.

@oehmke I have a PR that seems to do it: #2556

But new question: how much do I need to belt-and-suspender this? For example, do I need to first call ESMF_VMIsCreated? Or is that just overkill in that a VM is always created after a successful ESMF_Initialize call (which is right above this)?

Also, is there a preference between using ESMF_VMGetCurrent vs ESMF_VMGetGlobal? Or perhaps a better call to get the VM should be used?

I ask only because I rarely do ESMF coding and I figure I should do it "right" when I do. 😄

[theurich]

@mathomp4 - you are guaranteed a valid global VM after ESMF_Initialize() returns successfully. Right under ESMF_Initialize(), the current VM is identical to the global VM, so either of those calls would work.
Simpler even, just add the vm argument to your ESMF_Initialize() call, and it will return the global VM to you without any extra work. Then access esmfComm via ESMF_VMGet() as suggested by @oehmke, and you should be all set.

[mathomp4]

@theurich Oooh. Nice. One less call is always good! Thanks!