-
Notifications
You must be signed in to change notification settings - Fork 52
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Advice regarding PIO async in multi-component model #1853
Comments
You do not do the MPI_split. You call PIO_Init() on all processors. Those that are I/O will not return from PIO_Init() until all computational units call PIO_Finalize(). |
Also, make sure you use the current master. It has some very important async fixes. I am hoping to do a release this week with those fixes and improvements. |
Hi Nic,
Any use of MPI_COMM_WORLD for collectives will not work with async IO since
the IO processors will not participate in the call.
All uses of MPI_COMM_WORLD in the code need to be replaced with an explicit
derived communicator.
There are two interfaces to PIO_init_async in the first
<https://github.com/NCAR/ParallelIO/blob/master/src/clib/pioc.c#L1587> you
pass in COMM_WORLD and a COMP_COMM is returned.
In the second
<https://github.com/NCAR/ParallelIO/blob/master/src/clib/pioc.c#L2046> you
pass in existing COMMs that you have created and IO_COMM members do not
return.
They both do the same thing, choose the one that best fits your code. You
can use either all of comm_world or just the comm of the component which
will use PIO
but if you do the later you need to keep in mind that communications with
your component from the other components will be with the computational
split and not
the comm you passed to PIO_INIT and you will need to tell the other models
that somehow.
Jim
…On Mon, Feb 8, 2021 at 4:38 AM Edward Hartnett ***@***.***> wrote:
Also, make sure you use the current master. It has some very important
async fixes. I am hoping to do a release this week with those fixes and
improvements.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
<#1853 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ABOXUGFQJ7YVTGK7STY2GPTS57EL7ANCNFSM4XIJPGAA>
.
--
Jim Edwards
CESM Software Engineer
National Center for Atmospheric Research
Boulder, CO
|
Thank you very much @edwardhartnett and @jedwards4b - this is what I was after. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Hi,
I'd like to get some advice re implementing asynchronous IO for a single model in a multi-component model. I think my main query is where I should call pio_init() wrt. the existing call to MPI_Split that creates the individual communicators for the component models. The options I can think of:
Call pio_init() on all PEs before my existing MPI_Split. This will leave me with a new computation communicator which will become the new COMM_WORLD and an IO communicator. The new COMM_WORLD will then be split into component models.
Call pio_init() after the existing MPI_Split with only the PEs that are a part of the component model which uses the async IO. For various reasons I'm not sure how this would work, e.g. any subsequent use of MPI_COMM_WORLD for collectives in the code would break because the IO PEs are not going to join in.
Somehow replace the existing MPI_Split with the one that (presumably) is being called within pio_init()
Any thoughts or examples of how this is done elsewhere would be much appreciated.
The text was updated successfully, but these errors were encountered: