Open opal frameworks in opal_init not ompi_mpi_init

[hjelmn]

The allocator, rcache, and mpool frameworks were moved down to opal
some time ago. The framework open calls should have been moved as
well.

Signed-off-by: Nathan Hjelm hjelmn@lanl.gov

[hjelmn]

@jsquyres I just noticed something. ompi_mpi_init/finalize use opal_init/finalize_util and not the full opal_init/finalize. Do you know if there is a reason for that? Seems like a bug to me. We are getting lucky because orte does use opal_init/finalize.

[jsquyres]

Generally looks ok to me. @bosilca, can you confirm?

I'm trying to remember if there's a reason that the OMPI layer uses opal_init|finalize_util()... Best I can think of is that we wanted to initialize as little of OPAL as possible before initializing the RTE, which would assumedly initialize the rest of OPAL. But if we ran into a failure somewhere before ORTE initialized OPAL, then we might as well have initialized as little of OPAL as possible.

That being said, perhaps that's a false assumption, anyway: the RTE may not be ORTE. Hence, we should probably fully initialize OPAL, anyway.

@bosilca @hjelmn Does this sound right to you?

[bosilca]

1. The 3 frameworks you mentioned (allocator, rcache, and mpool) were opened by the MPI layer in purpose. The design was to minimize all impact on the code, in other words if you don't need them don't open them. For the same reason the BTLs are opened explicitly by the BML (and never by default by the OPAL layer).
2. opal_init vs. opal_init_utils. We wanted to be able to change the MCA parameters before OPAL opened and initialized all the frameworks. Thus, we opal_init_utils just to setup the MCA params. I have a vague recollection about some issue with the event library ...
3. It was expected that the RTE_open will do an opal_init. However, as we refcount the opal_init we can now do multiple init/fini, so we can add a safeguard opal_init in ompi_mpi_init.

[bosilca]

I don't see why we need to force the opening of all OPAL components from the beginning. I left the allocator, mpool, rcache and btl up in purpose in the OMPI layer as it was the first place where they were needed. Do we now have a reason to open everything down in OPAL?

[hjelmn]

i will close this. i plan to move the mpool open down to opal (since the mpool has changed quite a bit) but will leave the others where they are.