2.0.0rc2: Regression in MPI_Type_get_envelope() after MPI_File_get_view()

[dalcinl]

Querying the envelope of etype/filetype obtained with MPI_File_get_view() errors with MPI_ERR_INTERN.

from mpi4py import MPI
comm = MPI.COMM_SELF
mode = MPI.MODE_RDWR | MPI.MODE_CREATE
info = MPI.INFO_NULL
fh = MPI.File.Open(comm, "/tmp/datafile", mode, info)
disp, etype, ftype, datarep = fh.Get_view()
assert disp == 0
assert datarep == 'native'
print(etype.Get_envelope())
print(ftype.Get_envelope())

$ python test-ompi-3.py 
Traceback (most recent call last):
  File "test-ompi-3.py", line 9, in <module>
    print(etype.Get_envelope())
  File "MPI/Datatype.pyx", line 469, in mpi4py.MPI.Datatype.Get_envelope (src/mpi4py.MPI.c:65712)
mpi4py.MPI.Exception: MPI_ERR_INTERN: internal error

[edgargabriel]

I think ompio does the right thing here and the error comes from the datatype engine (there is the possibility that I might use the wrong function to duplicate the datatype after it moved to the opal layer, but in the debugger I can see the datatype as "dup dup MPI_Byte". @bosilca would you mind having a quick look at that? C version of this test is attached.

[edgargabriel]

#include <stdio.h>
#include "mpi.h"

int main ( int argc, char **argv )
{
    int rank, size, ret;
    MPI_File fh;
    MPI_Datatype etype, ftype;
    MPI_Offset disp;
    char datarep[64];

    MPI_Init ( &argc, &argv);
    MPI_Comm_rank ( MPI_COMM_WORLD, &rank );
    MPI_Comm_size ( MPI_COMM_WORLD, &size );

    MPI_File_open (MPI_COMM_WORLD, "test.out", MPI_MODE_RDWR | MPI_MODE_CREATE,                                                                                                                 
                   MPI_INFO_NULL, &fh );                                                                                                                                                        


    ret = MPI_File_get_view (fh, &disp, &etype, &ftype, datarep);                                                                                                                               
    if ( ret != MPI_SUCCESS ) {                                                                                                                                                                 
        printf("Error in MPI_File_get_view\n" );                                                                                                                                                
    }                                                                                                                                                                                           

    int num_integers, num_addresses, num_datatypes, combiner;                                                                                                                                   
    MPI_Type_get_envelope ( etype, &num_integers, &num_addresses,                                                                                                                               
                            &num_datatypes, &combiner);                                                                                                                                         
    printf("etype: num_integers=%d, num_addresses=%d, num_datatypes=%d, "                                                                                                                       
           "combiner=%d\n", num_integers, num_addresses, num_datatypes,                                                                                                                         
           combiner );                                                                                                                                                                          


    MPI_Finalize ();                                                                                                                                                                            
    return 0;                                                                                                                                                                                   
}

EDIT: Added verbatim block / C highlighting

[dalcinl]

>>> from mpi4py import MPI
>>> MPI.BYTE.Dup().Dup().Get_envelope()
(0, 0, 1, 1)

[edgargabriel]

I was referring to the name of the datatype as shown in the internal datatype structure of OMPI. Basically, it shows that MPI_BYTE was duplicated twice during the process. But then ompi_datatype_get_args returns with MPI_ERR_INTERNAL

[dalcinl]

I know, I'm just showing you that duplicating twice and getting the envelope seems to work just fine.

[edgargabriel]

so basically what it comes down to is whether

ompi_datatype_duplicate (&ompi_mpi_byte.dt, ...);

is identical to using MPI_Type_dup (MPI_BYTE,...)

[bosilca]

Calling ompi_datatype_duplicate is not enough, the args (which is the placeholder used for the enveloppe) is not created. In fact only the MPI level functions are creating the args, because returning a datatype is not a usual operation for any of the MPI functions.

[edgargabriel]

ok, so what else do I have for that? And I agree that it is fairly unusual scenario that I honestly have not seen in any of the MPI I/O applications or benchmarks

[edgargabriel]

looking at the code of type_dup.c, I see there is a function ompi_datatype_set_args, I will try to use this one. Following the logic in that function, I am however not sure whether I would also need to copy the attributes in a similar manner, i.e. if a user set some attributes on the filetype for example, and he retrieves a copy of the file type using get_view, would who expect the attributes to be available?

[edgargabriel]

I have a solution coded up for this one, but I have now a question to the MPI spec experts @bosilca and @jsquyres . What is the combiner supposed to be e.g. for the etype returned by MPI_File_get_envelope? Since I dup the datatypes on the handle, the combiner in my solution is COMBINER_DUP. If you run it with ROMIO, it is the original combiner (e.g. COMBINER NAMED if the original datatypes as e.g. MPI_BYTE).

MPI 3.1 says on page 506: The etype and filetype are new datatypes with
typemaps equal to the typemaps of the current etype and filetype, respectively.

which would support my solution. However, 3 lines later
If etype or filetype are derived datatypes, the user is responsible for freeing them. The etype and
filetype returned are both in a committed state

which would support ROMIOs solution.

[bosilca]

It doesn't say that it returns a duplicate of the datatype, which allows me to think that:

• we don't have to copy the attributes, and
• we should not return a duplicate of the datatype.

I would simply call ompi_datatype_copy_args on the datatype to be returned.

[edgargabriel]

well, ok, but that is then a bigger change, because right know I do dupliucate all the datatypes. Would the standard in your opinion support that? In the end, it does say that it returns a 'new' datatype