- make the split collective shared file pointer operations work

[edgargabriel]

• minor code restructering in io/ompio required for that.

[edgargabriel]

@hppritcha this is now the cleaned up version of the split collective operations, not containing any other fixes. If you have a chance to review it, please go ahead. If you prefer to review the pr to v2.x, let me know and I can merge this commit and file the pr.

[hppritcha]

should this be using opal_output instead?

[hppritcha]

I'm getting segfaults with filetest. Please don't merge yet.

[edgargabriel]

ok, will also incorporate your comments about using opal_output instead of printf tomorrow morning. You are testing on the cray? Can try on the stuttgart machine as well

[hppritcha]

i don't think this problem is specific to cray, but yes I"m using the NERSC systems hopper and edison.

[hppritcha]

If I don't use lustre, then filetest passes. By the way, what's going on with mca_fs_lustre_file_get_size?

[hppritcha]

I'm okay with this PR, but it appears that for Lustre things are pretty broken. Filetest fails at the very
first ompio_io_ompio_file_close owing to some mishandling of the f_converter at line 369. That's not the whole problem though because if I comment out that line, things still end up segfaulting later in the tests. The signature is what one would get with heap corruption.

Edison is using Lustre 2.5.0 (at least for the clients).
Hopper is using Lustre 2.4.1

I observe the same behavior on both systems' lustre file systems.

I'm pretty sure things were working for lustre several weeks ago when I tested one of the previous ompi i/o PRs. I'll try to do some bisecting to narrow down the problem when I have time.

[edgargabriel]

hm, ok, thanks! I will merge the branch and check on lustre as well. One simple test that you could run is to see whether things execute correctly if you exclude the lustre fs component, it should still work correctly, e.g.

mpirun --mca fs ^lustre -np 6 ./filetest

Also note, that I did modifications to the filetest testsuite in the last couple of days, maybe I introduced a bug there inadvertently.

[edgargabriel]

well, I will first fix the printf to opal_output stuff tomorrow morning, than I will merge.

[edgargabriel]

Will have to check mca_fs_lustre_file_get_size, did not look into that in a looong time

[edgargabriel]

Can confirm that I am able to reproduce the problem that you see on the Stuttgart Cray/Lustre, and am debugging it. I know at least part of the problem, it should be however unrelated to this pr.

[edgargabriel]

the problem definitely comes from the lustre fs component. If I exclude it (on the lustre system) and use the regular ufs component instead, everything works like a charm. I will merge this pr, and debug the lustre fs component to see what is going on. Thanks @hppritcha !