Can a known file mode encoding be declared on nc_open_par?

[mo-marqh]

hello Unidata netCDF folks

I'm in the process of performance & scalability analysing a parallel processing simulation code base using netCDF, HDF5 & MPI.

As part of exploring the I/O patterns inherent in the code, I am counting and tracing open calls.

A pattern which appears to be repeating when nc_open_par is used, is that each open call appears to be interpreted as two file system calls, in quick succession.

The first is calling nc_open_par but not calling HDF5
The second is calling nc_open_par which is then calling HDF5

Am I reading this right?

for example:

# open path="/my/unique/file/path/this_file.nc" flags=0x0000 rank=0
  #0   check_and_emit +0xf5  [0x1494c5eabb95]
  #1   open +0x13b  [0x1494c5eaba3b]
...
  #5   MPI_File_open +0x205  [0x1494c02e9fb5]
  #6   NC_infermodel +0x1128  [0x1494c5a9bc18]
  #7   NC_open +0xd4  [0x1494c5a7e7c4]
  #8   nc_open_par_fortran +0x24  [0x1494c5a7c5d4]
  #9   nf_open_par_ +0x127  [0x1494c5c6e717]
  #10  nf90_open$netcdf_ +0x15f  [0x1494c5c80c1f]
...
# open path="/my/unique/file/path/this_file.nc" flags=0x0000 rank=0
  #0   check_and_emit +0xf5  [0x1494c5eabb95]
  #1   open +0x13b  [0x1494c5eaba3b]
...
  #5   MPI_File_open +0x205  [0x1494c02e9fb5]
  #6   H5FD__mpio_open +0x2e6  [0x1494b66eebf6]
  #7   H5FD_open +0x23a  [0x1494b66d617a]
...
  #12  H5Fopen +0x55  [0x1494b66b2b25]
  #13  nc4_H5Fopen +0x3f  [0x1494c5ba779f]
  #14  NC4_open +0x3b9  [0x1494c5ba6989]
  #15  NC_open +0x216  [0x1494c5a7e906]
  #16  nc_open_par_fortran +0x24  [0x1494c5a7c5d4]
  #17  nf_open_par_ +0x127  [0x1494c5c6e717]
  #18  nf90_open$netcdf_ +0x15f  [0x1494c5c80c1f]
...

Is the netcdf-c implementation always:

1. opening the file to check if it may be a HDF5 encoded file,
2. closing the file,
3. reopening the file using HDF5
   ?

If so, is it plausible and/or sensible to pre-assert that the file should be opened using HDF5? (error if it fails?)
( nc_open && nc_open_par both take an omode argument, but this omode is not as rich as nc_create's cmode, which can also define encoding: NC_NETCDF4 causes netCDF to create a HDF5/NetCDF-4 file)

For large MPI_Comm_World counts using MPI-IO and communicating with parallel file systems that manage metadata servers and object stores, then such a pattern could bring a small amount of relief from high load scenarios.

Is it possible to do this already within netcdf-c & netcdf-fortran?
If not, could this be a valid feature development?
Has this been looked into in the past and deemed problematic? ... unnecessary?

many thanks
mark

[mo-marqh]

delving into some more implementation tracing, it appears that opening with nc_open does not hit this double open problem.

The first call I trace for this file path is :

# open path="/my/unique/file/path/this_file.nc" flags=0x0000 rank=0
  #0   check_and_emit +0xf5  [0x154db1a07b95]
  #1   open64 +0x13b  [0x154db1a07cfb]
  #2   H5FD__sec2_open +0x10f  [0x154da230439f]
  #3   H5FD_open +0x23a  [0x154da22d617a]
  #4   H5F_open +0x23b  [0x154da22bef2b]
  #5   H5VL__native_file_open +0x13  [0x154da254c7b3]
  #6   H5VL_file_open +0x53  [0x154da2539273]
  #7   H5F__open_api_common +0x1ae  [0x154da22b2f0e]
  #8   H5Fopen +0x55  [0x154da22b2b25]
  #9   nc4_H5Fopen +0x3f  [0x154db170379f]
  #10  NC4_open +0x3b9  [0x154db1702989]
  #11  NC_open +0x216  [0x154db15da906]
  #12  nc_open +0x14  [0x154db15da6e4]
  #13  nf_open_ +0x113  [0x154db17a6e63]
  #14  nf90_open$netcdf_ +0x17d  [0x154db17dcc3d]

... seemingly able to open the HDF5 encoded netCDF file.

So, I'm now exploring this as a difference between nc_open and nc_open_par for HDF5 encodings, I think