-
Notifications
You must be signed in to change notification settings - Fork 6
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Netcdf format from MOM6 #188
Comments
It looks like we don't use the MOM_netcdf module I linked and its controlled by FMS. For some reason we use the "deprecated_io" option from FMS. With "deprecated_io", I think the format is controlled by cpp macros, and so we need to turn off the "+large_file" option in spack, and then the default will be netcdf4. Alternatively, we could try using the "fm2_io" option, (i.e. turn off "+deprecated_io") but I assume there was a good reason this option is set (@micaeljtoliveira) ? |
@micaeljtoliveira confirmed removing Once we have done that, we can add
|
I'm moving this to |
I think you need to remove "+large_file +deprecated_io" from https://github.com/COSIMA/spack-config/blob/7a477783782e62b0f743937c7c512b5ca42835f4/environments/access-om3-0_x_0/spack.yaml#L20 and then build. Then in the config update |
Flag OCN_modelio::pio_* params as unused in nuopc.runconfig remove extra lines in nuopc.runconfig update comment for ocean nc type
Flag OCN_modelio::pio_* params as unused in nuopc.runconfig remove extra lines in nuopc.runconfig update comment for ocean nc type
Flag OCN_modelio::pio_* params as unused in nuopc.runconfig remove extra lines in nuopc.runconfig update comment for ocean nc type
Flag OCN_modelio::pio_* params as unused in nuopc.runconfig remove extra lines in nuopc.runconfig update comment for ocean nc type
Flag OCN_modelio::pio_* params as unused in nuopc.runconfig remove extra lines in nuopc.runconfig update comment for ocean nc type
Flag OCN_modelio::pio_* params as unused in nuopc.runconfig remove extra lines in nuopc.runconfig update comment for ocean nc type
Flag OCN_modelio::pio_* params as unused in nuopc.runconfig remove extra lines in nuopc.runconfig update comment for ocean nc type
Flag OCN_modelio::pio_* params as unused in nuopc.runconfig remove extra lines in nuopc.runconfig update comment for ocean nc type
Flag OCN_modelio::pio_* params as unused in nuopc.runconfig remove extra lines in nuopc.runconfig update comment for ocean nc type
Flag OCN_modelio::pio_* params as unused in nuopc.runconfig remove extra lines in nuopc.runconfig update comment for ocean nc type
Flag OCN_modelio::pio_* params as unused in nuopc.runconfig remove extra lines in nuopc.runconfig update comment for ocean nc type
Flag OCN_modelio::pio_* params as unused in nuopc.runconfig remove extra lines in nuopc.runconfig update comment for ocean nc type
Flag OCN_modelio::pio_* params as unused in nuopc.runconfig remove extra lines in nuopc.runconfig update comment for ocean nc type
Flag OCN_modelio::pio_* params as unused in nuopc.runconfig remove extra lines in nuopc.runconfig update comment for ocean nc type
Flag OCN_modelio::pio_* params as unused in nuopc.runconfig remove extra lines in nuopc.runconfig update comment for ocean nc type
Flag OCN_modelio::pio_* params as unused in nuopc.runconfig remove extra lines in nuopc.runconfig update comment for ocean nc type
When running the current 0.25 degree configs, MOM6 produces netcdf output of this form:
work/MOM_IC_1.nc work/MOM_IC_2.nc work/MOM_IC.nc
The files are of format "64-bit-offset" (aka cdf-2) and the first two files are size 3.6GB. Which is the maximum file size supported by this format, and therefore leads to the multiple netcdf files when it doesn't fit in one file.
Having multiple output files itself is not a problem, but this file format doesn't support compression, which leads to file sizes ~x2 as large as compressed files.
It doesn't look like there are configuration options in MOM to change this (see https://github.com/NCAR/MOM6/blob/2b1201a87259c912c93e4c11039b8e59971e3c26/src/framework/MOM_netcdf.F90#L127). To change it we would need to set the filetype (NF90_NETCDF4) in the
nf90_create()
calls and set deflate level innf90_def_var()
calls. We could do this as a patch, or add it to the MOM namelist?@angus-g Has there been any discussion of updating formats amongst the MOM developers? Do you know how other groups handle this?
The text was updated successfully, but these errors were encountered: