Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reading netCDF files with unconventional dimension names #344

Open
dkang2 opened this issue May 10, 2024 · 5 comments
Open

Reading netCDF files with unconventional dimension names #344

dkang2 opened this issue May 10, 2024 · 5 comments
Assignees

Comments

@dkang2
Copy link
Collaborator

dkang2 commented May 10, 2024

Is your feature request related to a problem? Please describe.
Yes, occasionally in some netCDF files there are variables that have dimensions other than those normally recognized by VERDI (columns, rows, layers, or south-north, west-east, and bottom-top), such at categories of crop types, soil types, or species types. Usually the variables have the dimensions as var(Time, var-cat, south-north, west-east). When this kind of files are opened by VERDI, the variables with the unconventional dimensions are ignored (no showing in the variable list).

Describe the solution you'd like
In conventional IOAPI (TSTEP, LAY, ROW, COL) or wrf netCDF (Times, bottom_top, south_north, west_east) files, verdi would automatically recognize the variables as 4-D time-space variables. But in the files with the unconventional dimension names, the first (Time) and the second and third (spatial) dimensions are generally the same. The only difference is the forth (LAY in case of IOAPI, bottom_top in case of wrf netCDF) dimension. One proposed solution is to treat the third (time-independent) or forth (time-dependent) unconventional dimension as the vertical layers as long as the other dimensions are conformable.

Describe alternatives you've considered
None.

Additional context
An example file can be found at: /work/MOD3DEV/dkj/Verdi/example_files/beld4_CMAQ12KM_2011.nc and look for variables with land_cat, crop_cat, and tree-cat as one of the dimensions.

@yadongxuEPA
Copy link
Collaborator

Tested with VERDI_2.1.5_linux64_20240717.tar.gz on Atmos, found that this issue has not resolved.
used this data file: /work/MOD3DEV/dkj/Verdi/example_files/beld4_CMAQ12KM_2011.nc

Found no differences in the app behaviors compared with loading the data file into "VERDI_2.1.5_linux64_20240524.tar.gz", the hidden variables (with unconventional dimension names) were not listed under "Variables" dropdown window.
image
image

@yadongxuEPA
Copy link
Collaborator

Add more details to the expected behavior for the above test case.
Use "ncdump", we can see these dimensions & variables in the data file:
image
testing_20240717_issue_344

We expected to see "TREEF", "CROPF" to be added under the variable dropdown window. Apparently, the older version VERDI (20240524 builds) already had code structure to replace "layer" with "land_cat" because we see "LANDUSEF" is listed and available for tile plot.

testing_20240717_issue_344_1

@yadongxuEPA
Copy link
Collaborator

Tested with VERDI_2.1.5_linux64_20240725.tar.gz on Atmos, found that this issue has not resolved.
used this data file: /work/MOD3DEV/dkj/Verdi/example_files/beld4_CMAQ12KM_2011.nc

Although "TREEF", "CROPF" are added to the variable list, but "tree_cat" and "crop_cat" have not been imported as the fourth dimension to replace "layer".
testing_20240725_issue_344_1
testing_20240725_issue_344_2

@dkang2
Copy link
Collaborator Author

dkang2 commented Aug 15, 2024

Just tested the 20240814 build on linux (atmos) and found that the issue has been nicely resolved. In addition, the SLF4J error has also resolved.

@yadongxuEPA
Copy link
Collaborator

Tested with VERDI_2.1.5_linux64_20240824.tar.gz on Atmos, found that this issue has resolved.
used this data file: /work/MOD3DEV/dkj/Verdi/example_files/beld4_CMAQ12KM_2011.nc
image
image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants