-
Notifications
You must be signed in to change notification settings - Fork 38
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Multi-model statistics on changed standard names through CMIP projects #1985
Comments
Thanks for the report, @bettina-gier! If we defer this to Iris, the only solution I can see is to go with the standard name that is not an alias, in case several standard names appear. But I think we should think about a solution within ESMValTool as well. After all, this is not only about the standard name, but also about the long name and, more generally, about changed variables from different sources. Perhaps we can have a "target project" (the project of the first dataset, the project of the majority of datasets, a user specified option, ...) and standardize all data to that? I'll also feed this upstream into the CMIP7 Task Teams for the next generation data request. |
thanks a lot @bettina-gier amd @zklaus 🍺 Tina, I am a bit confused since you mention OBS and OBS6 data, but your example recipe, and inherent issue describes a CMIP5 vs CMIP6 problem - if it's OBS-related then I wouldn't expect that to be an issue since we use the same standards to cmorize OBS data (I believe it is CMIP6 standards, but I might be wrong); if it's a CMIP-data-related issue then we have fixes for that, and by that I mean the on-the-fly CMIP5-vs-CMIP6 dictionary lookup and fixing, so either way I think it's something we can do to repair.
@zklaus that is indeed the root of our issues, please shake them all into (one standard) shape, holler back if you need muscle for that 😁 |
The currently available datasets for gpp are in the OBS project, which use the CMIP5 standard name, while the new data @schlunma and me added is in the OBS6 standard using the CMIP6 cmor data and thus CMIP6 standard name. I just used CMIP5 + CMIP6 model data in my example recipe as our OBS6 dataset isn't in the tool yet and the issue is the same. Sure we could remake all current OBS data into OBS6 data but we'd still encounter this problem if we ever want to merge cubes from the different CMIP projects for gpp. |
gotcha! @remi-kazeroni needs a summon here - my take would be to have CMIP5-standards OBS data transferred to CMIP6 standards; about the issue not working for the CMIP5/6 data case, am a bit surprised, since we have that mapping dict in and that should be done on the fly - maybe we need to add this there, @sloosvel needs a summon here then 😁 |
as far as I recall our current mapping is for changed short names but the standard names were the same, what I found is changed standard names with the same short name. So we might need to enlarge our mapping dict? I haven't checked what other variables are affected by this. |
Technically, aligning the names of variables and coordinates to a single project could be implemented as a preprocessor function, in the recipe you could then specify the name of the target dataset (or project?) as an argument to the function. We would need to take a bit of care to do this in a safe way though, i.e. don't allow renaming just anything to anything. |
When trying to finalize the PR with the diagnostics from my last paper I came up with the same problem.
For the Is someone working on a solution right now? Theoretically I promised to provide the code as soon as possible in the ESMValTool. |
There's an open PR in iris about handling standard name aliases: SciTools/iris#5313. This could be useful for us here. |
Describe the bug
I'm a walking bugfinder right now =D
I wanted to apply the multi_model_statistics preprocessor on gpp obs data for a "multi-obs mean", but encountered an error with the standard name
Turns out the standard name of gpp is different in CMIP5 and CMIP6
The CMIP5 standard name is considered an official alias according to the cf conventions, so it isn't wrong.
This occurs because two of the datasets are OBS while another is OBS6. As the OBS6 dataset isn't added yet, I've made a small example recipe just using CMIP5 and CMIP6 data to reproduce the error:
Example recipe
I assume this is more of an iris issue and I guess we should forward it to them to ask if they will allow the use of different standard_names on cube merges if they are official aliases @ESMValGroup/technical-lead-development-team? @zklaus probably interesting for you as our standards person!
In my current case I was using the multi_model_statistics in the diagnostic so I did a quick and dirty workaround, just wanted to raise this issue to make it known!
The text was updated successfully, but these errors were encountered: