Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Flatten ancestor lists for diag_spei.R and diag_spi.R. #1745

Merged
merged 1 commit into from
Jul 22, 2020

Conversation

katjaweigel
Copy link
Contributor

@katjaweigel katjaweigel commented Jul 22, 2020

Before you start, please read our contribution guidelines.

Please discuss your idea with the development team before getting started, to avoid disappointment later. The way to do this is to open a new issue on GitHub. If you are planning to modify an existing functionality, please discuss it with the original author(s) by tagging them in the issue.


Tasks

  • Create an issue to discuss what you are going to do, if you haven't done so already (and add the link at the bottom)
  • Give this pull request a descriptive title that can be used as a one line summary in a changelog
  • Make sure your code is composed of functions of no more than 50 lines and uses meaningful names for variables
  • Circle/CI tests pass. Status can be seen below your pull request. If the tests are failing, click the link to find out why.
  • Preferably Codacy code quality checks pass, however a few remaining hard to solve Codacy issues are still acceptable. Status can be seen below your pull request. If there is an error, click the link to find out why. If you suspect Codacy may be wrong, please ask by commenting.
  • Please use yamllint to check that your YAML files do not contain mistakes
  • (Only if really necessary) Add any additional dependencies needed for the diagnostic script to setup.py, esmvaltool/install/R/r_requirements.txt or esmvaltool/install/Julia/Project.toml (depending on the language of your script) and also to package/meta.yaml for conda dependencies (includes Python and others, but not R/Julia)
  • If new dependencies are introduced, check that the license is compatible with Apache2.0

New recipe/diagnostic

  • Add documentation for the recipe to the doc/sphinx/source/recipes folder and add a new entry to index.rst
  • Add provenance information

Modified recipe/diagnostic

  • Update documentation for the recipe to the doc/sphinx/source/recipes folder
  • Update provenance information if needed
  • Assign the author(s) of the affected recipe(s) as reviewer(s)

New data reformatting script

  • Test the CMORized data using recipes/example/recipe_check_obs.yml, to make sure the CMOR checks pass without errors
  • Add the new dataset to the table in the documentation
  • Tag @mattiarighi in this pull request, so that the new dataset can be added to the OBS data pool at DKRZ and synchronized with CEDA-Jasmin

Modified data reformatting script

  • Test the CMORized data using recipes/example/recipe_check_obs.yml, to make sure the CMOR checks pass without errors
  • Tag @mattiarighi in this pull request, so that the updated dataset can be added to the OBS data pool at DKRZ and synchronized with CEDA-Jasmin

If you need help with any of the tasks above, please do not hesitate to ask by commenting in the issue or pull request.


Closes #1717

@katjaweigel
Copy link
Contributor Author

Not completely tested due to Mistral issues.

@mattiarighi
Copy link
Contributor

Not completely tested due to Mistral issues.

You still can run the recipe in interactive mode (better on mistralpp).
I can try it now.

@katjaweigel
Copy link
Contributor Author

Yes, I did that yesterday, but it took forever (and I forgot something). I can restart it also again in there.

@mattiarighi
Copy link
Contributor

Searching for ancestors in the R diagnostics returns this list:

> grep -r ancestors --include="*.R" .

./droughtindex/diag_spi.R:  ancestors = list(""),
./droughtindex/diag_spi.R:  xprov$ancestors <- modfile[mod]
./droughtindex/diag_spi.R:xprov$ancestors <- modfile
./droughtindex/diag_save_spi.R:  ancestors = list(""),
./droughtindex/diag_save_spi.R:   xprov$ancestors <- list(modfile[mod])
./droughtindex/diag_spei.R:  ancestors = list(""),
./droughtindex/diag_spei.R:  xprov$ancestors <- list(modfile1[mod], modfile2[mod])
./droughtindex/diag_spei.R:xprov$ancestors <- c(modfile1, modfile2)
./quantilebias/quantilebias.R:    ancestors = list(infile, ref_data_file),
./miles/miles_regimes.R:    ancestors = infile,
./miles/miles_eof.R:    ancestors = infile,
./miles/miles_block.R:    ancestors = infile,
./hyint/hyint.R:      ancestors = prov_info[[fname]]$ancestors,
./hyint/hyint_plot_trends.R:              ancestors = prov_info[[infile]]$ancestors
./hyint/hyint_plot_trends.R:              prov_info[[figname]]$ancestors <-
./hyint/hyint_plot_trends.R:                c(prov_info[[figname]]$ancestors,
./hyint/hyint_plot_trends.R:                  prov_info[[infile]]$ancestors)
./hyint/hyint_plot_trends.R:                ancestors = prov_info[[infile]]$ancestors
./hyint/hyint_plot_trends.R:          ancestors = prov_info[[infile]]$ancestors
./hyint/hyint_plot_maps.R:                ancestors = prov_info[[infile]]$ancestors
./hyint/hyint_plot_maps.R:                ancestors = list(prov_info[[infile]]$ancestors,
./hyint/hyint_plot_maps.R:                  prov_info[[ref_filename]]$ancestors)
./hyint/hyint_plot_maps.R:              ancestors = list(prov_info[[infile]]$ancesstors,
./hyint/hyint_plot_maps.R:                prov_info[[ref_filename]]$ancestors)
./hyint/hyint_plot_maps.R:            ancestors = prov_info[[infile]]$ancestors
./hyint/hyint_etccdi_preproc.R:    prov_info[[hyint_file]]$ancestors <-
./hyint/hyint_etccdi_preproc.R:          c(prov_info[[hyint_file]]$ancestors, etccdi_files)
./hyint/hyint_diagnostic.R:    ancestors = climofiles[model_idx],
./hyint/hyint_trends.R:    ancestors = prov_info[[infile]]$ancestors,
./extreme_events/extreme_events.R:    ancestors = as.list(infile),
./extreme_events/extreme_events.R:    ancestors <- sapply(setdiff(models_name, reference_datasets),
./extreme_events/extreme_events.R:    xprov <- provenance_record(ancestors)
./magic_bsc/extreme_index.R:            ancestors = list(projection_filenames, reference_filenames),
./magic_bsc/extreme_index.R:  ancestors = list(fullpath_filenames),
./magic_bsc/multimodel_products.R:  ancestors = list(fullpath_filenames),
./magic_bsc/weather_regime.R:    ancestors = list(fullpath_filenames[reference_files]),
./magic_bsc/weather_regime.R:  ancestors = list(
./magic_bsc/weather_regime.R:  ancestors = list(filencdf1, filencdf2),
./magic_bsc/capacity_factor.R:    ancestors = list(
./magic_bsc/combined_indices.R:xprov <- list(ancestors = list(fullpath_filenames),
./magic_bsc/extreme_spells.R:      ancestors = list(projection_filenames, reference_filenames),
./magic_bsc/toymodel.R:  ancestors = list(fullpath_filenames),
./magic_bsc/diurnal_temp_index.R:      ancestors = list(

Do you see anything that can be quickly fixed in a similar way?

Sorry if I can't help, but I don't speak R. 😞

@mattiarighi
Copy link
Contributor

Tested successfully!

@mattiarighi mattiarighi marked this pull request as ready for review July 22, 2020 07:35
@mattiarighi
Copy link
Contributor

Merging this now.
Please open a new PR if you have the fixes for the other diags.

@mattiarighi mattiarighi merged commit 4525729 into master Jul 22, 2020
@mattiarighi mattiarighi deleted the fix_ancestors_for_recipe_spei branch July 22, 2020 07:36
@katjaweigel
Copy link
Contributor Author

ok ,thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Provenance of recipe_spei.yml broken
3 participants