diff --git a/doc/sphinx/source/api/esmvaltool.diag_scripts.seaborn_diag.rst b/doc/sphinx/source/api/esmvaltool.diag_scripts.seaborn_diag.rst new file mode 100644 index 0000000000..d250d0aaaf --- /dev/null +++ b/doc/sphinx/source/api/esmvaltool.diag_scripts.seaborn_diag.rst @@ -0,0 +1,9 @@ +.. _api.esmvaltool.diag_scripts.seaborn_diag: + +Seaborn Diagnostic +================== + +.. automodule:: esmvaltool.diag_scripts.seaborn_diag + :no-members: + :no-inherited-members: + :no-show-inheritance: diff --git a/doc/sphinx/source/api/esmvaltool.rst b/doc/sphinx/source/api/esmvaltool.rst index b770dc17eb..b080b81ac8 100644 --- a/doc/sphinx/source/api/esmvaltool.rst +++ b/doc/sphinx/source/api/esmvaltool.rst @@ -28,3 +28,4 @@ Diagnostic Scripts esmvaltool.diag_scripts.monitor esmvaltool.diag_scripts.ocean esmvaltool.diag_scripts.psyplot_diag + esmvaltool.diag_scripts.seaborn_diag diff --git a/doc/sphinx/source/faq.rst b/doc/sphinx/source/faq.rst index f38ed8e0f7..15d69192ca 100644 --- a/doc/sphinx/source/faq.rst +++ b/doc/sphinx/source/faq.rst @@ -121,3 +121,10 @@ Moreover, recipe :ref:`recipes_psyplot_diag` and the corresponding diagnostic :ref:`psyplot_diag.py ` provide a high-level interface to the `Psyplot `__ package which can be used to create a large variety of different plots. + +Similarly, recipe :ref:`recipes_seaborn_diag` and the corresponding diagnostic +:ref:`seaborn_diag.py ` provide a +high-level interface to the `Seaborn `__ package +which can also be used to create a large variety of different plots. + +See also :ref:`general_purpose_diags`. diff --git a/doc/sphinx/source/recipes/figures/seaborn/regional_pr_hists.jpg b/doc/sphinx/source/recipes/figures/seaborn/regional_pr_hists.jpg new file mode 100644 index 0000000000..da57977859 Binary files /dev/null and b/doc/sphinx/source/recipes/figures/seaborn/regional_pr_hists.jpg differ diff --git a/doc/sphinx/source/recipes/figures/seaborn/ta_vs_lat.jpg b/doc/sphinx/source/recipes/figures/seaborn/ta_vs_lat.jpg new file mode 100644 index 0000000000..c4929c33ca Binary files /dev/null and b/doc/sphinx/source/recipes/figures/seaborn/ta_vs_lat.jpg differ diff --git a/doc/sphinx/source/recipes/index.rst b/doc/sphinx/source/recipes/index.rst index a579032c46..779554c0fa 100644 --- a/doc/sphinx/source/recipes/index.rst +++ b/doc/sphinx/source/recipes/index.rst @@ -10,6 +10,21 @@ ESMValTool for all available recipes can be accessed `here .. toctree:: :maxdepth: 1 +.. _general_purpose_diags: + +General-purpose diagnostics +^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Recipes that use highly customizable diagnostics which are designed to plot a +large variety of input data. + +.. toctree:: + :maxdepth: 1 + + recipe_monitor + recipe_psyplot + recipe_seaborn + Atmosphere ^^^^^^^^^^ .. toctree:: @@ -121,9 +136,7 @@ Other recipe_ensclus recipe_esacci_lst recipe_examples - recipe_monitor recipe_multimodel_products - recipe_psyplot recipe_pv_capacity_factor recipe_rainfarm recipe_seaice diff --git a/doc/sphinx/source/recipes/recipe_bock20jgr.rst b/doc/sphinx/source/recipes/recipe_bock20jgr.rst index 7f0ebdabec..cb311249f9 100644 --- a/doc/sphinx/source/recipes/recipe_bock20jgr.rst +++ b/doc/sphinx/source/recipes/recipe_bock20jgr.rst @@ -82,7 +82,7 @@ User settings in recipe * time_avg: type of time average (currently only "yearly" and "monthly" are available). * ts_anomaly: calculates anomalies with respect to the defined reference - period; for each gird point by removing the mean for the given + period; for each grid point by removing the mean for the given calendar month (requiring at least 50% of the data to be non-missing) * ref_start: start year of reference period for anomalies @@ -151,7 +151,7 @@ User settings in recipe * Required settings (variables)* - * reference_dataset: name of reference datatset + * reference_dataset: name of reference dataset *Optional settings (variables)* @@ -229,8 +229,8 @@ User settings in recipe large multi-dimensional datasets, this might significantly reduce the computation time if only the multi-model mean dataset is relevant. * output_attributes: *dict*. Write additional attributes to netcdf files. - * seaborn_settings: *dict*. Options for :func:`seaborn.set` (affects all - plots). + * seaborn_settings: *dict*. Options for :func:`seaborn.set_theme` (affects + all plots). Variables @@ -375,14 +375,14 @@ Example plots :align: center :width: 9cm - Relative space-time root-mean-square deviation (RMSD) calculated from the - climatological seasonal cycle of the CMIP3, CMIP5, and CMIP6 simulations - (1980-1999) compared to observational data sets (Table 5). A relative - performance is displayed, with blue shading being better and red shading - worse than the median RMSD of all model results of all ensembles. A diagonal - split of a grid square shows the relative error with respect to the reference - data set (lower right triangle) and the alternative data set (upper left - triangle) which are marked in Table 5. White boxes are used when data are not + Relative space-time root-mean-square deviation (RMSD) calculated from the + climatological seasonal cycle of the CMIP3, CMIP5, and CMIP6 simulations + (1980-1999) compared to observational data sets (Table 5). A relative + performance is displayed, with blue shading being better and red shading + worse than the median RMSD of all model results of all ensembles. A diagonal + split of a grid square shows the relative error with respect to the reference + data set (lower right triangle) and the alternative data set (upper left + triangle) which are marked in Table 5. White boxes are used when data are not available for a given model and variable (Fig. 6). .. _fig_bock20jgr_5: @@ -390,5 +390,5 @@ Example plots :align: center :width: 9cm - Centered pattern correlations between models and observations for the annual - mean climatology over the period 1980–1999 (Fig. 7). + Centered pattern correlations between models and observations for the annual + mean climatology over the period 1980–1999 (Fig. 7). diff --git a/doc/sphinx/source/recipes/recipe_ecs.rst b/doc/sphinx/source/recipes/recipe_ecs.rst index b5dea18f21..d426e45b7c 100644 --- a/doc/sphinx/source/recipes/recipe_ecs.rst +++ b/doc/sphinx/source/recipes/recipe_ecs.rst @@ -54,8 +54,8 @@ User settings in recipe script or as absolute path. * ``savefig_kwargs``, *dict*, optional: Keyword arguments for :func:`matplotlib.pyplot.savefig`. - * ``seaborn_settings``, *dict*, optional: Options for :func:`seaborn.set` - (affects all plots). + * ``seaborn_settings``, *dict*, optional: Options for + :func:`seaborn.set_theme` (affects all plots). * ``sep_year``, *int*, optional (default: ``20``): Year to separate regressions of complex Gregory plot. Only effective if ``complex_gregory_plot`` is ``True``. @@ -79,8 +79,8 @@ User settings in recipe data. * ``savefig_kwargs``, *dict*, optional: Keyword arguments for :func:`matplotlib.pyplot.savefig`. - * ``seaborn_settings``, *dict*, optional: Options for :func:`seaborn.set` - (affects all plots). + * ``seaborn_settings``, *dict*, optional: Options for + :func:`seaborn.set_theme` (affects all plots). * ``sort_ascending``, *bool*, optional (default: ``False``): Sort bars in ascending order. * ``sort_descending``, *bool*, optional (default: ``False``): Sort bars in @@ -98,8 +98,8 @@ User settings in recipe * ``dataset_style``, *str*, optional: Name of the style file (located in :mod:`esmvaltool.diag_scripts.shared.plot.styles_python`). * ``pattern``, *str*, optional: Pattern to filter list of input files. - * ``seaborn_settings``, *dict*, optional: Options for :func:`seaborn.set` - (affects all plots). + * ``seaborn_settings``, *dict*, optional: Options for + :func:`seaborn.set_theme` (affects all plots). * ``y_range``, *list of float*, optional: Range for the Y axis of the plot. diff --git a/doc/sphinx/source/recipes/recipe_ipccwg1ar5ch9.rst b/doc/sphinx/source/recipes/recipe_ipccwg1ar5ch9.rst index a52d682a78..fc1c26464c 100644 --- a/doc/sphinx/source/recipes/recipe_ipccwg1ar5ch9.rst +++ b/doc/sphinx/source/recipes/recipe_ipccwg1ar5ch9.rst @@ -14,8 +14,8 @@ rather than constantly having to "re-invent the wheel". .. note:: - Please note that most recipes have been modified to include only models that are - (still) readily available via ESGF. Plots produced may therefore look different + Please note that most recipes have been modified to include only models that are + (still) readily available via ESGF. Plots produced may therefore look different than the original figures from IPCC AR5. The plots are produced collecting the diagnostics from individual recipes. The @@ -95,16 +95,16 @@ following figures from Flato et al. (2013) can currently be reproduced: calculated as the standard deviation of the annual means over the period 1986–2005. - * Figure 9.38: Seasonal cycle for the surface temperature or precipitation - over land within defined regions multi-model mean and difference to + * Figure 9.38: Seasonal cycle for the surface temperature or precipitation + over land within defined regions multi-model mean and difference to reference dataset or absolute annual cycle can be chosen. - * Figure 9.39: Seasonal bias box and whiskers plot + * Figure 9.39: Seasonal bias box and whiskers plot for surface temperature or precipitation within SREX (IPCC Special Report on Managing the Risks of Extreme Events and Disasters to Advance Climate Change Adaptation) regions. - * Figure 9.40: Seasonal bias box and whiskers plot for surface + * Figure 9.40: Seasonal bias box and whiskers plot for surface temperature or precipitation within defined polar and ocean regions. * Figure 9.41b: Comparison between observations and models for variable @@ -246,7 +246,7 @@ User settings in recipe * time_avg: type of time average (currently only "yearly" and "monthly" are available). * ts_anomaly: calculates anomalies with respect to the defined period; for - each gird point by removing the mean for the given calendar month + each grid point by removing the mean for the given calendar month (requiring at least 50% of the data to be non-missing) * ref_start: start year of reference period for anomalies * ref_end: end year of reference period for anomalies @@ -360,7 +360,7 @@ User settings in recipe * fig938_mip_MMM: mip to average, default "Amon" * fig938_names_MMM: names in legend i.e. (["CMIP5","CMIP3"]), default fig938_project_MMM * fig938_colors_MMM: Color for multi-model mean (e.g. ["red"]), default "red" - * If set fig938_mip_MMM, fig938_experiment_MMM, fig938_project_MMM, fig938_names_MMM, and fig938_colors_MMM must + * If set fig938_mip_MMM, fig938_experiment_MMM, fig938_project_MMM, fig938_names_MMM, and fig938_colors_MMM must have the same number of elements * fig938_refModel: Reference data set for differences default "ERA-Interim" @@ -453,7 +453,8 @@ User settings in recipe :mod:`esmvaltool.diag_scripts.shared.plot.styles_python.matplotlib`). * save: :obj:`dict` containing keyword arguments for the function :func:`matplotlib.pyplot.savefig`. - * seaborn_settings: Options for :func:`seaborn.set` (affects all plots). + * seaborn_settings: Options for :func:`seaborn.set_theme` (affects all + plots). .. _ch09_fig09_42b.py: @@ -476,7 +477,8 @@ User settings in recipe ``marker_column``). If a relative path is given, assumes that this is a pattern to search for ancestor files. * savefig_kwargs: Keyword arguments for :func:`matplotlib.pyplot.savefig`. - * seaborn_settings: Options for :func:`seaborn.set` (affects all plots). + * seaborn_settings: Options for :func:`seaborn.set_theme` (affects all + plots). * x_lim: Plot limits for X axis (ECS). * y_lim: Plot limits for Y axis (TCR). @@ -573,12 +575,12 @@ References Tignor, M., and Midgley, P. M., Cambridge University Press, Cambridge, UK, and New York, NY, USA, 109-230. -* Weigel, K., Bock, L., Gier, B. K., Lauer, A., Righi, M., Schlund, M., Adeniyi, K., - Andela, B., Arnone, E., Berg, P., Caron, L.-P., Cionni, I., Corti, S., Drost, N., - Hunter, A., Lledó, L., Mohr, C. W., Paçal, A., Pérez-Zanón, N., Predoi, V., Sandstad, - M., Sillmann, J., Sterl, A., Vegas-Regidor, J., von Hardenberg, J., and Eyring, V.: - Earth System Model Evaluation Tool (ESMValTool) v2.0 - diagnostics for extreme events, - regional and impact evaluation, and analysis of Earth system models in CMIP, +* Weigel, K., Bock, L., Gier, B. K., Lauer, A., Righi, M., Schlund, M., Adeniyi, K., + Andela, B., Arnone, E., Berg, P., Caron, L.-P., Cionni, I., Corti, S., Drost, N., + Hunter, A., Lledó, L., Mohr, C. W., Paçal, A., Pérez-Zanón, N., Predoi, V., Sandstad, + M., Sillmann, J., Sterl, A., Vegas-Regidor, J., von Hardenberg, J., and Eyring, V.: + Earth System Model Evaluation Tool (ESMValTool) v2.0 - diagnostics for extreme events, + regional and impact evaluation, and analysis of Earth system models in CMIP, Geosci. Model Dev., 14, 3159-3184, https://doi.org/10.5194/gmd-14-3159-2021, 2021. @@ -701,7 +703,7 @@ Example plots :align: center Figure 9.38tas: Mean seasonal cycle for surface temperature (tas) - as multi model mean of 38 CMIP5 and 22 CMIP6 models as well as + as multi model mean of 38 CMIP5 and 22 CMIP6 models as well as CRU and ERA-Interim reanalysis data averaged for 1980-2005 over land in different regions: Western North America (WNA), Eastern North America (ENA), @@ -710,7 +712,7 @@ Example plots North Africa (NAF), Central Africa (CAF), South Africa (SAF), North Asia (NAS), Central Asia (CAS), East Asia (EAS), South Asia (SAS), Southeast Asia (SEA), and Australia (AUS). - Similar to Fig. 9.38a from Flato et al. (2013), CMIP6 instead of CMIP3 and + Similar to Fig. 9.38a from Flato et al. (2013), CMIP6 instead of CMIP3 and set of CMIP5 models used different. @@ -718,7 +720,7 @@ Example plots :align: center Figure 9.38pr: Mean seasonal cycle for precipitation (pr) - as multi model mean of 38 CMIP5 and 22 CMIP6 models as well as + as multi model mean of 38 CMIP5 and 22 CMIP6 models as well as CRU and ERA-Interim reanalysis data averaged for 1980-1999 over land in different regions: Western North America (WNA), Eastern North America (ENA), @@ -727,7 +729,7 @@ Example plots North Africa (NAF), Central Africa (CAF), South Africa (SAF), North Asia (NAS), Central Asia (CAS), East Asia (EAS), South Asia (SAS), Southeast Asia (SEA), and Australia (AUS). - Similar to Fig. 9.38b from Flato et al. (2013), CMIP6 instead of CMIP3 and + Similar to Fig. 9.38b from Flato et al. (2013), CMIP6 instead of CMIP3 and set of CMIP5 models used different. .. figure:: /recipes/figures/ipccwg1ar5ch9/fig-9-38_regions.png @@ -740,7 +742,7 @@ Example plots Figure 9.39tas: Box and whisker plots showing the 5th, 25th, 50th, 75th and 95th percentiles of the seasonal- and annual mean biases for - surface temperature (tas) for 1980-2005 between 38 CMIP5 models + surface temperature (tas) for 1980-2005 between 38 CMIP5 models (box and whiskers) or 22 CMIP6 models (crosses) and CRU data. The regions are: Alaska/NW Canada (ALAs), Eastern Canada/Greenland/Iceland (CGIs), Western North America(WNAs), @@ -756,7 +758,7 @@ Example plots Southern Australia/New Zealand (SAUs). The positions of these regions are defined following (Seneviratne et al., 2012) and differ from the ones in Fig. 9.38. - Similar to Fig. 9.39 a,c,e from Flato et al. (2013), CMIP6 instead of CMIP3 and + Similar to Fig. 9.39 a,c,e from Flato et al. (2013), CMIP6 instead of CMIP3 and set of CMIP5 models used different. .. figure:: /recipes/figures/ipccwg1ar5ch9/fig-9-39-pr.png @@ -764,7 +766,7 @@ Example plots Figure 9.39pr: Box and whisker plots showing the 5th, 25th, 50th, 75th and 95th percentiles of the seasonal- and annual mean biases for - precipitation (pr) for 1980-2005 between 38 CMIP5 models + precipitation (pr) for 1980-2005 between 38 CMIP5 models (box and whiskers) or 22 CMIP6 models (crosses) and CRU data. The regions are: Alaska/NW Canada (ALAs), Eastern Canada/Greenland/Iceland (CGIs), Western North America(WNAs), @@ -780,7 +782,7 @@ Example plots Southern Australia/New Zealand (SAUs). The positions of these regions are defined following (Seneviratne et al., 2012) and differ from the ones in Fig. 9.38. - Similar to Fig. 9.39 b,d,f from Flato et al. (2013), CMIP6 instead of CMIP3 and + Similar to Fig. 9.39 b,d,f from Flato et al. (2013), CMIP6 instead of CMIP3 and set of CMIP5 models used different. .. figure:: /recipes/figures/ipccwg1ar5ch9/fig-9-39_regions.png diff --git a/doc/sphinx/source/recipes/recipe_seaborn.rst b/doc/sphinx/source/recipes/recipe_seaborn.rst new file mode 100644 index 0000000000..3c8fa64357 --- /dev/null +++ b/doc/sphinx/source/recipes/recipe_seaborn.rst @@ -0,0 +1,66 @@ +.. _recipes_seaborn_diag: + +Seaborn Diagnostics +=================== + +Overview +-------- + +These recipes showcase the use of the Seaborn diagnostic that provides a +high-level interface to `Seaborn `__ for ESMValTool +recipes. + + +Available recipes and diagnostics +--------------------------------- + +Recipes are stored in recipes/ + + * recipe_seaborn.yml + +Diagnostics are stored in diag_scripts/ + + * :ref:`seaborn_diag.py ` + + +Variables +--------- + +Arbitrary variables are supported. + + +Observations and reformat scripts +--------------------------------- + +Arbitrary datasets are supported. + + +References +---------- + +* Waskom, M. L. (2021), seaborn: statistical data visualization, Journal of + Open Source Software, 6(60), 3021, doi:10.21105/joss.03021. + + +Example plots +------------- + +.. _fig_seaborn_1: +.. figure:: /recipes/figures/seaborn/ta_vs_lat.jpg + :align: center + :width: 50% + + Monthly and zonal mean temperatures vs. latitude in the period 1991-2014 for + two Earth system models (CESM2-WACCM and GFDL-ESM4). + Colors visualize the corresponding pressure levels. + +.. _fig_seaborn_2: +.. figure:: /recipes/figures/seaborn/regional_pr_hists.jpg + :align: center + :width: 50% + + Spatiotemporal distribution of daily precipitation in the period 2005-2014 + for six IPCC AR6 regions simulated by two Earth system models (CESM2-WACCM + and GFDL-ESM4). + Each day in each grid cell in the corresponding regions is considered with + equal weight. diff --git a/doc/sphinx/source/recipes/recipe_tcr.rst b/doc/sphinx/source/recipes/recipe_tcr.rst index ec8c95edf0..58ea9e639b 100644 --- a/doc/sphinx/source/recipes/recipe_tcr.rst +++ b/doc/sphinx/source/recipes/recipe_tcr.rst @@ -51,8 +51,8 @@ User settings in recipe path can be given relative to this diagnostic script or as absolute path. * ``savefig_kwargs``, *dict*, optional: Keyword arguments for :func:`matplotlib.pyplot.savefig`. - * ``seaborn_settings``, *dict*, optional: Options for :func:`seaborn.set` - (affects all plots). + * ``seaborn_settings``, *dict*, optional: Options for + :func:`seaborn.set_theme` (affects all plots). * Script climate_metrics/create_barplot.py diff --git a/esmvaltool/diag_scripts/climate_metrics/create_barplot.py b/esmvaltool/diag_scripts/climate_metrics/create_barplot.py index e859780301..18759449e0 100644 --- a/esmvaltool/diag_scripts/climate_metrics/create_barplot.py +++ b/esmvaltool/diag_scripts/climate_metrics/create_barplot.py @@ -29,7 +29,7 @@ savefig_kwargs : dict, optional Keyword arguments for :func:`matplotlib.pyplot.savefig`. seaborn_settings : dict, optional - Options for :func:`seaborn.set` (affects all plots). + Options for :func:`seaborn.set_theme` (affects all plots). sort_ascending : bool, optional (default: False) Sort bars in ascending order. sort_descending : bool, optional (default: False) @@ -244,7 +244,7 @@ def main(cfg): 'orientation': 'landscape', 'bbox_inches': 'tight', }) - sns.set(**cfg.get('seaborn_settings', {})) + sns.set_theme(**cfg.get('seaborn_settings', {})) patterns = cfg.get('patterns') if patterns is None: input_files = io.get_all_ancestor_files(cfg) diff --git a/esmvaltool/diag_scripts/climate_metrics/create_scatterplot.py b/esmvaltool/diag_scripts/climate_metrics/create_scatterplot.py index 9f81046de7..71c9ed771c 100644 --- a/esmvaltool/diag_scripts/climate_metrics/create_scatterplot.py +++ b/esmvaltool/diag_scripts/climate_metrics/create_scatterplot.py @@ -23,7 +23,7 @@ pattern : str, optional Pattern to filter list of input data. seaborn_settings : dict, optional - Options for :func:`seaborn.set` (affects all plots). + Options for :func:`seaborn.set_theme` (affects all plots). y_range : list of float, optional Range for the y axis in the plot. @@ -118,7 +118,7 @@ def write_data(cfg, cube): def main(cfg): """Run the diagnostic.""" - sns.set(**cfg.get('seaborn_settings', {})) + sns.set_theme(**cfg.get('seaborn_settings', {})) input_files = io.get_all_ancestor_files(cfg, pattern=cfg.get('pattern')) if len(input_files) != 1: raise ValueError(f"Expected exactly 1 file, got {len(input_files)}") diff --git a/esmvaltool/diag_scripts/climate_metrics/ecs.py b/esmvaltool/diag_scripts/climate_metrics/ecs.py index e3a23065f5..ee015be568 100644 --- a/esmvaltool/diag_scripts/climate_metrics/ecs.py +++ b/esmvaltool/diag_scripts/climate_metrics/ecs.py @@ -34,7 +34,7 @@ savefig_kwargs : dict, optional Keyword arguments for :func:`matplotlib.pyplot.savefig`. seaborn_settings : dict, optional - Options for :func:`seaborn.set` (affects all plots). + Options for :func:`seaborn.set_theme` (affects all plots). sep_year : int, optional (default: 20) Year to separate regressions of complex Gregory plot. Only effective if ``complex_gregory_plot`` is ``True``. @@ -502,7 +502,7 @@ def write_data(cfg, ecs_data, feedback_parameter_data, ancestor_files): def main(cfg): """Run the diagnostic.""" cfg = set_default_cfg(cfg) - sns.set(**cfg.get('seaborn_settings', {})) + sns.set_theme(**cfg.get('seaborn_settings', {})) # Read external file if desired if cfg.get('read_external_file'): diff --git a/esmvaltool/diag_scripts/climate_metrics/feedback_parameters.py b/esmvaltool/diag_scripts/climate_metrics/feedback_parameters.py index f984fd04a1..db350982a2 100644 --- a/esmvaltool/diag_scripts/climate_metrics/feedback_parameters.py +++ b/esmvaltool/diag_scripts/climate_metrics/feedback_parameters.py @@ -27,7 +27,7 @@ output_attributes : dict, optional Write additional attributes to netcdf files. seaborn_settings : dict, optional - Options for :func:`seaborn.set` (affects all plots). + Options for :func:`seaborn.set_theme` (affects all plots). """ @@ -1016,7 +1016,7 @@ def set_default_cfg(cfg): def main(cfg): """Run the diagnostic.""" cfg = set_default_cfg(cfg) - sns.set(cfg['seaborn_settings']) + sns.set_theme(cfg['seaborn_settings']) check_input_data(cfg) year_indices = { 'all 150 years': slice(None), diff --git a/esmvaltool/diag_scripts/climate_metrics/tcr.py b/esmvaltool/diag_scripts/climate_metrics/tcr.py index c500133e53..c227a6fa6e 100644 --- a/esmvaltool/diag_scripts/climate_metrics/tcr.py +++ b/esmvaltool/diag_scripts/climate_metrics/tcr.py @@ -26,7 +26,7 @@ savefig_kwargs : dict, optional Keyword arguments for :func:`matplotlib.pyplot.savefig`. seaborn_settings : dict, optional - Options for :func:`seaborn.set` (affects all plots). + Options for :func:`seaborn.set_theme` (affects all plots). """ @@ -342,7 +342,7 @@ def write_data(cfg, tcr, external_file=None): def main(cfg): """Run the diagnostic.""" cfg = set_default_cfg(cfg) - sns.set(**cfg.get('seaborn_settings', {})) + sns.set_theme(**cfg.get('seaborn_settings', {})) # Read external file if desired if cfg.get('read_external_file'): diff --git a/esmvaltool/diag_scripts/emergent_constraints/ecs_scatter.py b/esmvaltool/diag_scripts/emergent_constraints/ecs_scatter.py index ca5c77312c..e0976306bc 100644 --- a/esmvaltool/diag_scripts/emergent_constraints/ecs_scatter.py +++ b/esmvaltool/diag_scripts/emergent_constraints/ecs_scatter.py @@ -33,7 +33,7 @@ savefig_kwargs: dict Keyword arguments for :func:`matplotlib.pyplot.savefig`. seaborn_settings: dict - Options for :func:`seaborn.set` (affects all plots). + Options for :func:`seaborn.set_theme` (affects all plots). """ @@ -792,7 +792,7 @@ def main(cfg): """Run the diagnostic.""" cfg = get_default_settings(cfg) diag = check_cfg(cfg) - sns.set(**cfg.get('seaborn_settings', {})) + sns.set_theme(**cfg.get('seaborn_settings', {})) # Get input data input_data = list(cfg['input_data'].values()) diff --git a/esmvaltool/diag_scripts/emergent_constraints/multiple_constraints.py b/esmvaltool/diag_scripts/emergent_constraints/multiple_constraints.py index 74e2e4c1d9..bdf621a697 100644 --- a/esmvaltool/diag_scripts/emergent_constraints/multiple_constraints.py +++ b/esmvaltool/diag_scripts/emergent_constraints/multiple_constraints.py @@ -57,7 +57,7 @@ savefig_kwargs: dict Keyword arguments for :func:`matplotlib.pyplot.savefig`. seaborn_settings: dict - Options for :func:`seaborn.set` (affects all plots). + Options for :func:`seaborn.set_theme` (affects all plots). """ @@ -94,7 +94,7 @@ def get_default_settings(cfg): def main(cfg): """Run the diagnostic.""" cfg = get_default_settings(cfg) - sns.set(**cfg['seaborn_settings']) + sns.set_theme(**cfg['seaborn_settings']) # Load data and perform PCA (training_data, prediction_data, attributes) = ec.get_input_data(cfg) diff --git a/esmvaltool/diag_scripts/emergent_constraints/single_constraint.py b/esmvaltool/diag_scripts/emergent_constraints/single_constraint.py index 17815c058f..9a95adb321 100644 --- a/esmvaltool/diag_scripts/emergent_constraints/single_constraint.py +++ b/esmvaltool/diag_scripts/emergent_constraints/single_constraint.py @@ -56,7 +56,7 @@ savefig_kwargs: dict Keyword arguments for :func:`matplotlib.pyplot.savefig`. seaborn_settings: dict - Options for :func:`seaborn.set` (affects all plots). + Options for :func:`seaborn.set_theme` (affects all plots). """ @@ -101,7 +101,7 @@ def get_default_settings(cfg): def main(cfg): """Run the diagnostic.""" cfg = get_default_settings(cfg) - sns.set(**cfg['seaborn_settings']) + sns.set_theme(**cfg['seaborn_settings']) # Load data (training_data, prediction_data, attributes) = ec.get_input_data(cfg) diff --git a/esmvaltool/diag_scripts/ipcc_ar5/ch09_fig09_42a.py b/esmvaltool/diag_scripts/ipcc_ar5/ch09_fig09_42a.py index 7f498adf5f..d0e23f4008 100644 --- a/esmvaltool/diag_scripts/ipcc_ar5/ch09_fig09_42a.py +++ b/esmvaltool/diag_scripts/ipcc_ar5/ch09_fig09_42a.py @@ -28,7 +28,7 @@ save : dict, optional Keyword arguments for :func:`matplotlib.pyplot.savefig`. seaborn_settings : dict, optional - Options for :func:`seaborn.set` (affects all plots). + Options for :func:`seaborn.set_theme` (affects all plots). """ @@ -164,7 +164,7 @@ def write_data(cfg, hist_cubes, pi_cubes, ecs_cube): def main(cfg): """Run the diagnostic.""" - sns.set(**cfg.get('seaborn_settings', {})) + sns.set_theme(**cfg.get('seaborn_settings', {})) input_data = deepcopy(list(cfg['input_data'].values())) input_data = sorted_metadata(input_data, ['short_name', 'exp', 'dataset']) project = list(group_metadata(input_data, 'project').keys()) diff --git a/esmvaltool/diag_scripts/ipcc_ar5/ch09_fig09_42b.py b/esmvaltool/diag_scripts/ipcc_ar5/ch09_fig09_42b.py index 019fddc587..ba49c8afa5 100644 --- a/esmvaltool/diag_scripts/ipcc_ar5/ch09_fig09_42b.py +++ b/esmvaltool/diag_scripts/ipcc_ar5/ch09_fig09_42b.py @@ -35,7 +35,7 @@ savefig_kwargs : dict, optional Keyword arguments for :func:`matplotlib.pyplot.savefig`. seaborn_settings : dict, optional - Options for :func:`seaborn.set` (affects all plots). + Options for :func:`seaborn.set_theme` (affects all plots). x_lim : list of float, optional (default: [1.5, 6.0]) Plot limits for X axis (ECS). y_lim : list of float, optional (default: [0.5, 3.5]) @@ -263,7 +263,7 @@ def write_data(cfg, ecs_cube, tcr_cube): def main(cfg): """Run the diagnostic.""" cfg = set_default_cfg(cfg) - sns.set(**cfg.get('seaborn_settings', {})) + sns.set_theme(**cfg.get('seaborn_settings', {})) ecs_file = io.get_ancestor_file(cfg, 'ecs.nc') tcr_file = io.get_ancestor_file(cfg, 'tcr.nc') ecs_cube = iris.load_cube(ecs_file) diff --git a/esmvaltool/diag_scripts/mlr/evaluate_residuals.py b/esmvaltool/diag_scripts/mlr/evaluate_residuals.py index 68b05b9c2a..e76f7c9eab 100644 --- a/esmvaltool/diag_scripts/mlr/evaluate_residuals.py +++ b/esmvaltool/diag_scripts/mlr/evaluate_residuals.py @@ -33,7 +33,7 @@ savefig_kwargs: dict, optional Keyword arguments for :func:`matplotlib.pyplot.savefig`. seaborn_settings: dict, optional - Options for :func:`seaborn.set` (affects all plots). + Options for :func:`seaborn.set_theme` (affects all plots). weighted_samples: dict If specified, use weighted root mean square error. The given keyword arguments are directly passed to @@ -188,7 +188,7 @@ def main(cfg): cfg = deepcopy(cfg) cfg.setdefault('weighted_samples', {'area_weighted': True, 'time_weighted': True}) - sns.set(**cfg.get('seaborn_settings', {})) + sns.set_theme(**cfg.get('seaborn_settings', {})) # Extract data residual_data = get_residual_data(cfg) diff --git a/esmvaltool/diag_scripts/mlr/models/__init__.py b/esmvaltool/diag_scripts/mlr/models/__init__.py index 4f60464289..71e44201d7 100644 --- a/esmvaltool/diag_scripts/mlr/models/__init__.py +++ b/esmvaltool/diag_scripts/mlr/models/__init__.py @@ -271,7 +271,7 @@ savefig_kwargs: dict Keyword arguments for :func:`matplotlib.pyplot.savefig`. seaborn_settings: dict - Options for :func:`seaborn.set` (affects all plots). + Options for :func:`seaborn.set_theme` (affects all plots). standardize_data: bool (default: True) Linearly standardize numerical input data by removing mean and scaling to unit variance. @@ -443,7 +443,7 @@ def __init__(self, input_datasets, **kwargs): self._random_state = np.random.RandomState(self._cfg['random_state']) # Seaborn - sns.set(**self._cfg.get('seaborn_settings', {})) + sns.set_theme(**self._cfg.get('seaborn_settings', {})) # Adapt output directories self._cfg['mlr_work_dir'] = os.path.join(self._cfg['work_dir'], diff --git a/esmvaltool/diag_scripts/mlr/plot.py b/esmvaltool/diag_scripts/mlr/plot.py index a4e944f473..e5155a9a9d 100644 --- a/esmvaltool/diag_scripts/mlr/plot.py +++ b/esmvaltool/diag_scripts/mlr/plot.py @@ -86,7 +86,7 @@ savefig_kwargs: dict, optional Keyword arguments for :func:`matplotlib.pyplot.savefig`. seaborn_settings: dict, optional - Options for :func:`seaborn.set` (affects all plots). + Options for :func:`seaborn.set_theme` (affects all plots). years_in_title: bool, optional (default: False) Print years in default title of plots. @@ -808,7 +808,7 @@ def plot_xy_with_errors(cfg, cube_dict): def main(cfg): """Run the diagnostic.""" - sns.set(**cfg.get('seaborn_settings', {})) + sns.set_theme(**cfg.get('seaborn_settings', {})) cfg = deepcopy(cfg) cfg.setdefault('group_by_attribute', 'mlr_model_name') cfg.setdefault('group_attribute_as_default_alias', True) diff --git a/esmvaltool/diag_scripts/mlr/rescale_with_emergent_constraint.py b/esmvaltool/diag_scripts/mlr/rescale_with_emergent_constraint.py index efca79f2dd..32610744d7 100644 --- a/esmvaltool/diag_scripts/mlr/rescale_with_emergent_constraint.py +++ b/esmvaltool/diag_scripts/mlr/rescale_with_emergent_constraint.py @@ -44,7 +44,7 @@ savefig_kwargs: dict, optional Keyword arguments for :func:`matplotlib.pyplot.savefig`. seaborn_settings: dict, optional - Options for :func:`seaborn.set` (affects all plots). + Options for :func:`seaborn.set_theme` (affects all plots). """ @@ -488,7 +488,7 @@ def rescale_labels(cfg, y_data, y_mean, y_std): def main(cfg): """Run the diagnostic.""" - sns.set(**cfg.get('seaborn_settings', {})) + sns.set_theme(**cfg.get('seaborn_settings', {})) cfg = deepcopy(cfg) cfg.setdefault('group_by_attributes', ['dataset']) cfg.setdefault('legend_kwargs', {}) diff --git a/esmvaltool/diag_scripts/monitor/multi_datasets.py b/esmvaltool/diag_scripts/monitor/multi_datasets.py index 35af284368..c57d09b9e6 100644 --- a/esmvaltool/diag_scripts/monitor/multi_datasets.py +++ b/esmvaltool/diag_scripts/monitor/multi_datasets.py @@ -83,7 +83,7 @@ Optional keyword arguments for :func:`matplotlib.pyplot.savefig`. By default, uses ``bbox_inches: tight, dpi: 300, orientation: landscape``. seaborn_settings: dict, optional - Options for :func:`seaborn.set` (affects all plots). By default, uses + Options for :func:`seaborn.set_theme` (affects all plots). By default, uses ``style: ticks``. Configuration options for plot type ``timeseries`` @@ -539,7 +539,7 @@ def __init__(self, config): f"the following dataset:\n{pformat(dataset)}") # Load seaborn settings - sns.set(**self.cfg['seaborn_settings']) + sns.set_theme(**self.cfg['seaborn_settings']) def _add_colorbar(self, plot_type, plot_left, plot_right, axes_left, axes_right, dataset_left, dataset_right): diff --git a/esmvaltool/diag_scripts/psyplot_diag.py b/esmvaltool/diag_scripts/psyplot_diag.py index 49787844d4..20016e21ef 100644 --- a/esmvaltool/diag_scripts/psyplot_diag.py +++ b/esmvaltool/diag_scripts/psyplot_diag.py @@ -33,7 +33,7 @@ Optional keyword arguments for :func:`matplotlib.pyplot.savefig`. By default, uses ``bbox_inches: tight, dpi: 300, orientation: landscape``. seaborn_settings: dict, optional - Options for :func:`seaborn.set` (affects all plots). + Options for :func:`seaborn.set_theme` (affects all plots). """ import logging @@ -104,7 +104,7 @@ def _get_psyplot_kwargs(cfg, dataset): def main(cfg): """Run diagnostic.""" cfg = _get_default_cfg(cfg) - sns.set(**cfg['seaborn_settings']) + sns.set_theme(**cfg['seaborn_settings']) plot_func = _get_plot_func(cfg) # Create individual plots for each dataset diff --git a/esmvaltool/diag_scripts/seaborn_diag.py b/esmvaltool/diag_scripts/seaborn_diag.py new file mode 100644 index 0000000000..aa6ca1b504 --- /dev/null +++ b/esmvaltool/diag_scripts/seaborn_diag.py @@ -0,0 +1,502 @@ +#!/usr/bin/env python +# -*- coding: utf-8 -*- +"""Create arbitrary Seaborn plots. + +Description +----------- +This diagnostic provides a high-level interface to Seaborn. For this, the input +data is arranged into a single :class:`pandas.DataFrame`, which is then used as +input for the Seaborn function defined by the option `seaborn_func`. + +Caveats +------- +All datasets of a given variable must have the same units (e.g., it is not +allowed to use datasets with units `K` and datasets with units `°C` for the +variable `tas`). + +Author +------ +Manuel Schlund (DLR, Germany) + +Configuration options in recipe +------------------------------- +add_ancillary_variables: bool, optional (default: False) + Add :meth:`~iris.cube.Cube.ancillary_variables` to the main data frame. + Note that this will assume that ancillary variables are identical across + cubes within a group (see option `groupby_facet`). This equality is not + checked! +add_aux_coords: bool, optional (default: False) + Add :attr:`~iris.cube.Cube.aux_coords` to the main data frame. Note that + this will assume that auxiliary coordinates are identical across cubes + within a group (see option `groupby_facet`). This equality is not checked! +add_cell_measures: bool, optional (default: False) + Add :meth:`~iris.cube.Cube.cell_measures` to the main data frame. Note that + this will assume that cell measures are identical across cubes within a + group (see option `groupby_facet`). This equality is not checked! +data_frame_ops: dict, optional + Perform additional operations on the main data frame. Allowed operations + are :meth:`pandas.DataFrame.query` (dict key `query`) and + :meth:`pandas.DataFrame.eval` (dict key `eval`). Operations are defined by + strings (dict values). Examples: ``{'query': 'latitude > 80', 'eval': + 'longitude = longitude - 180.0'}``. +dropna_kwargs: dict, optional + Optional keyword arguments for :meth:`pandas.DataFrame.dropna` to drop + missing values in the input data. If not given, do not drop NaNs. Note: + NaNs are dropped after potential `data_frame_ops`. +facets_as_columns: list of str, optional + Facets that will be added as a columns to the main data frame. Values for + these facets must be identical across all datasets within a group (see + option `groupby_facet`). +groupby_facet: str, optional (default: 'alias') + Facet which is used to group input datasets when creating the main data + frame. All datasets within a group are expected to have the same index + after calling :func:`iris.pandas.as_data_frame` on them. These datasets + within a group will then get merged (combined along axis 1, i.e., columns) + into a single data frame per group. Finally, the data frames for all groups + are concatenated (combined along axis 0, i.e., rows) into one main data + frame. `groupby_facet` is also added as a column to this main data frame. +legend_title: str, optional (default: None) + Title for legend. If ``None``, Seaborn will determine the legend title (if + possible). +plot_object_methods: dict, optional + Execute methods of the object returned by the plotting function + (`seaborn_func`). This object will either be a + :class:`matplotlib.axes.Axes` (e.g., :func:`~seaborn.scatterplot`, + :func:`~seaborn.lineplot`), a :class:`seaborn.FacetGrid` (e.g., + :func:`~seaborn.relplot`, :func:`~seaborn.displot`), a + :class:`seaborn.JointGrid` (e.g., :func:`~seaborn.jointplot`), or a + :class:`seaborn.PairGrid` (e.g., :func:`~seaborn.pairplot`). Dictionary + keys are method names, dictionary values function arguments (use a + :obj:`dict` to specify keyword arguments). Example (for + :func:`~seaborn.relplot`): ``{'set': {'xlabel': 'X [km]'}, 'set_titles': + 'Model {col_name}'}``. +reset_index: bool, optional (default: False) + Put coordinate information of datasets into columns instead of (multi-) + indices. This avoids the deletion of coordinate information if different + groups of datasets have different dimensions but increases the memory + footprint of this diagnostic. +savefig_kwargs: dict, optional + Optional keyword arguments for :func:`matplotlib.pyplot.savefig`. By + default, uses ``bbox_inches: tight, dpi: 300, orientation: landscape``. +seaborn_func: str + Function used to plot the data. Must be a function of Seaborn. An overview + of Seaborn's plotting functions is given `here + `__. +seaborn_kwargs: dict, optional + Optional keyword arguments for the plotting function given by + `seaborn_func`. Must not include an argument called `data`. Example: + ``{'x': 'variable_1', 'y': 'variable_2', 'hue': 'coord_1'}``. Note: + variables (here: `variable_1` and `variable_2` are identified by their + `variable_group` in the recipe, i.e., the keys that specify variable groups + in `variables`. +seaborn_settings: dict, optional + Options for :func:`seaborn.set_theme` (affects all plots). +suptitle: str or None, optional (default: None) + Suptitle for the plot (see :func:`matplotlib.pyplot.suptitle`). If + ``None``, do not create a suptitle. If the plot shows only a single panel, + use `plot_object_methods` with ``{'set': {'title': 'TITLE'}}`` instead. + +""" +from __future__ import annotations + +import logging +from copy import deepcopy +from pathlib import Path +from pprint import pformat + +import iris +import iris.pandas +import matplotlib.pyplot as plt +import pandas as pd +import seaborn as sns +from matplotlib.colors import LogNorm, Normalize + +from esmvaltool.diag_scripts.shared import ( + ProvenanceLogger, + get_plot_filename, + group_metadata, + run_diagnostic, +) + +logger = logging.getLogger(Path(__file__).stem) + +# Use the new behavior of :func:`iris.pandas.as_data_frame` +iris.FUTURE.pandas_ndim = True + +# Save units of different variables +# Note: units must be unique across datasets of the same variable +UNITS: dict[str, str] = {} + + +def _create_plot( + plot_func: callable, + data_frame: pd.DataFrame, + cfg: dict, +) -> None: + """Create plot.""" + logger.debug( + "Using main data frame as input for plotting:\n%s", data_frame + ) + + # Plot + plot_kwargs = cfg['seaborn_kwargs'] + plot_func_str = cfg['seaborn_func'] + if 'data' in plot_kwargs: + raise ValueError("'data' is an invalid argument for 'seaborn_kwargs'") + logger.info( + "Creating plot with\nseaborn.%s(\n data=main_data_frame,\n%s\n)", + plot_func_str, + _get_str_from_kwargs(plot_kwargs), + ) + plot_obj = plot_func(data=data_frame, **plot_kwargs) + + # Adjust plot appearance + if cfg['plot_object_methods']: + for (func_name, func_args) in cfg['plot_object_methods'].items(): + if isinstance(func_args, dict): + logger.debug( + "Running\n%s.%s(\n%s\n)", + type(plot_obj).__name__, + func_name, + _get_str_from_kwargs(func_args), + ) + getattr(plot_obj, func_name)(**func_args) + else: + logger.debug( + "Running %s.%s(%r)", + type(plot_obj).__name__, + func_name, + func_args, + ) + getattr(plot_obj, func_name)(func_args) + if cfg['suptitle'] is not None: + logger.debug("Setting `suptitle='%s'`", cfg['suptitle']) + plt.suptitle(cfg['suptitle'], y=1.05) + if cfg['legend_title'] is not None: + _set_legend_title(plot_obj, cfg['legend_title']) + + # Save plot + plot_path = get_plot_filename(f"seaborn_{plot_func_str}", cfg) + plt.savefig(plot_path, **cfg['savefig_kwargs']) + logger.info("Wrote %s", plot_path) + plt.close() + + # Provenance tracking + caption = f"Seaborn {cfg['seaborn_func']} for one or more dataset(s)" + ancestors = [d['filename'] for d in cfg['input_data'].values()] + provenance_record = { + 'ancestors': ancestors, + 'authors': ['schlund_manuel'], + 'caption': caption, + } + with ProvenanceLogger(cfg) as provenance_logger: + provenance_logger.log(plot_path, provenance_record) + + +def _get_grouped_data(cfg: dict) -> dict: + """Get grouped input data.""" + groupby_facet = cfg['groupby_facet'] + input_data = list(cfg['input_data'].values()) + + # Check if necessary facets are present + for dataset in input_data: + if groupby_facet not in dataset: + raise ValueError( + f"groupby_facet '{groupby_facet}' is not available for " + f"dataset {dataset['filename']}" + ) + for facet in cfg['facets_as_columns']: + if facet not in dataset: + raise ValueError( + f"Facet '{facet}' used for option 'facets_as_columns' is " + f"not available for dataset {dataset['filename']}" + ) + + # Group data accordingly + grouped_data = group_metadata( + input_data, + groupby_facet, + sort='filename', + ) + + return grouped_data + + +def _get_dataframe(cfg: dict) -> pd.DataFrame: + """Get main :class:`pandas.DataFrame` used as input for plotting. + + Note + ---- + Data is stored in long form, see also :func:`iris.pandas.as_data_frame`. + + """ + logger.info( + "Grouping datasets by '%s' to create main data frame (data frames " + "are merged [combined along axis 1, i.e., columns] within groups, " + "then concatenated [combined along axis 0, i.e., rows] across groups)", + cfg['groupby_facet'], + ) + if cfg['add_aux_coords']: + logger.info("Adding aux_coords as columns") + if cfg['add_cell_measures']: + logger.info("Adding cell_measures as columns") + if cfg['add_ancillary_variables']: + logger.info("Adding ancillary_variables as columns") + if cfg['facets_as_columns']: + logger.info("Adding facets as columns: %s", cfg['facets_as_columns']) + + grouped_data = _get_grouped_data(cfg) + + # Merge data frames within groups + df_dict = {} + for (group, datasets) in grouped_data.items(): + logger.info("Processing group '%s'", group) + df_group = _get_df_for_group(cfg, group, datasets) + df_dict[group] = df_group + + # Concatenate data frames across groups and use dtype 'category' for facet + # columns to reduce memory usage and decrease computation times + groupby_facet = cfg['groupby_facet'] + df_main = pd.concat(df_dict.values(), ignore_index=cfg['reset_index']) + df_main = df_main.astype({ + f: 'category' for f in cfg['facets_as_columns'] + [groupby_facet] + }) + + logger.info("Successfully retrieved main data frame from input data") + logger.debug("Got main data frame:\n%s", df_main) + return df_main + + +def _get_df_for_group( + cfg: dict, + group: str, + datasets: list[dict], +) -> pd.DataFrame: + """Extract :class:`pandas.DataFrame` for a single group of datasets. + + This merges (i.e., combines along axis 1 = columns) all data frames of + individual datasets of a group. + + """ + df_group = pd.DataFrame() + facets_as_columns: dict[str, str] = {} + for dataset in datasets: + filename = dataset['filename'] + logger.info("Reading %s", filename) + cube = iris.load_cube(filename) + + # Update units + variable_group = dataset['variable_group'] + units = dataset['units'] + if variable_group in UNITS and UNITS[variable_group] != units: + raise ValueError( + f"Got duplicate units for variable '{variable_group}': " + f"'{units}' and '{UNITS[variable_group]}'" + ) + UNITS.setdefault(variable_group, units) + + # Get data frame for individual dataset with proper name + df_dataset = iris.pandas.as_data_frame( + cube, + add_aux_coords=cfg['add_aux_coords'], + add_cell_measures=cfg['add_cell_measures'], + add_ancillary_variables=cfg['add_ancillary_variables'], + ) + df_dataset = df_dataset.rename( + {cube.name(): variable_group}, axis='columns' + ) + + # Merge + if df_group.empty: + df_group = df_dataset + facets_as_columns = { + f: dataset[f] for f in cfg['facets_as_columns'] + } + else: + # Make sure that dimensional coordinates match across cubes within + # a group + if not df_group.index.equals(df_dataset.index): + raise ValueError( + f"Dimensions of cube {filename} differ from other cubes " + f"of group '{group}'. Cubes of that group:\n" + f"{pformat([d['filename'] for d in datasets])}" + ) + + # Make sure that facet values used as columns match across datasets + # within a cube + for (facet, val) in facets_as_columns.items(): + if dataset[facet] != val: + raise ValueError( + f"Facet value for facet '{facet}' (used by option " + f"'facets_as_columns') of dataset {filename} differs " + f"from value of other datasets of group '{group}': " + f"expected '{val}', got '{dataset[facet]}'. Datasets " + f"of that group:\n" + f"{pformat([d['filename'] for d in datasets])}" + ) + df_group = pd.merge( + df_group, + df_dataset, + left_index=True, + right_index=True, + sort=False, + suffixes=[None, '_DUPLICATE'], + ) + + # Assume that aux_coords, cell_measures, and ancillary_variables + # (if requested) are equal across cubes within the group. Only add + # them when they first appear. + df_group = df_group.filter(regex='^(?!.*_DUPLICATE)') + + # Move dimensional coordinates from (multi-) index into columns if + # requested + if cfg['reset_index']: + df_group = df_group.reset_index() + + # Add additional information as column and save the data frame + for (facet, val) in facets_as_columns.items(): + df_group[facet] = val + if cfg['groupby_facet'] not in df_group.columns: + df_group[cfg['groupby_facet']] = group + + return df_group + + +def _get_default_cfg(cfg: dict) -> dict: + """Get default options for configuration dictionary.""" + cfg = deepcopy(cfg) + + cfg.setdefault('add_ancillary_variables', False) + cfg.setdefault('add_aux_coords', False) + cfg.setdefault('add_cell_measures', False) + cfg.setdefault('data_frame_ops', {}) + cfg.setdefault('dropna_kwargs', {}) + cfg.setdefault('facets_as_columns', []) + cfg.setdefault('groupby_facet', 'alias') + cfg.setdefault('legend_title', None) + cfg.setdefault('plot_object_methods', {}) + cfg.setdefault('reset_index', False) + cfg.setdefault('savefig_kwargs', { + 'bbox_inches': 'tight', + 'dpi': 300, + 'orientation': 'landscape', + }) + cfg.setdefault('seaborn_kwargs', {}) + cfg.setdefault('seaborn_settings', {}) + cfg.setdefault('suptitle', None) + + return cfg + + +def _get_str_from_kwargs(kwargs, separator='\n', prefix=' '): + """Get overview string for kwargs.""" + return separator.join(f"{prefix}{k}={v!r}," for (k, v) in kwargs.items()) + + +def _get_plot_func(cfg: dict) -> callable: + """Get seaborn plot function.""" + if 'seaborn_func' not in cfg: + raise ValueError("Necessary option 'seaborn_func' missing") + if not hasattr(sns, cfg['seaborn_func']): + raise AttributeError( + f"Invalid seaborn_func '{cfg['seaborn_func']}' (must be a " + f"function of the module seaborn; an overview of seaborn plotting " + f"functions is given here: https://seaborn.pydata.org/tutorial/" + f"function_overview.html)" + ) + logger.info("Using plotting function seaborn.%s", cfg['seaborn_func']) + return getattr(sns, cfg['seaborn_func']) + + +def _modify_dataframe(data_frame: pd.DataFrame, cfg: dict) -> pd.DataFrame: + """Modify data frame according to the option ``data_frame_ops``.""" + allowed_funcs = ('query', 'eval') + + # data_frame_ops + for (func, expr) in cfg['data_frame_ops'].items(): + if func not in allowed_funcs: + raise ValueError( + f"Got invalid operation '{func}' for option 'data_frame_ops', " + f"expected one of {allowed_funcs}" + ) + op_str = f"'{func}' with argument '{expr}'" + logger.info("Modifying main data frame through operation %s", op_str) + data_frame = getattr(data_frame, func)(expr) + logger.debug( + "Main data frame after operation %s:\n%s", op_str, data_frame + ) + + # dropna_kwargs + if cfg['dropna_kwargs']: + logger.debug( + "Running\ndata_frame.dropna(\n%s\n)", + _get_str_from_kwargs(cfg['dropna_kwargs']), + ) + data_frame = data_frame.dropna(**cfg['dropna_kwargs']) + logger.debug("Main data frame after dropna \n%s", data_frame) + return data_frame + + +def _set_legend_title(plot_obj, legend_title: str) -> None: + """Set legend title.""" + if hasattr(plot_obj, 'get_legend'): # Axes + legend = plot_obj.get_legend() + elif hasattr(plot_obj, 'legend'): # FacetGrid, PairGrid + legend = plot_obj.legend + else: + raise ValueError( + f"Cannot set legend title, `{type(plot_obj).__name__}` does not " + f"support legends" + ) + if legend is None: + raise ValueError( + "Cannot set legend title, plot does not contain legend" + ) + logger.debug("Setting `legend_title='%s'`", legend_title) + legend.set_title(legend_title) + + +def _validate_config(cfg: dict) -> dict: + """Validate configuration dictionary.""" + cfg = deepcopy(cfg) + + # seaborn_kwargs: hue_norm + if 'hue_norm' in cfg['seaborn_kwargs']: + hue_norm = cfg['seaborn_kwargs']['hue_norm'] + if isinstance(hue_norm, str): + vmin = cfg['seaborn_kwargs'].pop('vmin', None) + vmax = cfg['seaborn_kwargs'].pop('vmax', None) + if hue_norm == 'linear': + hue_norm = Normalize(vmin=vmin, vmax=vmax) + elif hue_norm == 'log': + hue_norm = LogNorm(vmin=vmin, vmax=vmax) + else: + raise ValueError( + f"String value for `hue_norm` can only be `linear` or " + f"`log`, got `{hue_norm}`" + ) + cfg['seaborn_kwargs']['hue_norm'] = hue_norm + if isinstance(hue_norm, list): + cfg['seaborn_kwargs']['hue_norm'] = tuple(hue_norm) + + return cfg + + +def main(cfg: dict) -> None: + """Run diagnostic.""" + cfg = _get_default_cfg(cfg) + cfg = _validate_config(cfg) + + sns.set_theme(**cfg['seaborn_settings']) + plot_func = _get_plot_func(cfg) + + df_main = _get_dataframe(cfg) + df_main = _modify_dataframe(df_main, cfg) + + _create_plot(plot_func, df_main, cfg) + + +if __name__ == '__main__': + + with run_diagnostic() as config: + main(config) diff --git a/esmvaltool/recipes/recipe_seaborn.yml b/esmvaltool/recipes/recipe_seaborn.yml new file mode 100644 index 0000000000..faf0f07085 --- /dev/null +++ b/esmvaltool/recipes/recipe_seaborn.yml @@ -0,0 +1,145 @@ +# ESMValTool +# recipe_seaborn.yml +--- +documentation: + title: > + Create arbitrary Seaborn plots. + + description: > + This recipe showcases the use of the Seaborn diagnostic that provides a + high-level interface to Seaborn for ESMValTool recipes. For this, the input + data is arranged into a single `pandas.DataFrame`, which is then used as + input for the Seaborn function defined by the option `seaborn_func`. + + authors: + - schlund_manuel + + maintainer: + - schlund_manuel + + references: + - waskom21joss + + projects: + - 4c + - esm2025 + - isenes3 + - usmile + + +preprocessors: + + zonal_mean: + zonal_statistics: + operator: mean + + extract_ar6_regions: + regrid: + target_grid: 5x5 + scheme: linear + extract_shape: + shapefile: ar6 + crop: true + decomposed: true + ids: + Name: ®ions_to_extract + - N.Europe + - West&Central-Europe + - Mediterranean + - Equatorial.Pacific-Ocean + - Equatorial.Atlantic-Ocean + - Equatorial.Indic-Ocean + convert_units: + units: mm day-1 + + +diagnostics: + + plot_temperature_vs_lat: + description: Plot air temperature vs. latitude (pressure levels = colors). + variables: + zonal_mean_ta: + short_name: ta + mip: Amon + preprocessor: zonal_mean + project: CMIP6 + exp: historical + timerange: '1991/2014' + additional_datasets: + - {dataset: CESM2-WACCM, grid: gn, ensemble: r1i1p1f1} + - {dataset: GFDL-ESM4, grid: gr1, ensemble: r1i1p1f1} + scripts: + plot: + script: seaborn_diag.py + seaborn_func: relplot + seaborn_kwargs: + x: latitude + y: zonal_mean_ta + col: alias + col_wrap: 2 + hue: air_pressure + hue_norm: log + palette: plasma + linewidth: 0.0 + marker: o + s: 1 + add_aux_coords: true + data_frame_ops: + eval: air_pressure = air_pressure / 100.0 + dropna_kwargs: + axis: 0 + how: any + legend_title: Pressure [hPa] + plot_object_methods: + set: + xlabel: 'Latitude [°]' + ylabel: 'Temperatute [K]' + set_titles: '{col_name}' + seaborn_settings: + style: ticks + rc: + axes.titlepad: 15.0 + suptitle: Simulated Temperature (1991-2014) + + plot_precipitation_histograms_region: + description: Plot precipitation histograms for different regions. + variables: + pr: + mip: day + preprocessor: extract_ar6_regions + project: CMIP6 + exp: historical + timerange: '2005/2014' + additional_datasets: + - {dataset: CESM2-WACCM, grid: gn, ensemble: r1i1p1f1} + - {dataset: GFDL-ESM4, grid: gr1, ensemble: r1i1p1f1} + scripts: + plot: + script: seaborn_diag.py + seaborn_func: displot + seaborn_kwargs: + kind: hist + stat: density + bins: 300 + x: pr + col: shape_id + col_order: *regions_to_extract + col_wrap: 3 + hue: alias + facet_kws: + sharey: false + add_aux_coords: true + dropna_kwargs: + axis: 0 + how: any + legend_title: Model + plot_object_methods: + set: + xlabel: 'Precipitation [mm/day]' + xlim: [0, 30] + set_titles: '{col_name}' + seaborn_settings: + style: ticks + rc: + axes.titlepad: 15.0 + suptitle: Simulated Precipitation (2005-2014) diff --git a/esmvaltool/references/waskom21joss.bibtex b/esmvaltool/references/waskom21joss.bibtex new file mode 100644 index 0000000000..386d270d8b --- /dev/null +++ b/esmvaltool/references/waskom21joss.bibtex @@ -0,0 +1,12 @@ +@article{waskom21joss, + doi = {10.21105/joss.03021}, + url = {https://doi.org/10.21105/joss.03021}, + year = {2021}, + publisher = {The Open Journal}, + volume = {6}, + number = {60}, + pages = {3021}, + author = {Michael L. Waskom}, + title = {seaborn: statistical data visualization}, + journal = {Journal of Open Source Software} +}