Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Solved issues in configuration files #1457

Merged
merged 8 commits into from Feb 3, 2022
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
153 changes: 74 additions & 79 deletions doc/quickstart/configure.rst
Expand Up @@ -46,85 +46,80 @@ with explanations in a commented line above each option:

.. code-block:: yaml

# Destination directory where all output will be written
# including log files and performance stats
output_dir: ./esmvaltool_output

# Directory for storing downloaded climate data
download_dir: ~/climate_data

# Disable the automatic download of missing CMIP3, CMIP5, CMIP6, CORDEX,
# and obs4MIPs data from ESGF [true]/false.
offline: true

# Auxiliary data directory, used by some recipes to look for additional datasets
auxiliary_data_dir: ~/auxiliary_data

# Rootpaths to the data from different projects (lists are also possible)
# these are generic entries to better allow you to enter your own
# For site-specific entries, see the default config-user.yml file
# that can be installed with the command `esmvaltool config get_config_user`.
# For each project, this can be either a single path or a list of paths.
# Comment out these when using a site-specific path
rootpath:
default: ~/climate_data

# Directory structure for input data: [default]/ESGF/BADC/DKRZ/ETHZ/etc
# See config-developer.yml for definitions.
# comment out/replace as per needed
drs:
CMIP3: ESGF
CMIP5: ESGF
CMIP6: ESGF
CORDEX: ESGF
obs4MIPs: ESGF

# Run at most this many tasks in parallel [null]/1/2/3/4/..
# Set to null to use the number of available CPUs.
# If you run out of memory, try setting max_parallel_tasks to 1 and check the
# amount of memory you need for that by inspecting the file
# run/resource_usage.txt in the output directory. Using the number there you
# can increase the number of parallel tasks again to a reasonable number for
# the amount of memory available in your system.
max_parallel_tasks: null

# Set the console log level debug, [info], warning, error
# for much more information printed to screen set log_level: debug
log_level: info

# Exit on warning (only for NCL diagnostic scripts)? true/[false]
exit_on_warning: false

# Plot file format? [png]/pdf/ps/eps/epsi
output_file_type: png

# Remove the ``preproc`` dir if the run was successful
# By default this option is set to "true", so all preprocessor output files
# will be removed after a successful run. Set to "false" if you need those files.
remove_preproc_dir: true

# Use netCDF compression true/[false]
compress_netcdf: false

# Save intermediary cubes in the preprocessor true/[false]
# set to true will save the output cube from each preprocessing step
# these files are numbered according to the preprocessing order
save_intermediary_cubes: false

# Use a profiling tool for the diagnostic run [false]/true
# A profiler tells you which functions in your code take most time to run.
# For this purpose we use vprof, see below for notes
# Only available for Python diagnostics
profile_diagnostic: false

# Path to custom config-developer file, to customise project configurations.
# See config-developer.yml for an example. Set to "null" to use the default
config_developer_file: null

.. code-block:: yaml

# Auxiliary data directory (used for some additional datasets)
auxiliary_data_dir: ~/auxiliary_data
# Directory where results are stored
output_dir: ~/esmvaltool_output

# Directory for storing downloaded climate data
download_dir: ~/climate_data

# Auxiliary data directory (used for some additional datasets)
auxiliary_data_dir: ~/auxiliary_data

# Run at most this many tasks in parallel [null]/1/2/3/4/..
# Set to null to use the number of available CPUs.
# If you run out of memory, try setting max_parallel_tasks to 1 and check the
# amount of memory you need for that by inspecting the file
# run/resource_usage.txt in the output directory. Using the number there you
# can increase the number of parallel tasks again to a reasonable number for
# the amount of memory available in your system.
max_parallel_tasks: null

# Disable the automatic download of missing CMIP3, CMIP5, CMIP6, CORDEX,
# and obs4MIPs data from ESGF by default. This is useful if you are working
# on a computer without an internet connection.
offline: true

# Log level of the console
log_level: info
schlunma marked this conversation as resolved.
Show resolved Hide resolved

# Exit on warning (only for NCL diagnostic scripts)? true/[false]
exit_on_warning: false

# Plot file format? [png]/pdf/ps/eps/epsi
output_file_type: png

# Remove the preproc dir if all fine
remove_preproc_dir: true

# Use netCDF compression true/[false]
compress_netcdf: false

# Save intermediary cubes in the preprocessor true/[false]
save_intermediary_cubes: false

# Path to custom config-developer file, to customise project configurations.
# See config-developer.yml for an example. Set to None to use the default
config_developer_file: null

# Get profiling information for diagnostics
# Only available for Python diagnostics
profile_diagnostic: false

# Rootpaths to the data from different projects (lists are also possible)
# these are generic entries to better allow you to enter your own
# For site-specific entries, see below
schlunma marked this conversation as resolved.
Show resolved Hide resolved
# Comment out these when using a site-specific path
rootpath:
CMIP3: [~/cmip3_inputpath1, ~/cmip3_inputpath2]
CMIP5: [~/cmip5_inputpath1, ~/cmip5_inputpath2]
CMIP6: [~/cmip6_inputpath1, ~/cmip6_inputpath2]
OBS: ~/obs_inputpath
OBS6: ~/obs6_inputpath
obs4MIPs: ~/obs4mips_inputpath
ana4mips: ~/ana4mips_inputpath
native6: ~/native6_inputpath
RAWOBS: ~/rawobs_inputpath
default: ~/default_inputpath

# Directory structure for input data: [default]/ESGF/BADC/DKRZ/ETHZ/etc
# See config-developer.yml for definitions.
# comment out/replace as per needed
drs:
CMIP3: default
CMIP5: default
CMIP6: default
CORDEX: default
obs4MIPs: default

The ``offline`` setting can be used to disable or enable automatic downloads from ESGF.
If ``offline`` is set to ``false``, the tool will automatically download
Expand Down
4 changes: 2 additions & 2 deletions esmvalcore/_config/_config.py
Expand Up @@ -104,7 +104,7 @@ def read_config_user_file(config_file, folder_name, options=None):

# set defaults
defaults = {
'auxiliary_data_dir': 'auxiliary_data',
'auxiliary_data_dir': '~/auxiliary_data',
'compress_netcdf': False,
'config_developer_file': None,
'drs': {},
Expand All @@ -114,7 +114,7 @@ def read_config_user_file(config_file, folder_name, options=None):
'max_parallel_tasks': None,
'offline': True,
'output_file_type': 'png',
'output_dir': 'esmvaltool_output',
'output_dir': '~/esmvaltool_output',
'profile_diagnostic': False,
'remove_preproc_dir': True,
'resume_from': [],
Expand Down
8 changes: 5 additions & 3 deletions esmvalcore/config-user.yml
Expand Up @@ -11,9 +11,11 @@
#
# Note for developers:
# -------------------
# Two identical copies of this file (``ESMValTool/config-user-example.yml`` and
# ``ESMValCore/esmvalcore/config-user.yml``) exist. If you change one of it,
# make sure to apply the changes to the other.
# Three (almost) identical copies of this file
# (``ESMValTool/config-user-example.yml``,
# ``ESMValCore/esmvalcore/config-user.yml`` and
# ``ESMValCore/doc/quickstart/configure.rst``) exist. If you change one of it,
# make sure to apply the changes to the others.
#
###############################################################################
---
Expand Down