Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Configuration management: Allow for creating / saving / loading configuration on the fly. #23

Merged
merged 34 commits into from
Jul 30, 2019
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
34 commits
Select commit Hold shift + click to select a range
4750e53
Add config module and .yaml file prototype
euronion Jul 16, 2019
992e186
config: Add license text
euronion Jul 16, 2019
4dc7cdb
config: Simplify yaml-config file locating.
euronion Jul 16, 2019
702a145
confg.yaml: Rename config default files.
euronion Jul 17, 2019
62c4431
config: Change default behaviour.
euronion Jul 17, 2019
2aa4c45
config: Change packages used internally.
euronion Jul 17, 2019
ea18f31
Update .atlite.default.config.yaml
euronion Jul 17, 2019
1ff9ba8
resource: Switch to config.py and absolute paths
euronion Jul 17, 2019
d59efa8
default.config.yaml: Change description
euronion Jul 17, 2019
add935d
config: Change config.yaml file name and location
euronion Jul 17, 2019
1ed75ff
config: Fix default.config.yaml name.
euronion Jul 17, 2019
f2ff96e
utils: Add function for determining relative paths.
euronion Jul 17, 2019
164b8ac
resource: Implement relative paths standard.
euronion Jul 17, 2019
d29539f
Restructure the way the config module works.
euronion Jul 19, 2019
afa27dd
Implement new config in all modules.
euronion Jul 20, 2019
4f525ef
Update example create_cutout for new config
euronion Jul 20, 2019
da00cde
cutout: Fix cutout_dir when not explicitly provided.
euronion Jul 20, 2019
e4a6202
config: Do not reset config_path with update(...)
euronion Jul 22, 2019
7e866f4
README: Include configuration management
euronion Jul 22, 2019
44ecbbf
default.config: Set reasonable defaults.
euronion Jul 22, 2019
ed8b789
README: Fix formatting issues.
euronion Jul 22, 2019
092ddf7
README: Fix more formatting
euronion Jul 22, 2019
d67cb2e
Update .gitignore
euronion Jul 22, 2019
d58ff0e
Revert "Update .gitignore"
euronion Jul 22, 2019
6bb2ec8
Revert "Revert "Update .gitignore""
euronion Jul 23, 2019
18c8107
Revert "Update .gitignore"
euronion Jul 23, 2019
2782a8b
config: Refractor style.
euronion Jul 24, 2019
04a6ef2
cutout: Set cutout_dir correctly based on constructor parameters.
euronion Jul 24, 2019
64e53c6
example/create_cutout: Adjust for changed function keywords.
euronion Jul 24, 2019
045e859
Update configuration management to always load defaults from package …
euronion Jul 29, 2019
760a106
cutout: Restructure cutout_dir and cutout_name extraction for portabi…
euronion Jul 29, 2019
cea09bb
create_cutout: Simplify example.
euronion Jul 30, 2019
5d70eaa
config.example.yaml: Correct typo.
euronion Jul 30, 2019
2123f5e
cutout: Adjust parsing of 'cutout_dir' from 'name' constructor argument.
euronion Jul 30, 2019
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
19 changes: 19 additions & 0 deletions atlite/config.default.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
# -------------------*DEFAULT SETTINGS*------------------- #
# --------------------*DO NOT CHANGE*--------------------- #
# For custom settings, see the 'config.example.yaml' file. #
# --------------------*DO NOT CHANGE*--------------------- #

# Folder for storing prepared cutout files
cutout_dir: <ATLITE>/cutouts

# Folder containing raw dataset data
gebco_path: <ATLITE>/data/gebco
ncep_dir: <ATLITE>/data/ncep
cordex_dir: <ATLITE>/data/cordex
sarah_dir: <ATLITE>/data/sarah

# Folder for different wind turbine configuration files
windturbine_dir: <ATLITE>/resources/windturbine

# Folder for different solar panel configuration files
solarpanel_dir: <ATLITE>/resources/solarpanel
35 changes: 35 additions & 0 deletions atlite/config.example.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
# Exemplary Settings for atlite


## How to use this file.
# 1. Copy this file into your home directory, usually that is
# "~" for linux users and "C:\Users\<Your Username>" for windows users.
# 2. Rename the file to ".atlite.config.yaml" (note the trailing dot).
# 3. Uncomment any setting you want to differ from the default settings.

## Remarks
# * Relative paths:
# Are relative to the location of the currently loaded config file,
# e.g. if the config file is located in your home directory,
# then any relative paths are considered relative to the homedirectory.
# Exception: Relative paths starting with "ATLITE/" (case-sensitive) are
euronion marked this conversation as resolved.
Show resolved Hide resolved
# considered relative to the atlite-package directory.
# * Custom location:
# You can use this configuration file and place it at custom locations with
# a custom file name. In this case you need to manually load the configuration
# after importing atlite with "atlite.config.read(<path with filename>)".

# Folder for storing prepared cutout files
# cutout_dir: <ATLITE>/cutouts

# Folder containing raw dataset data
# gebco_path: <ATLITE>/data/gebco
# ncep_dir: <ATLITE>/data/ncep
# cordex_dir: <ATLITE>/data/cordex
# sarah_dir: <ATLITE>/data/sarah

# Folder for different wind turbine configuration files
# windturbine_dir: <ATLITE>/resources/windturbine

# Folder for different solar panel configuration files
# solarpanel_dir: <ATLITE>/resources/solarpanel
48 changes: 22 additions & 26 deletions atlite/config.py
Original file line number Diff line number Diff line change
Expand Up @@ -21,18 +21,10 @@
logger = logging.getLogger(__name__)


_FILE_NAME = "config.yaml"
_DEFAULT_FILE_NAME = "default.config.yaml"

# Search paths for the yaml-configuration file
_SEARCH_PATHS = (
# User home directory - Custom
os.path.join(os.path.expanduser("~"), ".atlite", _FILE_NAME),
# Package install directory - Custom
pkg_resources.resource_filename(__name__, _FILE_NAME),
# Package install directory - Default
pkg_resources.resource_filename(__name__, _DEFAULT_FILE_NAME)
)
_FILE_NAME = ".atlite.config.yaml"
_FILE_SEARCH_PATH = os.path.join(os.path.expanduser("~"), _FILE_NAME)
_DEFAULT_FILE_NAME = "config.default.yaml"
_DEFAULT_SEARCH_PATH = pkg_resources.resource_filename(__name__, _DEFAULT_FILE_NAME)

# List of all supported attributes for the config
ATTRS = []
Expand All @@ -47,7 +39,7 @@

# Path of the configuration file.
# Automatically updated when using provided API.
config_path = None
config_path = ""

def read(path):
"""Read and set the configuration based on the file in 'path'."""
Expand Down Expand Up @@ -97,7 +89,21 @@ def update(config_dict):

globals().update(config_dict)
_update_variables()


def reset():
"""Reset the configuration to its initial values."""

# Test for file existence in order to not try to read
# non-existing configuration files at this point (do not confuse the user)
for path in [_DEFAULT_SEARCH_PATH, _FILE_SEARCH_PATH]:
if os.path.isfile(path):
read(path)

# Notify user of empty config
if not config_path:
logger.warn("No valid configuration file found in default and home directories. "
"No configuration is loaded, manual configuration required.")

def _update_variables():
"""Update list of provided attributes by the module."""

Expand All @@ -110,15 +116,5 @@ def _update_variables():
"logger", "os", "pkg_resources", "yaml"}


# Try to load configuration from standard paths
for path in _SEARCH_PATHS:
if os.path.isfile(path):
# Don't check if the file is actually what it claims to be
# also: consider a read without error a success.
read(path)
break

# Notify user of empty config
if not config_path:
logger.warn("No valid configuration file found in default paths. "
"No configuration is loaded, manual configuration required.")
# Load the configuration at first module import
reset()
25 changes: 0 additions & 25 deletions atlite/default.config.yaml

This file was deleted.

3 changes: 3 additions & 0 deletions atlite/utils.py
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,7 @@
import xarray as xr
import sys
import os
import pkg_resources

from . import config

Expand Down Expand Up @@ -79,6 +80,8 @@ def construct_filepath(path):

if os.path.isabs(path):
return path
elif path.startswith('<ATLITE>'):
return pkg_resources.resource_filename(__name__, path[8:])
elif config.config_path is None:
# If config_path is not defined assume the user know what per does
return path
Expand Down