Skip to content

Commit

Permalink
Merge pull request #184 from monocongo/issue_182
Browse files Browse the repository at this point in the history 
Updates for division/grid processing, docs, and tests
  • Loading branch information
@monocongo
monocongo committed Sep 14, 2018
2 parents a04aa37 + 09e1175 commit 4096124
Show file tree
Hide file tree
Showing 8 changed files with 685 additions and 551 deletions.
23 changes: 19 additions & 4 deletions climate_indices/indices.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -160,7 +160,9 @@ def spei(scale,
(and shape?) as the input precipitation array, must be unspecified or None if using an array
of PET values as input
:param data_start_year: the initial year of the input datasets (assumes that the two inputs cover the same period)
:param latitude_degrees: the latitude of the location, in degrees north, must be unspecified or None if using
:param calibration_year_initial: initial year of the calibration period
:param calibration_year_final: final year of the calibration period
:param latitude_degrees: the latitude of the location, in degrees north, must be unspecified or None if using
an array of PET values as an input, and must be specified if using an array of temperatures
as input, valid range is -90.0 to 90.0 (inclusive)
:return: an array of SPEI values
Expand All @@ -174,9 +176,11 @@ def spei(scale,
# validate the function's argument combinations
if temps_celsius is not None:

# since we have temperature then it's expected that we'll compute PET internally, so we shouldn't have PET as an input
# since we have temperature then it's expected that we'll compute PET internally,
# so we shouldn't have PET as an input
if pet_mm is not None:
message = 'Incompatible arguments: either temperature or PET arrays can be specified as arguments, but not both'
message = 'Incompatible arguments: either temperature or PET arrays can be '\
'specified as arguments, but not both'
_logger.error(message)
raise ValueError(message)

Expand Down Expand Up @@ -249,7 +253,12 @@ def spei(scale,
calibration_year_initial,
calibration_year_final,
periodicity)


else:
message = "Unsupported distribution argument: {dist}".format(dist=distribution)
_logger.error(message)
raise ValueError(message)

# clip values to within the valid range, reshape the array back to 1-D
spei = np.clip(transformed_fitted_values, _FITTED_INDEX_VALID_MIN, _FITTED_INDEX_VALID_MAX).flatten()

Expand Down Expand Up @@ -351,6 +360,12 @@ def percentage_of_normal(values,
:rtype: numpy.ndarray of type float
'''

# validate the scale argument
if (scale is None) or (scale < 1):
message = 'Invalid scale argument: \'{0}\''.format(scale)
_logger.error(message)
raise ValueError(message)

# if doing monthly then we'll use 12 periods, corresponding to calendar months, if daily assume years w/366 days
if periodicity is compute.Periodicity.monthly:
periodicity = 12
Expand Down
10 changes: 5 additions & 5 deletions climate_indices/utils.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -211,9 +211,9 @@ def reshape_to_divs_years_months(monthly_values):

#-----------------------------------------------------------------------------------------------------------------------
@numba.jit
def transform_to_366day(original,
year_start,
total_years):
def to_366day(original,
year_start,
total_years):
'''
Takes an array of daily values with only actual leap years represented as 366 day years (non-leap years with 365 days)
and converts it to an array of daily values represented as containing full 366 day years as if each year is a leap year
Expand Down Expand Up @@ -278,8 +278,8 @@ def transform_to_366day(original,

#-----------------------------------------------------------------------------------------------------------------------
@numba.jit
def transform_to_gregorian(original,
year_start):
def to_gregorian(original,
year_start):
'''
Takes an array of daily values represented as full 366 day years (as if each year is a leap year with
fill/faux values for the Feb. 29th of each non-leap year) and converts it to an array of daily values
Expand Down
102 changes: 82 additions & 20 deletions docs/index.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -19,8 +19,7 @@
.. |License| image:: https://img.shields.io/badge/license-Unlicense-green.svg
:target: https://unlicense.org/


|Build| |Coverage| |CodeFactor| |License|
|Build| |Coverage| |CodeFactor| |License|

Climate Indices in Python
=============================================
Expand Down Expand Up @@ -153,33 +152,55 @@ Indices Processing

Included are scripts which interact with the core computational package to compute
one or more climate indices. These are ``process_grid.py`` which is used
to compute indices from gridded NetCDF datasets, and ``process_divisions.py``
which is used to compute indices from US climate division NetCDF datasets.
to compute indices corresponding to gridded NetCDF datasets, and ``process_divisions.py``
which is used to compute indices corresponding to US climate division NetCDF datasets.

These Python scripts are written to be run via bash shell commands, i.e.

``$ python process_grid.py <options>``

The options for these scripts are described below:


+------------------------+-------------------------------------------------+
| Option | Description |
+========================+=================================================+
| index | Which of the available indices to compute. |
| index | Which of the climate indices to compute. |
| | Valid values are 'spi', 'spei', 'pnp', 'scaled',|
| | and 'palmers'. 'scaled' indicates all three |
| | scaled indices (SPI, SPEI, and PNP) and |
| | 'pet', and 'palmers'. 'scaled' indicates all |
| | three scaled indices (SPI, SPEI, and PNP) and |
| | 'palmers' indicates all Palmer indices (PDSI, |
| | PHDI, PMDI, SCPDSI, and Z-Index). |
| | |
| | **NOTE**: Only used for grid processing, as |
| | the divisions processing will compute all |
| | indices. |
+------------------------+-------------------------------------------------+
| periodicity | The periodicity of the input dataset files. |
| | Valid values are 'monthly' and 'daily'. |
| | Note: only SPI and PNP support daily inputs. |
| | |
| | **NOTE**: Only used for grid processing, and |
| | only SPI and PNP support daily inputs. |
+------------------------+-------------------------------------------------+
| netcdf_divs | Input NetCDF file containing a US climate |
| | divisions dataset with precipitation, |
| | temperature, and AWC variables. Computed indices|
| | variables will be written into this file, with |
| | existing indices variables (if any) overwritten |
| | with the computed values. This file serves as |
| | both input and output for the divisions |
| | processing. |
| | |
| | **NOTE**: Only used for US climate divisions |
| | processing. |
+------------------------+-------------------------------------------------+
| netcdf_precip | Input NetCDF file containing a |
| | precipitation dataset, required for all |
| | indices except for PET. Requires the use of |
| | **var_name_temp** in conjunction so as to |
| | identify the NetCDF's precipitation variable. |
| | |
| | **NOTE**: Only used for grid processing. |
+------------------------+-------------------------------------------------+
| var_name_precip | Name of the precipitation variable within |
| | the input precipitation NetCDF. |
Expand All @@ -197,6 +218,8 @@ These Python scripts are written to be run via bash shell commands, i.e.
| | Palmers. Requires the use of |
| | **var_name_temp** in conjunction so as to |
| | identify the NetCDF's temperature variable. |
| | |
| | **NOTE**: Only used for grid processing. |
+------------------------+-------------------------------------------------+
| var_name_temp | Name of the temperature variable within the |
| | input temperature NetCDF. |
Expand All @@ -210,33 +233,51 @@ These Python scripts are written to be run via bash shell commands, i.e.
| | Palmers. Requires the use of |
| | **var_name_pet** in conjunction so as to |
| | identify the NetCDF's PET variable. |
| | |
| | **NOTE**: Only used for grid processing. |
+------------------------+-------------------------------------------------+
| var_name_pet | Name of the PET variable within the input PET |
| | NetCDF. |
| | |
| | **NOTE**: Only used for grid processing. |
+------------------------+-------------------------------------------------+
| netcdf_awc | Input NetCDF file containing an available water |
| | capacity, required for Palmers. Requires the |
| | use of **var_name_awc** in conjunction so as to |
| | identify the NetCDF's AWC variable. |
| | |
| | **NOTE**: Only used for grid processing. |
+------------------------+-------------------------------------------------+
| awc_var_name | Name of the available water capacity variable |
| | within the input AWC NetCDF. |
+------------------------+-------------------------------------------------+
| output_file_base | Base file name for all computed output files. |
| | Each computed index will have an output file |
| | whose name will begin with this base name plus |
| | the index's abbreviation plus a month scale |
| output_file_base | Base file name for all grid processing |
| | output files. |
| | |
| | Each computed index will have a corresponding |
| | output file whose name will begin with |
| | this base name plus the index's |
| | abbreviation plus a month scale |
| | (if applicable), connected with underscores, |
| | plus the '.nc' extension. For example |
| | for SPI at 3-month |
| | scale the resulting output files will be |
| | for SPI at 3-month scale |
| | the resulting output files will be |
| | named **<output_file_base>_spi_gamma_03.nc** |
| | and **<output_file_base>_spi_pearson_03.nc**. |
| | |
| | **NOTE**: Only used for grid processing. |
+------------------------+-------------------------------------------------+
| scales | Time step scales over which the PNP, SPI, and |
| | SPEI values are to be computed. Required when |
| | the **index** argument is 'spi', 'spei', |
| | 'pnp', or 'scaled'. |
| | 'pnp', or 'scaled'. If the **periodicity** |
| | option is specified then that option will infer |
| | that the scales used here either month or day |
| | scales. |
| | |
| | **NOTE**: When used for US climate divisions |
| | processing this option on specifies to month |
| | scales. |
+------------------------+-------------------------------------------------+
| calibration_start_year | Initial year of the calibration period. |
+------------------------+-------------------------------------------------+
Expand All @@ -247,6 +288,27 @@ These Python scripts are written to be run via bash shell commands, i.e.
Example Command Line Invocations
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

US Climate Divisions (all indices)
""""""""""""""""""""""""""""""""""

``$ python process_divisions.py --scales 3 6
--netcdf_divs ../example_inputs/nclimdiv.nc
--var_name_precip prcp
--var_name_temp tavg --var_name_awc awc
--calibration_start_year 1951 --calibration_end_year 2010``

The above command will compute all indices from an input NetCDF dataset containing
precipitation, temperature, and available water capacity variables (in this case,
the US Climate Divisions NetCDF dataset provided in the example inputs directory).
The input dataset is monthly data and the calibration period used will be
Jan. 1951 through Dec. 2010. The indices will be computed at 3-month and 6-month scales.
Upon completion the NetCDF file will contain variables for all computed indices:
`pet`, `pnp_03`, `pnp_06`, `spi_gamma_03`, `spi_gamma_06`, `spi_pearson_03`, `spi_pearson_06`,
`spei_gamma_03`, `spei_gamma_06`, `spei_pearson_03`, `spei_pearson_06`, `pdsi`, `phdi`, `pmdi`,
`scpdsi`, and `zindex`. If any or all of these index variables exist in the file before the
processing then existing values are overwritten with computed values, otherwise new variables
are created and populated in the NetCDF dataset file.

PET monthly
""""""""""""

Expand All @@ -256,9 +318,9 @@ PET monthly

The above command will compute PET (potential evapotranspiration) using the
Thornthwaite method from an input temperature dataset (in this case, the reduced
resolution nClimGrid temperature dataset provided as an example input). The input
dataset is monthly data and the calibration period used will be Jan. 1951 through
Dec. 2010. The output file will be `<out_dir>/nclimgrid_lowres_pet.nc`.
resolution nClimGrid temperature dataset provided in the example inputs directory).
The input dataset is monthly data and the calibration period used will be Jan. 1951
through Dec. 2010. The output file will be `<out_dir>/nclimgrid_lowres_pet.nc`.

SPI daily
""""""""""
Expand All @@ -270,7 +332,7 @@ prcp --output_file_base <out_dir>/cmorph_lowres_daily_conus --scales 30 90

The above command will compute SPI (standardized precipitation index, both gamma
and Pearson Type III distributions) from an input precipitation dataset (in this case,
the reduced resolution CMORPH precipitation dataset provided in the example inputs
the reduced resolution CMORPH precipitation dataset provided in the example inputs
directory). The input dataset is daily data and the calibration period used will be
Jan. 1st, 1998 through Dec. 31st, 2016. The index will be computed at 30-day and
90-day timescales. The output files will be `<out_dir>/cmorph_lowres_daily_conus_spi_gamma_30.nc`,
Expand All @@ -288,7 +350,7 @@ SPI monthly

The above command will compute SPI (standardized precipitation index, both gamma and
Pearson Type III distributions) from an input precipitation dataset (in this case,
the reduced resolution nClimGrid precipitation dataset provided in the example inputs directory).
the reduced resolution nClimGrid precipitation dataset provided in the example inputs directory).
The input dataset is monthly data and the calibration period used will be
Jan. 1951 through Dec. 2010. The index will be computed at 6-month and 12-month timescales.
The output files will be `<out_dir>/nclimgrid_lowres_spi_gamma_06.nc`,
Expand Down
Binary file modified example_inputs/nclimdiv.nc
View file
Binary file not shown.
Loading

0 comments on commit 4096124

Please sign in to comment.