Skip to content

Commit

Permalink
v0.4
Browse files Browse the repository at this point in the history
  • Loading branch information
seanhaythorne committed Oct 26, 2016
1 parent 2e53cd1 commit 1370615
Show file tree
Hide file tree
Showing 92 changed files with 4,673 additions and 3,682 deletions.
72 changes: 72 additions & 0 deletions Bias Corrections/MSLP/Trace21_2.5x2.5_MA_Delta.txt

Large diffs are not rendered by default.

72 changes: 72 additions & 0 deletions Bias Corrections/MSLP/Trace21_2.5x2.5_MB_Delta.txt

Large diffs are not rendered by default.

72 changes: 72 additions & 0 deletions Bias Corrections/MSLP/Trace21_2.5x2.5_MC_Delta.txt

Large diffs are not rendered by default.

72 changes: 72 additions & 0 deletions Bias Corrections/MSLP/Trace21_2.5x2.5_MD_Delta.txt

Large diffs are not rendered by default.

72 changes: 72 additions & 0 deletions Bias Corrections/MSLP/Trace21_2.5x2.5_ME_Delta.txt

Large diffs are not rendered by default.

72 changes: 72 additions & 0 deletions Bias Corrections/MSLP/Trace21_2.5x2.5_MF_Delta.txt

Large diffs are not rendered by default.

72 changes: 72 additions & 0 deletions Bias Corrections/MSLP/Trace21_2.5x2.5_MG_Delta.txt

Large diffs are not rendered by default.

72 changes: 72 additions & 0 deletions Bias Corrections/MSLP/Trace21_2.5x2.5_MH_Delta.txt

Large diffs are not rendered by default.

72 changes: 72 additions & 0 deletions Bias Corrections/MSLP/Trace21_2.5x2.5_MI_Delta.txt

Large diffs are not rendered by default.

72 changes: 72 additions & 0 deletions Bias Corrections/MSLP/Trace21_2.5x2.5_MJ_Delta.txt

Large diffs are not rendered by default.

72 changes: 72 additions & 0 deletions Bias Corrections/MSLP/Trace21_2.5x2.5_MK_Delta.txt

Large diffs are not rendered by default.

72 changes: 72 additions & 0 deletions Bias Corrections/MSLP/Trace21_2.5x2.5_ML_Delta.txt

Large diffs are not rendered by default.

72 changes: 0 additions & 72 deletions Bias Corrections/Precip/New/Trace21_2.5x2.5_PA_Error.txt

This file was deleted.

72 changes: 0 additions & 72 deletions Bias Corrections/Precip/New/Trace21_2.5x2.5_PB_Error.txt

This file was deleted.

72 changes: 0 additions & 72 deletions Bias Corrections/Precip/New/Trace21_2.5x2.5_PC_Error.txt

This file was deleted.

72 changes: 0 additions & 72 deletions Bias Corrections/Precip/New/Trace21_2.5x2.5_PD_Error.txt

This file was deleted.

72 changes: 0 additions & 72 deletions Bias Corrections/Precip/New/Trace21_2.5x2.5_PE_Error.txt

This file was deleted.

72 changes: 0 additions & 72 deletions Bias Corrections/Precip/New/Trace21_2.5x2.5_PF_Error.txt

This file was deleted.

72 changes: 0 additions & 72 deletions Bias Corrections/Precip/New/Trace21_2.5x2.5_PG_Error.txt

This file was deleted.

72 changes: 0 additions & 72 deletions Bias Corrections/Precip/New/Trace21_2.5x2.5_PH_Error.txt

This file was deleted.

72 changes: 0 additions & 72 deletions Bias Corrections/Precip/New/Trace21_2.5x2.5_PI_Error.txt

This file was deleted.

72 changes: 0 additions & 72 deletions Bias Corrections/Precip/New/Trace21_2.5x2.5_PJ_Error.txt

This file was deleted.

72 changes: 0 additions & 72 deletions Bias Corrections/Precip/New/Trace21_2.5x2.5_PK_Error.txt

This file was deleted.

72 changes: 0 additions & 72 deletions Bias Corrections/Precip/New/Trace21_2.5x2.5_PL_Error.txt

This file was deleted.

72 changes: 0 additions & 72 deletions Bias Corrections/Precip/Old/Trace21_2.5x2.5_PA_Error.txt

This file was deleted.

72 changes: 0 additions & 72 deletions Bias Corrections/Precip/Old/Trace21_2.5x2.5_PB_Error.txt

This file was deleted.

72 changes: 0 additions & 72 deletions Bias Corrections/Precip/Old/Trace21_2.5x2.5_PC_Error.txt

This file was deleted.

72 changes: 0 additions & 72 deletions Bias Corrections/Precip/Old/Trace21_2.5x2.5_PD_Error.txt

This file was deleted.

72 changes: 0 additions & 72 deletions Bias Corrections/Precip/Old/Trace21_2.5x2.5_PE_Error.txt

This file was deleted.

72 changes: 0 additions & 72 deletions Bias Corrections/Precip/Old/Trace21_2.5x2.5_PF_Error.txt

This file was deleted.

72 changes: 0 additions & 72 deletions Bias Corrections/Precip/Old/Trace21_2.5x2.5_PG_Error.txt

This file was deleted.

72 changes: 0 additions & 72 deletions Bias Corrections/Precip/Old/Trace21_2.5x2.5_PH_Error.txt

This file was deleted.

72 changes: 0 additions & 72 deletions Bias Corrections/Precip/Old/Trace21_2.5x2.5_PI_Error.txt

This file was deleted.

72 changes: 0 additions & 72 deletions Bias Corrections/Precip/Old/Trace21_2.5x2.5_PJ_Error.txt

This file was deleted.

72 changes: 0 additions & 72 deletions Bias Corrections/Precip/Old/Trace21_2.5x2.5_PK_Error.txt

This file was deleted.

72 changes: 0 additions & 72 deletions Bias Corrections/Precip/Old/Trace21_2.5x2.5_PL_Error.txt

This file was deleted.

Binary file removed Bias Corrections/Precip/Precip.7z
Binary file not shown.
72 changes: 0 additions & 72 deletions Bias Corrections/Precip/Precip/Trace21_2.5x2.5_PA_Error.txt

This file was deleted.

72 changes: 0 additions & 72 deletions Bias Corrections/Precip/Precip/Trace21_2.5x2.5_PB_Error.txt

This file was deleted.

72 changes: 0 additions & 72 deletions Bias Corrections/Precip/Precip/Trace21_2.5x2.5_PC_Error.txt

This file was deleted.

72 changes: 0 additions & 72 deletions Bias Corrections/Precip/Precip/Trace21_2.5x2.5_PD_Error.txt

This file was deleted.

72 changes: 0 additions & 72 deletions Bias Corrections/Precip/Precip/Trace21_2.5x2.5_PE_Error.txt

This file was deleted.

72 changes: 0 additions & 72 deletions Bias Corrections/Precip/Precip/Trace21_2.5x2.5_PF_Error.txt

This file was deleted.

72 changes: 0 additions & 72 deletions Bias Corrections/Precip/Precip/Trace21_2.5x2.5_PG_Error.txt

This file was deleted.

72 changes: 0 additions & 72 deletions Bias Corrections/Precip/Precip/Trace21_2.5x2.5_PH_Error.txt

This file was deleted.

72 changes: 0 additions & 72 deletions Bias Corrections/Precip/Precip/Trace21_2.5x2.5_PI_Error.txt

This file was deleted.

72 changes: 0 additions & 72 deletions Bias Corrections/Precip/Precip/Trace21_2.5x2.5_PJ_Error.txt

This file was deleted.

72 changes: 0 additions & 72 deletions Bias Corrections/Precip/Precip/Trace21_2.5x2.5_PK_Error.txt

This file was deleted.

72 changes: 0 additions & 72 deletions Bias Corrections/Precip/Precip/Trace21_2.5x2.5_PL_Error.txt

This file was deleted.

72 changes: 72 additions & 0 deletions Bias Corrections/RelativeHumidity/Trace21_2.5x2.5_HA_Modelled.txt

Large diffs are not rendered by default.

72 changes: 72 additions & 0 deletions Bias Corrections/RelativeHumidity/Trace21_2.5x2.5_HA_Observed.txt

Large diffs are not rendered by default.

72 changes: 72 additions & 0 deletions Bias Corrections/RelativeHumidity/Trace21_2.5x2.5_HB_Modelled.txt

Large diffs are not rendered by default.

72 changes: 72 additions & 0 deletions Bias Corrections/RelativeHumidity/Trace21_2.5x2.5_HB_Observed.txt

Large diffs are not rendered by default.

72 changes: 72 additions & 0 deletions Bias Corrections/RelativeHumidity/Trace21_2.5x2.5_HC_Modelled.txt

Large diffs are not rendered by default.

72 changes: 72 additions & 0 deletions Bias Corrections/RelativeHumidity/Trace21_2.5x2.5_HC_Observed.txt

Large diffs are not rendered by default.

72 changes: 72 additions & 0 deletions Bias Corrections/RelativeHumidity/Trace21_2.5x2.5_HD_Modelled.txt

Large diffs are not rendered by default.

72 changes: 72 additions & 0 deletions Bias Corrections/RelativeHumidity/Trace21_2.5x2.5_HD_Observed.txt

Large diffs are not rendered by default.

72 changes: 72 additions & 0 deletions Bias Corrections/RelativeHumidity/Trace21_2.5x2.5_HE_Modelled.txt

Large diffs are not rendered by default.

72 changes: 72 additions & 0 deletions Bias Corrections/RelativeHumidity/Trace21_2.5x2.5_HE_Observed.txt

Large diffs are not rendered by default.

72 changes: 72 additions & 0 deletions Bias Corrections/RelativeHumidity/Trace21_2.5x2.5_HF_Modelled.txt

Large diffs are not rendered by default.

72 changes: 72 additions & 0 deletions Bias Corrections/RelativeHumidity/Trace21_2.5x2.5_HF_Observed.txt

Large diffs are not rendered by default.

72 changes: 72 additions & 0 deletions Bias Corrections/RelativeHumidity/Trace21_2.5x2.5_HG_Modelled.txt

Large diffs are not rendered by default.

72 changes: 72 additions & 0 deletions Bias Corrections/RelativeHumidity/Trace21_2.5x2.5_HG_Observed.txt

Large diffs are not rendered by default.

72 changes: 72 additions & 0 deletions Bias Corrections/RelativeHumidity/Trace21_2.5x2.5_HH_Modelled.txt

Large diffs are not rendered by default.

72 changes: 72 additions & 0 deletions Bias Corrections/RelativeHumidity/Trace21_2.5x2.5_HH_Observed.txt

Large diffs are not rendered by default.

72 changes: 72 additions & 0 deletions Bias Corrections/RelativeHumidity/Trace21_2.5x2.5_HI_Modelled.txt

Large diffs are not rendered by default.

72 changes: 72 additions & 0 deletions Bias Corrections/RelativeHumidity/Trace21_2.5x2.5_HI_Observed.txt

Large diffs are not rendered by default.

72 changes: 72 additions & 0 deletions Bias Corrections/RelativeHumidity/Trace21_2.5x2.5_HJ_Modelled.txt

Large diffs are not rendered by default.

72 changes: 72 additions & 0 deletions Bias Corrections/RelativeHumidity/Trace21_2.5x2.5_HJ_Observed.txt

Large diffs are not rendered by default.

72 changes: 72 additions & 0 deletions Bias Corrections/RelativeHumidity/Trace21_2.5x2.5_HK_Modelled.txt

Large diffs are not rendered by default.

72 changes: 72 additions & 0 deletions Bias Corrections/RelativeHumidity/Trace21_2.5x2.5_HK_Observed.txt

Large diffs are not rendered by default.

72 changes: 72 additions & 0 deletions Bias Corrections/RelativeHumidity/Trace21_2.5x2.5_HL_Modelled.txt

Large diffs are not rendered by default.

72 changes: 72 additions & 0 deletions Bias Corrections/RelativeHumidity/Trace21_2.5x2.5_HL_Observed.txt

Large diffs are not rendered by default.

72 changes: 72 additions & 0 deletions Bias Corrections/SpecificHumidity/Trace21_2.5x2.5_SA_Delta.txt

Large diffs are not rendered by default.

72 changes: 72 additions & 0 deletions Bias Corrections/SpecificHumidity/Trace21_2.5x2.5_SB_Delta.txt

Large diffs are not rendered by default.

72 changes: 72 additions & 0 deletions Bias Corrections/SpecificHumidity/Trace21_2.5x2.5_SC_Delta.txt

Large diffs are not rendered by default.

72 changes: 72 additions & 0 deletions Bias Corrections/SpecificHumidity/Trace21_2.5x2.5_SD_Delta.txt

Large diffs are not rendered by default.

72 changes: 72 additions & 0 deletions Bias Corrections/SpecificHumidity/Trace21_2.5x2.5_SE_Delta.txt

Large diffs are not rendered by default.

72 changes: 72 additions & 0 deletions Bias Corrections/SpecificHumidity/Trace21_2.5x2.5_SF_Delta.txt

Large diffs are not rendered by default.

72 changes: 72 additions & 0 deletions Bias Corrections/SpecificHumidity/Trace21_2.5x2.5_SG_Delta.txt

Large diffs are not rendered by default.

72 changes: 72 additions & 0 deletions Bias Corrections/SpecificHumidity/Trace21_2.5x2.5_SH_Delta.txt

Large diffs are not rendered by default.

72 changes: 72 additions & 0 deletions Bias Corrections/SpecificHumidity/Trace21_2.5x2.5_SI_Delta.txt

Large diffs are not rendered by default.

72 changes: 72 additions & 0 deletions Bias Corrections/SpecificHumidity/Trace21_2.5x2.5_SJ_Delta.txt

Large diffs are not rendered by default.

72 changes: 72 additions & 0 deletions Bias Corrections/SpecificHumidity/Trace21_2.5x2.5_SK_Delta.txt

Large diffs are not rendered by default.

72 changes: 72 additions & 0 deletions Bias Corrections/SpecificHumidity/Trace21_2.5x2.5_SL_Delta.txt

Large diffs are not rendered by default.

82 changes: 58 additions & 24 deletions PaleoclimateToolDataFileHelper.py
Expand Up @@ -67,8 +67,7 @@ def __init__(self, application_gui=None) :

# Climate data parameters in presented order
self.data_parameters = ['mean_temperature', 'minimum_temperature', 'maximum_temperature',
'specific_humidity', 'relative_humidity',
'precipitation', 'sea_level_pressure']
'specific_humidity', 'relative_humidity', 'precipitation', 'sea_level_pressure']

# Parameter directory code map
self.parameter_directory_code_map = { 'mean_temperature' : 'T', 'minimum_temperature' : 'Tmin', 'maximum_temperature' : 'Tmax',
Expand Down Expand Up @@ -132,7 +131,7 @@ def __init__(self, application_gui=None) :
'precipitation' : 'Precip', 'sea_level_pressure' : 'MSLP' }
# Bias correction file code map
self.bias_correction_file_code_map = { 'mean_temperature' : 'Delta', 'minimum_temperature' : 'Delta', 'maximum_temperature' : 'Delta',
'specific_humidity' : 'Delta', 'relative_humidity' : 'Delta',
'specific_humidity' : 'Delta', 'relative_humidity' : ['Modelled', 'Observed'],
'precipitation' : 'Error', 'sea_level_pressure' : 'Delta' }

# Bias correction file template
Expand All @@ -146,7 +145,11 @@ def __init__(self, application_gui=None) :
'minimum_temperature' : 'climate_data_grids + bias_correction_data_grids',
'maximum_temperature' : 'climate_data_grids + bias_correction_data_grids',
'specific_humidity' : 'climate_data_grids*bias_correction_data_grids',
'relative_humidity' : '(1 - (1 - climate_data_grids/100)**bias_correction_data_grids)*100',
'relative_humidity' : ('(climate_data_grids <= bias_correction_data_grids[:,0])' + # Muncorr <= Mref
'* (climate_data_grids*(bias_correction_data_grids[:,1]/bias_correction_data_grids[:,0]))+' +
'(climate_data_grids > bias_correction_data_grids[:,0])' + # Muncorr > Mref
'* (bias_correction_data_grids[:,1]+((100-bias_correction_data_grids[:,1])/(100-bias_correction_data_grids[:,0]))*(climate_data_grids-bias_correction_data_grids[:,0]))'
),
'precipitation' : 'climate_data_grids*bias_correction_data_grids',
'sea_level_pressure' : 'climate_data_grids + bias_correction_data_grids' }

Expand Down Expand Up @@ -178,6 +181,8 @@ def setClimateDataSource(self, code) : # url or local

# Set climate data url
def setClimateDataUrl(self, url) :
if url :
url = url.rstrip()
if url and url[-1] != '/' :
url += '/'
self.climate_data_url = url
Expand Down Expand Up @@ -219,13 +224,28 @@ def getClimateDataUrl(self) :
# Check climate data url
def checkClimateDataUrl(self) :
try :
#print 'check 0', self.climate_data_url
current_url_file = url.urlopen((self.climate_data_url + 'current_url.txt'))
current_url = current_url_file.readline().rstrip()
current_url_file.close()
return True
if self.climate_data_url == current_url :
#print 'check 1', self.climate_data_url, current_url
return True
else :
self.setClimateDataUrl(current_url)
#print 'check 2'
return False
except Exception, e :
current_url_file = url.urlopen('http://homepage.cs.latrobe.edu.au/shaythorne/paleoview/current_url.txt')
self.climate_data_url = current_url_file.readline()
current_url_file.close()
self.setClimateDataUrl(current_url_file.readline().rstrip())
try :
current_first_url_file = url.urlopen((self.climate_data_url + 'current_url.txt'))
current_first_url_file.close()
#print 'check 3'
except Exception, e :
self.setClimateDataUrl(current_url_file.readline().rstrip()) # select second url
#print 'check 4'
current_url_file.close()
return False

# Climate data is present
Expand Down Expand Up @@ -847,21 +867,31 @@ def loadBiasCorrectionDataGrid(self, parameter, month_index) :
if self.cached_bias_correction_data_grids.has_key(parameter) and self.cached_bias_correction_data_grids[parameter].has_key(month_index) :
return self.cached_bias_correction_data_grids[parameter][month_index]
elif self.bias_correction_directory_code_map.has_key(parameter) :
data_file = self.bias_correction_file_template.replace('{bias_correction_directory_code}', self.bias_correction_directory_code_map[parameter])
data_file = data_file.replace('{parameter_file_code}', self.parameter_file_code_map[parameter])
data_file = data_file.replace('{month_code}', self.month_codes[month_index])
data_file = data_file.replace('{bias_correction_file_code}', self.bias_correction_file_code_map[parameter])
data_file = path.join(self.bias_correction_directory['path'], data_file)
if path.exists(data_file) :
data_grid = np.genfromtxt(data_file)
if not self.cached_bias_correction_data_grids.has_key(parameter) :
self.cached_bias_correction_data_grids[parameter] = {}
self.cached_bias_correction_data_grids[parameter][month_index] = data_grid
return data_grid
if type(self.bias_correction_file_code_map[parameter]) == list :
bias_correction_file_codes = self.bias_correction_file_code_map[parameter]
else :
bias_correction_file_codes = [self.bias_correction_file_code_map[parameter]]
data_grids = []
for bias_correction_file_code in bias_correction_file_codes :
data_file = self.bias_correction_file_template.replace('{bias_correction_directory_code}', self.bias_correction_directory_code_map[parameter])
data_file = data_file.replace('{parameter_file_code}', self.parameter_file_code_map[parameter])
data_file = data_file.replace('{month_code}', self.month_codes[month_index])
data_file = data_file.replace('{bias_correction_file_code}', bias_correction_file_code)
data_file = path.join(self.bias_correction_directory['path'], data_file)
if path.exists(data_file) :
data_grids.append(np.genfromtxt(data_file))
else :
#print 'TODO: handle missing climate data:', path.join(self.splitPath(split_path['directory'])['name'], split_path['name'])
exception_message = 'Could not find ' + parameter.replace('_', ' ').title() + ' bias correction data for ' + self.month_names[month_index] + '. Expected bias correction data file: \n' + data_file
raise Exception(exception_message)
if not self.cached_bias_correction_data_grids.has_key(parameter) :
self.cached_bias_correction_data_grids[parameter] = {}
if type(self.bias_correction_file_code_map[parameter]) == list :
self.cached_bias_correction_data_grids[parameter][month_index] = np.array(data_grids)
return data_grids
else :
#print 'TODO: handle missing climate data:', path.join(self.splitPath(split_path['directory'])['name'], split_path['name'])
exception_message = 'Could not find ' + parameter.replace('_', ' ').title() + ' bias correction data for ' + self.month_names[month_index] + '. Expected bias correction data file: \n' + data_file
raise Exception(exception_message)
self.cached_bias_correction_data_grids[parameter][month_index] = data_grids[0]
return data_grids[0]
else :
raise Exception('The bias correction data location for ' + parameter.replace('_', ' ').title() + ' has not been defined yet')

Expand Down Expand Up @@ -1093,7 +1123,8 @@ def convertAdIntervalToDataLabel(self, from_year_ad, until_year_ad) :
return from_year_str + '-' + until_year_str

# Method generates a NetCDF file for a climate data parameter for the specified interval
def generateNetCdfClimateData(self, parameter, from_year_ad, until_year_ad, min_year_ad=(1950-22000), max_year_ad=1989, zlib=True, decimals=None) :
def generateNetCdfClimateData(self, parameter, from_year_ad, until_year_ad, min_year_ad=(1950-22000),
max_year_ad=1989, zlib=True, decimals=None, correction_factor=1) :

# Construct NetCDF file path
data_interval_str = self.convertAdIntervalToDataLabel(from_year_ad, until_year_ad)
Expand All @@ -1111,7 +1142,7 @@ def generateNetCdfClimateData(self, parameter, from_year_ad, until_year_ad, min_
if decimals :
data_decimals = decimals
else :
data_decimals = len(first_line.split('.')[1].split(' '))
data_decimals = len(first_line.split('.')[1].split(' ')) # not working for Relative Humidity data: set decimals=6

# Construct NetCDF dataset file
rootgrp = Dataset(output_file_path, 'w')
Expand Down Expand Up @@ -1185,7 +1216,10 @@ def generateNetCdfClimateData(self, parameter, from_year_ad, until_year_ad, min_
data.units = self.parameter_unit_string[parameter]
data.long_name = parameter.replace('_',' ').title()
data.standard_name = parameter
data[:,:,:] = np.array(year_grids)
if correction_factor != 1 :
data[:,:,:] = np.array(year_grids)*correction_factor
else :
data[:,:,:] = np.array(year_grids)

if year_ad % 100 == 0 : ###
print year_str, strftime("%Y-%m-%d %H:%M", localtime())
Expand Down

0 comments on commit 1370615

Please sign in to comment.