Worksheet 1 correction from Tajikistan

[hsteptoe]

Following the Tajikistan workshop we found a few issues and have a few suggestions:

• Check that the ls cahpaa.pmi2???.pp pattern matching is correct. We ended up having to match to cahpaa.pmi2???.rr8.pp instead, but i’m not clear why.
• It would be helpful to add a pictorial explanation of rim removal, or maybe add one of the pictures from the presentations to help remind people why we need to remove the rim.
• More information is needed to explain why these steps are needed. The reasons for doing these steps weren’t always clear. Possible adding a quick plotting step at the end might be useful to explain what we can now do...?
• Make it clearer which parts are shell commands and which are python commands.
• Add a challenge question at the end for those participants who work quickly through this material. Can they repeat the process for a different set of data (eg. CORDEX)?

[SiTucker]

Here are my observations, to add to Hamish's list. For context, the Tajikistan participants had no/very little previous experience.

General comments:

• Some cells will be better split in to smaller ones, to make it easier to understand what is going on. In the first few worksheet cells, we should refrain from using loops (repeating cells is good, if it aids understanding). The first time loops (or any new coding feature) are introduced then they should be explained.
• A simple plot early on would be good for maintaining interest
• We should focus on explaining what is happening, rather than writing 'comprehensive' solutions that involve large blocks of code. This can wait until we have a python library where the code can be hidden in functions (teaching new people is a different activity to writing code for our own needs )
• We should try to make sure that we use informative variable names. For example, an english speaker might guess that 'fn' is short for filename, but its not obvious.

Specific comments:
Worksheets1.1:

%cd ..
%cd ..
!pwd
!ls
`
Its better to provide 4 empty cells here, rather than retyping in an old one

1.3. Rim removal. This code is far too complex to go into the first worksheet. Instead of removing the rim on all files, I suggest the following:

• Pick a variable where the 'rim effect' is particulary noticeable.
• Read this in to a cube, print it, and make a quick contour plot of this variable
• Use basic indexing to remove the rim (even if this means its not a general solution). e.g. tas_rr8=tas[:,8:-8, 8:-8]
• Make a plot of tas_rr8, so that participants can see the difference.
• Save tas_rr8 as a NetCDF file to illustrate how to do this

1.4. If we save the rim removed file as NetCDF, then I don't think they need this,

Worksheet2a

Section2.1
The following would be better as seven separate cells:
Cell1:
# Ensure you have iris 1.10 or greater
import iris

Cell2:
# Feed in the names of the directories where the netCDF model files are stored
path_in_cahpa = 'monthly/cahpa/05216/'
path_in_cahpb = 'monthly/cahpb/05216/'
path_in_APHRO = 'monthly/APHRODITE/'

Cell3:
# Load and print the HadCM3Q0 (cahpa) model cube data
cahpaData = iris.load_cube(path_in_cahpa + 'cahpaa.pm.6190.05216.rr8.nc')
print 'This is the HadCM3Q0 cube'
print cahpaData
print

Cell4:
# Load and print the ECHAM5 (cahpb) model cube data
cahpbData = iris.load_cube(path_in_cahpb + 'cahpba.pm.6190.05216.rr8.nc')
print 'This is the ECHAM5 cube'
print cahpbData
print

Cell5:
# Load and print the APHRODITE observation cube data
aphroData = iris.load_cube(path_in_APHRO + 'aphro.mon.6190.nc')
print 'This is the APHRODITE cube'
print aphroData
print

Cell6:
# The coord objects of the data cubes contain the co-ordinate points of the data
print 'This is the APHRODITE cube latitude coordinate data'
print aphroData.coord('latitude').points

Cell7
print 'This is the APHRODITE cube longitude coordinate data'
print aphroData.coord('longitude').points
`

We should try to give our cubes meaningful names 'aphroData' caused some confusion as to whether it was a cube, or just the data. In addition we should follow Carwyn's advice on the use of capital letters

There is a camelcase convention for class names, not objects.
https://www.python.org/dev/peps/pep-0008/#class-names
https://www.python.org/dev/peps/pep-0008/#function-and-variable-names

The purpose of this is so that you can easily distinguish between a class and an object within a namespace.

Example: I might import 'iris.cube.Cube' into my namespace and I might call something like 'mycube' in my code. 'Cube' is the class while 'mycube' is clearly the object. I can tell by this convention rather than relying on inspection.

Names with uppercase (not camelcase) are commonly global variables though I'm not entirely sure whether this is by explicit convention.

Re 2.2)
I would argue against using a loop here, but if we are then can 'fn' be replaced with something more informative?
I would vote instead of a loop to do it separately for present day and future files. For the second file, one big cell is OK, but for the first, I think splitting to five cells would be better:
Cell1:
Read in file
Cell 2:
Print units, and summary statistics about the data
Cell3:
Change units
Cell4:
Print units, and summary statistics about the data, post unit change
Cell5:
Save file

It may also be better to do this with temperature cube first. Temperature is more straightforward as we don't need to make assumptions re density of water (which makes the change units a two stage process)

I have similar comments for the rest of the worksheet.

[balazagi]

I have branched out (fix/worksheet2a) to do the changes proposed above, but I am not sure where the netcdf files we point at in the following locations come from?

Feed in the names of the directories where the netCDF model files are stored

path_in_cahpa = 'monthly/cahpa/05216/'
path_in_cahpb = 'monthly/cahpb/05216/'
path_in_APHRO = 'monthly/APHRODITE/'

are these somewhere in Mass or the product of some post-processing code?

[SiTucker]

They look like post processed PRECIS data. Perhaps its worth archiving them to MASS as backup

[balazagi]

They do indeed, but am I correct that the post-processing is done out of the worksheet context, i.e. we are not missing any steps that is meant to generate these files and they are only used for the purpose of demonstrating Iris cube operation in this worksheet?

[SiTucker]

Yes and no. The first worksheet shows how the files could be produced from PRECIS output, but the actual files we bring with us

[balazagi]

But the files generated in the first worksheet are named differently:

worsheet1: rr8_air_temperature_m01s03i236_cahpaa.pmi2jul.03236.nc
vs
worksheet2a: cahpaa.pm.6190.05216.rr8.nc

what does the 6190 in the filename signifies?

[balazagi]

Judging by the code in section 2.2 it seems that the content of the following directories
path_in_cahpa = 'monthly/cahpa/05216/'
path_in_cahpb = 'monthly/cahpb/05216/'
path_in_APHRO = 'monthly/APHRODITE/'

under these path: /data/users/ssadri/data-local/PRECIS_WORK/monthly are not correct. Could you please point me to the correct one or the ones used in the last workshop?

[balazagi]

I have implemented the changes proposed proposed by Simon to section 2.3. I have made a pull request for Hamish to review the changes so far but Simon if you have more suggestions for the rest of the worksheet please let me know. Personally I think we should move some of the remaining sections in worksheet2a to worksheet2b but not sure where would the best logical break up point should be!

[hsteptoe]

Have we now actioned all of the changes to Worksheet 1 following the Tajikistan trip? If so we can close this issue...