overlap_solution

Problem: station netcdf files are overlapping

Solution:

Files:

• rec_grep
• ncdate

there are two types of overlaps – ok-overlaps, where data is just repeated,

and not-that-ok-overlaps – these cases are due to the fact that ROMS doesn't stop when it runs out of atmospheric forcing

(about expired climatology it complains!). For me it happens sometimes when it progresses from December to January.

I can suggest an approach how to handle this issue – by making a concatenated time-records consistent file.

So here is the example:

-rw-r--r-- 1 mitya mitya 119M Jun 14 11:35 ocean_sta_2006_02_06.nc

-rw-r--r-- 1 mitya mitya  31M Jun 16 00:53 ocean_sta_2006_01_10.nc

I put these files into a test directory:

/global/work/apn/Arctic-4km_results/1993_2010_jens/test

ocean_sta_2006_02_06.nc -> 1.nc

ocean_sta_2006_01_10.nc -> 2.nc

you can use my python utilities from: /home/mitya/bin/

rec_grep, ncdate

[mitya@stallo-2 test]$ ncdate 1.nc

1.nc

ocean_time

2832 1129032000 1139223600

2005-10-11 12:00:00 2006-02-06 11:00:00

[mitya@stallo-2 test]$ ncdate 2.nc

2.nc

ocean_time

739 1134216000 1136872800

2005-12-10 12:00:00 2006-01-10 06:00:00

Thus you need to have the first file to be finished by 2005-12-10 11:00:00. To learn which record corresponds to this date_time (NB: my script is format sensitive):

[mitya@stallo-2 test]$ rec_grep -dt 2005-12-10 11:00:00 1.nc

you want to grep 2005-12-10 11:00:00

 

In the file 1.nc

variable ocean_time exists

(2832,)

a time record for the date_time  2005-12-10 11:00:00  exists

the record number is  1440

Then you may use an NCO tool:

[mitya@stallo-2 test]$ ncks -d ocean_time,0,1439 1.nc 1_cons.nc

The result:

[mitya@stallo-2 test]$ ncdate 1_cons.nc

1_cons.nc

ocean_time

1440 1129032000 1134212400

2005-10-11 12:00:00 2005-12-10 11:00:00

[mitya@stallo-2 test]$ ncrcat 1_cons.nc 2.nc 1_2_cons.nc

[mitya@stallo-2 test]$ ncdate 1_2_cons.nc

1_2_cons.nc

ocean_time

2179 1129032000 1136872800

2005-10-11 12:00:00 2006-01-10 06:00:00

1440+739=2179, I think it worked out.

P.S. you may need to load the following modules on stallo to make my python utilities work (they have python2.7.3 bundled with netcdf4):

module load nco netcdf ncview

P.P.S. all above can be put into a shell (or whatever) script – just an iteration over the time ordered list (ls –ltr ocean_sta*) is required I suppose.