Skip to content

Latest commit

 

History

History
187 lines (135 loc) · 4.7 KB

simQuickRedo.md

File metadata and controls

187 lines (135 loc) · 4.7 KB
Raw

Quick Restart

This document should help to quickly redo all analysis steps when new simulations are available.

Synsats (Mistral)

Calculation

  • Go to the synsat scripts directory
cd ${HOME}/proj/2015-12_synsat/synsats4icon_nawdex
  • Make a list (also change listname...)
EXPDIR=/work/bm0834/b380459/NAWDEX/ICON_OUTPUT_NWP/nawdexnwp-2km-mis-0002-shcon/
find ${EXPDIR} -iname '*fg*nc' | sort > lists/nawdexnwp-2km-mis-0002-shcon.lst
  • Split list into parts if runs should be distributed across several mistralpp nodes
split -d -l 48 nawdexnwp-2km-mis-0002-shcon.lst nawdexnwp-2km-mis-0002-shcon.lst
  • Open screen
screen -S synsat
. ~/.profile
  • Parallel synsat runs
 start_proc_from_list -n 4 ./synsat_icon-nawdex.py lists/nawdexnwp-2km-mis-0001-shcon.lst01
  • Wait

Regridding

After all synsat runs are finished: Regridding is done in parts (memory and speed issue)

  • go to the dir
cd ${HOME}/proj/2017-07_nawdex_analysis/inout
./save_reproj_synsat.py ${HOME}/data/synsat/nawdex/nawdexnwp-80km-mis-0001
  • make a list file that looks like this (default: 6 lines = 6 chunks). vim regrid_synsat10.lst
/pf/b/b380352/data/synsat/nawdex/nawdexnwp-2km-mis-0010-shcon 0 
/pf/b/b380352/data/synsat/nawdex/nawdexnwp-2km-mis-0010-shcon 1
/pf/b/b380352/data/synsat/nawdex/nawdexnwp-2km-mis-0010-shcon 2
/pf/b/b380352/data/synsat/nawdex/nawdexnwp-2km-mis-0010-shcon 3
/pf/b/b380352/data/synsat/nawdex/nawdexnwp-2km-mis-0010-shcon 4
/pf/b/b380352/data/synsat/nawdex/nawdexnwp-2km-mis-0010-shcon 5

  • open a screen screen -S regrid_synsat10

  • run

start_proc_from_list -n 6 ./save_reproj_synsat_in_parts.py regrid_synsat10.lst
  • goto the parts output dir
cd ~/data/nawdex/synsat-parts/
  • combine file parts with cdo
cdo -z zip_4 cat synsat-nawdexnwp-2km-mis-0010-shcon-part?.nc synsat-nawdexnwp-2km-mis-0010-shcon.nc
  • move the combine into the ../synsat directory

Combined Radiation Fluxes

Regridding (Mistral)

Similar to synsat regridding!!!

Regridding is done in parts (memory and speed issue)

  • go to the dir
cd ${HOME}/proj/2017-07_nawdex_analysis/inout/
  • make a list file that looks like this (default: 6 lines = 6 chunks). vim regrid_radflux10.lst
/work/bm0834/b380459/NAWDEX/ICON_OUTPUT_NWP/nawdexnwp-2km-mis-0010-shcon 0
/work/bm0834/b380459/NAWDEX/ICON_OUTPUT_NWP/nawdexnwp-2km-mis-0010-shcon 1
/work/bm0834/b380459/NAWDEX/ICON_OUTPUT_NWP/nawdexnwp-2km-mis-0010-shcon 2
/work/bm0834/b380459/NAWDEX/ICON_OUTPUT_NWP/nawdexnwp-2km-mis-0010-shcon 3
/work/bm0834/b380459/NAWDEX/ICON_OUTPUT_NWP/nawdexnwp-2km-mis-0010-shcon 4
/work/bm0834/b380459/NAWDEX/ICON_OUTPUT_NWP/nawdexnwp-2km-mis-0010-shcon 5

  • open a screen screen -S regrid_radflux10

  • run

start_proc_from_list -n 6 ./save_reproj_sim_raddata_in_parts.py regrid_radflux10.lst
  • goto the parts output dir
cd ~/data/nawdex/sim-toarad-parts/
  • combine file parts with cdo
cdo -z zip_4 cat toa_clear_radflux-nawdexnwp-2km-mis-0010-shcon-part?.nc toa_clear_radflux-nawdexnwp-2km-mis-0010-shcon.nc
cdo -z zip_4 cat toa_radflux-nawdexnwp-2km-mis-0010-shcon-part?.nc toa_radflux-nawdexnwp-2km-mis-0006-shcon.nc
  • move the combine into the ../sim-toarad directory

Transfer to TROPOS (Gauss)

Copy data to Gauss

cd ${LHOME}/data/icon/nawdex/sim-toarad
scp -r b380352@mistralpp.dkrz.de:/pf/b/b380352/data/nawdex/sim-toarad/*shcon.nc .

Cloud Typing (Gauss)

Transfer of Synsats to Gauss

Copy data to Gauss

cd  ${LHOME}/data/icon/nawdex/synsat/
scp -r b380352@mistralpp.dkrz.de:/pf/b/b380352/data/nawdex/synsat/*shcon.nc .

Run Cloud Typing

cd ${LHOME}/proj/2019-08_NWCSAF/synNWCSAF/scripts
./run_NWCSAF4synsat ${LHOME}/data/icon/nawdex/synsat/synsat-nawdexnwp-2km-mis-0001.nc

CRE (Gauss)

If regridded Radflux files & Synsat files are complete, then derived average rad-fluxes per CT (level3 data)

cd ${LHOME}/proj/2017-07_icon-nawdex/inout
./save_ave_radfluxes.py ${LHOME}/data/icon/nawdex/sim-toarad/toa_radflux-nawdexnwp-2km-mis-0001-shcon.nc

where the full filename refers to the regidded radflux file.

Adjusting the Naming Scheme

The new expermients might have new names, e.g. nawdexnwp-2km-mis-0001-shcon. The new experiment need to be mapped into experiment categories.

nawdex_analysis.io.selector has to be modified.

  • Go to the nawdex_analysis package development path
cd ${LHOME}/proj/2018-05_pypackage_devel/nawdex_analysis/nawdex_analysis/io
  • Modify selector.py
emacs selector.py
  • Change Version Number
cd ${LHOME}/proj/2018-05_pypackage_devel/nawdex_analysis/nawdex_analysis
vim _version.py
  • Install new package version into thew system
cd ..
pip install --upgrade .