Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cads for andrew #616

Merged
merged 80 commits into from
Dec 4, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
80 commits
Select commit Hold shift + click to select a range
b83d543
Added co2_slicing technique as a subroutine to qc_irsnd. Needed to a…
wx20jjung Jun 21, 2022
8c47ad4
Added instrument flags to qc_irsnd subroutine for the new co2_slicing…
wx20jjung Jun 21, 2022
24db10c
Added comments and documentation for the two new subroutines CO2_clou…
wx20jjung Jun 23, 2022
096aeb4
Change surface channel test back to cris and ocean only after testing.
wx20jjung Jun 24, 2022
066d95b
Some structural changes to the new CO2_slicing subroutine
wx20jjung Aug 1, 2022
1f4415a
Added quality control procedures to read_cris for the new CO2_slicing…
wx20jjung Aug 1, 2022
1ae6050
Added quality control procedures to read_iasi for the new CO2_slicing…
wx20jjung Aug 1, 2022
b224b43
Added flags to use the co2_cloud_detect for cris and iasi instruments.
wx20jjung Sep 2, 2022
ab3424b
added flags to namelist obsqc to use co2_cloud_detect instead of defa…
wx20jjung Sep 2, 2022
2d317a8
Minor tuning to the cloud thresholds for iasi and cris based on the V…
wx20jjung Sep 2, 2022
0e7986e
Modified to look for channels used by CO2_cloud_detect in any CrIS su…
wx20jjung Sep 15, 2022
caedef8
Added logic to check for required CO2_cloud_detect channels in any IA…
wx20jjung Sep 15, 2022
9506020
Modified to look for channels used by CO2_cloud_detect in any AIRS s…
wx20jjung Sep 15, 2022
f2beb83
Added logic so that specific channels used by CO2_cloud_detect must b…
wx20jjung Sep 15, 2022
d6a702f
This commit contains a bug fix to the CO2_cloud_detect subroutine. E…
wx20jjung Oct 13, 2022
fd3bf06
Tuning of the co2_slicing thresholds. Other cosmetic changes.
wx20jjung Oct 24, 2022
ecbfc2e
Changes were added to derive channel height assignments for the new c…
wx20jjung Nov 21, 2022
f95f2ce
Namelist variables were added to call the cloud_and_Aerosol_Detection…
wx20jjung Nov 21, 2022
acc615b
The addition of chan_level in crtm_interface.f90 (the call_crtm() sub…
wx20jjung Nov 21, 2022
11d92fa
chan_level is a pass-through variable in setuprad and is sent to subr…
wx20jjung Nov 21, 2022
ea25e3d
A new module was added containing the cloud_and_aerosol_detection sof…
wx20jjung Nov 21, 2022
967e656
Extensive changes were made to the subroutine qc_irsnd. This version…
wx20jjung Nov 21, 2022
e38a3f0
This is a new module containing the subroutines for the new cloud_and…
wx20jjung Nov 21, 2022
8ca9a66
Reorganized various subroutines for CADS. Moved CADS setup to gsimod.…
wx20jjung May 2, 2023
de56c94
Added code to read the AVHRR cluster (IASI) and the VIIRS cluser (CrI…
wx20jjung May 2, 2023
7ebf0fe
Added the CADS namelist files to this script. They should be located…
wx20jjung May 2, 2023
21b7dcd
Merge remote-tracking branch 'emc/develop' into jung_cloud_tests_dr
wx20jjung May 2, 2023
3d20ace
Removed the CO2_Slicing code and various comments.
wx20jjung May 2, 2023
aa7024d
These changes incorporate the imager cluster information from IASI an…
wx20jjung Jun 16, 2023
9d8a7f7
Exponent consolidation for the VIIRS cluster information. Does not c…
wx20jjung Jun 26, 2023
ef37d7a
Cleaned up setuprad code and changed some variable names to include i…
wx20jjung Aug 7, 2023
a274631
Added some quality control steps for when the imager (viirs, avhrr) c…
wx20jjung Aug 7, 2023
f4e119c
modified read_prepbufr with Russ' change to use bufr library v12.0
wx20jjung Aug 7, 2023
de3e664
Removes changes needed to use bufrlib_v12.0
wx20jjung Aug 30, 2023
2fb0a00
merge commit
wx20jjung Aug 30, 2023
5938dd0
Changes to qcmod to fix conflict with merge
wx20jjung Aug 31, 2023
97e3fc0
Made changes to qcmod.f90 and setuprad.f90 to fix conflicts from the …
wx20jjung Aug 31, 2023
084eafc
Added test for the existence of the imager (VIIRS or AVHRR) spectral …
wx20jjung Sep 4, 2023
4717c1e
Added logic for j2 (n21) for viirs.
wx20jjung Sep 5, 2023
b405117
Potential bug fix
wx20jjung Sep 8, 2023
4492129
Removed un-used variables as identified by the debug compiler flag
wx20jjung Sep 8, 2023
567c511
Merge branch 'NOAA-EMC:develop' into CADS_for_Andrew
wx20jjung Sep 8, 2023
af86c2f
Updating to latest develop
wx20jjung Sep 8, 2023
cb94211
Merge branch 'CADS_for_Andrew' of https://github.com/wx20jjung/GSI in…
wx20jjung Sep 8, 2023
ad6c011
Update fix submodule to 6a42a29
RussTreadon-NOAA Sep 8, 2023
d8bf8c9
Merge remote-tracking branch 'emc/develop' into CADS_for_Andrew
wx20jjung Sep 14, 2023
9d436fa
Merge remote-tracking branch 'emc/develop' into CADS_for_Andrew
wx20jjung Sep 15, 2023
e6d5e34
removed various comments
wx20jjung Sep 17, 2023
fd99173
removed various comments
wx20jjung Sep 17, 2023
e712434
fixed a bug with identifier Obs_Time and revoved various comments.
wx20jjung Sep 17, 2023
93d68b5
Merge remote-tracking branch 'emc/develop' into CADS_for_Andrew
wx20jjung Sep 20, 2023
1196943
Changed some CADS write statements to use verbose (print only when ve…
wx20jjung Sep 21, 2023
d9284b0
Removed write statements from the sensor parameter read section.
wx20jjung Sep 21, 2023
ebd2989
Merge remote-tracking branch 'emc/develop' into CADS_for_Andrew
wx20jjung Sep 22, 2023
b3b02f7
Two bug fixes both are array out of bounds issues. One with CADS in …
wx20jjung Sep 26, 2023
f07a4e9
Merge remote-tracking branch 'emc/develop' into CADS_for_Andrew
wx20jjung Sep 26, 2023
06f953f
Fixed openmp
wx20jjung Oct 3, 2023
6df65e7
Changes made to the emc_legacy_cloud_detect
wx20jjung Oct 3, 2023
67014cf
re-arranged some of the imager cloud detection code.
wx20jjung Oct 3, 2023
c88c1c6
Merge remote-tracking branch 'emc/develop' into CADS_for_Andrew
wx20jjung Oct 3, 2023
dd24102
fixed avhrr and viirs cloud detection bug
wx20jjung Oct 6, 2023
5e774ea
Merge remote-tracking branch 'emc/develop' into CADS_for_Andrew
wx20jjung Oct 6, 2023
2383ead
Put if "CADS" around the calculaton of chan_level.
wx20jjung Oct 6, 2023
d3fbfe7
Restructured the infrared cloud derivation section in qc_irsnd.
wx20jjung Oct 6, 2023
c2e5652
rearranged the cads - legacy cloud detection section to be more effic…
wx20jjung Oct 12, 2023
7ec4619
Returned read_bufrtovs.f90 to the develop version.
wx20jjung Oct 12, 2023
b04d004
Merge remote-tracking branch 'emc/develop' into CADS_for_Andrew
wx20jjung Oct 12, 2023
099b90f
Returned abi2km to original coding. There are indexing problems and …
wx20jjung Oct 13, 2023
a436130
More changes to the way the imager cluster information is handled for…
wx20jjung Oct 25, 2023
8ba50c8
Moved dtime_setup() to above the CADS section.
wx20jjung Oct 25, 2023
ed9cad7
Merge remote-tracking branch 'emc/develop' into CADS_for_Andrew
wx20jjung Oct 25, 2023
e4646fd
Merge remote-tracking branch 'emc/develop' into CADS_for_Andrew
wx20jjung Nov 2, 2023
1d1715b
Changed a section of CADS code inside setuprad to a subdirectory.
wx20jjung Nov 7, 2023
3858863
Merge remote-tracking branch 'emc/develop' into CADS_for_Andrew
wx20jjung Nov 7, 2023
ac16db4
CADS for CrIS instruments requires band 1 ( channels 1 -713 ) to exis…
wx20jjung Nov 13, 2023
3968508
Moved the subroutine cads_imager_calc from setuprad.f90 to cads.f90 (…
wx20jjung Nov 21, 2023
7e250a6
Merge remote-tracking branch 'emc/develop' into CADS_for_Andrew
wx20jjung Nov 21, 2023
9c0a736
Code review changes from Dave Huber. In qcmod.f90, definition of tsi…
wx20jjung Nov 29, 2023
af80355
Merge remote-tracking branch 'emc/develop' into CADS_for_Andrew
wx20jjung Nov 29, 2023
88bf2ec
Merge remote-tracking branch 'upstream/develop' into CADS_for_Andrew
RussTreadon-NOAA Dec 1, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2,230 changes: 2,230 additions & 0 deletions src/gsi/cads.f90

Large diffs are not rendered by default.

18 changes: 16 additions & 2 deletions src/gsi/crtm_interface.f90
Original file line number Diff line number Diff line change
Expand Up @@ -977,7 +977,7 @@ end subroutine destroy_crtm
subroutine call_crtm(obstype,obstime,data_s,nchanl,nreal,ich, &
h,q,qs,clw_guess,ciw_guess,rain_guess,snow_guess,prsl,prsi, &
trop5,tzbgr,dtsavg,sfc_speed,&
tsim,emissivity,ptau5,ts, &
tsim,emissivity,chan_level,ptau5,ts, &
emissivity_k,temp,wmix,jacobian,error_status,tsim_clr,tcc, &
tcwv,hwp_ratio,stability,layer_od,jacobian_aero)
!$$$ subprogram documentation block
Expand Down Expand Up @@ -1097,6 +1097,7 @@ subroutine call_crtm(obstype,obstime,data_s,nchanl,nreal,ich, &
real(r_kind) ,intent( out) :: sfc_speed,dtsavg
real(r_kind),dimension(nchanl+nreal) ,intent(in ) :: data_s
real(r_kind),dimension(nchanl) ,intent( out) :: tsim,emissivity,ts,emissivity_k
real(r_kind),dimension(nchanl) ,intent( out) :: chan_level
character(10) ,intent(in ) :: obstype
integer(i_kind) ,intent( out) :: error_status
real(r_kind),dimension(nsig,nchanl) ,intent( out) :: temp,ptau5,wmix
Expand Down Expand Up @@ -1150,6 +1151,7 @@ subroutine call_crtm(obstype,obstime,data_s,nchanl,nreal,ich, &
real(r_kind):: sno00,sno01,sno10,sno11,secant_term
real(r_kind):: hwp_total,theta_700,theta_sfc,hs
real(r_kind):: dlon,dlat,dxx,dyy,yy,zz,garea
real(r_kind):: radiance, radiance_overcast, radiance_ratio
real(r_kind),dimension(0:3):: wgtavg
real(r_kind),dimension(nsig,nchanl):: omix
real(r_kind),dimension(nsig,nchanl,n_aerosols_jac):: jaero
Expand Down Expand Up @@ -2217,8 +2219,10 @@ subroutine call_crtm(obstype,obstime,data_s,nchanl,nreal,ich, &
end do
end if

chan_level = zero

!$omp parallel do schedule(dynamic,1) private(i) &
!$omp private(total_od,k,kk,m,term,ii,cwj)
!$omp private(total_od,k,kk,m,term,ii,cwj,radiance,radiance_overcast,radiance_ratio)
do i=1,nchanl
! Zero jacobian and transmittance arrays
do k=1,nsig
Expand All @@ -2228,6 +2232,16 @@ subroutine call_crtm(obstype,obstime,data_s,nchanl,nreal,ich, &
wmix(k,i)=zero
end do

radiance=rtsolution(i,1)%radiance
do k=msig, 1, -1
radiance_overcast = rtsolution(i,1)%upwelling_overcast_radiance(k)
radiance_ratio = abs(radiance_overcast/radiance)
if (radiance_ratio < 0.99_r_kind) then
chan_level(i) = atmosphere(1)%pressure(k) / r10
exit
endif
enddo

! Simulated brightness temperatures
tsim(i)=rtsolution(i,1)%brightness_temperature

Expand Down
1 change: 1 addition & 0 deletions src/gsi/gsi_files.cmake
Original file line number Diff line number Diff line change
Expand Up @@ -101,6 +101,7 @@ bkgvar_rewgt.f90
blacklist.f90
blendmod.f90
buddycheck_mod.f90
cads.f90
calc_fov_conical.f90
calc_fov_crosstrk.f90
calctends.f90
Expand Down
55 changes: 53 additions & 2 deletions src/gsi/gsimod.F90
Original file line number Diff line number Diff line change
Expand Up @@ -93,8 +93,14 @@ module gsimod
erradar_inflate,tdrerr_inflate,use_poq7,qc_satwnds,&
init_qcvars,vadfile,noiqc,c_varqc,gps_jacqc,qc_noirjaco3,qc_noirjaco3_pole,&
buddycheck_t,buddydiag_save,njqc,vqc,nvqc,hub_norm,vadwnd_l2rw_qc, &
pvis,pcldch,scale_cv,estvisoe,estcldchoe,vis_thres,cldch_thres,cao_check
pvis,pcldch,scale_cv,estvisoe,estcldchoe,vis_thres,cldch_thres,cao_check, &
cris_cads, iasi_cads, airs_cads
use qcmod, only: troflg,lat_c,nrand
use cads, only: M__Sensor,N__Num_Bands,N__GradChkInterval,N__Band_Size,N__Bands,N__Window_Width, &
N__Window_Bounds,R__BT_Threshold,R__Grad_Threshold,R__Window_Grad_Threshold, L__Do_Quick_Exit, &
L__Do_CrossBand, N__BandToUse,L__Do_Imager_Cloud_Detection, N__Num_Imager_Chans, &
N__Num_Imager_Clusters,N__Imager_Chans,R__Stddev_Threshold,R__Coverage_Threshold, &
R__FG_Departure_Threshold, CADS_Setup_Cloud
use pcpinfo, only: npredp,diag_pcp,dtphys,deltim,init_pcp
use jfunc, only: iout_iter,iguess,miter,factqmin,factqmax,superfact,limitqobs, &
factql,factqi,factqr,factqs,factqg, &
Expand Down Expand Up @@ -507,6 +513,9 @@ module gsimod
! 2. fv3_regional = .true.
! 3. fv3_cmaq_regional = .true.
! 4. berror_fv3_cmaq_regional = .true.
! 09-02-2022 Jung Added namelist entries to call a new IR cloud detection routine
! the original cloud detection routine is the default. To use the new
! cloud detection routine, set the flags to .true.
! 09-15-2022 yokota - add scale/variable/time-dependent localization
! 2023-07-30 Zhao - added namelist options for analysis of significant wave height
! (aka howv in GSI code): corp_howv, hwllp_howv
Expand Down Expand Up @@ -1051,6 +1060,13 @@ module gsimod
! wind observations.

! vad_near_analtime - assimilate newvadwnd obs around analysis time only
!
! Flags to use the new IR cloud detection routine. Flag must be set to true to use the new routine. The default
! (no flag or .false.) will use the default.
! airs_cads: use the clod and aerosool detection software for the AIRS instrument
! cris_cads: use the cloud and aerosol detection software for CrIS instruments
! iasi_cads: use the cloud and aerosol detection software for IASI instruments
!

namelist/obsqc/dfact,dfact1,erradar_inflate,tdrerr_inflate,oberrflg,&
vadfile,noiqc,c_varqc,blacklst,use_poq7,hilbert_curve,tcp_refps,tcp_width,&
Expand All @@ -1061,7 +1077,7 @@ module gsimod
q_doe_a_136,q_doe_a_137,q_doe_b_136,q_doe_b_137, &
t_doe_a_136,t_doe_a_137,t_doe_b_136,t_doe_b_137, &
uv_doe_a_236,uv_doe_a_237,uv_doe_a_213,uv_doe_b_236,uv_doe_b_237,uv_doe_b_213, &
vad_near_analtime
vad_near_analtime,airs_cads,cris_cads,iasi_cads

! OBS_INPUT (controls input data):
! dmesh(max(dthin))- thinning mesh for each group
Expand Down Expand Up @@ -1663,6 +1679,40 @@ module gsimod
! fac_tsl - index to apply thermal skin layer or not: 0 = no; 1 = yes.
namelist/nst/nst_gsi,nstinfo,zsea1,zsea2,fac_dtl,fac_tsl

! Initialize the Cloud and Aerosol Detection Software (CADS)
!
! M__Sensor Unique ID for sensor
! N__Num_Bands Number of channel bands
! N__Band_Size(:) Number of channels in each band
! N__Bands(:,:) Channel lists
! N__Window_Width(:) Smoothing filter window widths per band
! N__Window_Bounds(:,:) Channels in the spectral window gradient check
! N__GradChkInterval(:) Window width used in gradient calculation
! R__BT_Threshold(:) BT threshold for cloud contamination
! R__Grad_Threshold(:) Gradient threshold for cloud contamination
! R__Window_Grad_Threshold(:) Threshold for window gradient check in QE
! L__Do_Quick_Exit On/off switch for the Quick Exit scenario
! L__Do_CrossBand On/off switch for the cross-band method
! N__BandToUse(:) Band number assignment for each channel
! L__Do_Imager_Cloud_Detection On/off switch for the imager cloud detection
! N__Num_Imager_Chans No. of imager channels
! N__Num_Imager_Clusters No. of clusters to be expected
! N__Imager_Chans(:) List of imager channels
! R__Stddev_Threshold(:) St. Dev. threshold, one for each imager channel
! R__Coverage_Threshold Threshold for fractional coverage of a cluster
! R__FG_Departure_Threshold Threshold for imager FG departure

NAMELIST / Cloud_Detect_Coeffs / M__Sensor, N__Num_Bands, &
N__Band_Size, N__Bands, N__Window_Width, N__Window_Bounds, &
N__GradChkInterval, R__BT_Threshold, R__Grad_Threshold, &
R__Window_Grad_Threshold, L__Do_Quick_Exit, &
L__Do_CrossBand, N__BandToUse, &
L__Do_Imager_Cloud_Detection, N__Num_Imager_Chans, &
N__Num_Imager_Clusters, N__Imager_Chans, &
R__Stddev_Threshold, R__Coverage_Threshold, &
R__FG_Departure_Threshold


!EOC

!---------------------------------------------------------------------------
Expand Down Expand Up @@ -1749,6 +1799,7 @@ subroutine gsimain_initialize
call set_fgrid2agrid
call gsi_nstcoupler_init_nml
call init_radaruse_directDA
call CADS_Setup_Cloud

if(mype==0) write(6,*)' at 0 in gsimod, use_gfs_stratosphere,nems_nmmb_regional = ', &
use_gfs_stratosphere,nems_nmmb_regional
Expand Down
Loading