Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve exaqm_bias_correction_o3 solution to avoid failure #23

Open
lgannoaa opened this issue Mar 4, 2024 · 0 comments
Open

Improve exaqm_bias_correction_o3 solution to avoid failure #23

lgannoaa opened this issue Mar 4, 2024 · 0 comments

Comments

@lgannoaa
Copy link

lgannoaa commented Mar 4, 2024

Job exaqm_bias_correction_o3 00Z running in AQM implementation ecflow parallel for 2024030400 failed.

Request for developer to implement the following fixes:
(1) Developer should use production utility "cpreq" instead of "cp" in exaqm_bias_correction_o3.sh
(2) exaqm_bias_correction_o3.sh[222] should copy data used for the job from its DATA location instead of from dcom again. Because those files were checked, used, and copied from dcom to DATA. Therefore, at the end of the job. It should be copy from DATA to ensure it is actually a true copy of the file been processed in the current job.
(3) exaqm_bias_correction_o3.sh[149] only give warning message. But line 213 copy those files with no exception handling logic to avoid file not presented.

Debug information follow:
line 4342 show:
exaqm_bias_correction_o3.sh[149] statement:
cp ${DCOMINairnow}/${PDYm1}/airnow/HourlyAQObs_${PDYm1}*.dat ${COMOUTbicor}/bcdata.${yyyymm_m1}/airnow/csv/${yyyy_m1}/${PDYm1}
Failed

  • exaqm_bias_correction_o3.sh[222]: cp ...
    cp: cannot stat '/lfs/h1/ops/prod/dcom/20240303/airnow/HourlyAQObs_2024030320.dat': No such file or directory
    Debug information:
    line 149:
    if [ "$(ls -A ${DCOMINairnow}/${cvt_pdy}/airnow)" ]; then
    cp ${DCOMINairnow}/${cvt_pdy}/airnow/HourlyAQObs_${cvt_pdy}.dat "${cvt_input_dir}/${cvt_yyyy}/${cvt_pdy}"
    else
    message_warning="WARNING: airnow data missing. skip this date ${cvt_pdy}"
    print_info_msg "${message_warning}"
    fi
    line 2883 show HourlyAQObs_2024030320.dat was found in statement:
    ls -A /lfs/h1/ops/prod/dcom/20240303/airnow/HourlyAQObs_${cvt_pdy}
    .dat

exaqm_bias_correction_o3.sh[149] statement:
cp ${DCOMINairnow}/${cvt_pdy}/airnow/HourlyAQObs_${cvt_pdy}*.dat "${cvt_input_dir}/${cvt_yyyy}/${cvt_pdy}"
has already copy files to
/lfs/h2/emc/stmp/lin.gan/aqm/ecflow_aqm/aqm_bias_correction_o3_00.120073811.cbqs01/data/bcdata.202403/airnow/csv/2024/20240303
-rw-r--r-- 1 lin.gan emc 1.1M Mar 4 02:08 HourlyAQObs_2024030320.dat

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant