Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Changed to use run_and_check function (found in test_utils.bash) in #206

Merged
merged 2 commits into from
Apr 6, 2023

Conversation

ngs333
Copy link
Contributor

@ngs333 ngs333 commented Feb 17, 2023

This PR modifies test scripts that checked for answers or function status to use the new run_and_check() function
found in test_utils.bash. Said function was created in previous PR to echo $output if the status of the
executed command was not 0.

For an example of the output produced in the github CI actions/runs that encounter failures,
test 32 was forced to fail by giving it a threshold of zero on line 100. This line :
run_and_check expr ${var_34_0} \< 0.00001
was changed to this:
run_and_check expr ${var_34_0} \< 0
The output produced is:

FAIL: Test32-fregrid_no_stretched.sh 1 Test no stretched grid data lats 32.0 34.0 35.4
# Test32-fregrid_no_stretched.sh: (from function `run_and_check' in file /__w/FRE-NCtools/FRE-NCtools/t/test_utils.bash, line 95,
# Test32-fregrid_no_stretched.sh: in test file /__w/FRE-NCtools/FRE-NCtools/t/Test32-fregrid_no_stretched.sh, line 100)
# Test32-fregrid_no_stretched.sh: `run_and_check expr ${var_34_0} \< 0' failed
# Test32-fregrid_no_stretched.sh: 9.97102e-07
# Test32-fregrid_no_stretched.sh: 9.97102e-07
# Test32-fregrid_no_stretched.sh: failed: expr 0.000000997 < 0

most test scripts that checked for awnsers or function status.
@ngs333 ngs333 linked an issue Feb 17, 2023 that may be closed by this pull request
@ngs333 ngs333 marked this pull request as ready for review February 17, 2023 00:42
@ceblanton
Copy link
Contributor

With these updates the test failure details, either from make check with the VERBOSE=1 option or always in the test-suite.log, is improved.

The old message:

not ok 1 Test no stretched grid data lats 32.0 34.0 35.4
FAIL: Test32-fregrid_no_stretched.sh 1 Test no stretched grid data lats 32.0 34.0 35.4
# (in test file Test32-fregrid_no_stretched.sh, line 68) 
#   `expr ${var_32_0} \< 0' failed
# 9.97102e-07
# 0
ERROR: Test32-fregrid_no_stretched.sh - exited with status 1

compared to the new message:

not ok 1 Test no stretched grid data lats 32.0 34.0 35.4
FAIL: Test32-fregrid_no_stretched.sh 1 Test no stretched grid data lats 32.0 34.0 35.4
# (from function `run_and_check' in file test_utils.bash, line 90, 
#  in test file Test32-fregrid_no_stretched.sh, line 85) 
#   `run_and_check expr ${var_32_0} \< 0' failed
# 9.97102e-07
# failed: expr 0.000000997 < 0 
# 0
ERROR: Test32-fregrid_no_stretched.sh - exited with status 1

@ceblanton ceblanton merged commit 8df52bf into NOAA-GFDL:master Apr 6, 2023
@ceblanton ceblanton assigned ngs333 and unassigned ceblanton Apr 7, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Need to add echo "$output" to test scripts
2 participants