Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

sharness: test_under_flux: add fatal error if modules loaded during test aren't unloaded at test completion #1175

Merged
merged 2 commits into from Aug 31, 2017

Conversation

Projects
None yet
4 participants
@grondo
Copy link
Contributor

grondo commented Aug 31, 2017

This PR adds a minor enhancement to test_under_flux to detect when modules are not unloaded that were loaded manually during the course of the tests as suggested in #1020.

Unfortunately, the only place that we can insert code at test completion is by adding another "final cleanup" hook. These run after the final test summary, so this is not ideal, however we can still get the test overall to exit with non-zero status, and a fatal error message emitted to stdout so that it appears in output and logfiles.

grondo@flux-core:~/workspace/t (test_under_flux) $ cat test.t
#!/bin/sh
test_description="test_under_flux with module loaded, but not unloaded"
. ./sharness.sh
test_under_flux 2 minimal
test_expect_success "flux module load kvs" "
  flux module load -r 0 kvs
"
test_done
grondo@flux-core:~/workspace/t (test_under_flux) $ ./test.t
ok 1 - flux module load kvs
# passed all 1 test(s)
1..1
error: manually loaded module(s) not unloaded: kvs
2017-08-31T15:18:42.249361Z broker.err[0]: Run level 2 Exited with non-zero status (rc=1) 0.1s
lt-flux-broker: module 'kvs' was not cleanly shutdown
flux-start: 0 (pid 428474) exited with rc=1

A test is added to t0001-basic.t to ensure imbalanced module load is detected.

grondo added some commits Aug 31, 2017

flux-sharness.sh: detect failure to unload modules
Detect failure to unload modules when test_under_flux is used
and modules are loaded manually during testing. Failure to unload
modules can cause lack of coverage and is considered an error.

Unfortunately, the only practical way to generate an error is
to add a new "cleanup" hook, and fail during this hook with an
"exit 1" if any modules are loaded that were not also loaded
initially (these will presumably be unloaded during rc3).

Fixes #1020
t0001-basic.t: add test for module unload failure detection
Add a test to detect that test_under_flux can detect when modules
loaded during testing are not unloaded after test_done.

@grondo grondo force-pushed the grondo:test_under_flux branch from 8ac6543 to 9ede5d6 Aug 31, 2017

@coveralls

This comment has been minimized.

Copy link

coveralls commented Aug 31, 2017

Coverage Status

Coverage increased (+0.01%) to 78.131% when pulling 9ede5d6 on grondo:test_under_flux into 2eb43ea on flux-framework:master.

@codecov-io

This comment has been minimized.

Copy link

codecov-io commented Aug 31, 2017

Codecov Report

Merging #1175 into master will decrease coverage by <.01%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master    #1175      +/-   ##
==========================================
- Coverage   77.75%   77.74%   -0.01%     
==========================================
  Files         158      158              
  Lines       29266    29263       -3     
==========================================
- Hits        22756    22751       -5     
- Misses       6510     6512       +2
Impacted Files Coverage Δ
src/common/libflux/mrpc.c 85.15% <0%> (-1.18%) ⬇️
src/broker/module.c 83.28% <0%> (-0.28%) ⬇️
src/modules/kvs/kvs.c 62.72% <0%> (-0.26%) ⬇️
src/common/libkvs/kvs.c 64.62% <0%> (-0.25%) ⬇️
src/common/libflux/handle.c 83.66% <0%> (-0.25%) ⬇️
src/common/libflux/message.c 81.29% <0%> (ø) ⬆️
src/cmd/flux-module.c 84.45% <0%> (+0.3%) ⬆️
src/broker/overlay.c 72.02% <0%> (+0.34%) ⬆️
src/common/libcompat/handle.c 85.34% <0%> (+0.36%) ⬆️
src/common/libkvs/kvs_watch.c 87.22% <0%> (+0.88%) ⬆️
... and 1 more
@coveralls

This comment has been minimized.

Copy link

coveralls commented Aug 31, 2017

Coverage Status

Coverage decreased (-0.007%) to 78.11% when pulling 9ede5d6 on grondo:test_under_flux into 2eb43ea on flux-framework:master.

@grondo

This comment has been minimized.

Copy link
Contributor Author

grondo commented Aug 31, 2017

One occurrence of write error. Restarted.

@garlick

This comment has been minimized.

Copy link
Member

garlick commented Aug 31, 2017

Nice enhancement.

@garlick garlick merged commit e255d0f into flux-framework:master Aug 31, 2017

3 of 4 checks passed

codecov/project 77.74% (-0.01%) compared to 2eb43ea
Details
codecov/patch Coverage not affected when comparing 2eb43ea...9ede5d6
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
coverage/coveralls Coverage decreased (-0.007%) to 78.11%
Details

@grondo grondo deleted the grondo:test_under_flux branch Oct 25, 2017

@grondo grondo referenced this pull request May 10, 2018

Closed

0.9.0 Release #1479

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.