tests: fix raciness in stop mode tests #5526
Merged
Conversation
This fixes the issue where shell disables signal handlers while running external commands. This was causing the signal handler established via "trap" do nothing at all while the signal was delivered during "sleep 1" process lifetime. Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>
This fixes this issue:
Traceback (most recent call last):
File "./start-stop-mode", line 49, in <module>
main()
File "./start-stop-mode", line 45, in main
time.sleep(1)
File "./start-stop-mode", line 29, in on_signal
print("got {}".format(signal.Signals(signum).name.lower()))
File "./start-stop-mode", line 29, in on_signal
print("got {}".format(signal.Signals(signum).name.lower()))
RuntimeError: reentrant call inside <_io.BufferedWriter name='<stdout>'>
Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>
This allows the test to either use system synchronization or hand-made synchronization but in either case, not be racy. Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>
Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>
I've seen the light now. I can sleep. Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>
Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>
Codecov Report
@@ Coverage Diff @@
## master #5526 +/- ##
==========================================
+ Coverage 79.06% 79.29% +0.23%
==========================================
Files 514 514
Lines 38949 39492 +543
==========================================
+ Hits 30795 31316 +521
- Misses 5681 5688 +7
- Partials 2473 2488 +15
Continue to review full report at Codecov.
|
|
As a general comment, the test still suffers some issues but we suspect those are in our general log extracting helpers. We've decided to merge this as is as it fixes real issues. Investigation of the log extractor bug continues in the branch referenced by mvo above. |
|
Looks good, thank you! Much nicer than the original shell version. |
mvo5
added a commit
to mvo5/snappy
that referenced
this pull request
Jul 19, 2018
Disable for now, until snapcore#5526 gets merged.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
This branch fixes raciness in the stop-mode tests. This test was failing for
the following reasons:
The first issue is fixed with a "stamp" like file that we look for in the test.
This ensures that before we attempt to stop the service we wait for it to
initialize fully, that is to register the signal handlers.
The second issue is fixed by re-writing the test in Python 3. This requires
some more explanation. We had established signal handlers using the "trap"
shell builtin and were expecting them to run when a signal (sent by systemd as
a result of stopping the service) arrives. What we can see by running the shell
script with "strace -e %signal" indicates that shell disables trap handlers
before executing any command. A trivial test involved increasing the sleep time
to a large value and sending the expected signal at that time. The signal
handler never runs in that case.
This is fixed by writing the test service in Python and not relying on
additional processes. This has the expected semantics and works well in
practice.
Shell is tricky, more than we though, and we keep learning this the hard way.
Signed-off-by: Zygmunt Krynicki zygmunt.krynicki@canonical.com