Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[CI] Fix spuriously-failing end-to-end test #7103

Merged
merged 3 commits into from Oct 24, 2019
Merged

Conversation

@christophermaier
Copy link
Contributor

christophermaier commented Oct 24, 2019

No description provided.

These were missed in #7100 😦

Signed-off-by: Christopher Maier <cmaier@chef.io>
@christophermaier christophermaier self-assigned this Oct 24, 2019
@christophermaier christophermaier changed the title Fix spuriously-failing end-to-end test [CI] Fix spuriously-failing end-to-end test Oct 24, 2019
We were seeing failures for this test in CI, but not locally. I
hypothesize that this is because the test was waiting 5 seconds after
starting a Supervisor to observe the desired state. However, the bulk
of that time was actually occupied by downloading and installing
`core/hab-sup` and `core/hab-launcher` packages, which are not present
on the clean systems we test with. As a result, we were failing the
test unnecessarily.

Here, we preinstall these packages before starting the Supervisor and
entering our waiting period.

Additionally, we make some additional assertions about the nature of
the observed error by comparing the Supervisor's output with a desired
string. While this does couple the test to log output, it provides us
a bit more certainty that the test is actually testing what we want it
to, and not "passing" for some unexpected reason. The error message in
the event that this match no longer works should help guide developers
to a suitable fix quickly, should the need arise.

Signed-off-by: Christopher Maier <cmaier@chef.io>
Signed-off-by: Christopher Maier <cmaier@chef.io>
@christophermaier christophermaier force-pushed the cm/fix-e2e-test branch from 91a4f23 to 2a0dfdc Oct 24, 2019
@christophermaier christophermaier merged commit 37b4251 into master Oct 24, 2019
4 of 5 checks passed
4 of 5 checks passed
buildkite/habitat-sh-habitat-master-verify Build #3902 started
Details
DCO This commit has a DCO Signed-off-by
Details
buildkite/habitat-sh-habitat-master-website Build #982 passed (48 seconds)
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
expeditor/config-validation Validated your Expeditor config file
Details
@chef-expeditor chef-expeditor bot deleted the cm/fix-e2e-test branch Oct 24, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
2 participants
You can’t perform that action at this time.