End to end tests: hab-svc-load and hup-does-not-abandon-services #6849

baumanj · 2019-08-14T00:08:11Z

Add/fix these two tests for our end-to-end CI. This will merge back into #6780.

See also #6816

chef-expeditor · 2019-08-14T00:08:28Z

Hello baumanj! Thanks for the pull request!

Here is what will happen next:

Your PR will be reviewed by the maintainers.
If everything looks good, one of them will approve it, and your PR will be merged.

Thank you for contributing!

christophermaier

Looks good, but it would be good to address the race condition.

christophermaier · 2019-08-15T15:01:43Z

.expeditor/end_to_end.pipeline.yml

+  - label: "[:linux: hup-does-not-abandon-services]"
+    command:
+      - .expeditor/scripts/setup_environment.sh DEV
+      - hab pkg install --binlink core/expect/5.45.4/20190115014137


The reason the fully-qualified ident is needed here is because the environment contains HAB_BLDR_CHANNEL=DEV (see line 13 above), and there isn't a core/expect in that channel.

Since the exact release of expect isn't really important here, we could simplify this to

hab pkg install --binlink --channel=stable core/expect/5.45.4

or, simpler still, to

hab pkg install --binlink --channel=stable core/expect

What's the advantage? I don't think expect is likely to change in a way that's relevant to this test. Pinning it has the advantage to eliminating a potential difference when running this in different environments.

It's just that when I see a pin, I immediately wonder what the significance of that specific release is.

Fair enough. I'll try changing it.

christophermaier · 2019-08-15T15:06:08Z

test/end-to-end/hup-does-not-abandon-services.exp

@@ -99,7 +99,7 @@ expect {
        error "Redis not ready to accept connections!"
    }
 }
-set redis_pid [exec pgrep redis-server]
+set redis_pid [exec cat /hab/svc/redis/PID]


When I run this test locally, it will sometimes fail at this part; it looks like there might be a race between when the Supervisor writes this PID file and when the test tries to read it.

If I add a sleep 1 immediately before this check, for instance, I see no failures.

0d5487c should address this

That works.

Out of curiosity, what prompted you to move away from the pgrep approach to the PID file? Not having to worry about whether pgrep would be there?

As I mentioned in c53bfa1:

there may be other redis-servers running locally.

To wit:

jbauman@ubuntu:~/habitat$ pgrep -afl redis-server 77398 /opt/opscode/embedded/bin/redis-server 127.0.0.1:16379

christophermaier · 2019-08-15T15:18:35Z

test/end-to-end/hab-svc-load.exp

+sleep 1
+
+# Load up Redis on that Supervisor
+spawn hab svc load core/redis


Initially, I was surprised to see that this package installation was coming from stable and not DEV, but then I discovered that the HAB_BLDR_CHANNEL environment variable is ignored in hab svc load.

christophermaier · 2019-08-15T15:30:40Z

Incidentally, I think you can rebase your work directly on master and merge from there. I don't think any of your changes are dependent on the ones from my PR.

baumanj · 2019-08-15T18:32:04Z

Incidentally, I think you can rebase your work directly on master and merge from there. I don't think any of your changes are dependent on the ones from my PR.

Any reason not to merge back into your PR? All the e2e work is captured in the same issue, and there are some things in my changes that may be useful to the other e2e tests. Also, it'll just be less work for me to merge it as is since my branch was based on yours from the beginning.

christophermaier · 2019-08-15T22:27:49Z

Incidentally, I think you can rebase your work directly on master and merge from there. I don't think any of your changes are dependent on the ones from my PR.

Any reason not to merge back into your PR? All the e2e work is captured in the same issue, and there are some things in my changes that may be useful to the other e2e tests. Also, it'll just be less work for me to merge it as is since my branch was based on yours from the beginning.

If I were to rework and force-push my branch then that could mess yours up. They're not inherently coupled. If your PR has something I (or anyone else) can use, then I'll rebase mine when yours merges.

Some minor tweaks were necessary to the expect file. Notably, replace `pgrep redis-server` with accessing /hab/svc/redis/PID, since there may be other `redis-server`s running locally. Signed-off-by: Jon Bauman <5906042+baumanj@users.noreply.github.com>

Signed-off-by: Jon Bauman <5906042+baumanj@users.noreply.github.com>

We discovered #6852 as part of hup-does-not-abandon-services.exp, but that test is more complex. It's preferable to have a simpler, more targeted test as well which makes the nature of the failure (and potential regression) clearer. Signed-off-by: Jon Bauman <5906042+baumanj@users.noreply.github.com>

Signed-off-by: Jon Bauman <5906042+baumanj@users.noreply.github.com>

Signed-off-by: Christopher Maier <cmaier@chef.io>

Signed-off-by: Jon Bauman <5906042+baumanj@users.noreply.github.com>

baumanj · 2019-08-16T00:13:02Z

I think I've addressed all the feedback now.

scotthain · 2019-08-19T16:26:05Z

This is drive-by post-merge commentary that may have been resolved somewhere else, but it looks like ACCEPTANCE_HAB_BLDR_URL was changed to HAB_BLDR_URL and not all instances were change later in the pipeline.

baumanj force-pushed the end-to-end/hup-does-not-abandon-services branch 2 times, most recently from 99229bf to 1911efa Compare August 14, 2019 22:54

baumanj changed the base branch from master to cm/end-to-end August 14, 2019 23:25

baumanj force-pushed the end-to-end/hup-does-not-abandon-services branch from 9b38e38 to eab5307 Compare August 14, 2019 23:35

baumanj self-assigned this Aug 14, 2019

baumanj changed the title ~~End to end/hup does not abandon services~~ End to end tests: hab-svc-load and hup-does-not-abandon-services Aug 14, 2019

baumanj marked this pull request as ready for review August 14, 2019 23:39

baumanj added the Expeditor: Exclude from Changelog label Aug 14, 2019

baumanj requested review from christophermaier, nellshamrell, scotthain and smacfarlane August 14, 2019 23:39

christophermaier mentioned this pull request Aug 14, 2019

Add existing end-to-end tests to pipeline #6816

Closed

8 tasks

christophermaier suggested changes Aug 15, 2019

View reviewed changes

baumanj force-pushed the end-to-end/hup-does-not-abandon-services branch from eab5307 to 0d5487c Compare August 15, 2019 18:27

baumanj requested review from chefsalim, eeyun and raskchanky as code owners August 15, 2019 18:27

baumanj force-pushed the end-to-end/hup-does-not-abandon-services branch from 0d5487c to 91bc0d2 Compare August 15, 2019 22:56

baumanj changed the base branch from cm/end-to-end to master August 15, 2019 22:56

baumanj added 4 commits August 15, 2019 16:09

Add hab user in setup_environment.sh and turn on tracing

2d8419b

Signed-off-by: Jon Bauman <5906042+baumanj@users.noreply.github.com>

Wait for redis PID file to be created

aaac068

Signed-off-by: Jon Bauman <5906042+baumanj@users.noreply.github.com>

baumanj force-pushed the end-to-end/hup-does-not-abandon-services branch from 91bc0d2 to aaac068 Compare August 15, 2019 23:15

Further streamlining of setup_environment.sh

b217351

Signed-off-by: Christopher Maier <cmaier@chef.io>

baumanj added 2 commits August 15, 2019 16:48

Set HAB_BLDR_URL to acceptance for all of end-to-end pipeline

68d4e68

Signed-off-by: Jon Bauman <5906042+baumanj@users.noreply.github.com>

Simplify expect-based end-to-end test execution

b399808

Signed-off-by: Jon Bauman <5906042+baumanj@users.noreply.github.com>

baumanj force-pushed the end-to-end/hup-does-not-abandon-services branch from 51919f1 to b399808 Compare August 16, 2019 00:08

baumanj requested a review from christophermaier August 16, 2019 00:12

christophermaier approved these changes Aug 16, 2019

View reviewed changes

baumanj merged commit 003b835 into master Aug 16, 2019

baumanj deleted the end-to-end/hup-does-not-abandon-services branch August 16, 2019 19:31

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

End to end tests: hab-svc-load and hup-does-not-abandon-services #6849

End to end tests: hab-svc-load and hup-does-not-abandon-services #6849

baumanj commented Aug 14, 2019 •

edited

Loading

chef-expeditor bot commented Aug 14, 2019

christophermaier left a comment

christophermaier Aug 15, 2019

baumanj Aug 15, 2019

christophermaier Aug 15, 2019

baumanj Aug 15, 2019

christophermaier Aug 15, 2019

baumanj Aug 15, 2019

christophermaier Aug 15, 2019

baumanj Aug 15, 2019

christophermaier Aug 15, 2019

christophermaier commented Aug 15, 2019

baumanj commented Aug 15, 2019

christophermaier commented Aug 15, 2019

baumanj commented Aug 16, 2019

scotthain commented Aug 19, 2019

End to end tests: hab-svc-load and hup-does-not-abandon-services #6849

End to end tests: hab-svc-load and hup-does-not-abandon-services #6849

Conversation

baumanj commented Aug 14, 2019 • edited Loading

chef-expeditor bot commented Aug 14, 2019

christophermaier left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

christophermaier commented Aug 15, 2019

baumanj commented Aug 15, 2019

christophermaier commented Aug 15, 2019

baumanj commented Aug 16, 2019

scotthain commented Aug 19, 2019

baumanj commented Aug 14, 2019 •

edited

Loading