Extend the timeout on initial validation of reboot tests.

[brendandburns]

An attempt to improve #14772

All flakes that I've seen recently are waiting for a pod to transition from (Running, ready=false) to (Running, read=true)

No reason to not just extend the timeout.

[brendandburns]

@fabioy @quinton-hoole

[ghost]

I'd be more inclined to increase podReadyBeforeTimeout (currently 20 sec) to, say 40 sec, for two reasons:

1. I can't see any reason why the pod startup time for these pods should be any different than other pods?
2. increasing the timeout 6x from 20 sec to 2 minutes seems excessive.

[brendandburns]

1. Done
2. Because if we make it long, we won't (hopefully) have to tune this again. As with other timeouts of this style, since we are polling, most of the time we won't burn that much time waiting for the condition to become true.

[ghost]

LGTM once comment addressed.
I'd suggest removing the "fixes" in the description, and closing #14772 only once we've seen 20 successive runs of this test (given that flakyness is only in the 10% range). I'll do that now and update description accordingly.

[brendandburns]

Comment addressed. Though gce-reboot is running closer to 50% flaky, so I'm ok w/ marking it closed after 5 clean runs...

--brendan

[k8s-bot]

Unit, integration and GCE e2e test build/test passed for commit f626b6ed38afec866d606f6c5d69f119772881d3.

• Build Log
• Test Artifacts
• Internal Jenkins Results

[k8s-github-robot]

Labelling this PR as size/XS

[ghost]

LGTM. As discussed, this will cause the minimum failure time for the test suite to be about 100 x 2 min (3.5 hours), but we can adjust the timeout down again as necessary in one central place.

[k8s-bot]

Unit, integration and GCE e2e test build/test passed for commit 09337d1.

• Build Log
• Test Artifacts
• Internal Jenkins Results

[fabioy]

For better or worse, reboot tests complete in around ~30 min right now. With this change, a failed or flaky test could stop merges for 7 hours (a failed run followed by a success)? Ick...

[fabioy]

NVM. Forgot that Jenkins let you configure build time limit, so there's that extra control knob (currently at 90 min).

Should I go ahead and merge this manually?

[ghost]

Yup
On Sep 29, 2015 6:08 PM, "Fabio Yeon" notifications@github.com wrote:

NVM. Forgot that Jenkins let you configure build time limit, so there's
that extra control knob (currently at 90 min).

Should I go ahead and merge this manually?

—
Reply to this email directly or view it on GitHub
#14784 (comment)
.

[fabioy]

Manually merging this PR.