We were running into the issue the wait is meant to resolve so I tried it out. Looks like the logic got reversed somewhere along the way.
Fix logic for waitUntilContainerIsReady
@philk: Thanks for catching this. I tried that and it looks good!
@philk @iocanel @carlossg I'm not sure this is entirely fixed.
Before, I was also seeing the timeout issues as described in #93 (comment).
Now, I've got a situation where I've got some sh logic wrapped in a retry block, and I'm seeing the timeout error happen the first time the retry block is executed, and then once the block is retried, the script moves forward as expected:
Still waiting to schedule task
kubernetes-cbc4406893db41a9aef2d9e5d526fca1-6e9ef3076ae06 is offline
Running on kubernetes-cbc4406893db41a9aef2d9e5d526fca1-6e9ef3076ae06 in /home/jenkins/workspace/blendle_core-api_jenkins-JAM5PI6ZOUNKGTGYGMINDN4KNK2PFRGQPKWNXCRMGAOHJME4OVZA
Sleeping for 1 sec
[blendle_core-api_jenkins-JAM5PI6ZOUNKGTGYGMINDN4KNK2PFRGQPKWNXCRMGAOHJME4OVZA] Running shell script
Waiting for container container [instance-1] of pod [kubernetes-cbc4406893db41a9aef2d9e5d526fca1-6e9ef3076ae06] to become ready.
ERROR: Execution failed
java.io.IOException: Failed to execute shell script inside container [instance-1] of pod [kubernetes-cbc4406893db41a9aef2d9e5d526fca1-6e9ef3076ae06]. Timed out waiting for container to become ready!
Sleeping for 1 sec
... from here on out the build continued as expected
It's almost as if the watcher logic here never triggers the MODIFIED event, and thus it just waits for the timeout to finish.
The timeout does take quite a long time to happen (maybe 60 seconds?) before my retry logic kicks in.
@JeanMertz: You are right! The watcher implementation does not check the pod reference by the event, but the pod in its previous state (before) the watch! That will never work!
Let me fix that!
@JeanMertz: It should look like this: #97
Do you want to give it a spin?
Nice catch, we had a retry in our Jenkinsfile from before (when there was a race for the pod to become ready) so we didn't notice this. Thanks for the fix!