Sleep before retrying SSH#establish_connection. #399
Conversation
This addresses a race condition where the remote instance might be restarting its sshd process and is therefore unavailable for a second or two.
| @@ -124,6 +124,7 @@ def establish_connection | |||
| rescue *rescue_exceptions => e | |||
| if (retries -= 1) > 0 | |||
| logger.info("[SSH] connection failed, retrying (#{e.inspect})") | |||
| sleep 1 | |||
There was a problem hiding this comment.
👍 but should this actually be a configurable? It feels dirty to hardcode it when we have such a nice config 😄
There was a problem hiding this comment.
That race condition is quite funny when your running low on memory or CPU.
sleep 1 gets in; then ssh closes and then you can log in actually.
There was a problem hiding this comment.
retries is currently hard coded, perhaps we should get this in to address the immediate issue, and create an issue to make both configurable. Of course there will be cases where we cannot save the run/connection.
|
@sethvargo, @damm: agreed, I'd like to make this tunable at least to Driver authors and possibly to end-users. This was a bit of a placeholder PR as @portertech and I were testing init support in the Docker driver today and found that sshd could restart just after the A tiny bit of cleanup, then should be ready to merge. |
|
@fnichol worst part is ssh restarting is expected behavior on Ubuntu. :( |
|
I believe this fixes #418. 👍 |
Sleep before retrying SSH#establish_connection.
|
I wish this fixed #418. I updated my bundle to git master and I'm still hanging on this. :( |
This addresses a race condition where the remote instance might be
restarting its sshd process and is therefore unavailable for a second or
two.