HydraWrapper sometimes fails on all(?) hosts

[jaywink]

Sometimes, HydraWrapper fails to connect to hosts seemingly in bulk - something fails and then all the hosts fail. These are well working pods which is very improbable that they are all down at the same time.

I've had this for a long time remember seeing it in the logs maybe half a year ago) and reporting it now because just saw it in the diaspora-fr.org logs (ping @Flaburgan ).

[2015-10-10T12:23:29] WARN  PID-11675 TID-1893246424160 HydraWrapper: event=http_multi_fail sender_id=297 url=https://iliketoast.net/receive/public return_code=coul
dnt_resolve_host response_code=0
[2015-10-10T12:23:29] WARN  PID-11675 TID-169436084700 HydraWrapper: event=http_multi_fail sender_id=853 url=https://iliketoast.net/receive/users/7ed1555bc6ae03db r
eturn_code=couldnt_resolve_host response_code=0
[2015-10-10T12:23:29] WARN  PID-11675 TID-1893246424160 HydraWrapper: event=http_multi_fail sender_id=297 url=https://diaspora.lebarjack.com/receive/public return_c
ode=couldnt_connect response_code=0
[2015-10-10T12:23:29] WARN  PID-11675 TID-169436084700 HydraWrapper: event=http_multi_fail sender_id=853 url=https://joindiaspora.com/receive/users/eb2e0ec0ec0d04bc
 return_code=couldnt_resolve_host response_code=0
[2015-10-10T12:23:29] WARN  PID-11675 TID-1893246424160 HydraWrapper: event=http_multi_fail sender_id=297 url=https://social.mrzyx.de/receive/public return_code=cou
ldnt_resolve_host response_code=0
[2015-10-10T12:23:29] WARN  PID-11675 TID-169436084700 HydraWrapper: event=http_multi_fail sender_id=853 url=https://free-beer.ch/receive/users/c5937eaddec02679 ret
urn_code=couldnt_connect response_code=0
[2015-10-10T12:23:29] WARN  PID-11675 TID-1893246424160 HydraWrapper: event=http_multi_fail sender_id=297 url=https://diasp.org/receive/public return_code=couldnt_r
esolve_host response_code=0
[2015-10-10T12:23:29] WARN  PID-11675 TID-169436084700 HydraWrapper: event=http_multi_fail sender_id=853 url=https://diasp.org/receive/users/b580a76b92760595 return
_code=couldnt_resolve_host response_code=0
[2015-10-10T12:23:29] WARN  PID-11675 TID-1893246424160 HydraWrapper: event=http_multi_fail sender_id=297 url=https://free-beer.ch/receive/public return_code=couldn
t_resolve_host response_code=0
[2015-10-10T12:23:29] WARN  PID-11675 TID-1893246424160 HydraWrapper: event=http_multi_fail sender_id=297 url=https://diasp.eu/receive/public return_code=couldnt_re
solve_host response_code=0
[2015-10-10T12:23:29] WARN  PID-11675 TID-169436084700 HydraWrapper: event=http_multi_fail sender_id=853 url=https://free-beer.ch/receive/users/f021e6dde9f70a62 ret
urn_code=couldnt_resolve_host response_code=0

This is probably an upstream issue but I'd like to track it here and possibly identify how to log to upstream :)

iliketoast.net
Ubuntu 14.04.3 LTS
curl 7.35.0 (x86_64-pc-linux-gnu) libcurl/7.35.0 OpenSSL/1.0.1f zlib/1.2.8 libidn/1.28 librtmp/2.3
Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps pop3 pop3s rtmp rtsp smtp smtps telnet tftp
Features: AsynchDNS GSS-Negotiate IDN IPv6 Largefile NTLM NTLM_WB SSL libz TLS-SRP

diaspora-fr.org
Ubuntu 12.04.5 LTS
curl 7.32.0 (x86_64-pc-linux-gnu) libcurl/7.32.0 OpenSSL/1.0.1 zlib/1.2.3.4 libidn/1.23 libssh2/1.2.8 librtmp/2.3
Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps pop3 pop3s rtmp rtsp scp sftp smtp smtps telnet tftp
Features: AsynchDNS GSS-Negotiate IDN Largefile NTLM NTLM_WB SSL libz TLS-SRP

Any ideas to debug?

[jhass]

If you have a way to trigger it, https://github.com/diaspora/diaspora/blob/develop/config/diaspora.yml.example#L507-L510 Careful, that produces a metric ton of debug output.