go DNS resolution might be a cause of acmetool fails to reconcile

[drbolsen]

[acmetool]

go version go1.7 linux/amd64 gc cgo=false
built by travis
git github.com/alecthomas/template a0175ee3bccc567396460bf5acd36800cb10c49c heads/master
git github.com/alecthomas/units 2efee857e7cfd4f3d0138cc3cbb1b4966962b93a heads/master
git github.com/coreos/go-systemd 43e4800a6165b4e02bb2a36673c54b230d6f7b26 v12
git github.com/godbus/dbus 9998ce8520caaa4a287fa5122ecb5d1ca1aeaede heads/master
git github.com/hlandau/acme a4d55ea51a8782633d7ca477d24c5da9a5c6147b v0.0.58
git github.com/hlandau/buildinfo b25d4b0e518fdb8bcbefaa3d52d77473bebe08fd heads/master
git github.com/hlandau/dexlogconfig 055e2e35f21ef605ada9e9af4e36597678678bf1 heads/master
git github.com/hlandauf/gspt 25f3bd3f5948489aa5f31c949310ae9f2b0e956c heads/master
git github.com/hlandau/goutils 0cdb66aea5b843822af6fdffc21286b8fe8379c4 heads/master
git github.com/hlandau/xlog 197ef798aed28e08ed3e176e678fda81be993a31 v1.0.0
git github.com/jmhodges/clock 880ee4c335489bc78d01e4d0a254ae880734bc15 heads/master
git github.com/mattn/go-isatty 66b8e73f3f5cda9f96b69efd03dd3d7fc4a5cdb8 heads/master
git github.com/mitchellh/go-wordwrap ad45545899c7b13c020ea92b2072220eefad42b8 heads/master
git github.com/ogier/pflag 45c278ab3607870051a2ea9040bb85fcb8557481 heads/master
git github.com/peterhellberg/link d1cebc7ea14a5fc0de7cb4a45acae773161642c6 v1.0.0
git github.com/satori/go.uuid 0aa62d5ddceb50dbcb909d790b5345affd3669b6 heads/master
git github.com/shiena/ansicolor a422bbe96644373c5753384a59d678f7d261ff10 heads/master
git github.com/square/go-jose d00415a0a4fdbcfbdf69deae1fe07fc953d9e76d heads/master
git golang.org/x/crypto 9e590154d2353f3f5e1b24da7275686040dcf491 heads/master
git golang.org/x/net 1358eff22f0dd0c54fc521042cc607f6ff4b531a heads/master
git gopkg.in/alecthomas/kingpin.v2 e9044be3ab2a8e11d4e1f418d12f0790d57e8d70 tags/v2.2.3
git gopkg.in/cheggaaa/pb.v1 9453b2db37f4d8bc63751daca63bbe7049eb5e74 v1.0.5
git gopkg.in/hlandau/configurable.v1 41496864a1fe3e0fef2973f22372b755d2897402 v1.0.1
git gopkg.in/hlandau/easyconfig.v1 bc5afaa18a1a72fe424da647d6bb57ca4d7f83c4 v1.0.14
git gopkg.in/hlandau/service.v2 601cce2a79c1e61856e27f43c28ed4d7d2c7a619 v2.0.15
git gopkg.in/hlandau/svcutils.v1 88dbd6d288dcde4c17ea6048d9f3da23f263571c v1.0.8
git gopkg.in/square/go-jose.v1 6e6dedab09005ebebd66b7cd4af9e8eada723c61 v1.0.4
git gopkg.in/tylerb/graceful.v1 50a48b6e73fcc75b45e22c05b79629a67c79e938 tags/v1.2.13
git gopkg.in/yaml.v2 e4d366fc3c7938e2958e662b4258c7a89e1f0e3e heads/v2

Just finished debuting a weird acmetool behaviour that was causing reconcile task to fail with the following error messages:

error satisfying Target(devops.xxx.xxx;https://acme-v01.api.letsencrypt.org/directory;0): Get https://acme-v01.api.letsencrypt.org/directory: dial tcp: lookup acme-v01.api.letsencrypt.org on 8.8.8.8:53: dial udp 8.8.8.8:53: i/o timeout

I believe the issue is related to golang/go#17329 and it seems that it has been resolved in GO 1.7.1

In our environment acmetool is running in a docker container and by default resolv.conf in the container is based on the host resolv.conf:

nameserver 10.0.0.4
nameserver 10.0.0.5
nameserver 8.8.8.8

With this configuration acmetool fails however when the container is set with --dns=8.8.8.8 which forces resolv.conf to a single name server entry:

nameserver 8.8.8.8

it works just fine.

I guess the issue is due to how go library treats any failure of any local DNS server and incorrectly reports it as a total fail to resolve DNS name with the last nameserver in the list. Totally misleading.

For time being we are using --dns flag for acmetool container as a quick fix, but planning to try to recompile acmetool with GO 1.7.1, hopefully it will fix the issue.

Cheers

[hlandau]

Thanks. Future builds will use Go 1.8.