Investigate flaky test-https-connect-address-family.js and test-tls-connect-address-family

[gibfahn]

• Version:v6.2.1
• Platform:Linux (Ubuntu 14.04, 15.04, Debian 8, others?)
• Subsystem: test

See #6654 (comment) and nodejs/build#415

These tests were added in v6.2.1 and were marked as flaky due to a problem with resolving localhost on IPv6 on some Linux distros.

Basically /etc/hosts needs to contain the line ::1 localhost in order for localhost to be properly resolved. Most distros seem to have it (e.g. Ubuntu 16.04: ::1 localhost ip6-localhost ip6-loopback) but some, (e.g. Ubuntu 14.04/15.04 ::1 ip6-localhost ip6-loopback) don't. Given the number of people that will be testing their pull requests with make test, it seems important to make sure that this test passes on machines with other strings in their ::1 section.

As ./test/common.js contains this list of options, it should in theory work.

@bnoordhuis @jbergstroem I know you were/are looking into this, I just wanted to make sure we had an issue open for it.

[bnoordhuis]

As ./test/common.js contains this list of options, it should in theory work.

Theory != practice, unfortunately. The initial version of #6654 used that list but it only caused more CI fallout, not less.

[Trott]

FWIW, I think that once nodejs/build#415 happens, using the list of IPv6 local hostnames in common.js should become workable.

[mhdawson]

I have to agree this is quite annoying as this fails 100% on my ubuntu 14 box. The result is that other tests are not run, and you have to know to run things like make lint on your own. I wonder if the default runs should exclude all flaky tests ?

[gibfahn]

@Trott as I understand it, nodejs/build#415 means localhost will resolve to ::1 on the build machines, but it won't make a difference for anyone running make test on their own machine.

[Trott]

@gibm I think it will make the "try all the items in the IPv6 local hostnames in common.js" strategy work on CI. Which means we can implement that in the test, and that should fix it for people running make test. I could be wrong though.

[Trott]

Another option is to confirm that localhost maps to ::1 with something like this:

dns.lookup('localhost', {family:6, all: true}, (err, addresses) => {
  // code that skips the test if err is true or addresses does not contain { address: '::1', family: 6 }
  // otherwise, this kicks off the test
});

[Trott]

Simple workaround: #7605

Might be an argument for cycling through common.localIPv6Hosts. Counter-argument might be simplicity. Middle-ground: future enhancement if needed!