autobuilds: new test 2600 failing on Solaris

[MarcelRaad]

I did this

Look at the autobuilds at https://curl.se/dev/builds.html.

I expected the following

All tests successful.

On the Solaris builders, test 2600 (added in d55de24) is failing, e.g. here:
https://curl.se/dev/log.cgi?id=20230214093837-2682677#prob112

The failures are:

/tests/unit/unit2600.c:219 test failed: '1: expected 1 ipv4 creations, but got 0'
/tests/unit/unit2600.c:230 test failed: '1: expected min duration of 200ms, but took 114ms'
[...]
/tests/unit/unit2600.c:219 test failed: '2: expected 2 ipv4 creations, but got 0'
/tests/unit/unit2600.c:230 test failed: '2: expected min duration of 400ms, but took 3ms'
[...]
/tests/unit/unit2600.c:224 test failed: '3: expected 1 ipv6 creations, but got 0'
/tests/unit/unit2600.c:230 test failed: '3: expected min duration of 200ms, but took 3ms'
[...]
/tests/unit/unit2600.c:224 test failed: '4: expected 2 ipv6 creations, but got 0'
/tests/unit/unit2600.c:230 test failed: '4: expected min duration of 400ms, but took 4ms'
[...]
/tests/unit/unit2600.c:219 test failed: '5: expected 1 ipv4 creations, but got 0'
/tests/unit/unit2600.c:224 test failed: '5: expected 1 ipv6 creations, but got 0'
/tests/unit/unit2600.c:230 test failed: '5: expected min duration of 350ms, but took 4ms'
[...]
/tests/unit/unit2600.c:219 test failed: '6: expected 1 ipv4 creations, but got 0'
/tests/unit/unit2600.c:224 test failed: '6: expected 1 ipv6 creations, but got 0'
/tests/unit/unit2600.c:230 test failed: '6: expected min duration of 350ms, but took 4ms'
[...]
/tests/unit/unit2600.c:224 test failed: '7: expected 3 ipv6 creations, but got 0'
/tests/unit/unit2600.c:230 test failed: '7: expected min duration of 350ms, but took 4ms'

Info from configure

Host setup: sparc-sun-solaris2.10
Install prefix: /usr/local
Compiler: /opt/csw/bin/gcc
CFLAGS: -Werror-implicit-function-declaration -g -O0 -std=gnu89 -pedantic -Wall -W -Wpointer-arith -Wwrite-strings -Wunused -Wshadow -Winline -Wnested-externs -Wmissing-declarations -Wmissing-prototypes -Wno-long-long -Wbad-function-cast -Wfloat-equal -Wno-multichar -Wsign-compare -Wundef -Wno-format-nonliteral -Wendif-labels -Wstrict-prototypes -Wdeclaration-after-statement -Wold-style-definition -Wstrict-aliasing=3 -Wcast-align -Wtype-limits -Wold-style-declaration -Wmissing-parameter-type -Wempty-body -Wclobbered -Wignored-qualifiers -Wconversion -Wno-sign-conversion -Wvla -ftree-vrp -Wdouble-promotion -Wformat=2 -Warray-bounds=2 -Wno-system-headers
CPPFLAGS: -isystem /opt/csw/include -isystem /opt/csw/include -isystem /opt/csw/include -isystem /opt/csw/include
LDFLAGS: -L/opt/csw/lib -L/opt/csw/lib -L/opt/csw/lib -L/opt/csw/lib -L/opt/csw/lib
LIBS: -lnghttp2 -lidn2 -lrtmp -lgsasl -lpsl -lssl -lcrypto -lssl -lcrypto -lldap -llber -lzstd -lbrotlidec -lz -lrt -lsocket -lnsl

curl version: 7.87.1-DEV
SSL: enabled (OpenSSL)
SSH: no (--with-{libssh,libssh2})
zlib: enabled
brotli: enabled (libbrotlidec)
zstd: enabled (libzstd)
GSS-API: no (--with-gssapi)
GSASL: enabled
TLS-SRP: enabled
resolver: default (--enable-ares / --enable-threaded-resolver)
IPv6: enabled
Unix sockets: enabled
IDN: enabled (libidn2)
Build libcurl: Shared=yes, Static=yes
Built-in manual: enabled
--libcurl option: enabled (--disable-libcurl-option)
Verbose errors: enabled (--disable-verbose)
Code coverage: disabled
SSPI: no (--enable-sspi)
ca cert bundle: no
ca cert path: no
ca fallback: no
LDAP: enabled (OpenLDAP)
LDAPS: enabled
RTSP: enabled
RTMP: enabled (librtmp)
PSL: enabled
Alt-svc: enabled (--disable-alt-svc)
Headers API: enabled (--disable-headers-api)
HSTS: enabled (--disable-hsts)
HTTP1: enabled (internal)
HTTP2: enabled (nghttp2)
HTTP3: no (--with-ngtcp2, --with-quiche --with-msh3)
ECH: no (--enable-ech)
WebSockets: no (--enable-websockets)
Protocols: DICT FILE FTP FTPS GOPHER GOPHERS HTTP HTTPS IMAP IMAPS LDAP LDAPS MQTT POP3 POP3S RTMP RTSP SMB SMBS SMTP SMTPS TELNET TFTP
Features: GSASL HSTS HTTP2 HTTPS-proxy IDN IPv6 Largefile NTLM NTLM_WB PSL SSL TLS-SRP UnixSockets alt-svc brotli libz threadsafe zstd

CC @icing

[MarcelRaad]

Continuing from #10410, half of which was fixed.

[icing]

I believe, unit2600.c does not include the necessary system header files to detect that ALARM_TIMEOUT is in use.

Therefore the test runs on a platform with minimal timer resolution of 1 second and that does nor work. Since the test itself is otherwise rather platform independent, there is no need to make it work there, I believe.

Could you try #10513 on your machine to see if the change works on solaris? Thanks.

[MarcelRaad]

I don't personally have a Solaris machine to test, the autobuilds are run by @dago . They seem to run for some PRs too, but only for @bagder 's it seems? (no, there are also very few others)
@bagder do you know how to trigger them?

[bagder]

@bagder do you know how to trigger them?

Sorry, no clue.

[dago]

Hi @MarcelRaad @bagder
the builds should be triggered by any PR, I am not sure why buildbot does not pick them up. If you want I can set up an upstream developer account for you on the buildfarm or you can probably use one of the Solaris machines from https://cfarm.tetaneutral.net Feel free to send me a private notice for details.

[MarcelRaad]

Thanks a lot @dago, the Solaris 11 machine there worked fine! (After fighting with SSH to get a connection - is OpenSSH 6.6 really the latest version available there? 😞 )
FYI, I just noticed this in the autobuild output:

configure: WARNING: unrecognized options: --with-spnego

The option should be --with-gssapi instead of --with-spnego.

[MarcelRaad]

@icing The "expected min duration" errors are gone, but unfortunately the test is still failing:

test 2600...[llist unit tests]
./unit/unit2600 - >log/stdout2600 2>log/stderr2600
CMD (2304): ./unit/unit2600 - >log/stdout2600 2>log/stderr2600

unit2600 returned 9, when expecting 0
 exit FAILED
== Contents of files in the log/ dir after test 2600
=== Start of file commands.log
 ./unit/unit2600 - >log/stdout2600 2>log/stderr2600
=== End of file commands.log
=== Start of file ftpserver.cmd
 Testnum 2600
=== End of file ftpserver.cmd
=== Start of file stderr2600
 URL: -
 * !!! WARNING !!
 * This is a debug build of libcurl, do not use in production.
 * Added test.com:123:192.0.2.1 to DNS cache
 * STATE: INIT => CONNECT handle 0x16a2f0; line 1933 (connection #-5000)
 * Added connection 0. The cache now contains 1 members
 * remaining timeout of 2 too small to resolve via SIGALRM method
 * Failed to resolve host 'test.com' with timeout after 0 ms
 * The cache now contains 0 members
 * Curl_disconnect(conn #0, dead=1)
 * Closing connection 0
 * Expire cleared (transfer 0x16a2f0)
 unit2600.c:254 test failed: '1: expected 1 ipv4 creations, but got 0'
 * RESOLVE test.com:123 is - old addresses discarded
 * Added test.com:123:192.0.2.1,192.0.2.2 to DNS cache
 * STATE: INIT => CONNECT handle 0x16a2f0; line 1933 (connection #-5000)
 * Added connection 1. The cache now contains 1 members
 * remaining timeout of 2 too small to resolve via SIGALRM method
 * Failed to resolve host 'test.com' with timeout after 0 ms
 * The cache now contains 0 members
 * Curl_disconnect(conn #1, dead=1)
 * Closing connection 1
 * Expire cleared (transfer 0x16a2f0)
 unit2600.c:254 test failed: '2: expected 2 ipv4 creations, but got 0'
 * RESOLVE test.com:123 is - old addresses discarded
 * Added test.com:123:::1 to DNS cache
 * STATE: INIT => CONNECT handle 0x16a2f0; line 1933 (connection #-5000)
 * Added connection 2. The cache now contains 1 members
 * remaining timeout of 2 too small to resolve via SIGALRM method
 * Failed to resolve host 'test.com' with timeout after 0 ms
 * The cache now contains 0 members
 * Curl_disconnect(conn #2, dead=1)
 * Closing connection 2
 * Expire cleared (transfer 0x16a2f0)
 unit2600.c:259 test failed: '3: expected 1 ipv6 creations, but got 0'
 * RESOLVE test.com:123 is - old addresses discarded
 * Added test.com:123:::1,::2 to DNS cache
 * STATE: INIT => CONNECT handle 0x16a2f0; line 1933 (connection #-5000)
 * Added connection 3. The cache now contains 1 members
 * remaining timeout of 2 too small to resolve via SIGALRM method
 * Failed to resolve host 'test.com' with timeout after 0 ms
 * The cache now contains 0 members
 * Curl_disconnect(conn #3, dead=1)
 * Closing connection 3
 * Expire cleared (transfer 0x16a2f0)
 unit2600.c:259 test failed: '4: expected 2 ipv6 creations, but got 0'
 * RESOLVE test.com:123 is - old addresses discarded
 * Added test.com:123:192.0.2.1,::1 to DNS cache
 * STATE: INIT => CONNECT handle 0x16a2f0; line 1933 (connection #-5000)
 * Added connection 4. The cache now contains 1 members
 * remaining timeout of 2 too small to resolve via SIGALRM method
 * Failed to resolve host 'test.com' with timeout after 0 ms
 * The cache now contains 0 members
 * Curl_disconnect(conn #4, dead=1)
 * Closing connection 4
 * Expire cleared (transfer 0x16a2f0)
 unit2600.c:254 test failed: '5: expected 1 ipv4 creations, but got 0'
 unit2600.c:259 test failed: '5: expected 1 ipv6 creations, but got 0'
 * RESOLVE test.com:123 is - old addresses discarded
 * Added test.com:123:::1,192.0.2.1 to DNS cache
 * STATE: INIT => CONNECT handle 0x16a2f0; line 1933 (connection #-5000)
 * Added connection 5. The cache now contains 1 members
 * remaining timeout of 2 too small to resolve via SIGALRM method
 * Failed to resolve host 'test.com' with timeout after 0 ms
 * The cache now contains 0 members
 * Curl_disconnect(conn #5, dead=1)
 * Closing connection 5
 * Expire cleared (transfer 0x16a2f0)
 unit2600.c:254 test failed: '6: expected 1 ipv4 creations, but got 0'
 unit2600.c:259 test failed: '6: expected 1 ipv6 creations, but got 0'
 * RESOLVE test.com:123 is - old addresses discarded
 * Added test.com:123:::1,192.0.2.1,::2,::3 to DNS cache
 * STATE: INIT => CONNECT handle 0x16a2f0; line 1933 (connection #-5000)
 * Added connection 6. The cache now contains 1 members
 * remaining timeout of 2 too small to resolve via SIGALRM method
 * Failed to resolve host 'test.com' with timeout after 0 ms
 * The cache now contains 0 members
 * Curl_disconnect(conn #6, dead=1)
 * Closing connection 6
 * Expire cleared (transfer 0x16a2f0)
 unit2600.c:259 test failed: '7: expected 3 ipv6 creations, but got 0'
=== End of file stderr2600

The "remaining timeout of 2 too small to resolve via SIGALRM method" messages look suspicious, don't they?

[dago]

Hi @MarcelRaad

Thanks a lot @dago, the Solaris 11 machine there worked fine! (After fighting with SSH to get a connection - is OpenSSH 6.6 really the latest version available there? 😞 )

Unfortunately yes, building a newer version would be a major effort.

FYI, I just noticed this in the autobuild output:

configure: WARNING: unrecognized options: --with-spnego

The option should be --with-gssapi instead of --with-spnego.

Ah yes, thanks for noticing, this should be fixed now!

[icing]

The "remaining timeout of 2 too small to resolve via SIGALRM method" messages look suspicious, don't they?

Indeed, it looks suspiciously as if I got the values wrong.😬

I pushed a fix to the PR, also taking care of the unused var.

[MarcelRaad]

Looks much better now! Only test 7 is still failing:

 unit2600.c:247 test failed: '7: expected result 7 but got 7'
 unit2600.c:252 test failed: '7: expected 0 ipv4 creations, but got 1'

For the first failure, the message is also buggy and always outputs CURLE_COULDNT_CONNECT.

Full output:

test 2600...[llist unit tests]
./unit/unit2600 - >log/stdout2600 2>log/stderr2600
CMD (512): ./unit/unit2600 - >log/stdout2600 2>log/stderr2600

unit2600 returned 2, when expecting 0
 exit FAILED
== Contents of files in the log/ dir after test 2600
=== Start of file commands.log
 ./unit/unit2600 - >log/stdout2600 2>log/stderr2600
=== End of file commands.log
=== Start of file ftpserver.cmd
 Testnum 2600
=== End of file ftpserver.cmd
=== Start of file stderr2600
 URL: -
 * !!! WARNING !!
 * This is a debug build of libcurl, do not use in production.
 * Added test.com:123:192.0.2.1 to DNS cache
 * STATE: INIT => CONNECT handle 0x16a330; line 1933 (connection #-5000)
 * Added connection 0. The cache now contains 1 members
 * Hostname test.com was found in DNS cache
 * STATE: CONNECT => CONNECTING handle 0x16a330; line 1989 (connection #0)
 * 0002ms: cf[v4-0] created
 * 0002ms: cf[v4-0] continuing
 * 1002ms: cf[v4-0] fail delay reached
 * Failed to connect to test.com port 123 after 1001 ms: Couldn't connect to server
 * multi_done: status: 7 prem: 1 done: 0
 * multi_done, not re-using connection=0, forbid=0, close=1, premature=1, conn_multiplex=0
 * The cache now contains 0 members
 * Curl_disconnect(conn #0, dead=1)
 * Closing connection 0
 * 1003ms: cf[v4-0] destroyed
 * Expire cleared (transfer 0x16a330)
 * RESOLVE test.com:123 is - old addresses discarded
 * Added test.com:123:192.0.2.1,192.0.2.2 to DNS cache
 * STATE: INIT => CONNECT handle 0x16a330; line 1933 (connection #-5000)
 * Added connection 1. The cache now contains 1 members
 * Hostname test.com was found in DNS cache
 * STATE: CONNECT => CONNECTING handle 0x16a330; line 1989 (connection #1)
 * 0001ms: cf[v4-0] created
 * 0001ms: cf[v4-0] continuing
 * 1001ms: cf[v4-0] fail delay reached
 * 1001ms: cf[v4-1] created
 * 1001ms: cf[v4-0] destroyed
 * 2000ms: cf[v4-1] fail delay reached
 * Failed to connect to test.com port 123 after 2000 ms: Couldn't connect to server
 * multi_done: status: 7 prem: 1 done: 0
 * multi_done, not re-using connection=1, forbid=0, close=1, premature=1, conn_multiplex=0
 * The cache now contains 0 members
 * Curl_disconnect(conn #1, dead=1)
 * Closing connection 1
 * 2000ms: cf[v4-1] destroyed
 * Expire cleared (transfer 0x16a330)
 * RESOLVE test.com:123 is - old addresses discarded
 * Added test.com:123:::1 to DNS cache
 * STATE: INIT => CONNECT handle 0x16a330; line 1933 (connection #-5000)
 * Added connection 2. The cache now contains 1 members
 * Hostname test.com was found in DNS cache
 * STATE: CONNECT => CONNECTING handle 0x16a330; line 1989 (connection #2)
 * 0001ms: cf[v6-0] created
 * 0001ms: cf[v6-0] continuing
 * 1001ms: cf[v6-0] fail delay reached
 * Failed to connect to test.com port 123 after 1000 ms: Couldn't connect to server
 * multi_done: status: 7 prem: 1 done: 0
 * multi_done, not re-using connection=2, forbid=0, close=1, premature=1, conn_multiplex=0
 * The cache now contains 0 members
 * Curl_disconnect(conn #2, dead=1)
 * Closing connection 2
 * 1001ms: cf[v6-0] destroyed
 * Expire cleared (transfer 0x16a330)
 * RESOLVE test.com:123 is - old addresses discarded
 * Added test.com:123:::1,::2 to DNS cache
 * STATE: INIT => CONNECT handle 0x16a330; line 1933 (connection #-5000)
 * Added connection 3. The cache now contains 1 members
 * Hostname test.com was found in DNS cache
 * STATE: CONNECT => CONNECTING handle 0x16a330; line 1989 (connection #3)
 * 0001ms: cf[v6-0] created
 * 0001ms: cf[v6-0] continuing
 * 1001ms: cf[v6-0] fail delay reached
 * 1001ms: cf[v6-1] created
 * 1001ms: cf[v6-0] destroyed
 * 2000ms: cf[v6-1] fail delay reached
 * Failed to connect to test.com port 123 after 2000 ms: Couldn't connect to server
 * multi_done: status: 7 prem: 1 done: 0
 * multi_done, not re-using connection=3, forbid=0, close=1, premature=1, conn_multiplex=0
 * The cache now contains 0 members
 * Curl_disconnect(conn #3, dead=1)
 * Closing connection 3
 * 2001ms: cf[v6-1] destroyed
 * Expire cleared (transfer 0x16a330)
 * RESOLVE test.com:123 is - old addresses discarded
 * Added test.com:123:192.0.2.1,::1 to DNS cache
 * STATE: INIT => CONNECT handle 0x16a330; line 1933 (connection #-5000)
 * Added connection 4. The cache now contains 1 members
 * Hostname test.com was found in DNS cache
 * STATE: CONNECT => CONNECTING handle 0x16a330; line 1989 (connection #4)
 * 0001ms: cf[v4-0] created
 * 0001ms: cf[v4-0] continuing
 * 1001ms: cf[v4-0] fail delay reached
 * 1001ms: cf[v6-0] created
 * 1001ms: cf[v6-0] continuing
 * 2000ms: cf[v6-0] fail delay reached
 * Failed to connect to test.com port 123 after 2000 ms: Couldn't connect to server
 * multi_done: status: 7 prem: 1 done: 0
 * multi_done, not re-using connection=4, forbid=0, close=1, premature=1, conn_multiplex=0
 * The cache now contains 0 members
 * Curl_disconnect(conn #4, dead=1)
 * Closing connection 4
 * 2001ms: cf[v4-0] destroyed
 * 2001ms: cf[v6-0] destroyed
 * Expire cleared (transfer 0x16a330)
 * RESOLVE test.com:123 is - old addresses discarded
 * Added test.com:123:::1,192.0.2.1 to DNS cache
 * STATE: INIT => CONNECT handle 0x16a330; line 1933 (connection #-5000)
 * Added connection 5. The cache now contains 1 members
 * Hostname test.com was found in DNS cache
 * STATE: CONNECT => CONNECTING handle 0x16a330; line 1989 (connection #5)
 * 0001ms: cf[v6-0] created
 * 0001ms: cf[v6-0] continuing
 * 1002ms: cf[v6-0] fail delay reached
 * 1002ms: cf[v4-0] created
 * 1002ms: cf[v4-0] continuing
 * 2000ms: cf[v4-0] fail delay reached
 * Failed to connect to test.com port 123 after 2000 ms: Couldn't connect to server
 * multi_done: status: 7 prem: 1 done: 0
 * multi_done, not re-using connection=5, forbid=0, close=1, premature=1, conn_multiplex=0
 * The cache now contains 0 members
 * Curl_disconnect(conn #5, dead=1)
 * Closing connection 5
 * 2001ms: cf[v6-0] destroyed
 * 2001ms: cf[v4-0] destroyed
 * Expire cleared (transfer 0x16a330)
 * RESOLVE test.com:123 is - old addresses discarded
 * Added test.com:123:::1,192.0.2.1,::2,::3 to DNS cache
 * STATE: INIT => CONNECT handle 0x16a330; line 1933 (connection #-5000)
 * Added connection 6. The cache now contains 1 members
 * Hostname test.com was found in DNS cache
 * STATE: CONNECT => CONNECTING handle 0x16a330; line 1989 (connection #6)
 * 0001ms: cf[v6-0] created
 * 0001ms: cf[v6-0] continuing
 * 1001ms: cf[v6-0] fail delay reached
 * 1001ms: cf[v6-1] created
 * 1001ms: cf[v6-0] destroyed
 * 1001ms: cf[v4-0] created
 * 1001ms: cf[v6-1] continuing
 * 1001ms: cf[v4-0] continuing
 * 1501ms: cf[v6-1] fail delay reached
 * 1501ms: cf[v6-2] created
 * 1501ms: cf[v6-1] destroyed
 * 1501ms: cf[v4-0] fail delay reached
 * 1502ms: cf[v6-2] continuing
 * 2001ms: cf[v6-2] fail delay reached
 * Failed to connect to test.com port 123 after 2000 ms: Couldn't connect to server
 * multi_done: status: 7 prem: 1 done: 0
 * multi_done, not re-using connection=6, forbid=0, close=1, premature=1, conn_multiplex=0
 * The cache now contains 0 members
 * Curl_disconnect(conn #6, dead=1)
 * Closing connection 6
 * 2001ms: cf[v6-2] destroyed
 * 2002ms: cf[v4-0] destroyed
 * Expire cleared (transfer 0x16a330)
 unit2600.c:247 test failed: '7: expected result 7 but got 7'
 unit2600.c:252 test failed: '7: expected 0 ipv4 creations, but got 1'
=== End of file stderr2600

[icing]

Thanks, that expected result (and the check) is indeed wrong. I just pushed an update.

Rebased, CI is running.

[icing]

@MarcelRaad would you have time to take the PR out for a spin again? Thanks!

[MarcelRaad]

@icing There's still one failure left: unit2600.c:253 test failed: '7: expected 0 ipv4 creations, but got 1'

Full output:

test 2600...[llist unit tests]
./unit/unit2600 - >log/stdout2600 2>log/stderr2600
CMD (256): ./unit/unit2600 - >log/stdout2600 2>log/stderr2600

unit2600 returned 1, when expecting 0
 exit FAILED
== Contents of files in the log/ dir after test 2600
=== Start of file commands.log
 ./unit/unit2600 - >log/stdout2600 2>log/stderr2600
=== End of file commands.log
=== Start of file ftpserver.cmd
 Testnum 2600
=== End of file ftpserver.cmd
=== Start of file stderr2600
 URL: -
 * !!! WARNING !!
 * This is a debug build of libcurl, do not use in production.
 * Added test.com:123:192.0.2.1 to DNS cache
 * STATE: INIT => CONNECT handle 0x16a4f0; line 1933 (connection #-5000)
 * Added connection 0. The cache now contains 1 members
 * Hostname test.com was found in DNS cache
 * STATE: CONNECT => CONNECTING handle 0x16a4f0; line 1989 (connection #0)
 * 0002ms: cf[v4-0] created
 * 0002ms: cf[v4-0] continuing
 * 1002ms: cf[v4-0] fail delay reached
 * Failed to connect to test.com port 123 after 1001 ms: Couldn't connect to server
 * multi_done: status: 7 prem: 1 done: 0
 * multi_done, not re-using connection=0, forbid=0, close=1, premature=1, conn_multiplex=0
 * The cache now contains 0 members
 * Curl_disconnect(conn #0, dead=1)
 * Closing connection 0
 * 1003ms: cf[v4-0] destroyed
 * Expire cleared (transfer 0x16a4f0)
 * RESOLVE test.com:123 is - old addresses discarded
 * Added test.com:123:192.0.2.1,192.0.2.2 to DNS cache
 * STATE: INIT => CONNECT handle 0x16a4f0; line 1933 (connection #-5000)
 * Added connection 1. The cache now contains 1 members
 * Hostname test.com was found in DNS cache
 * STATE: CONNECT => CONNECTING handle 0x16a4f0; line 1989 (connection #1)
 * 0001ms: cf[v4-0] created
 * 0001ms: cf[v4-0] continuing
 * 1001ms: cf[v4-0] fail delay reached
 * 1001ms: cf[v4-1] created
 * 1001ms: cf[v4-0] destroyed
 * 2000ms: cf[v4-1] fail delay reached
 * Failed to connect to test.com port 123 after 2000 ms: Couldn't connect to server
 * multi_done: status: 7 prem: 1 done: 0
 * multi_done, not re-using connection=1, forbid=0, close=1, premature=1, conn_multiplex=0
 * The cache now contains 0 members
 * Curl_disconnect(conn #1, dead=1)
 * Closing connection 1
 * 2001ms: cf[v4-1] destroyed
 * Expire cleared (transfer 0x16a4f0)
 * RESOLVE test.com:123 is - old addresses discarded
 * Added test.com:123:::1 to DNS cache
 * STATE: INIT => CONNECT handle 0x16a4f0; line 1933 (connection #-5000)
 * Added connection 2. The cache now contains 1 members
 * Hostname test.com was found in DNS cache
 * STATE: CONNECT => CONNECTING handle 0x16a4f0; line 1989 (connection #2)
 * 0001ms: cf[v6-0] created
 * 0001ms: cf[v6-0] continuing
 * 1001ms: cf[v6-0] fail delay reached
 * Failed to connect to test.com port 123 after 1000 ms: Couldn't connect to server
 * multi_done: status: 7 prem: 1 done: 0
 * multi_done, not re-using connection=2, forbid=0, close=1, premature=1, conn_multiplex=0
 * The cache now contains 0 members
 * Curl_disconnect(conn #2, dead=1)
 * Closing connection 2
 * 1001ms: cf[v6-0] destroyed
 * Expire cleared (transfer 0x16a4f0)
 * RESOLVE test.com:123 is - old addresses discarded
 * Added test.com:123:::1,::2 to DNS cache
 * STATE: INIT => CONNECT handle 0x16a4f0; line 1933 (connection #-5000)
 * Added connection 3. The cache now contains 1 members
 * Hostname test.com was found in DNS cache
 * STATE: CONNECT => CONNECTING handle 0x16a4f0; line 1989 (connection #3)
 * 0001ms: cf[v6-0] created
 * 0001ms: cf[v6-0] continuing
 * 1001ms: cf[v6-0] fail delay reached
 * 1001ms: cf[v6-1] created
 * 1001ms: cf[v6-0] destroyed
 * 2000ms: cf[v6-1] fail delay reached
 * Failed to connect to test.com port 123 after 2000 ms: Couldn't connect to server
 * multi_done: status: 7 prem: 1 done: 0
 * multi_done, not re-using connection=3, forbid=0, close=1, premature=1, conn_multiplex=0
 * The cache now contains 0 members
 * Curl_disconnect(conn #3, dead=1)
 * Closing connection 3
 * 2001ms: cf[v6-1] destroyed
 * Expire cleared (transfer 0x16a4f0)
 * RESOLVE test.com:123 is - old addresses discarded
 * Added test.com:123:192.0.2.1,::1 to DNS cache
 * STATE: INIT => CONNECT handle 0x16a4f0; line 1933 (connection #-5000)
 * Added connection 4. The cache now contains 1 members
 * Hostname test.com was found in DNS cache
 * STATE: CONNECT => CONNECTING handle 0x16a4f0; line 1989 (connection #4)
 * 0001ms: cf[v4-0] created
 * 0001ms: cf[v4-0] continuing
 * 1001ms: cf[v4-0] fail delay reached
 * 1001ms: cf[v6-0] created
 * 1001ms: cf[v6-0] continuing
 * 2000ms: cf[v6-0] fail delay reached
 * Failed to connect to test.com port 123 after 2000 ms: Couldn't connect to server
 * multi_done: status: 7 prem: 1 done: 0
 * multi_done, not re-using connection=4, forbid=0, close=1, premature=1, conn_multiplex=0
 * The cache now contains 0 members
 * Curl_disconnect(conn #4, dead=1)
 * Closing connection 4
 * 2001ms: cf[v4-0] destroyed
 * 2001ms: cf[v6-0] destroyed
 * Expire cleared (transfer 0x16a4f0)
 * RESOLVE test.com:123 is - old addresses discarded
 * Added test.com:123:::1,192.0.2.1 to DNS cache
 * STATE: INIT => CONNECT handle 0x16a4f0; line 1933 (connection #-5000)
 * Added connection 5. The cache now contains 1 members
 * Hostname test.com was found in DNS cache
 * STATE: CONNECT => CONNECTING handle 0x16a4f0; line 1989 (connection #5)
 * 0001ms: cf[v6-0] created
 * 0001ms: cf[v6-0] continuing
 * 1001ms: cf[v6-0] fail delay reached
 * 1001ms: cf[v4-0] created
 * 1001ms: cf[v4-0] continuing
 * 2000ms: cf[v4-0] fail delay reached
 * Failed to connect to test.com port 123 after 2000 ms: Couldn't connect to server
 * multi_done: status: 7 prem: 1 done: 0
 * multi_done, not re-using connection=5, forbid=0, close=1, premature=1, conn_multiplex=0
 * The cache now contains 0 members
 * Curl_disconnect(conn #5, dead=1)
 * Closing connection 5
 * 2001ms: cf[v6-0] destroyed
 * 2001ms: cf[v4-0] destroyed
 * Expire cleared (transfer 0x16a4f0)
 * RESOLVE test.com:123 is - old addresses discarded
 * Added test.com:123:::1,192.0.2.1,::2,::3 to DNS cache
 * STATE: INIT => CONNECT handle 0x16a4f0; line 1933 (connection #-5000)
 * Added connection 6. The cache now contains 1 members
 * Hostname test.com was found in DNS cache
 * STATE: CONNECT => CONNECTING handle 0x16a4f0; line 1989 (connection #6)
 * 0001ms: cf[v6-0] created
 * 0001ms: cf[v6-0] continuing
 * 1001ms: cf[v6-0] fail delay reached
 * 1001ms: cf[v6-1] created
 * 1001ms: cf[v6-0] destroyed
 * 1001ms: cf[v4-0] created
 * 1001ms: cf[v6-1] continuing
 * 1001ms: cf[v4-0] continuing
 * 1503ms: cf[v6-1] fail delay reached
 * 1503ms: cf[v6-2] created
 * 1503ms: cf[v6-1] destroyed
 * 1503ms: cf[v4-0] fail delay reached
 * 2000ms: cf[v6-2] fail delay reached
 * Failed to connect to test.com port 123 after 2000 ms: Couldn't connect to server
 * multi_done: status: 7 prem: 1 done: 0
 * multi_done, not re-using connection=6, forbid=0, close=1, premature=1, conn_multiplex=0
 * The cache now contains 0 members
 * Curl_disconnect(conn #6, dead=1)
 * Closing connection 6
 * 2001ms: cf[v6-2] destroyed
 * 2001ms: cf[v4-0] destroyed
 * Expire cleared (transfer 0x16a4f0)
 unit2600.c:253 test failed: '7: expected 0 ipv4 creations, but got 1'
 * RESOLVE test.com:123 is - old addresses discarded
 * Added test.com:123:192.0.2.1,::1 to DNS cache
 * STATE: INIT => CONNECT handle 0x16a4f0; line 1933 (connection #-5000)
 * Added connection 7. The cache now contains 1 members
 * Hostname test.com was found in DNS cache
 * STATE: CONNECT => CONNECTING handle 0x16a4f0; line 1989 (connection #7)
 * 0001ms: cf[v4-0] created
 * 0001ms: cf[v4-0] continuing
 * 1001ms: cf[v4-0] fail delay reached
 * Failed to connect to test.com port 123 after 1000 ms: Couldn't connect to server
 * multi_done: status: 7 prem: 1 done: 0
 * multi_done, not re-using connection=7, forbid=0, close=1, premature=1, conn_multiplex=0
 * The cache now contains 0 members
 * Curl_disconnect(conn #7, dead=1)
 * Closing connection 7
 * 1001ms: cf[v4-0] destroyed
 * Expire cleared (transfer 0x16a4f0)
 * RESOLVE test.com:123 is - old addresses discarded
 * Added test.com:123:::1,192.0.2.1 to DNS cache
 * STATE: INIT => CONNECT handle 0x16a4f0; line 1933 (connection #-5000)
 * Added connection 8. The cache now contains 1 members
 * Hostname test.com was found in DNS cache
 * STATE: CONNECT => CONNECTING handle 0x16a4f0; line 1989 (connection #8)
 * 0001ms: cf[v6-0] created
 * 0001ms: cf[v6-0] continuing
 * 1001ms: cf[v6-0] fail delay reached
 * Failed to connect to test.com port 123 after 1000 ms: Couldn't connect to server
 * multi_done: status: 7 prem: 1 done: 0
 * multi_done, not re-using connection=8, forbid=0, close=1, premature=1, conn_multiplex=0
 * The cache now contains 0 members
 * Curl_disconnect(conn #8, dead=1)
 * Closing connection 8
 * 1001ms: cf[v6-0] destroyed
 * Expire cleared (transfer 0x16a4f0)

This one also appears in the autotools jobs on AppVeyor, so Solaris is now as good as these :-)
https://ci.appveyor.com/project/curlorg/curl/builds/46261808/job/urm8s0xyvata7qym#L8183
https://ci.appveyor.com/project/curlorg/curl/builds/46261808/job/b8o4c44pi36o9hra#L8204
https://ci.appveyor.com/project/curlorg/curl/builds/46261808/job/0lwvr6mhgd266j9c#L8482

[icing]

Thanks! I identified the problem with test 7 and updated the PR.

[bagder]

Fixed via #10513