4.3.4: test suite is failing in two units

[kloczek]

Issue description

Cmake test suite is failing in two units.

Environment

• libzmq version: 4.3.4
• OS: Linux x86/64

Test suite is failing:

+ cd libzmq-4.3.4
+ /usr/bin/make -O -j48 V=1 VERBOSE=1 -C x86_64-redhat-linux-gnu test ARGS=--output-on-failure -j1
make: Entering directory '/home/tkloczko/rpmbuild/BUILD/libzmq-4.3.4/x86_64-redhat-linux-gnu'
Running tests...
/usr/bin/ctest --force-new-ctest-process --output-on-failure
Test project /home/tkloczko/rpmbuild/BUILD/libzmq-4.3.4/x86_64-redhat-linux-gnu
        Start   1: test_ancillaries
  1/105 Test   #1: test_ancillaries .................   Passed    0.01 sec
        Start   2: test_system
  2/105 Test   #2: test_system ......................   Passed    0.02 sec
        Start   3: test_pair_inproc
  3/105 Test   #3: test_pair_inproc .................   Passed    0.01 sec
        Start   4: test_pair_tcp
  4/105 Test   #4: test_pair_tcp ....................   Passed    0.01 sec
        Start   5: test_reqrep_inproc
  5/105 Test   #5: test_reqrep_inproc ...............   Passed    0.01 sec
        Start   6: test_reqrep_tcp
  6/105 Test   #6: test_reqrep_tcp ..................   Passed    0.03 sec
        Start   7: test_hwm
  7/105 Test   #7: test_hwm .........................   Passed    1.56 sec
        Start   8: test_hwm_pubsub
  8/105 Test   #8: test_hwm_pubsub ..................   Passed    3.22 sec
        Start   9: test_reqrep_device
  9/105 Test   #9: test_reqrep_device ...............   Passed    0.01 sec
        Start  10: test_sub_forward
 10/105 Test  #10: test_sub_forward .................   Passed    0.31 sec
        Start  11: test_invalid_rep
 11/105 Test  #11: test_invalid_rep .................   Passed    0.01 sec
        Start  12: test_msg_flags
 12/105 Test  #12: test_msg_flags ...................   Passed    0.01 sec
        Start  13: test_msg_ffn
 13/105 Test  #13: test_msg_ffn .....................   Passed    1.21 sec
        Start  14: test_connect_resolve
 14/105 Test  #14: test_connect_resolve .............   Passed    0.02 sec
        Start  15: test_immediate
 15/105 Test  #15: test_immediate ...................   Passed    3.84 sec
        Start  16: test_last_endpoint
 16/105 Test  #16: test_last_endpoint ...............   Passed    0.01 sec
        Start  17: test_term_endpoint
 17/105 Test  #17: test_term_endpoint ...............   Passed    0.62 sec
        Start  18: test_router_mandatory
 18/105 Test  #18: test_router_mandatory ............   Passed    0.01 sec
        Start  19: test_probe_router
 19/105 Test  #19: test_probe_router ................   Passed    0.02 sec
        Start  20: test_stream
 20/105 Test  #20: test_stream ......................   Passed    0.02 sec
        Start  21: test_stream_empty
 21/105 Test  #21: test_stream_empty ................   Passed    0.01 sec
        Start  22: test_stream_disconnect
 22/105 Test  #22: test_stream_disconnect ...........   Passed    0.01 sec
        Start  23: test_disconnect_inproc
 23/105 Test  #23: test_disconnect_inproc ...........   Passed    0.61 sec
        Start  24: test_unbind_wildcard
 24/105 Test  #24: test_unbind_wildcard .............   Passed    0.02 sec
        Start  25: test_ctx_options
 25/105 Test  #25: test_ctx_options .................   Passed    0.01 sec
        Start  26: test_ctx_destroy
 26/105 Test  #26: test_ctx_destroy .................   Passed    0.32 sec
        Start  27: test_security_no_zap_handler
 27/105 Test  #27: test_security_no_zap_handler .....   Passed    0.02 sec
        Start  28: test_security_null
 28/105 Test  #28: test_security_null ...............   Passed    1.27 sec
        Start  29: test_security_plain
 29/105 Test  #29: test_security_plain ..............   Passed    2.27 sec
        Start  30: test_security_zap
 30/105 Test  #30: test_security_zap ................   Passed   34.63 sec
        Start  31: test_iov
 31/105 Test  #31: test_iov .........................   Passed    0.32 sec
        Start  32: test_spec_req
 32/105 Test  #32: test_spec_req ....................   Passed    0.87 sec
        Start  33: test_spec_rep
 33/105 Test  #33: test_spec_rep ....................   Passed    0.62 sec
        Start  34: test_spec_dealer
 34/105 Test  #34: test_spec_dealer .................   Passed    1.73 sec
        Start  35: test_spec_router
 35/105 Test  #35: test_spec_router .................   Passed    1.22 sec
        Start  36: test_spec_pushpull
 36/105 Test  #36: test_spec_pushpull ...............   Passed    6.24 sec
        Start  37: test_req_correlate
 37/105 Test  #37: test_req_correlate ...............   Passed    0.02 sec
        Start  38: test_req_relaxed
 38/105 Test  #38: test_req_relaxed .................***Failed    6.63 sec
WARNING: Forced closure of 2 sockets, this is an implementation error unless the test case failed
/home/tkloczko/rpmbuild/BUILD/libzmq-4.3.4/tests/test_req_relaxed.cpp:228:test_case_1:PASS
/home/tkloczko/rpmbuild/BUILD/libzmq-4.3.4/tests/test_req_relaxed.cpp:229:test_case_2:PASS
/home/tkloczko/rpmbuild/BUILD/libzmq-4.3.4/tests/test_req_relaxed.cpp:230:test_case_3:PASS
/home/tkloczko/rpmbuild/BUILD/libzmq-4.3.4/tests/test_req_relaxed.cpp:209:test_case_4:FAIL: zmq_bind (router, ENDPOINT_0) failed, errno = 98 (Address already in use)

-----------------------
4 Tests 1 Failures 0 Ignored
FAIL

        Start  39: test_conflate
 39/105 Test  #39: test_conflate ....................   Passed    0.32 sec
        Start  40: test_inproc_connect
 40/105 Test  #40: test_inproc_connect ..............   Passed    0.34 sec
        Start  41: test_issue_566
 41/105 Test  #41: test_issue_566 ...................   Passed    0.25 sec
        Start  42: test_shutdown_stress
 42/105 Test  #42: test_shutdown_stress .............   Passed    0.17 sec
        Start  43: test_timeo
 43/105 Test  #43: test_timeo .......................   Passed    0.26 sec
        Start  44: test_many_sockets
 44/105 Test  #44: test_many_sockets ................   Passed    0.13 sec
        Start  45: test_diffserv
 45/105 Test  #45: test_diffserv ....................   Passed    0.02 sec
        Start  46: test_connect_rid
 46/105 Test  #46: test_connect_rid .................   Passed    0.63 sec
        Start  47: test_xpub_nodrop
 47/105 Test  #47: test_xpub_nodrop .................   Passed    0.27 sec
        Start  48: test_pub_invert_matching
 48/105 Test  #48: test_pub_invert_matching .........   Passed    1.22 sec
        Start  49: test_setsockopt
 49/105 Test  #49: test_setsockopt ..................   Passed    0.02 sec
        Start  50: test_sockopt_hwm
 50/105 Test  #50: test_sockopt_hwm .................   Passed    0.92 sec
        Start  51: test_heartbeats
 51/105 Test  #51: test_heartbeats ..................   Passed   36.36 sec
        Start  52: test_atomics
 52/105 Test  #52: test_atomics .....................   Passed    0.02 sec
        Start  53: test_bind_src_address
 53/105 Test  #53: test_bind_src_address ............   Passed    0.02 sec
        Start  54: test_capabilities
 54/105 Test  #54: test_capabilities ................   Passed    0.02 sec
        Start  55: test_metadata
 55/105 Test  #55: test_metadata ....................   Passed    0.02 sec
        Start  56: test_router_handover
 56/105 Test  #56: test_router_handover .............   Passed    0.93 sec
        Start  57: test_srcfd
 57/105 Test  #57: test_srcfd .......................   Passed    0.32 sec
        Start  58: test_stream_timeout
 58/105 Test  #58: test_stream_timeout ..............   Passed    0.22 sec
        Start  59: test_xpub_manual
 59/105 Test  #59: test_xpub_manual .................   Passed    3.93 sec
        Start  60: test_xpub_welcome_msg
 60/105 Test  #60: test_xpub_welcome_msg ............   Passed    0.02 sec
        Start  61: test_xpub_verbose
 61/105 Test  #61: test_xpub_verbose ................   Passed    0.02 sec
        Start  62: test_base85
 62/105 Test  #62: test_base85 ......................   Passed    0.02 sec
        Start  63: test_bind_after_connect_tcp
 63/105 Test  #63: test_bind_after_connect_tcp ......   Passed    0.21 sec
        Start  64: test_sodium
 64/105 Test  #64: test_sodium ......................   Passed    0.03 sec
        Start  65: test_monitor
 65/105 Test  #65: test_monitor .....................   Passed    0.02 sec
        Start  66: test_socket_null
 66/105 Test  #66: test_socket_null .................   Passed    0.02 sec
        Start  67: test_reconnect_ivl
 67/105 Test  #67: test_reconnect_ivl ...............   Passed    7.54 sec
        Start  68: test_reconnect_options
 68/105 Test  #68: test_reconnect_options ...........***Failed    0.02 sec
WARNING: Forced closure of 1 sockets, this is an implementation error unless the test case failed
WARNING: Forced closure of 1 sockets, this is an implementation error unless the test case failed
/home/tkloczko/rpmbuild/BUILD/libzmq-4.3.4/tests/test_reconnect_options.cpp:43:reconnect_default:FAIL: zmq_bind (pub, ENDPOINT_0) failed, errno = 98 (Address already in use)
/home/tkloczko/rpmbuild/BUILD/libzmq-4.3.4/tests/test_reconnect_options.cpp:96:reconnect_success:FAIL: zmq_bind (pub, ENDPOINT_0) failed, errno = 98 (Address already in use)

-----------------------
2 Tests 2 Failures 0 Ignored
FAIL

        Start  69: test_tcp_accept_filter
 69/105 Test  #69: test_tcp_accept_filter ...........   Passed    1.03 sec
        Start  70: test_mock_pub_sub
 70/105 Test  #70: test_mock_pub_sub ................   Passed    0.03 sec
        Start  71: test_security_gssapi
 71/105 Test  #71: test_security_gssapi .............   Passed    0.02 sec
        Start  72: test_socks
 72/105 Test  #72: test_socks .......................   Passed    3.04 sec
        Start  73: test_connect_null_fuzzer
 73/105 Test  #73: test_connect_null_fuzzer .........***Skipped   0.02 sec
        Start  74: test_bind_null_fuzzer
 74/105 Test  #74: test_bind_null_fuzzer ............***Skipped   0.02 sec
        Start  75: test_connect_fuzzer
 75/105 Test  #75: test_connect_fuzzer ..............***Skipped   0.02 sec
        Start  76: test_bind_fuzzer
 76/105 Test  #76: test_bind_fuzzer .................***Skipped   0.02 sec
        Start  77: test_security_curve
 77/105 Test  #77: test_security_curve ..............   Passed   12.06 sec
        Start  78: test_connect_curve_fuzzer
 78/105 Test  #78: test_connect_curve_fuzzer ........***Skipped   0.02 sec
        Start  79: test_bind_curve_fuzzer
 79/105 Test  #79: test_bind_curve_fuzzer ...........***Skipped   0.02 sec
        Start  80: test_z85_decode_fuzzer
 80/105 Test  #80: test_z85_decode_fuzzer ...........***Skipped   0.02 sec
        Start  81: test_ipc_wildcard
 81/105 Test  #81: test_ipc_wildcard ................   Passed    0.03 sec
        Start  82: test_pair_ipc
 82/105 Test  #82: test_pair_ipc ....................   Passed    0.02 sec
        Start  83: test_reqrep_ipc
 83/105 Test  #83: test_reqrep_ipc ..................   Passed    0.33 sec
        Start  84: test_rebind_ipc
 84/105 Test  #84: test_rebind_ipc ..................   Passed    0.21 sec
        Start  85: test_proxy
 85/105 Test  #85: test_proxy .......................   Passed    1.03 sec
        Start  86: test_proxy_hwm
 86/105 Test  #86: test_proxy_hwm ...................   Passed    3.13 sec
        Start  87: test_proxy_single_socket
 87/105 Test  #87: test_proxy_single_socket .........   Passed    0.03 sec
        Start  88: test_proxy_terminate
 88/105 Test  #88: test_proxy_terminate .............   Passed    0.93 sec
        Start  89: test_getsockopt_memset
 89/105 Test  #89: test_getsockopt_memset ...........   Passed    0.02 sec
        Start  90: test_filter_ipc
 90/105 Test  #90: test_filter_ipc ..................   Passed    1.84 sec
        Start  91: test_stream_exceeds_buffer
 91/105 Test  #91: test_stream_exceeds_buffer .......   Passed    0.03 sec
        Start  92: test_router_mandatory_hwm
 92/105 Test  #92: test_router_mandatory_hwm ........   Passed    1.03 sec
        Start  93: test_use_fd
 93/105 Test  #93: test_use_fd ......................   Passed    0.03 sec
        Start  94: test_zmq_poll_fd
 94/105 Test  #94: test_zmq_poll_fd .................   Passed    0.03 sec
        Start  95: test_fork
 95/105 Test  #95: test_fork ........................   Passed    0.03 sec
        Start  96: test_abstract_ipc
 96/105 Test  #96: test_abstract_ipc ................   Passed    0.03 sec
        Start  97: test_address_tipc
 97/105 Test  #97: test_address_tipc ................***Skipped   0.03 sec
        Start  98: test_pair_tipc
 98/105 Test  #98: test_pair_tipc ...................***Skipped   0.03 sec
        Start  99: test_reqrep_device_tipc
 99/105 Test  #99: test_reqrep_device_tipc ..........***Skipped   0.03 sec
        Start 100: test_reqrep_tipc
100/105 Test #100: test_reqrep_tipc .................***Skipped   0.03 sec
        Start 101: test_router_mandatory_tipc
101/105 Test #101: test_router_mandatory_tipc .......   Passed    0.03 sec
        Start 102: test_sub_forward_tipc
102/105 Test #102: test_sub_forward_tipc ............***Skipped   0.03 sec
        Start 103: test_connect_delay_tipc
103/105 Test #103: test_connect_delay_tipc ..........***Skipped   0.03 sec
        Start 104: test_shutdown_stress_tipc
104/105 Test #104: test_shutdown_stress_tipc ........***Skipped   0.03 sec
        Start 105: test_term_endpoint_tipc
105/105 Test #105: test_term_endpoint_tipc ..........   Passed    0.03 sec

98% tests passed, 2 tests failed out of 105

Total Test time (real) = 147.53 sec

The following tests did not run:
         73 - test_connect_null_fuzzer (Skipped)
         74 - test_bind_null_fuzzer (Skipped)
         75 - test_connect_fuzzer (Skipped)
         76 - test_bind_fuzzer (Skipped)
         78 - test_connect_curve_fuzzer (Skipped)
         79 - test_bind_curve_fuzzer (Skipped)
         80 - test_z85_decode_fuzzer (Skipped)
         97 - test_address_tipc (Skipped)
         98 - test_pair_tipc (Skipped)
         99 - test_reqrep_device_tipc (Skipped)
        100 - test_reqrep_tipc (Skipped)
        102 - test_sub_forward_tipc (Skipped)
        103 - test_connect_delay_tipc (Skipped)
        104 - test_shutdown_stress_tipc (Skipped)

The following tests FAILED:
         38 - test_req_relaxed (Failed)
         68 - test_reconnect_options (Failed)
Errors while running CTest
make: *** [Makefile:74: test] Error 8

Please let me know if you need more details.

[yourhumblebugspotter]

We have a potential issue using 4.3.4 with Ubuntu22.04.1 and seeing ZMQ dropping connection at random.

Details:
Package: libzmq5
Versions:
4.3.4-2 (/var/lib/apt/lists/archive.ubuntu.com_ubuntu_dists_jammy_universe_binary-amd64_Packages.lz4) (/var/lib/dpkg/status)
Description Language:
File: /var/lib/apt/lists/archive.ubuntu.com_ubuntu_dists_jammy_universe_binary-amd64_Packages.lz4
MD5: 8f2756f773da0ff44ace3832dbb30dbb
Dependencies:
4.3.4-2 - libbsd0 (2 0.0) libc6 (2 2.34) libgcc-s1 (2 3.3.1) libgssapi-krb5-2 (2 1.17) libnorm1 (2 1.5r6) libpgm-5.3-0 (2 5.1.116~dfsg) libsodium23 (2 1.0.3) libstdc++6 (2 11)
Provides:
4.3.4-2 -
Reverse Provides:

Package: libzmq3-dev
Versions:
4.3.4-2 (/var/lib/apt/lists/archive.ubuntu.com_ubuntu_dists_jammy_universe_binary-amd64_Packages.lz4) (/var/lib/dpkg/status)
Description Language:
File: /var/lib/apt/lists/archive.ubuntu.com_ubuntu_dists_jammy_universe_binary-amd64_Packages.lz4
MD5: 7b2c1e47f6d305566aebc0e65e04f5ee
Dependencies:
4.3.4-2 - libzmq5 (5 4.3.4-2) libpgm-dev (2 5.2.122~dfsg) libsodium-dev (0 (null)) libnorm-dev (0 (null)) libkrb5-dev (0 (null)) libbsd-dev (0 (null)) libzmq-dev (0 (null)) libzmq5-dev (0 (null)) libzmq5-dev (0 (null))
Provides:
4.3.4-2 - libzmq5-dev (= )
Reverse Provides:

[aniketor]

I'm having a test fail issue as well with Ubuntu 22.04.

  CXX      tests/test_timers-test_timers.o
  CXX      tests/test_radio_dish-test_radio_dish.o
  CXX      tests/test_scatter_gather-test_scatter_gather.o
  CXX      tests/test_dgram-test_dgram.o
  CXX      tests/test_app_meta-test_app_meta.o
  CXX      tests/test_xpub_manual_last_value-test_xpub_manual_last_value.o
  CXX      tests/test_router_notify-test_router_notify.o
  CXX      tests/test_peer-test_peer.o
  CXX      tests/test_reconnect_options-test_reconnect_options.o
cc1plus: all warnings being treated as errors
make[2]: *** [Makefile:7493: tests/test_security_curve-test_security_curve.o] Error 1
make[2]: *** Waiting for unfinished jobs....
make[2]: Leaving directory '/mnt/zeromq'
make[1]: *** [Makefile:9803: check-am] Error 2
make[1]: Leaving directory '/mnt/zeromq'
make: *** [Makefile:8369: check-recursive] Error 1

Probably suppressing warning will solve it but doesn't seem ideal.