test failures with openssl-3.0.0

[zarniwhoop73]

I've been building an experimental from-source system using openssl-3.0.0 and failed to notice that a lot of tests in Net-SSLeay-1.90 were failing (this is beyond linuxfromscratch, someone reported that one specific external test could fail, and I didn't want that to stop the build). Now, I've been asked if they all passed because IO-Socket-SSL depends on that.

Details:

#   at t/local/33_x509_create_cert.t line 98.
#                   undef
#     doesn't match '(?^:-----BEGIN (ENCRYPTED|RSA) PRIVATE KEY-----)'
# Looks like you failed 1 test of 139.
t/local/33_x509_create_cert.t .......... 
Dubious, test returned 1 (wstat 256, 0x100)
Failed 1/139 subtests 
        (less 1 skipped subtest: 137 okay)
#   Failed test 'X509_NAME_oneline [2/4]'
#   at t/local/39_pkcs12.t line 43.
#          got: '/C=PL/O=Net-SSLeay/OU=Test Suite/CN=Intermediate CA'
#     expected: '/C=PL/O=Net-SSLeay/OU=Test Suite/CN=Root CA'

#   Failed test 'X509_NAME_oneline [2/5]'
#   at t/local/39_pkcs12.t line 44.
#          got: '/C=PL/O=Net-SSLeay/OU=Test Suite/CN=Root CA'
#     expected: '/C=PL/O=Net-SSLeay/OU=Test Suite/CN=Intermediate CA'
# Looks like you failed 2 tests of 19.
t/local/39_pkcs12.t .................... 
Dubious, test returned 2 (wstat 512, 0x200)
Failed 2/19 subtests 
#   Failed test 'CTX set partially bad ciphersuites'
#   at t/local/43_misc_functions.t line 260.
#          got: '1'
#     expected: '0'

#   Failed test 'SSL set partially bad ciphersuites'
#   at t/local/43_misc_functions.t line 275.
#          got: '1'
#     expected: '0'
# Looks like you failed 2 tests of 46.
t/local/43_misc_functions.t ............ 
Dubious, test returned 2 (wstat 512, 0x200)
Failed 2/46 subtests 

#   Failed test 'Server TLSv1 new_cb call count'
#   at t/local/44_sess.t line 307.
#          got: undef
#     expected: '1'

#   Failed test 'Server TLSv1 new_cb params were correct'
#   at t/local/44_sess.t line 308.
#          got: undef
#     expected: '1'

#   Failed test 'Server TLSv1 remove_cb call count'
#   at t/local/44_sess.t line 309.
#          got: '2'
#     expected: '1'

#   Failed test 'Client TLSv1 new_cb call count'
#   at t/local/44_sess.t line 312.
#          got: undef
#     expected: '1'

#   Failed test 'Client TLSv1 new_cb params were correct'
#   at t/local/44_sess.t line 313.
#          got: undef
#     expected: '1'

#   Failed test 'Client TLSv1 remove_cb call count'
#   at t/local/44_sess.t line 314.
#          got: undef
#     expected: '1'
   Failed test 'Client TLSv1 remove_cb params were correct'
#   at t/local/44_sess.t line 315.
#          got: undef
#     expected: '1'

#   Failed test 'Server TLSv1 session is resumable'
#   at t/local/44_sess.t line 321.
#          got: undef
#     expected: '1'

#   Failed test 'Client TLSv1 session is resumable'
#   at t/local/44_sess.t line 324.
#          got: undef
#     expected: '1'

#   Failed test 'Server TLSv1.1 new_cb call count'
#   at t/local/44_sess.t line 307.
#          got: undef
#     expected: '1'

#   Failed test 'Server TLSv1.1 new_cb params were correct'
#   at t/local/44_sess.t line 308.
#          got: undef
#     expected: '1'

#   Failed test 'Server TLSv1.1 remove_cb call count'
#   at t/local/44_sess.t line 309.
#          got: '2'
#     expected: '1'
#   Failed test 'Client TLSv1.1 new_cb call count'
#   at t/local/44_sess.t line 312.
#          got: undef
#     expected: '1'

#   Failed test 'Client TLSv1.1 new_cb params were correct'
#   at t/local/44_sess.t line 313.
#          got: undef
#     expected: '1'

#   Failed test 'Client TLSv1.1 remove_cb call count'
#   at t/local/44_sess.t line 314.
#          got: undef
#     expected: '1'

#   Failed test 'Client TLSv1.1 remove_cb params were correct'
#   at t/local/44_sess.t line 315.
#          got: undef
#     expected: '1'

#   Failed test 'Server TLSv1.1 session is resumable'
#   at t/local/44_sess.t line 321.
#          got: undef
#     expected: '1'

#   Failed test 'Client TLSv1.1 session is resumable'
#   at t/local/44_sess.t line 324.
#          got: undef
#     expected: '1'
# Looks like you failed 18 tests of 58.
t/local/44_sess.t ......................
Dubious, test returned 18 (wstat 4608, 0x1200)
Failed 18/58 subtests 
Use of uninitialized value $bytes2_0 in substr at t/local/45_exporter.t line 117.

#   Failed test 'TLSv1: Got enough for bytes1_0'
#   at t/local/45_exporter.t line 119.
#          got: undef
#     expected: '64'

#   Failed test 'TLSv1: Got enough for bytes1_1'
#   at t/local/45_exporter.t line 120.
#          got: undef
#     expected: '64'

#   Failed test 'TLSv1: Got enough for bytes1_2'
#   at t/local/45_exporter.t line 121.
#          got: undef
#     expected: '64'

#   Failed test 'TLSv1: Got enough for bytes1_3'
#   at t/local/45_exporter.t line 122.
#          got: undef
#     expected: '64'

#   Failed test 'TLSv1: Got enough for bytes2_0'
#   at t/local/45_exporter.t line 123.
#          got: undef
#     expected: '128'
Use of uninitialized value $bytes1_0 in unpack at t/local/45_exporter.t line 125.
Use of uninitialized value $bytes1_1 in unpack at t/local/45_exporter.t line 126.
Use of uninitialized value $bytes1_2 in unpack at t/local/45_exporter.t line 127.
Use of uninitialized value $bytes1_3 in unpack at t/local/45_exporter.t line 128.
Use of uninitialized value $bytes2_0 in unpack at t/local/45_exporter.t line 129.
#   Failed test 'TLSv1: empty and undefined context yields different values'
#   at t/local/45_exporter.t line 141.
#          got: ''
#     expected: anything else

#   Failed test 'TLSv1: different context'
#   at t/local/45_exporter.t line 145.
#          got: ''
#     expected: anything else
Use of uninitialized value in subroutine entry at t/local/45_exporter.t line 89.
Use of uninitialized value $bytes2_0 in substr at t/local/45_exporter.t line 117.

#   Failed test 'TLSv1.1: Got enough for bytes1_0'
#   at t/local/45_exporter.t line 119.
#          got: undef
#     expected: '64'

#   Failed test 'TLSv1.1: Got enough for bytes1_1'
#   at t/local/45_exporter.t line 120.
#          got: undef
#     expected: '64'

#   Failed test 'TLSv1.1: Got enough for bytes1_2'
#   at t/local/45_exporter.t line 121.
#          got: undef
#     expected: '64'

#   Failed test 'TLSv1.1: Got enough for bytes1_3'
#   at t/local/45_exporter.t line 122.
#          got: undef
#     expected: '64'
#   Failed test 'TLSv1.1: Got enough for bytes2_0'
#   at t/local/45_exporter.t line 123.
#          got: undef
#     expected: '128'
Use of uninitialized value $bytes1_0 in unpack at t/local/45_exporter.t line 125.
Use of uninitialized value $bytes1_1 in unpack at t/local/45_exporter.t line 126.
Use of uninitialized value $bytes1_2 in unpack at t/local/45_exporter.t line 127.
Use of uninitialized value $bytes1_3 in unpack at t/local/45_exporter.t line 128.
Use of uninitialized value $bytes2_0 in unpack at t/local/45_exporter.t line 129.

#   Failed test 'TLSv1.1: empty and undefined context yields different values'
#   at t/local/45_exporter.t line 141.
#          got: ''
#     expected: anything else

#   Failed test 'TLSv1.1: different context'
#   at t/local/45_exporter.t line 145.
#          got: ''
#     expected: anything else
Use of uninitialized value in subroutine entry at t/local/45_exporter.t line 89.
# Looks like you failed 14 tests of 36.
t/local/45_exporter.t .................. 
Dubious, test returned 14 (wstat 3584, 0xe00)
Failed 14/36 subtests 

Test Summary Report
-------------------
t/local/33_x509_create_cert.t        (Wstat: 256 Tests: 139 Failed: 1)
  Failed test:  37
  Non-zero exit status: 1
t/local/39_pkcs12.t                  (Wstat: 512 Tests: 19 Failed: 2)
  Failed tests:  11-12
  Non-zero exit status: 2
t/local/43_misc_functions.t          (Wstat: 512 Tests: 46 Failed: 2)
  Failed tests:  4, 9
  Non-zero exit status: 2
t/local/44_sess.t                    (Wstat: 4608 Tests: 58 Failed: 18)
  Failed tests:  2-4, 6-10, 12, 14-16, 18-22, 24
  Non-zero exit status: 18
t/local/45_exporter.t                (Wstat: 3584 Tests: 36 Failed: 14)
  Failed tests:  1-5, 7, 9-14, 16, 18
  Non-zero exit status: 14
Files=43, Tests=1835,  7 wallclock secs ( 0.20 usr  0.04 sys +  4.64 cusr  0.37 csys =  5.25 CPU)
Result: FAIL
Failed 5/43 test programs. 37/1835 subtests failed.
make: *** [Makefile:1110: test_dynamic] Error 255```

[zarniwhoop73]

I've been pointed to ​https://github.com/radiator-software/p5-net-ssleay/commit/7d7d74a409493e59850541c2a78519de768d2848.patch
​https://github.com/radiator-software/p5-net-ssleay/commit/0dd5d3d92d9e35e56ccb9d88108adb7a59fbcbca.patch
​https://github.com/radiator-software/p5-net-ssleay/commit/327550f61f5e1e932ea911e59ccc496ebb307030.patch
​https://github.com/radiator-software/p5-net-ssleay/pull/273.patch

With those applied it almost fixes it, 1 subtest fails:

#   at t/local/33_x509_create_cert.t line 98.
#                   undef
#     doesn't match '(?^:-----BEGIN (ENCRYPTED|RSA) PRIVATE KEY-----)'
# Looks like you failed 1 test of 139.
t/local/33_x509_create_cert.t ..........
Dubious, test returned 1 (wstat 256, 0x100)
Failed 1/139 subtests
        (less 1 skipped subtest: 137 okay)

Test Summary Report
------------------------------
t/local/33_x509_create_cert.t        (Wstat: 256 Tests: 139 Failed: 1)
  Failed test:  37
  Non-zero exit status: 1
Files=43, Tests=2817,  8 wallclock secs ( 0.27 usr  0.03 sys +  5.05 cusr  0.39 csys =  5.74 CPU)
Result: FAIL```

[h-vn]

Please try 1.91_01, the current developer release, https://metacpan.org/release/CHRISN/Net-SSLeay-1.91_01, which is also tagged in this repository and currently where the head is at. We'd be interested to hear how it goes.

There have been a lot of changes since 1.90 and these changes address, for example, OpenSSL 3.0 compatibility.

[zarniwhoop73]

(comment in passing - for people like me, it's easy to forget to hit the 'Comment' button after managing to get a comemt to render sensibly: I'd pointed to 3 upstream commits I'd been told about which fixed all but one of the failures. That is now water under the bridge.)

Yes, with 1.91_01 all tests pass for me, so people using openssl-3 should use that version or later. Thanks.