Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

tests fail with OpenSSL-3.x #431

Closed
mouse07410 opened this issue Nov 14, 2021 · 8 comments
Closed

tests fail with OpenSSL-3.x #431

mouse07410 opened this issue Nov 14, 2021 · 8 comments

Comments

@mouse07410
Copy link
Contributor

###Config
MacOS Big Sur 11.6.1, Xcode-13.1, OpenSSL-3.0.0 (Macports-installed to /opt/local) and 3.1.dev (locally installed into $HOME/openssl-3).

Problem

Majority of the tests are failing now, like this:

$ cat tests/rsa-evp-sign.softhsm.log
Current directory: /Users/ur20980/src/libp11/tests
Source directory: .
Output directory: output.45386
* Initializing smart card libp11-test...ok
Importing
Finished
***************
Listing objects
***************
Using slot 0 with a present token (0x63f35e2)
Public Key Object; RSA 2048 bits
  label:      server-key
  ID:         01020304
  Usage:      encrypt, verify, wrap
  Access:     none
Certificate Object; type = X.509 cert
  label:      server-key
  subject:    DN: CN=server-1
  ID:         01020304
Private Key Object; RSA 
  label:      server-key
  ID:         01020304
  Usage:      decrypt, sign, unwrap
  Access:     sensitive
pkcs11:token=libp11-test;id=%01%02%03%04;object=server-key;type=private;pin-value=1234
Invalid command '--version'; type "help" for a list.
./rsa-evp-sign.softhsm: line 35: 45399 Segmentation fault: 11  ./evp-sign ctrl false "${outdir}/engines.cnf" ${PRIVATE_KEY} ${PUBLIC_KEY} ${MODULE}
Basic PKCS #11 test, using ctrl failed
FAIL rsa-evp-sign.softhsm (exit status: 1)

Crash report:

Process:               evp-sign [45399]
Path:                  /Users/USER/*/evp-sign
Identifier:            evp-sign
Version:               0
Code Type:             X86-64 (Native)
Parent Process:        bash [45386]
Responsible:           Terminal [813]
User ID:               501

Date/Time:             2021-11-14 12:04:53.870 -0500
OS Version:            macOS 11.6.1 (20G224)
Report Version:        12
Bridge OS Version:     6.0 (19P548)
Anonymous UUID:        BD844EB9-9C6F-867E-78EB-1ACDA55970A0


Time Awake Since Boot: 350000 seconds

System Integrity Protection: enabled

Crashed Thread:        0  Dispatch queue: com.apple.main-thread

Exception Type:        EXC_BAD_ACCESS (SIGSEGV)
Exception Codes:       KERN_INVALID_ADDRESS at 0x0000000000000000
Exception Note:        EXC_CORPSE_NOTIFY

Termination Signal:    Segmentation fault: 11
Termination Reason:    Namespace SIGNAL, Code 0xb
Terminating Process:   exc handler [45399]

VM Regions Near 0:
--> 
    __TEXT                      10a644000-10a648000    [   16K] r-x/r-x SM=COW  /Users/*

Thread 0 Crashed:: Dispatch queue: com.apple.main-thread
0   libsystem_pthread.dylib       	0x00007fff20511b7b pthread_rwlock_wrlock + 0
1   libcrypto.3.dylib             	0x000000010af0f9ac CRYPTO_THREAD_write_lock + 9
2   libcrypto.3.dylib             	0x000000010aec6152 engine_table_register + 52
3   libcrypto.3.dylib             	0x000000010aec6a91 ENGINE_register_ciphers + 75
4   gost.dylib                    	0x000000010ad00e22 bind_engine + 562
5   libcrypto.3.dylib             	0x000000010a8822a0 dynamic_load + 203 (eng_dyn.c:487) [inlined]
6   libcrypto.3.dylib             	0x000000010a8822a0 dynamic_ctrl + 1840 (eng_dyn.c:344)
7   libcrypto.3.dylib             	0x000000010a881a0c ENGINE_ctrl_cmd_string + 604
8   libcrypto.3.dylib             	0x000000010a880f55 int_engine_configure + 625 [inlined]
9   libcrypto.3.dylib             	0x000000010a880f55 int_engine_module_init + 805 (eng_cnf.c:164)
10  libcrypto.3.dylib             	0x000000010a7d2960 module_init + 131 (conf_mod.c:375) [inlined]
11  libcrypto.3.dylib             	0x000000010a7d2960 module_run + 488 (conf_mod.c:239) [inlined]
12  libcrypto.3.dylib             	0x000000010a7d2960 CONF_modules_load + 800 (conf_mod.c:138)
13  libcrypto.3.dylib             	0x000000010a7d2ca2 CONF_modules_load_file_ex + 114 (conf_mod.c:181)
14  libcrypto.3.dylib             	0x000000010a7d3563 ossl_config_int + 67 (conf_sap.c:64)
15  libcrypto.3.dylib             	0x000000010a8d3fab ossl_init_config + 7 (init.c:242) [inlined]
16  libcrypto.3.dylib             	0x000000010a8d3fab ossl_init_config_ossl_ + 11 (init.c:240)
17  libsystem_pthread.dylib       	0x00007fff205111e1 __pthread_once_handler + 65
18  libsystem_platform.dylib      	0x00007fff205575c6 _os_once_callout + 18
19  libsystem_pthread.dylib       	0x00007fff2051118f pthread_once + 74
20  libcrypto.3.dylib             	0x000000010a8e1cf9 CRYPTO_THREAD_run_once + 9 (threads_pthread.c:144)
21  libcrypto.3.dylib             	0x000000010a8d3bee OPENSSL_init_crypto + 1134 (init.c:584)
22  evp-sign                      	0x000000010a64754f main + 399 (evp-sign.c:190)
23  libdyld.dylib                 	0x00007fff2052ff3d start + 1

Thread 0 crashed with X86 Thread State (64-bit):
  rax: 0x0000000000000011  rbx: 0x00007f9c23d07230  rcx: 0x000000010add3310  rdx: 0x00007f9c23d07230
  rdi: 0x0000000000000000  rsi: 0x000000010aec6aa0  rbp: 0x00007ffee55b90c0  rsp: 0x00007ffee55b90b8
   r8: 0x0000000000000011   r9: 0x0000000000000000  r10: 0x00000000fffffffc  r11: 0x00007f9b18f3dd4c
  r12: 0x000000010b097ac8  r13: 0x000000010aec6aa0  r14: 0x0000000000000011  r15: 0x000000010b099808
  rip: 0x00007fff20511b7b  rfl: 0x0000000000010202  cr2: 0x0000000000000000
  
Logical CPU:     4
Error Code:      0x00000004 (no mapping for user data read)
Trap Number:     14

Thread 0 instruction stream:
  20 89 c0 48 09 c8 48 09-da 44 89 c9 48 09 d9 48   ..H..H..D..H..H
  c1 e6 20 44 89 d3 48 09-f3 f0 49 0f c7 08 40 0f  .. D..H...I...@.
  95 c6 48 89 c1 48 c1 e9-20 48 89 d3 48 c1 eb 20  ..H..H.. H..H.. 
  40 84 f6 74 27 eb 9c 89-ce 41 89 c2 a8 07 74 b7  @..t'....A....t.
  41 89 cb 41 81 e3 00 ff-ff ff 89 ce 41 89 c2 45  A..A........A..E
  39 d9 75 a3 31 f6 5b e9-6d 18 00 00 31 c0 5b c3  9.u.1.[.m...1.[.
 [48]81 3f 4b 4c 57 52 75-6c 4c 8d 57 2f 49 83 e2  H.?KLWRulL.W/I..	<==
  f0 49 8b 02 a8 02 75 5d-49 b9 00 00 00 00 ff ff  .I....u]I.......
  ff ff 49 b8 00 00 00 00-00 01 00 00 48 89 c6 a8  ..I.........H...
  40 74 42 48 89 f2 4c 21-ca 89 c1 48 09 d1 25 00  @tBH..L!...H..%.
  ff ff ff 05 83 01 00 00-4c 01 c6 4c 21 ce 48 09  ........L..L!.H.
  c6 48 89 c8 f0 49 0f b1-32 48 89 c6 75 d1 48 83  .H...I..2H..u.H.
  
Thread 0 last branch register state not available.


Binary Images:
       0x10a644000 -        0x10a647fff +evp-sign (0) <7027854C-8906-3774-8D3A-DA317330DFB6> /Users/USER/*/evp-sign
       0x10a659000 -        0x10a664fff +libp11.2.dylib (0) <FAF64B14-733F-3B7D-9C4C-94CE3F4CD8E5> /Users/USER/*/libp11.2.dylib
       0x10a67c000 -        0x10a6effff +libssl.3.dylib (0) <AFE0892C-2B50-39E7-BDDE-6A340FD5DACD> /Users/USER/*/libssl.3.dylib
       0x10a744000 -        0x10aa47fff +libcrypto.3.dylib (0) <BA2C2CB6-38C3-3EBB-A62A-24D5486A7880> /Users/USER/*/libcrypto.3.dylib
       0x10aca9000 -        0x10acb8fff +legacy.dylib (0) <5CC3E84C-EA7E-3716-9C03-29802DA977FD> /Users/USER/*/legacy.dylib
       0x10acd9000 -        0x10ace8fff +pkcs11.dylib (0) <716CE783-DC4D-36C4-A478-EA3B160F03E9> /Users/USER/*/pkcs11.dylib
       0x10acfd000 -        0x10adccfff +gost.dylib (0) <5ED05BB0-09DE-3245-B12C-0B84D56130FD> /opt/local/libexec/*/gost.dylib
       0x10ade1000 -        0x10b03cfff +libcrypto.3.dylib (0) <A43BACD0-8559-3294-9EF3-042FE26EA638> /opt/local/libexec/*/libcrypto.3.dylib
       0x10b14d000 -        0x10b160fff +libz.1.dylib (0) <1DE622A3-18A9-348E-B8C0-C68B7BF55B85> /opt/local/lib/libz.1.dylib
       0x10d659000 -        0x10d6f4fff  dyld (852.2) <14C5D17F-5032-37A2-9CD8-F437EB4A3992> /usr/lib/dyld
    0x7fff2024c000 -     0x7fff2024dfff  libsystem_blocks.dylib (79) <4DF6D8F5-D9C2-3A96-8DE4-5E99D6B73DC8> /usr/lib/system/libsystem_blocks.dylib
    0x7fff2024e000 -     0x7fff20283fff  libxpc.dylib (2038.120.1) <0397FC9F-BD11-31FC-882E-9EDAA1E5CA65> /usr/lib/system/libxpc.dylib
    0x7fff20284000 -     0x7fff2029bfff  libsystem_trace.dylib (1277.120.1) <0A24EB90-5396-36B0-A7E6-E9288702856D> /usr/lib/system/libsystem_trace.dylib
    0x7fff2029c000 -     0x7fff20339fff  libcorecrypto.dylib (1000.140.4) <57E7471E-3960-3398-8918-20DF446EA99B> /usr/lib/system/libcorecrypto.dylib
    0x7fff2033a000 -     0x7fff20366fff  libsystem_malloc.dylib (317.140.5) <10C22FD0-FC7B-3325-852E-FEC4E88D2BC5> /usr/lib/system/libsystem_malloc.dylib
    0x7fff20367000 -     0x7fff203abfff  libdispatch.dylib (1271.120.2) <BA7AD614-F2C2-3E89-9043-43DD548AE5B1> /usr/lib/system/libdispatch.dylib
    0x7fff203ac000 -     0x7fff203e5fff  libobjc.A.dylib (824) <6C842797-3EA9-30B4-A36D-E948DAF0EC89> /usr/lib/libobjc.A.dylib
    0x7fff203e6000 -     0x7fff203e8fff  libsystem_featureflags.dylib (28.60.1) <6EB33926-8E22-3000-B2F1-C6182B8CBD8F> /usr/lib/system/libsystem_featureflags.dylib
    0x7fff203e9000 -     0x7fff20471fff  libsystem_c.dylib (1439.141.1) <8447A4B8-0751-3EF1-AA9B-042E40EFA07D> /usr/lib/system/libsystem_c.dylib
    0x7fff20472000 -     0x7fff204c7fff  libc++.1.dylib (905.6) <81674ADB-670F-3B19-AF5D-F3F66CB93D44> /usr/lib/libc++.1.dylib
    0x7fff204c8000 -     0x7fff204ddfff  libc++abi.dylib (905.6) <78CE7863-E224-3D0B-98F1-E5E3E382322D> /usr/lib/libc++abi.dylib
    0x7fff204de000 -     0x7fff2050dfff  libsystem_kernel.dylib (7195.141.8) <8DFE5B9F-B718-3D48-ABB1-238122989218> /usr/lib/system/libsystem_kernel.dylib
    0x7fff2050e000 -     0x7fff20519fff  libsystem_pthread.dylib (454.120.2) <49670AEC-4D5D-3383-906C-23F568351FCB> /usr/lib/system/libsystem_pthread.dylib
    0x7fff2051a000 -     0x7fff20555fff  libdyld.dylib (852.2) <5FBD0E1A-ACCE-36DB-B11C-622F26C85132> /usr/lib/system/libdyld.dylib
    0x7fff20556000 -     0x7fff2055ffff  libsystem_platform.dylib (254.80.2) <A85D12B6-6213-34EE-84D8-8E993C19E330> /usr/lib/system/libsystem_platform.dylib
    0x7fff20560000 -     0x7fff2058bfff  libsystem_info.dylib (542.40.3) <36329FC6-9982-306A-87F9-D018E7F49F4B> /usr/lib/system/libsystem_info.dylib
    0x7fff2295b000 -     0x7fff22964fff  libsystem_darwin.dylib (1439.141.1) <29F82ABE-E1A0-3BC2-B91E-ADC010CF23FA> /usr/lib/system/libsystem_darwin.dylib
    0x7fff22d7a000 -     0x7fff22d85fff  libsystem_notify.dylib (279.40.4) <608B5A07-D31A-3BEC-86BF-C2E498C085AF> /usr/lib/system/libsystem_notify.dylib
    0x7fff24d12000 -     0x7fff24d20fff  libsystem_networkextension.dylib (1295.140.3) <022BE26C-C058-3CC7-8E0B-348B3D3B639C> /usr/lib/system/libsystem_networkextension.dylib
    0x7fff24d7f000 -     0x7fff24d95fff  libsystem_asl.dylib (385) <47B99A49-749E-356F-9D81-6D3A0B1544AB> /usr/lib/system/libsystem_asl.dylib
    0x7fff2645c000 -     0x7fff26463fff  libsystem_symptoms.dylib (1431.140.1) <EA6435E7-8F85-315B-8AED-C20A07DE7F96> /usr/lib/system/libsystem_symptoms.dylib
    0x7fff2846b000 -     0x7fff2847bfff  libsystem_containermanager.dylib (318.100.4) <E0A89571-6D3E-3184-9F39-C6094C87B92B> /usr/lib/system/libsystem_containermanager.dylib
    0x7fff2917f000 -     0x7fff29182fff  libsystem_configuration.dylib (1109.140.1) <89E70992-616F-3DD3-9430-67025F759A1B> /usr/lib/system/libsystem_configuration.dylib
    0x7fff29183000 -     0x7fff29187fff  libsystem_sandbox.dylib (1441.141.4) <087B0ADD-EF39-3E80-800F-F111FD65ABBA> /usr/lib/system/libsystem_sandbox.dylib
    0x7fff29e79000 -     0x7fff29e7bfff  libquarantine.dylib (119.40.2) <B1418CB0-CDC7-3EB2-ABC3-8609ADC92BC4> /usr/lib/system/libquarantine.dylib
    0x7fff2a424000 -     0x7fff2a428fff  libsystem_coreservices.dylib (127.1) <7AE405FC-6A44-34A0-86AD-6BD80B0050BB> /usr/lib/system/libsystem_coreservices.dylib
    0x7fff2a638000 -     0x7fff2a67ffff  libsystem_m.dylib (3186.100.3) <EA0354A3-8618-3D76-A760-E550AC60CE95> /usr/lib/system/libsystem_m.dylib
    0x7fff2a681000 -     0x7fff2a686fff  libmacho.dylib (980) <1FCE2BE3-4F6F-3EAA-9BC5-A9892A45CF0D> /usr/lib/system/libmacho.dylib
    0x7fff2a6a3000 -     0x7fff2a6aefff  libcommonCrypto.dylib (60178.120.3) <7E242F29-1CB6-30EF-8C9A-C768A90FDBA0> /usr/lib/system/libcommonCrypto.dylib
    0x7fff2a6af000 -     0x7fff2a6b9fff  libunwind.dylib (201) <A5B040A8-847F-36EE-B13D-5DD1F5CD5BED> /usr/lib/system/libunwind.dylib
    0x7fff2a6ba000 -     0x7fff2a6c1fff  liboah.dylib (203.58) <26D08622-69F5-32DB-80D2-9B4651A9F0CC> /usr/lib/liboah.dylib
    0x7fff2a6c2000 -     0x7fff2a6ccfff  libcopyfile.dylib (173.40.2) <8C783785-0F5F-3DC5-B815-B29CEBA75737> /usr/lib/system/libcopyfile.dylib
    0x7fff2a6cd000 -     0x7fff2a6d4fff  libcompiler_rt.dylib (102.2) <1FDC92D1-8A17-30AF-8E72-4F0517AEA157> /usr/lib/system/libcompiler_rt.dylib
    0x7fff2a6d5000 -     0x7fff2a6d7fff  libsystem_collections.dylib (1439.141.1) <11D5775A-AD4C-35ED-BC05-616AB67ACBBE> /usr/lib/system/libsystem_collections.dylib
    0x7fff2a6d8000 -     0x7fff2a6dafff  libsystem_secinit.dylib (87.60.1) <27982311-637E-3308-9F55-4871762736F4> /usr/lib/system/libsystem_secinit.dylib
    0x7fff2a6db000 -     0x7fff2a6ddfff  libremovefile.dylib (49.120.1) <5973CED7-797B-3288-9589-C1856752F91A> /usr/lib/system/libremovefile.dylib
    0x7fff2a6de000 -     0x7fff2a6defff  libkeymgr.dylib (31) <AAA929A0-45E5-3186-8ABD-37EB25B2C939> /usr/lib/system/libkeymgr.dylib
    0x7fff2a6df000 -     0x7fff2a6e6fff  libsystem_dnssd.dylib (1310.140.1) <4A85F13B-E3AD-3C44-B327-091F56D35CC1> /usr/lib/system/libsystem_dnssd.dylib
    0x7fff2a6e7000 -     0x7fff2a6ecfff  libcache.dylib (83) <435668CB-666B-3379-AD65-F604A72099F4> /usr/lib/system/libcache.dylib
    0x7fff2a6ed000 -     0x7fff2a6eefff  libSystem.B.dylib (1292.120.1) <2553EE0F-3C42-3ACE-9400-EF055BD5BE8E> /usr/lib/libSystem.B.dylib
    0x7fff2daf1000 -     0x7fff2daf1fff  liblaunch.dylib (2038.120.1) <A3F46CC6-BD89-39DB-8732-C885B881A635> /usr/lib/system/liblaunch.dylib
    0x7fff2ff88000 -     0x7fff2ff88fff  libsystem_product_info_filter.dylib (8.40.1) <6CA8DEA4-5BD4-375F-9AA7-3338135306C5> /usr/lib/system/libsystem_product_info_filter.dylib

engine.cnf:

$ cat tests/output.45386/engines.cnf 
HOME			= .
RANDFILE		= $ENV::HOME/.rnd

openssl_conf = openssl_def

[openssl_def]
engines = engine_section

[engine_section]
pkcs11 = pkcs11_section

[pkcs11_section]
engine_id = pkcs11
dynamic_path = ../src/.libs/pkcs11.so
MODULE_PATH = /opt/local/lib/softhsm/libsofthsm2.so
init = 0

Same results/errors from my fork and your master.
test-conf-out.txt
test-check-out.txt
test-suite.log
ec-cert-store.softhsm.log
search-all-matching-tokens.softhsm.log
ec-check-privkey.softhsm.log
rsa-oaep.softhsm.log
rsa-evp-sign.softhsm.log

Process:               evp-sign [59722]
Path:                  /Users/USER/*/evp-sign
Identifier:            evp-sign
Version:               0
Code Type:             X86-64 (Native)
Parent Process:        bash [59700]
Responsible:           Terminal [813]
User ID:               501

Date/Time:             2021-11-14 12:32:10.603 -0500
OS Version:            macOS 11.6.1 (20G224)
Report Version:        12
Bridge OS Version:     6.0 (19P548)
Anonymous UUID:        BD844EB9-9C6F-867E-78EB-1ACDA55970A0


Time Awake Since Boot: 350000 seconds

System Integrity Protection: enabled

Crashed Thread:        0  Dispatch queue: com.apple.main-thread

Exception Type:        EXC_BAD_ACCESS (SIGSEGV)
Exception Codes:       KERN_INVALID_ADDRESS at 0x0000000000000000
Exception Note:        EXC_CORPSE_NOTIFY

Termination Signal:    Segmentation fault: 11
Termination Reason:    Namespace SIGNAL, Code 0xb
Terminating Process:   exc handler [59722]

VM Regions Near 0:
--> 
    __TEXT                      1015f9000-1015fd000    [   16K] r-x/r-x SM=COW  /Users/*

Thread 0 Crashed:: Dispatch queue: com.apple.main-thread
0   libsystem_pthread.dylib       	0x00007fff20511b7b pthread_rwlock_wrlock + 0
1   libcrypto.3.dylib             	0x0000000101df49ac CRYPTO_THREAD_write_lock + 9
2   libcrypto.3.dylib             	0x0000000101dab152 engine_table_register + 52
3   libcrypto.3.dylib             	0x0000000101daba91 ENGINE_register_ciphers + 75
4   gost.dylib                    	0x0000000101be5e22 bind_engine + 562
5   libcrypto.3.dylib             	0x000000010176b2a0 dynamic_load + 203 (eng_dyn.c:487) [inlined]
6   libcrypto.3.dylib             	0x000000010176b2a0 dynamic_ctrl + 1840 (eng_dyn.c:344)
7   libcrypto.3.dylib             	0x000000010176aa0c ENGINE_ctrl_cmd_string + 604
8   libcrypto.3.dylib             	0x0000000101769f55 int_engine_configure + 625 [inlined]
9   libcrypto.3.dylib             	0x0000000101769f55 int_engine_module_init + 805 (eng_cnf.c:164)
10  libcrypto.3.dylib             	0x00000001016bb960 module_init + 131 (conf_mod.c:375) [inlined]
11  libcrypto.3.dylib             	0x00000001016bb960 module_run + 488 (conf_mod.c:239) [inlined]
12  libcrypto.3.dylib             	0x00000001016bb960 CONF_modules_load + 800 (conf_mod.c:138)
13  libcrypto.3.dylib             	0x00000001016bbca2 CONF_modules_load_file_ex + 114 (conf_mod.c:181)
14  libcrypto.3.dylib             	0x00000001016bc563 ossl_config_int + 67 (conf_sap.c:64)
15  libcrypto.3.dylib             	0x00000001017bcfab ossl_init_config + 7 (init.c:242) [inlined]
16  libcrypto.3.dylib             	0x00000001017bcfab ossl_init_config_ossl_ + 11 (init.c:240)
17  libsystem_pthread.dylib       	0x00007fff205111e1 __pthread_once_handler + 65
18  libsystem_platform.dylib      	0x00007fff205575c6 _os_once_callout + 18
19  libsystem_pthread.dylib       	0x00007fff2051118f pthread_once + 74
20  libcrypto.3.dylib             	0x00000001017cacf9 CRYPTO_THREAD_run_once + 9 (threads_pthread.c:144)
21  libcrypto.3.dylib             	0x00000001017bcbee OPENSSL_init_crypto + 1134 (init.c:584)
22  evp-sign                      	0x00000001015fc54f main + 399
23  libdyld.dylib                 	0x00007fff2052ff3d start + 1
@mouse07410
Copy link
Contributor Author

On another machine, tests with OpenSSL-3 pass (except pkcs11-uri-without-token.softhsm and search-all-matching-tokens.softhsm).

Any recommendation how to track this problem down?

@Jakuje
Copy link
Member

Jakuje commented Nov 16, 2021

Do you have master version or latest release? There are were couple of issues that surfaced with 3.0, for example #406

With the above change, libp11 works fine for us in Fedora with OpenSSL 3.0.

Invalid command '--version'; type "help" for a list.

This sounds like some command/tool is missing on your machine. Running the script with "set -x" or something to log what is actually being called might give you some more information.

@mouse07410
Copy link
Contributor Author

Do you have master version or latest release?

Master version. Compiled and installed from the source.

invalid command '--version'; type "help" for a list.

This sounds like some command/tool is missing on your machine

That was caused by my silly mistake - I added openssl --version to the tests/rsa-evp-sign.softhsm script, forgetting that it should be openssl version.

And, to my surprise, the tests with OpenSSL-3.0.0 all pass.

However, with OpenSSL-3.1.dev 13 out of 15 still fail.

ossl3-build.txt
test-suite.log

Crash reports for some tests:

Process:               evp-sign [63899]
Path:                  /Users/USER/*/evp-sign
Identifier:            evp-sign
Version:               0
Code Type:             X86-64 (Native)
Parent Process:        bash [63869]
Responsible:           Terminal [813]
User ID:               501

Date/Time:             2021-11-16 12:16:53.505 -0500
OS Version:            macOS 11.6.1 (20G224)
Report Version:        12
Bridge OS Version:     6.0 (19P548)
Anonymous UUID:        BD844EB9-9C6F-867E-78EB-1ACDA55970A0


Time Awake Since Boot: 530000 seconds

System Integrity Protection: enabled

Crashed Thread:        0  Dispatch queue: com.apple.main-thread

Exception Type:        EXC_BAD_ACCESS (SIGSEGV)
Exception Codes:       KERN_INVALID_ADDRESS at 0x0000000000000000
Exception Note:        EXC_CORPSE_NOTIFY

Termination Signal:    Segmentation fault: 11
Termination Reason:    Namespace SIGNAL, Code 0xb
Terminating Process:   exc handler [63899]

VM Regions Near 0:
--> 
    __TEXT                      10f53e000-10f542000    [   16K] r-x/r-x SM=COW  /Users/*

Thread 0 Crashed:: Dispatch queue: com.apple.main-thread
0   libsystem_pthread.dylib       	0x00007fff20511b7b pthread_rwlock_wrlock + 0
1   libcrypto.3.dylib             	0x000000010fe999ac CRYPTO_THREAD_write_lock + 9
2   libcrypto.3.dylib             	0x000000010fe4e773 ENGINE_finish + 30
3   libsofthsm2.so                	0x000000010fce2b58 OSSLCryptoFactory::~OSSLCryptoFactory() + 28
4   libsofthsm2.so                	0x000000010fce2bf4 OSSLCryptoFactory::~OSSLCryptoFactory() + 14
5   libsystem_c.dylib             	0x00007fff20443d25 __cxa_finalize_ranges + 316
6   libsystem_c.dylib             	0x00007fff20444010 exit + 53
7   libdyld.dylib                 	0x00007fff2052ff44 start + 8

Process:               rsa-oaep [64054]
Path:                  /Users/USER/*/rsa-oaep
Identifier:            rsa-oaep
Version:               0
Code Type:             X86-64 (Native)
Parent Process:        bash [64041]
Responsible:           Terminal [813]
User ID:               501

Date/Time:             2021-11-16 12:16:59.916 -0500
OS Version:            macOS 11.6.1 (20G224)
Report Version:        12
Bridge OS Version:     6.0 (19P548)
Anonymous UUID:        BD844EB9-9C6F-867E-78EB-1ACDA55970A0


Time Awake Since Boot: 530000 seconds

System Integrity Protection: enabled

Crashed Thread:        0  Dispatch queue: com.apple.main-thread

Exception Type:        EXC_BAD_ACCESS (SIGSEGV)
Exception Codes:       KERN_INVALID_ADDRESS at 0x0000000000000000
Exception Note:        EXC_CORPSE_NOTIFY

Termination Signal:    Segmentation fault: 11
Termination Reason:    Namespace SIGNAL, Code 0xb
Terminating Process:   exc handler [64054]

VM Regions Near 0:
--> 
    __TEXT                      1086fe000-108702000    [   16K] r-x/r-x SM=COW  /Users/*

Thread 0 Crashed:: Dispatch queue: com.apple.main-thread
0   libsystem_pthread.dylib       	0x00007fff20511b7b pthread_rwlock_wrlock + 0
1   libcrypto.3.dylib             	0x000000010905a9ac CRYPTO_THREAD_write_lock + 9
2   libcrypto.3.dylib             	0x000000010900f773 ENGINE_finish + 30
3   libsofthsm2.so                	0x0000000108ea3b58 OSSLCryptoFactory::~OSSLCryptoFactory() + 28
4   libsofthsm2.so                	0x0000000108ea3bf4 OSSLCryptoFactory::~OSSLCryptoFactory() + 14
5   libsystem_c.dylib             	0x00007fff20443d25 __cxa_finalize_ranges + 316
6   libsystem_c.dylib             	0x00007fff20444010 exit + 53
7   libdyld.dylib                 	0x00007fff2052ff44 start + 8

It looks like all the crashes are related to pthread locking???

@mouse07410
Copy link
Contributor Author

Here's a bit more details. It looks like the problem comes when the engine finishes:

OpenSSL crypto demo completed.
AddressSanitizer:DEADLYSIGNAL
=================================================================
==18400==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000000 (pc 0x7fff2035db7b bp 0x7ffeea264ae0 sp 0x7ffeea264ad8 T0)
==18400==The signal is caused by a READ memory access.
==18400==Hint: address points to the zero page.
    #0 0x7fff2035db7b in pthread_rwlock_wrlock+0x0 (libsystem_pthread.dylib:x86_64+0x3b7b)
    #1 0x109ad3772 in ENGINE_finish+0x1d (libcrypto.3.dylib:x86_64+0xe3772)
    #2 0x109967b57 in OSSLCryptoFactory::~OSSLCryptoFactory()+0x1b (libsofthsm2.so:x86_64+0x3bb57)
    #3 0x109967bf3 in OSSLCryptoFactory::~OSSLCryptoFactory()+0xd (libsofthsm2.so:x86_64+0x3bbf3)
    #4 0x7fff2028fd24 in __cxa_finalize_ranges+0x13b (libsystem_c.dylib:x86_64+0x5ad24)
    #5 0x7fff2029000f in exit+0x34 (libsystem_c.dylib:x86_64+0x5b00f)
    #6 0x7fff2037bf43 in start+0x7 (libdyld.dylib:x86_64+0x15f43)

==18400==Register values:
rax = 0x00000001099aed50  rbx = 0x0000611000020340  rcx = 0x00007fff80657238  rdx = 0x0000000000000000  
rdi = 0x0000000000000000  rsi = 0x00000001059c6030  rbp = 0x00007ffeea264ae0  rsp = 0x00007ffeea264ad8  
 r8 = 0x000000000073cf3a   r9 = 0xffffffff00000000  r10 = 0x00007fff80657230  r11 = 0x00000fffffffffff  
r12 = 0x0000000000000002  r13 = 0x0000000000000000  r14 = 0x0000000109ca8808  r15 = 0x000000000000000d  
AddressSanitizer can not provide additional info.
SUMMARY: AddressSanitizer: SEGV (libsystem_pthread.dylib:x86_64+0x3b7b) in pthread_rwlock_wrlock+0x0
==18400==ABORTING
Abort trap: 6

Somewhere between libp11 and OpenSSL there's double-free during cleanup when engine finishes.

Any help, please?

@Jakuje
Copy link
Member

Jakuje commented Nov 21, 2021

I would assume there are already some unstable stuff in the 3.1 branch. I would suggest you to stick to the 3.0 branch if you want to be up to date. 3.1 branch is not even in alpha as far as I know so I would bet on some issue/incompatibility on the openssl side.

@mouse07410
Copy link
Contributor Author

Understood, but I observe the same issue with OpenSSL-1.1.1.

Somehow, 3.0.0 doesn't exhibit it.

@mouse07410
Copy link
Contributor Author

I found the problem.

  • When I install Macports-built SoftHSMv2, OpenSSL dies with SEGV upon engine cleanup.
  • When I build and install OpenHSMv2 from sources (linking it against OpenSSL-1.1.1), all is good for OpenSSL 1.1.1, 3.0.0, and 3.1.dev:
$ openssl11 pkeyutl -engine pkcs11 -keyform engine -decrypt -pkeyopt rsa_padding_mode:oaep -pkeyopt rsa_oaep_md:sha384 -pkeyopt rsa_mgf1_md:sha384 -inkey "pkcs11:model=YubiHSM;manufacturer=Yubico%20%28www.yubico.com%29;token=YubiHSM;id=%04%02;object=RSA-OAEP;type=private" -in /tmp/oaep.oaep
engine "pkcs11" set.
Enter PKCS#11 token PIN for YubiHSM:
6e6a8f11eddcec5c2c4761b5d5a4389afab878c48e790cf3
$ openssl3 version
OpenSSL 3.1.0-dev  (Library: OpenSSL 3.1.0-dev )
$ openssl3 pkeyutl -engine pkcs11 -keyform engine -decrypt -pkeyopt rsa_padding_mode:oaep -pkeyopt rsa_oaep_md:sha384 -pkeyopt rsa_mgf1_md:sha384 -inkey "pkcs11:model=YubiHSM;manufacturer=Yubico%20%28www.yubico.com%29;token=YubiHSM;id=%04%02;object=RSA-OAEP;type=private" -in /tmp/oaep.oaep
Engine "pkcs11" set.
Enter PKCS#11 token PIN for YubiHSM:
6e6a8f11eddcec5c2c4761b5d5a4389afab878c48e790cf3
$

@mtrojnar
Copy link
Member

mtrojnar commented Mar 10, 2022 •
edited

Does your Macports-built SoftHSMv2 include this commit? opendnssec/SoftHSMv2@2793f3c
Otherwise, we're fighting the non-reentrant OpenSSL's engine initialization/finalization again.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@Jakuje @mouse07410 @mtrojnar