[WIP] Add a test case for the engine crash with AES-256-CTR

[bernd-edlinger]

Implement the AES-256-CTR cipher in the dasync engine.

Use that to reproduce the reported problems with the
devcrypto engine in our normal test environment.

See #17995 and #17532 for details.

WIP because it is not easy to fix, two attempts
failed miserably: 47f3506 and 178535e

[bernd-edlinger]

@paulidale or @mspncp maybe that's something for you?

[bernd-edlinger]

In case that is not clear, this is where the crash happens (see https://github.com/openssl/openssl/runs/5811306565?check_suite_focus=true):

Engine "dasync" set.
MemorySanitizer:DEADLYSIGNAL
==20144==ERROR: MemorySanitizer: SEGV on unknown address 0x000000000018 (pc 0x7fcec07cdd46 bp 0x7ffe01059440 sp 0x7ffe010593f0 T20144)
==20144==The signal is caused by a READ memory access.
==20144==Hint: address points to the zero page.
    #0 0x7fcec07cdd46 in pthread_rwlock_wrlock (/lib/x86_64-linux-gnu/libpthread.so.0+0xed46)
    #1 0x7fcec0bc51ea in CRYPTO_THREAD_write_lock /home/runner/work/openssl/openssl/crypto/threads_pthread.c:100:9
    #2 0x7fcec0ac986b in ENGINE_finish /home/runner/work/openssl/openssl/crypto/engine/eng_init.c:106:10
    #3 0x7fcec0b2034f in EVP_CIPHER_CTX_reset /home/runner/work/openssl/openssl/crypto/evp/evp_enc.c:65:5
    #4 0x7fcec0b205b2 in EVP_CIPHER_CTX_free /home/runner/work/openssl/openssl/crypto/evp/evp_enc.c:88:5
    #5 0x7fcec0e0b7bd in drbg_ctr_free /home/runner/work/openssl/openssl/providers/implementations/rands/drbg_ctr.c:618:9
    #6 0x7fcec0b36fe8 in EVP_RAND_CTX_free /home/runner/work/openssl/openssl/crypto/evp/evp_rand.c:385:5
    #7 0x7fcec0c33e1d in ossl_rand_ctx_free /home/runner/work/openssl/openssl/crypto/rand/rand_lib.c:483:5
    #8 0x7fcec0b93cea in context_deinit_objs /home/runner/work/openssl/openssl/crypto/context.c:223:9
    #9 0x7fcec0b92253 in context_deinit /home/runner/work/openssl/openssl/crypto/context.c:334:5
    #10 0x7fcec0b921d0 in ossl_lib_ctx_default_deinit /home/runner/work/openssl/openssl/crypto/context.c:370:5
    #11 0x7fcec0b9c5d4 in OPENSSL_cleanup /home/runner/work/openssl/openssl/crypto/init.c:423:5
    #12 0x449e8b in MSanCxaAtExitWrapper(void*) (/home/runner/work/openssl/openssl/apps/openssl+0x449e8b)
    #13 0x7fcec049fa26  (/lib/x86_64-linux-gnu/libc.so.6+0x49a26)
    #14 0x7fcec049fbdf in exit (/lib/x86_64-linux-gnu/libc.so.6+0x49bdf)
    #15 0x51f7ba in main /home/runner/work/openssl/openssl/apps/openssl.c:311:5
    #16 0x7fcec047d0b2 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x270b2)
    #17 0x43d21d in _start (/home/runner/work/openssl/openssl/apps/openssl+0x43d21d)
MemorySanitizer can not provide additional info.
SUMMARY: MemorySanitizer: SEGV (/lib/x86_64-linux-gnu/libpthread.so.0+0xed46) in pthread_rwlock_wrlock

So the DRBG picks up the AES-256-CTR from the dasync engine,
but the DRBG is cleaned up in ossl_lib_ctx_default_deinit() long after
engine_cleanup_int() destroyed the global_engine_lock.

In 1.1.1 this does not happen because rand_drbg_cleanup_int() used to
delete the master_drbg before engine_cleanup_int() was called.

[paulidale]

I think it unlikely I'll get anything but review time allocated for this 😢

[jaingaurav2712]

In case that is not clear, this is where the crash happens (see https://github.com/openssl/openssl/runs/5811306565?check_suite_focus=true):

Engine "dasync" set.
MemorySanitizer:DEADLYSIGNAL
==20144==ERROR: MemorySanitizer: SEGV on unknown address 0x000000000018 (pc 0x7fcec07cdd46 bp 0x7ffe01059440 sp 0x7ffe010593f0 T20144)
==20144==The signal is caused by a READ memory access.
==20144==Hint: address points to the zero page.
    #0 0x7fcec07cdd46 in pthread_rwlock_wrlock (/lib/x86_64-linux-gnu/libpthread.so.0+0xed46)
    #1 0x7fcec0bc51ea in CRYPTO_THREAD_write_lock /home/runner/work/openssl/openssl/crypto/threads_pthread.c:100:9
    #2 0x7fcec0ac986b in ENGINE_finish /home/runner/work/openssl/openssl/crypto/engine/eng_init.c:106:10
    #3 0x7fcec0b2034f in EVP_CIPHER_CTX_reset /home/runner/work/openssl/openssl/crypto/evp/evp_enc.c:65:5
    #4 0x7fcec0b205b2 in EVP_CIPHER_CTX_free /home/runner/work/openssl/openssl/crypto/evp/evp_enc.c:88:5
    #5 0x7fcec0e0b7bd in drbg_ctr_free /home/runner/work/openssl/openssl/providers/implementations/rands/drbg_ctr.c:618:9
    #6 0x7fcec0b36fe8 in EVP_RAND_CTX_free /home/runner/work/openssl/openssl/crypto/evp/evp_rand.c:385:5
    #7 0x7fcec0c33e1d in ossl_rand_ctx_free /home/runner/work/openssl/openssl/crypto/rand/rand_lib.c:483:5
    #8 0x7fcec0b93cea in context_deinit_objs /home/runner/work/openssl/openssl/crypto/context.c:223:9
    #9 0x7fcec0b92253 in context_deinit /home/runner/work/openssl/openssl/crypto/context.c:334:5
    #10 0x7fcec0b921d0 in ossl_lib_ctx_default_deinit /home/runner/work/openssl/openssl/crypto/context.c:370:5
    #11 0x7fcec0b9c5d4 in OPENSSL_cleanup /home/runner/work/openssl/openssl/crypto/init.c:423:5
    #12 0x449e8b in MSanCxaAtExitWrapper(void*) (/home/runner/work/openssl/openssl/apps/openssl+0x449e8b)
    #13 0x7fcec049fa26  (/lib/x86_64-linux-gnu/libc.so.6+0x49a26)
    #14 0x7fcec049fbdf in exit (/lib/x86_64-linux-gnu/libc.so.6+0x49bdf)
    #15 0x51f7ba in main /home/runner/work/openssl/openssl/apps/openssl.c:311:5
    #16 0x7fcec047d0b2 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x270b2)
    #17 0x43d21d in _start (/home/runner/work/openssl/openssl/apps/openssl+0x43d21d)
MemorySanitizer can not provide additional info.
SUMMARY: MemorySanitizer: SEGV (/lib/x86_64-linux-gnu/libpthread.so.0+0xed46) in pthread_rwlock_wrlock

So the DRBG picks up the AES-256-CTR from the dasync engine, but the DRBG is cleaned up in ossl_lib_ctx_default_deinit() long after engine_cleanup_int() destroyed the global_engine_lock.

In 1.1.1 this does not happen because rand_drbg_cleanup_int() used to delete the master_drbg before engine_cleanup_int() was called.

why the sequence of cleanup is reversed in OpenSSL 3.0?

[jaingaurav2712]

@bernd-edlinger @paulidale : This issue is critical for our release. can you please help to provide an update?