Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Not released memory if legacy provider or own library context is used. #16516

Open
mbroz opened this issue Sep 6, 2021 · 15 comments
Open

Not released memory if legacy provider or own library context is used. #16516

mbroz opened this issue Sep 6, 2021 · 15 comments
Labels
branch: master Merge to master branch branch: 3.0 Merge to openssl-3.0 branch triaged: bug The issue/pr is/fixes a bug triaged: OTC evaluated This issue/pr was triaged by OTC

Comments

@mbroz
Copy link
Contributor

mbroz commented Sep 6, 2021

Hi,
we have to use legacy provider in libcryptsetup and when running under valgrind, a lot of still-in-use memory is reported.
I reproduced it with current openssl-3.0 branch on Debian, configured and installed just as

./Configure shared --prefix=/usr --openssldir=/usr/lib/ssl --libdir=lib/x86_64-linux-gnu linux-x86_64

and this simple C standalone reproducer:

#include <stdio.h>
#include <openssl/crypto.h>
#include <openssl/provider.h>

static OSSL_PROVIDER *ossl_legacy = NULL;
static OSSL_LIB_CTX  *ossl_ctx = NULL;

int main(int argc, char *argv[])
{
	printf("OpenSSL (%s)\n", OpenSSL_version(OPENSSL_VERSION));

	ossl_ctx = OSSL_LIB_CTX_new();
	if (!ossl_ctx) {
		printf("CTX fail.\n");
		return 1;
	}

	ossl_legacy  = OSSL_PROVIDER_load(ossl_ctx, "legacy");
	if (!ossl_legacy)
		printf("Cannot load legacy provider.\n");
	printf("App EXIT\n");

	OSSL_PROVIDER_unload(ossl_legacy);
	OSSL_LIB_CTX_free(ossl_ctx);

	return 0;
}

Running under valgrind, I see

==18670== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.
==18670== Using Valgrind-3.14.0 and LibVEX; rerun with -h for copyright info
==18670== Command: ./test
==18670== 
OpenSSL (OpenSSL 3.0.0-beta3-dev )
App EXIT
==18670== 
==18670== HEAP SUMMARY:
==18670==     in use at exit: 4,431 bytes in 60 blocks
==18670==   total heap usage: 225 allocs, 165 frees, 17,572 bytes allocated
==18670== 
==18670== LEAK SUMMARY:
==18670==    definitely lost: 0 bytes in 0 blocks
==18670==    indirectly lost: 0 bytes in 0 blocks
==18670==      possibly lost: 0 bytes in 0 blocks
==18670==    still reachable: 4,431 bytes in 60 blocks
==18670==         suppressed: 0 bytes in 0 blocks
==18670== Rerun with --leak-check=full to see details of leaked memory
==18670== 
==18670== For counts of detected and suppressed errors, rerun with: -v
==18670== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)

What is wrong here? Shouldn't it release all context and provider used memory?

Full valgrind log:

==18667== Memcheck, a memory error detector
==18667== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.
==18667== Using Valgrind-3.14.0 and LibVEX; rerun with -h for copyright info
==18667== Command: ./test
==18667== 
OpenSSL (OpenSSL 3.0.0-beta3-dev )
App EXIT
==18667== 
==18667== HEAP SUMMARY:
==18667==     in use at exit: 4,431 bytes in 60 blocks
==18667==   total heap usage: 225 allocs, 165 frees, 17,572 bytes allocated
==18667== 
==18667== 16 bytes in 1 blocks are still reachable in loss record 1 of 32
==18667==    at 0x483577F: malloc (vg_replace_malloc.c:299)
==18667==    by 0x4A69DC8: CRYPTO_zalloc (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A69526: create_global_tevent_register_ossl_ (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4E8F996: __pthread_once_slow (pthread_once.c:116)
==18667==    by 0x4A76EE8: CRYPTO_THREAD_run_once (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A69687: init_thread_deregister (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A74A10: ossl_provider_free (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4AE8F52: OPENSSL_sk_pop_free (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A73FE1: provider_store_free.part.14 (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A688CE: CRYPTO_free_ex_data (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A65380: context_deinit (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A65787: OSSL_LIB_CTX_free (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667== 
==18667== 18 bytes in 1 blocks are still reachable in loss record 2 of 32
==18667==    at 0x483577F: malloc (vg_replace_malloc.c:299)
==18667==    by 0x4AA629F: ossl_property_string (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4AA5AD3: ossl_property_parse_init (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A6548A: context_init.part.1 (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A654F4: default_context_do_init_ossl_ (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4E8F996: __pthread_once_slow (pthread_once.c:116)
==18667==    by 0x4A76EE8: CRYPTO_THREAD_run_once (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A6571B: ossl_lib_ctx_is_default (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A74C47: ossl_provider_find (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A720A9: OSSL_PROVIDER_try_load (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x109202: main (openssl-test.c:18)
==18667== 
==18667== 19 bytes in 1 blocks are still reachable in loss record 3 of 32
==18667==    at 0x483577F: malloc (vg_replace_malloc.c:299)
==18667==    by 0x4AA629F: ossl_property_string (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4AA5AA2: ossl_property_parse_init (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A6548A: context_init.part.1 (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A654F4: default_context_do_init_ossl_ (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4E8F996: __pthread_once_slow (pthread_once.c:116)
==18667==    by 0x4A76EE8: CRYPTO_THREAD_run_once (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A6571B: ossl_lib_ctx_is_default (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A74C47: ossl_provider_find (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A720A9: OSSL_PROVIDER_try_load (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x109202: main (openssl-test.c:18)
==18667== 
==18667== 24 bytes in 1 blocks are still reachable in loss record 4 of 32
==18667==    at 0x483577F: malloc (vg_replace_malloc.c:299)
==18667==    by 0x4A64905: OPENSSL_LH_insert (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4AA62E4: ossl_property_string (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4AA5AA2: ossl_property_parse_init (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A6548A: context_init.part.1 (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A654F4: default_context_do_init_ossl_ (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4E8F996: __pthread_once_slow (pthread_once.c:116)
==18667==    by 0x4A76EE8: CRYPTO_THREAD_run_once (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A6571B: ossl_lib_ctx_is_default (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A74C47: ossl_provider_find (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A720A9: OSSL_PROVIDER_try_load (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x109202: main (openssl-test.c:18)
==18667== 
==18667== 24 bytes in 1 blocks are still reachable in loss record 5 of 32
==18667==    at 0x483577F: malloc (vg_replace_malloc.c:299)
==18667==    by 0x4A64905: OPENSSL_LH_insert (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4AA62E4: ossl_property_string (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4AA5AD3: ossl_property_parse_init (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A6548A: context_init.part.1 (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A654F4: default_context_do_init_ossl_ (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4E8F996: __pthread_once_slow (pthread_once.c:116)
==18667==    by 0x4A76EE8: CRYPTO_THREAD_run_once (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A6571B: ossl_lib_ctx_is_default (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A74C47: ossl_provider_find (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A720A9: OSSL_PROVIDER_try_load (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x109202: main (openssl-test.c:18)
==18667== 
==18667== 32 bytes in 1 blocks are still reachable in loss record 6 of 32
==18667==    at 0x483577F: malloc (vg_replace_malloc.c:299)
==18667==    by 0x4A69DC8: CRYPTO_zalloc (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4AE8E2E: OPENSSL_sk_new_reserve (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A67FE4: ossl_crypto_get_ex_new_index_ex (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A659F8: ossl_lib_ctx_get_data (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4AA650B: ossl_property_name (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4AA5A81: ossl_property_parse_init (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A6548A: context_init.part.1 (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A654F4: default_context_do_init_ossl_ (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4E8F996: __pthread_once_slow (pthread_once.c:116)
==18667==    by 0x4A76EE8: CRYPTO_THREAD_run_once (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A6571B: ossl_lib_ctx_is_default (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667== 
==18667== 32 bytes in 1 blocks are still reachable in loss record 7 of 32
==18667==    at 0x483577F: malloc (vg_replace_malloc.c:299)
==18667==    by 0x4A69DC8: CRYPTO_zalloc (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4AE8497: OPENSSL_sk_insert (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A67FF7: ossl_crypto_get_ex_new_index_ex (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A659F8: ossl_lib_ctx_get_data (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4AA650B: ossl_property_name (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4AA5A81: ossl_property_parse_init (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A6548A: context_init.part.1 (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A654F4: default_context_do_init_ossl_ (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4E8F996: __pthread_once_slow (pthread_once.c:116)
==18667==    by 0x4A76EE8: CRYPTO_THREAD_run_once (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A6571B: ossl_lib_ctx_is_default (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667== 
==18667== 32 bytes in 1 blocks are still reachable in loss record 8 of 32
==18667==    at 0x483577F: malloc (vg_replace_malloc.c:299)
==18667==    by 0x4A69DC8: CRYPTO_zalloc (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4AA636B: property_string_data_new (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A652D5: ossl_lib_ctx_generic_new (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A68107: ossl_crypto_alloc_ex_data_intern (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A65A28: ossl_lib_ctx_get_data (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4AA650B: ossl_property_name (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4AA5A81: ossl_property_parse_init (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A6548A: context_init.part.1 (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A654F4: default_context_do_init_ossl_ (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4E8F996: __pthread_once_slow (pthread_once.c:116)
==18667==    by 0x4A76EE8: CRYPTO_THREAD_run_once (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667== 
==18667== 32 bytes in 1 blocks are still reachable in loss record 9 of 32
==18667==    at 0x483577F: malloc (vg_replace_malloc.c:299)
==18667==    by 0x4A69DC8: CRYPTO_zalloc (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4AE8E2E: OPENSSL_sk_new_reserve (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A68244: CRYPTO_set_ex_data (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A65305: ossl_lib_ctx_generic_new (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A68107: ossl_crypto_alloc_ex_data_intern (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A65A28: ossl_lib_ctx_get_data (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4AA650B: ossl_property_name (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4AA5A81: ossl_property_parse_init (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A6548A: context_init.part.1 (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A654F4: default_context_do_init_ossl_ (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4E8F996: __pthread_once_slow (pthread_once.c:116)
==18667== 
==18667== 32 bytes in 1 blocks are still reachable in loss record 10 of 32
==18667==    at 0x483577F: malloc (vg_replace_malloc.c:299)
==18667==    by 0x4A69DC8: CRYPTO_zalloc (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4AE8497: OPENSSL_sk_insert (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A681A3: CRYPTO_set_ex_data (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A65305: ossl_lib_ctx_generic_new (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A68107: ossl_crypto_alloc_ex_data_intern (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A65A28: ossl_lib_ctx_get_data (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4AA650B: ossl_property_name (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4AA5A81: ossl_property_parse_init (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A6548A: context_init.part.1 (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A654F4: default_context_do_init_ossl_ (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4E8F996: __pthread_once_slow (pthread_once.c:116)
==18667== 
==18667== 32 bytes in 1 blocks are still reachable in loss record 11 of 32
==18667==    at 0x483577F: malloc (vg_replace_malloc.c:299)
==18667==    by 0x4A69DC8: CRYPTO_zalloc (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4AE8E2E: OPENSSL_sk_new_reserve (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A6953C: create_global_tevent_register_ossl_ (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4E8F996: __pthread_once_slow (pthread_once.c:116)
==18667==    by 0x4A76EE8: CRYPTO_THREAD_run_once (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A69687: init_thread_deregister (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A74A10: ossl_provider_free (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4AE8F52: OPENSSL_sk_pop_free (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A73FE1: provider_store_free.part.14 (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A688CE: CRYPTO_free_ex_data (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A65380: context_deinit (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667== 
==18667== 48 bytes in 1 blocks are still reachable in loss record 12 of 32
==18667==    at 0x483577F: malloc (vg_replace_malloc.c:299)
==18667==    by 0x4A67ED7: ossl_crypto_get_ex_new_index_ex (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A659F8: ossl_lib_ctx_get_data (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4AA650B: ossl_property_name (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4AA5A81: ossl_property_parse_init (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A6548A: context_init.part.1 (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A654F4: default_context_do_init_ossl_ (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4E8F996: __pthread_once_slow (pthread_once.c:116)
==18667==    by 0x4A76EE8: CRYPTO_THREAD_run_once (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A6571B: ossl_lib_ctx_is_default (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A74C47: ossl_provider_find (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A720A9: OSSL_PROVIDER_try_load (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667== 
==18667== 49 bytes in 1 blocks are still reachable in loss record 13 of 32
==18667==    at 0x483577F: malloc (vg_replace_malloc.c:299)
==18667==    by 0x401AAD9: strdup (strdup.c:42)
==18667==    by 0x4008A08: _dl_map_object (dl-load.c:2398)
==18667==    by 0x4012FEA: dl_open_worker (dl-open.c:228)
==18667==    by 0x4DEEB2E: _dl_catch_exception (dl-error-skeleton.c:196)
==18667==    by 0x4012BB9: _dl_open (dl-open.c:599)
==18667==    by 0x4E7C255: dlopen_doit (dlopen.c:66)
==18667==    by 0x4DEEB2E: _dl_catch_exception (dl-error-skeleton.c:196)
==18667==    by 0x4DEEBBE: _dl_catch_error (dl-error-skeleton.c:215)
==18667==    by 0x4E7C974: _dlerror_run (dlerror.c:163)
==18667==    by 0x4E7C2E5: dlopen@@GLIBC_2.2.5 (dlopen.c:87)
==18667==    by 0x49B4F94: dlfcn_load (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667== 
==18667== 49 bytes in 1 blocks are still reachable in loss record 14 of 32
==18667==    at 0x483577F: malloc (vg_replace_malloc.c:299)
==18667==    by 0x400B58F: _dl_new_object (dl-object.c:163)
==18667==    by 0x4005E47: _dl_map_object_from_fd (dl-load.c:1001)
==18667==    by 0x4008A8C: _dl_map_object (dl-load.c:2466)
==18667==    by 0x4012FEA: dl_open_worker (dl-open.c:228)
==18667==    by 0x4DEEB2E: _dl_catch_exception (dl-error-skeleton.c:196)
==18667==    by 0x4012BB9: _dl_open (dl-open.c:599)
==18667==    by 0x4E7C255: dlopen_doit (dlopen.c:66)
==18667==    by 0x4DEEB2E: _dl_catch_exception (dl-error-skeleton.c:196)
==18667==    by 0x4DEEBBE: _dl_catch_error (dl-error-skeleton.c:215)
==18667==    by 0x4E7C974: _dlerror_run (dlerror.c:163)
==18667==    by 0x4E7C2E5: dlopen@@GLIBC_2.2.5 (dlopen.c:87)
==18667== 
==18667== 56 bytes in 1 blocks are still reachable in loss record 15 of 32
==18667==    at 0x483577F: malloc (vg_replace_malloc.c:299)
==18667==    by 0x4A69DC8: CRYPTO_zalloc (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A76E16: CRYPTO_THREAD_lock_new (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A654DC: default_context_do_init_ossl_ (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4E8F996: __pthread_once_slow (pthread_once.c:116)
==18667==    by 0x4A76EE8: CRYPTO_THREAD_run_once (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A6571B: ossl_lib_ctx_is_default (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A74C47: ossl_provider_find (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A720A9: OSSL_PROVIDER_try_load (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x109202: main (openssl-test.c:18)
==18667== 
==18667== 56 bytes in 1 blocks are still reachable in loss record 16 of 32
==18667==    at 0x483577F: malloc (vg_replace_malloc.c:299)
==18667==    by 0x4A69DC8: CRYPTO_zalloc (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A76E16: CRYPTO_THREAD_lock_new (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A653FD: context_init.part.1 (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A654F4: default_context_do_init_ossl_ (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4E8F996: __pthread_once_slow (pthread_once.c:116)
==18667==    by 0x4A76EE8: CRYPTO_THREAD_run_once (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A6571B: ossl_lib_ctx_is_default (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A74C47: ossl_provider_find (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A720A9: OSSL_PROVIDER_try_load (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x109202: main (openssl-test.c:18)
==18667== 
==18667== 56 bytes in 1 blocks are still reachable in loss record 17 of 32
==18667==    at 0x483577F: malloc (vg_replace_malloc.c:299)
==18667==    by 0x4A69DC8: CRYPTO_zalloc (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A76E16: CRYPTO_THREAD_lock_new (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A67CD4: ossl_do_ex_data_init (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A65467: context_init.part.1 (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A654F4: default_context_do_init_ossl_ (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4E8F996: __pthread_once_slow (pthread_once.c:116)
==18667==    by 0x4A76EE8: CRYPTO_THREAD_run_once (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A6571B: ossl_lib_ctx_is_default (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A74C47: ossl_provider_find (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A720A9: OSSL_PROVIDER_try_load (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x109202: main (openssl-test.c:18)
==18667== 
==18667== 56 bytes in 1 blocks are still reachable in loss record 18 of 32
==18667==    at 0x483577F: malloc (vg_replace_malloc.c:299)
==18667==    by 0x4A69DC8: CRYPTO_zalloc (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A76E16: CRYPTO_THREAD_lock_new (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4AA6378: property_string_data_new (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A652D5: ossl_lib_ctx_generic_new (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A68107: ossl_crypto_alloc_ex_data_intern (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A65A28: ossl_lib_ctx_get_data (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4AA650B: ossl_property_name (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4AA5A81: ossl_property_parse_init (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A6548A: context_init.part.1 (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A654F4: default_context_do_init_ossl_ (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4E8F996: __pthread_once_slow (pthread_once.c:116)
==18667== 
==18667== 56 bytes in 1 blocks are still reachable in loss record 19 of 32
==18667==    at 0x483577F: malloc (vg_replace_malloc.c:299)
==18667==    by 0x4A69DC8: CRYPTO_zalloc (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A76E16: CRYPTO_THREAD_lock_new (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A68DE8: ossl_init_base_ossl_ (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4E8F996: __pthread_once_slow (pthread_once.c:116)
==18667==    by 0x4A76EE8: CRYPTO_THREAD_run_once (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A68FE7: OPENSSL_init_crypto (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A186CF: do_err_strings_init_ossl_ (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4E8F996: __pthread_once_slow (pthread_once.c:116)
==18667==    by 0x4A76EE8: CRYPTO_THREAD_run_once (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A1BEF3: ERR_get_next_error_library (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A74E4E: ossl_provider_new (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667== 
==18667== 56 bytes in 1 blocks are still reachable in loss record 20 of 32
==18667==    at 0x483577F: malloc (vg_replace_malloc.c:299)
==18667==    by 0x4A69DC8: CRYPTO_zalloc (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A76E16: CRYPTO_THREAD_lock_new (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A186EC: do_err_strings_init_ossl_ (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4E8F996: __pthread_once_slow (pthread_once.c:116)
==18667==    by 0x4A76EE8: CRYPTO_THREAD_run_once (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A1BEF3: ERR_get_next_error_library (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A74E4E: ossl_provider_new (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A720E6: OSSL_PROVIDER_try_load (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x109202: main (openssl-test.c:18)
==18667== 
==18667== 56 bytes in 1 blocks are still reachable in loss record 21 of 32
==18667==    at 0x483577F: malloc (vg_replace_malloc.c:299)
==18667==    by 0x4A69DC8: CRYPTO_zalloc (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A76E16: CRYPTO_THREAD_lock_new (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A6954B: create_global_tevent_register_ossl_ (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4E8F996: __pthread_once_slow (pthread_once.c:116)
==18667==    by 0x4A76EE8: CRYPTO_THREAD_run_once (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A69687: init_thread_deregister (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A74A10: ossl_provider_free (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4AE8F52: OPENSSL_sk_pop_free (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A73FE1: provider_store_free.part.14 (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A688CE: CRYPTO_free_ex_data (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A65380: context_deinit (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667== 
==18667== 120 bytes in 1 blocks are still reachable in loss record 22 of 32
==18667==    at 0x4837B65: calloc (vg_replace_malloc.c:752)
==18667==    by 0x4010A1F: _dl_check_map_versions (dl-version.c:274)
==18667==    by 0x4013095: dl_open_worker (dl-open.c:277)
==18667==    by 0x4DEEB2E: _dl_catch_exception (dl-error-skeleton.c:196)
==18667==    by 0x4012BB9: _dl_open (dl-open.c:599)
==18667==    by 0x4E7C255: dlopen_doit (dlopen.c:66)
==18667==    by 0x4DEEB2E: _dl_catch_exception (dl-error-skeleton.c:196)
==18667==    by 0x4DEEBBE: _dl_catch_error (dl-error-skeleton.c:215)
==18667==    by 0x4E7C974: _dlerror_run (dlerror.c:163)
==18667==    by 0x4E7C2E5: dlopen@@GLIBC_2.2.5 (dlopen.c:87)
==18667==    by 0x49B4F94: dlfcn_load (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x49B5988: DSO_load (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667== 
==18667== 128 bytes in 1 blocks are still reachable in loss record 23 of 32
==18667==    at 0x483577F: malloc (vg_replace_malloc.c:299)
==18667==    by 0x4A69DC8: CRYPTO_zalloc (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A6460D: OPENSSL_LH_new (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4AA6393: property_string_data_new (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A652D5: ossl_lib_ctx_generic_new (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A68107: ossl_crypto_alloc_ex_data_intern (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A65A28: ossl_lib_ctx_get_data (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4AA650B: ossl_property_name (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4AA5A81: ossl_property_parse_init (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A6548A: context_init.part.1 (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A654F4: default_context_do_init_ossl_ (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4E8F996: __pthread_once_slow (pthread_once.c:116)
==18667== 
==18667== 128 bytes in 1 blocks are still reachable in loss record 24 of 32
==18667==    at 0x483577F: malloc (vg_replace_malloc.c:299)
==18667==    by 0x4A69DC8: CRYPTO_zalloc (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A6460D: OPENSSL_LH_new (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4AA63AF: property_string_data_new (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A652D5: ossl_lib_ctx_generic_new (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A68107: ossl_crypto_alloc_ex_data_intern (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A65A28: ossl_lib_ctx_get_data (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4AA650B: ossl_property_name (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4AA5A81: ossl_property_parse_init (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A6548A: context_init.part.1 (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A654F4: default_context_do_init_ossl_ (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4E8F996: __pthread_once_slow (pthread_once.c:116)
==18667== 
==18667== 128 bytes in 1 blocks are still reachable in loss record 25 of 32
==18667==    at 0x483577F: malloc (vg_replace_malloc.c:299)
==18667==    by 0x4A69DC8: CRYPTO_zalloc (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A6460D: OPENSSL_LH_new (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A1870B: do_err_strings_init_ossl_ (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4E8F996: __pthread_once_slow (pthread_once.c:116)
==18667==    by 0x4A76EE8: CRYPTO_THREAD_run_once (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A1BEF3: ERR_get_next_error_library (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A74E4E: ossl_provider_new (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A720E6: OSSL_PROVIDER_try_load (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x109202: main (openssl-test.c:18)
==18667== 
==18667== 135 bytes in 6 blocks are still reachable in loss record 26 of 32
==18667==    at 0x483577F: malloc (vg_replace_malloc.c:299)
==18667==    by 0x4AA629F: ossl_property_string (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4AA5A81: ossl_property_parse_init (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A6548A: context_init.part.1 (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A654F4: default_context_do_init_ossl_ (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4E8F996: __pthread_once_slow (pthread_once.c:116)
==18667==    by 0x4A76EE8: CRYPTO_THREAD_run_once (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A6571B: ossl_lib_ctx_is_default (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A74C47: ossl_provider_find (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A720A9: OSSL_PROVIDER_try_load (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x109202: main (openssl-test.c:18)
==18667== 
==18667== 144 bytes in 6 blocks are still reachable in loss record 27 of 32
==18667==    at 0x483577F: malloc (vg_replace_malloc.c:299)
==18667==    by 0x4A64905: OPENSSL_LH_insert (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4AA62E4: ossl_property_string (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4AA5A81: ossl_property_parse_init (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A6548A: context_init.part.1 (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A654F4: default_context_do_init_ossl_ (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4E8F996: __pthread_once_slow (pthread_once.c:116)
==18667==    by 0x4A76EE8: CRYPTO_THREAD_run_once (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A6571B: ossl_lib_ctx_is_default (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A74C47: ossl_provider_find (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A720A9: OSSL_PROVIDER_try_load (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x109202: main (openssl-test.c:18)
==18667== 
==18667== 176 bytes in 1 blocks are still reachable in loss record 28 of 32
==18667==    at 0x483577F: malloc (vg_replace_malloc.c:299)
==18667==    by 0x4A69DC8: CRYPTO_zalloc (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A645EF: OPENSSL_LH_new (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4AA6393: property_string_data_new (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A652D5: ossl_lib_ctx_generic_new (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A68107: ossl_crypto_alloc_ex_data_intern (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A65A28: ossl_lib_ctx_get_data (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4AA650B: ossl_property_name (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4AA5A81: ossl_property_parse_init (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A6548A: context_init.part.1 (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A654F4: default_context_do_init_ossl_ (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4E8F996: __pthread_once_slow (pthread_once.c:116)
==18667== 
==18667== 176 bytes in 1 blocks are still reachable in loss record 29 of 32
==18667==    at 0x483577F: malloc (vg_replace_malloc.c:299)
==18667==    by 0x4A69DC8: CRYPTO_zalloc (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A645EF: OPENSSL_LH_new (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4AA63AF: property_string_data_new (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A652D5: ossl_lib_ctx_generic_new (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A68107: ossl_crypto_alloc_ex_data_intern (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A65A28: ossl_lib_ctx_get_data (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4AA650B: ossl_property_name (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4AA5A81: ossl_property_parse_init (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A6548A: context_init.part.1 (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A654F4: default_context_do_init_ossl_ (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4E8F996: __pthread_once_slow (pthread_once.c:116)
==18667== 
==18667== 176 bytes in 1 blocks are still reachable in loss record 30 of 32
==18667==    at 0x483577F: malloc (vg_replace_malloc.c:299)
==18667==    by 0x4A69DC8: CRYPTO_zalloc (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A645EF: OPENSSL_LH_new (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A1870B: do_err_strings_init_ossl_ (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4E8F996: __pthread_once_slow (pthread_once.c:116)
==18667==    by 0x4A76EE8: CRYPTO_THREAD_run_once (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A1BEF3: ERR_get_next_error_library (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A74E4E: ossl_provider_new (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A720E6: OSSL_PROVIDER_try_load (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x109202: main (openssl-test.c:18)
==18667== 
==18667== 1,064 bytes in 19 blocks are still reachable in loss record 31 of 32
==18667==    at 0x483577F: malloc (vg_replace_malloc.c:299)
==18667==    by 0x4A69DC8: CRYPTO_zalloc (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A76E16: CRYPTO_THREAD_lock_new (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A6541E: context_init.part.1 (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A654F4: default_context_do_init_ossl_ (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4E8F996: __pthread_once_slow (pthread_once.c:116)
==18667==    by 0x4A76EE8: CRYPTO_THREAD_run_once (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A6571B: ossl_lib_ctx_is_default (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A74C47: ossl_provider_find (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x4A720A9: OSSL_PROVIDER_try_load (in /usr/lib/x86_64-linux-gnu/libcrypto.so.3)
==18667==    by 0x109202: main (openssl-test.c:18)
==18667== 
==18667== 1,225 bytes in 1 blocks are still reachable in loss record 32 of 32
==18667==    at 0x4837B65: calloc (vg_replace_malloc.c:752)
==18667==    by 0x400B2AD: _dl_new_object (dl-object.c:73)
==18667==    by 0x4005E47: _dl_map_object_from_fd (dl-load.c:1001)
==18667==    by 0x4008A8C: _dl_map_object (dl-load.c:2466)
==18667==    by 0x4012FEA: dl_open_worker (dl-open.c:228)
==18667==    by 0x4DEEB2E: _dl_catch_exception (dl-error-skeleton.c:196)
==18667==    by 0x4012BB9: _dl_open (dl-open.c:599)
==18667==    by 0x4E7C255: dlopen_doit (dlopen.c:66)
==18667==    by 0x4DEEB2E: _dl_catch_exception (dl-error-skeleton.c:196)
==18667==    by 0x4DEEBBE: _dl_catch_error (dl-error-skeleton.c:215)
==18667==    by 0x4E7C974: _dlerror_run (dlerror.c:163)
==18667==    by 0x4E7C2E5: dlopen@@GLIBC_2.2.5 (dlopen.c:87)
==18667== 
==18667== LEAK SUMMARY:
==18667==    definitely lost: 0 bytes in 0 blocks
==18667==    indirectly lost: 0 bytes in 0 blocks
==18667==      possibly lost: 0 bytes in 0 blocks
==18667==    still reachable: 4,431 bytes in 60 blocks
==18667==         suppressed: 0 bytes in 0 blocks
==18667== 
==18667== For counts of detected and suppressed errors, rerun with: -v
==18667== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)
@mbroz mbroz added the issue: bug report The issue was opened to report a bug label Sep 6, 2021
@t8m t8m added branch: 3.0 Merge to openssl-3.0 branch branch: master Merge to master branch triaged: bug The issue/pr is/fixes a bug and removed issue: bug report The issue was opened to report a bug labels Sep 6, 2021
@mbroz
Copy link
Contributor Author

mbroz commented Sep 6, 2021

@t8m OPENSSL_init_crypto(0, NULL); does not help (reply to irc question :)

I also tried own atexit call with OPENSSL_cleanup() and some leaks are still there.

@t8m t8m added this to the 3.0.0 milestone Sep 7, 2021
@t8m
Copy link
Member

t8m commented Sep 7, 2021

I do not think this is a release blocker but OTC should look at this.

@t8m t8m removed this from the 3.0.0 milestone Sep 7, 2021
@t8m t8m added the triaged: OTC evaluated This issue/pr was triaged by OTC label Sep 7, 2021
@t8m
Copy link
Member

t8m commented Sep 7, 2021

OTC: not a blocker for 3.0.0

@mbroz
Copy link
Contributor Author

mbroz commented Sep 7, 2021

To not forget, some findings:

  • most of leaks disappear if some crypto operation is used (I tried PBKDF2-WHIRLPOOL using new KDF API; this need adding defaut provider to the mix also)
  • the DSO related leaks disappear if compiled with no-pinshared

@paulidale
Copy link
Contributor

Could you test this against 3.0?
I'm not able to reproduce this locally. A similar looking problem was addressed late in the beta phase.

@t8m
Copy link
Member

t8m commented Sep 8, 2021

@mbroz could it be that you're loading legacy.so from the Debian build given the paths you've configured? Could you strace the reproducer to verify? OPENSSL_MODULES=/path/to/dir/with/providers should be used to override the path.

@mbroz
Copy link
Contributor Author

mbroz commented Sep 8, 2021 •
edited

Could you test this against 3.0?

I tested everything against 3.0 branch (on yesterday's git) - I am pretty sure that the issue was there.

@t8m well, I did that mistake in the beginning :) But now I have Debian without any openssl3 files (purged everything) and I use just "make install" from the devel git (it only requires Debianish paths for Config) - so it is loading the devel-compiled legacy.so. I'll try strace.

@mbroz
Copy link
Contributor Author

mbroz commented Sep 8, 2021

For openssl-3.0 branch configured with

./Configure shared --prefix=/usr --openssldir=/usr/lib/ssl --libdir=lib/x86_64-linux-gnu linux-x86_64

I get this strace:

execve("./test", ["./test"], 0x7ffe54b16510 /* 46 vars */) = 0
brk(NULL)                               = 0x558a412e0000
access("/etc/ld.so.preload", R_OK)      = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=87814, ...}) = 0
mmap(NULL, 87814, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7efc02dc1000
close(3)                                = 0
openat(AT_FDCWD, "/lib/x86_64-linux-gnu/tls/haswell/x86_64/libcrypto.so.3", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
stat("/lib/x86_64-linux-gnu/tls/haswell/x86_64", 0x7ffc21181350) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/lib/x86_64-linux-gnu/tls/haswell/libcrypto.so.3", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
stat("/lib/x86_64-linux-gnu/tls/haswell", 0x7ffc21181350) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/lib/x86_64-linux-gnu/tls/x86_64/libcrypto.so.3", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
stat("/lib/x86_64-linux-gnu/tls/x86_64", 0x7ffc21181350) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/lib/x86_64-linux-gnu/tls/libcrypto.so.3", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
stat("/lib/x86_64-linux-gnu/tls", 0x7ffc21181350) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/lib/x86_64-linux-gnu/haswell/x86_64/libcrypto.so.3", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
stat("/lib/x86_64-linux-gnu/haswell/x86_64", 0x7ffc21181350) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/lib/x86_64-linux-gnu/haswell/libcrypto.so.3", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
stat("/lib/x86_64-linux-gnu/haswell", 0x7ffc21181350) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/lib/x86_64-linux-gnu/x86_64/libcrypto.so.3", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
stat("/lib/x86_64-linux-gnu/x86_64", 0x7ffc21181350) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/lib/x86_64-linux-gnu/libcrypto.so.3", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\0000\v\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0755, st_size=5260560, ...}) = 0
mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7efc02dbf000
mmap(NULL, 4585896, PROT_READ, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7efc0295f000
mmap(0x7efc02a11000, 2568192, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0xb2000) = 0x7efc02a11000
mmap(0x7efc02c84000, 892928, PROT_READ, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x325000) = 0x7efc02c84000
mmap(0x7efc02d5e000, 385024, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x3fe000) = 0x7efc02d5e000
mmap(0x7efc02dbc000, 10664, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7efc02dbc000
close(3)                                = 0
openat(AT_FDCWD, "/lib/x86_64-linux-gnu/libc.so.6", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\3\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\260A\2\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0755, st_size=1824496, ...}) = 0
mmap(NULL, 1837056, PROT_READ, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7efc0279e000
mprotect(0x7efc027c0000, 1658880, PROT_NONE) = 0
mmap(0x7efc027c0000, 1343488, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x22000) = 0x7efc027c0000
mmap(0x7efc02908000, 311296, PROT_READ, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x16a000) = 0x7efc02908000
mmap(0x7efc02955000, 24576, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1b6000) = 0x7efc02955000
mmap(0x7efc0295b000, 14336, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7efc0295b000
close(3)                                = 0
openat(AT_FDCWD, "/lib/x86_64-linux-gnu/libdl.so.2", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0000\21\0\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0644, st_size=14592, ...}) = 0
mmap(NULL, 16656, PROT_READ, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7efc02799000
mmap(0x7efc0279a000, 4096, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1000) = 0x7efc0279a000
mmap(0x7efc0279b000, 4096, PROT_READ, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x2000) = 0x7efc0279b000
mmap(0x7efc0279c000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x2000) = 0x7efc0279c000
close(3)                                = 0
openat(AT_FDCWD, "/lib/x86_64-linux-gnu/libpthread.so.0", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\3\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0@l\0\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0755, st_size=146968, ...}) = 0
mmap(NULL, 132288, PROT_READ, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7efc02778000
mmap(0x7efc0277e000, 61440, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x6000) = 0x7efc0277e000
mmap(0x7efc0278d000, 24576, PROT_READ, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x15000) = 0x7efc0278d000
mmap(0x7efc02793000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1a000) = 0x7efc02793000
mmap(0x7efc02795000, 13504, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7efc02795000
close(3)                                = 0
mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7efc02776000
arch_prctl(ARCH_SET_FS, 0x7efc02776b80) = 0
mprotect(0x7efc02955000, 16384, PROT_READ) = 0
mprotect(0x7efc02793000, 4096, PROT_READ) = 0
mprotect(0x7efc0279c000, 4096, PROT_READ) = 0
mprotect(0x7efc02d5e000, 372736, PROT_READ) = 0
mprotect(0x558a40bb2000, 4096, PROT_READ) = 0
mprotect(0x7efc02dfe000, 4096, PROT_READ) = 0
munmap(0x7efc02dc1000, 87814)           = 0
set_tid_address(0x7efc02776e50)         = 3074
set_robust_list(0x7efc02776e60, 24)     = 0
rt_sigaction(SIGRTMIN, {sa_handler=0x7efc0277e6b0, sa_mask=[], sa_flags=SA_RESTORER|SA_SIGINFO, sa_restorer=0x7efc0278a730}, NULL, 8) = 0
rt_sigaction(SIGRT_1, {sa_handler=0x7efc0277e740, sa_mask=[], sa_flags=SA_RESTORER|SA_RESTART|SA_SIGINFO, sa_restorer=0x7efc0278a730}, NULL, 8) = 0
rt_sigprocmask(SIG_UNBLOCK, [RTMIN RT_1], NULL, 8) = 0
prlimit64(0, RLIMIT_STACK, NULL, {rlim_cur=8192*1024, rlim_max=RLIM64_INFINITY}) = 0
fstat(1, {st_mode=S_IFREG|0644, st_size=6202, ...}) = 0
brk(NULL)                               = 0x558a412e0000
brk(0x558a41301000)                     = 0x558a41301000
futex(0x7efc02dbbcc8, FUTEX_WAKE_PRIVATE, 2147483647) = 0
futex(0x7efc02dbc1b8, FUTEX_WAKE_PRIVATE, 2147483647) = 0
futex(0x7efc02dbbbf8, FUTEX_WAKE_PRIVATE, 2147483647) = 0
futex(0x7efc0279d0c8, FUTEX_WAKE_PRIVATE, 2147483647) = 0
openat(AT_FDCWD, "/usr/lib/x86_64-linux-gnu/ossl-modules/legacy.so", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\240v\0\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0755, st_size=157880, ...}) = 0
mmap(NULL, 131944, PROT_READ, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7efc02755000
mprotect(0x7efc0275c000, 86016, PROT_NONE) = 0
mmap(0x7efc0275c000, 57344, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x7000) = 0x7efc0275c000
mmap(0x7efc0276a000, 24576, PROT_READ, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x15000) = 0x7efc0276a000
mmap(0x7efc02771000, 20480, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1b000) = 0x7efc02771000
close(3)                                = 0
mprotect(0x7efc02771000, 16384, PROT_READ) = 0
futex(0x7efc02dbc1e4, FUTEX_WAKE_PRIVATE, 2147483647) = 0
write(1, "OpenSSL (OpenSSL 3.0.1-dev )\nApp"..., 38OpenSSL (OpenSSL 3.0.1-dev )
App EXIT
) = 38
exit_group(0)                           = ?
+++ exited with 0 +++

And I verified that /usr/lib/x86_64-linux-gnu/ossl-modules/legacy.so is installed with make install form openssl tree.
(Rest of the system uses OpenSSL1.1 so it should not interfere here...)

@t8m
Copy link
Member

t8m commented Sep 8, 2021

Could you test this against 3.0?
I'm not able to reproduce this locally. A similar looking problem was addressed late in the beta phase.

I am able to reproduce this with valgrind --leak-check=full --show-leak-kinds=all

@paulidale - please note that you need these options to see the leaks. There are basically two kinds of leaks - one from the initialization done inside the cleanup - probably should be fixed by doing initialization in OSSL_PROVIDER_load() and OSSL_LIB_CTX_new(). And then there are the shared library associated leaks which are eliminated when ./Configure no-pinshared is used - so I believe those can be ignored.

@paulidale
Copy link
Contributor

Something weird is going on: even with those valgrind options, I'm not seing the problem...

@t8m
Copy link
Member

t8m commented Sep 8, 2021

@paulidale And the legacy provider successfully loads for you?

@paulidale
Copy link
Contributor

Ah, that's the issue -- the load fails. With the environment set up properly, I see the leaks.

t8m added a commit to t8m/openssl that referenced this issue Sep 8, 2021
@t8m
Always init with loading config when loading a provider
8b154ac 
Fixes openssl#16516
@t8m t8m mentioned this issue Sep 8, 2021
Closed
@t8m
Copy link
Member

t8m commented Sep 8, 2021

Please try #16558

@mbroz
Copy link
Contributor Author

mbroz commented Sep 8, 2021

Please try #16558

Yes, it fixes all the leaks (except remaining 4 related to DSO load - I guess this is just side effect of not using no-pinshared).

Thanks!

@mbroz
Copy link
Contributor Author

mbroz commented Oct 7, 2021

While #16558 has not been merged... any plans to fix this issue with a different approach?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
branch: master Merge to master branch branch: 3.0 Merge to openssl-3.0 branch triaged: bug The issue/pr is/fixes a bug triaged: OTC evaluated This issue/pr was triaged by OTC
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Always init with loading config when loading a provider t8m/openssl
3 participants
@mbroz @t8m @paulidale