Let's stop sending gmt_unix_time #23

nmathewson · 2013-09-20T05:08:28Z

Hi, all.

Ben Laurie asked me to do this as a pull request. I've sent the most ambitious version by asking that it go into 1.0.1.

This patch's goal is to simply stop sending gmt_unix_time as a part of client or server Random fields.

We discussed this on the tls-wg mailing list last week, and the best use of the gmt_unix_time field anybody found was the tlsdate program -- but tlsdate is also from Tor, and tlsdate is easily adapted to speak HTTPS instead: see my tlsdate repository.

The main rationale for having gmt_unix_time in SSL3 in the first place was to prevent collisions if the RNG was broken. But it's neither necessary nor sufficient, and it leads to bad linkability issues in practice. See the tls-wg mailing list for further discussion.

Instead, send random bytes. While the gmt_unix_time record was added in an ostensible attempt to mitigate the dangers of a bad RNG, its presence leaks the host's view of the current time in the clear. This minor leak can help fingerprint TLS instances across networks and protocols... and what's worse, it's doubtful thet the gmt_unix_time record does any good at all for its intended purpose, since: * It's quite possible to open two TLS connections in one second. * If the PRNG output is prone to repeat itself, ephemeral * handshakes (and who knows what else besides) are broken.

Instead, send random bytes.

I'll be using this to make an option for randomizing the time.

(I'd rather use an option, but it appears that the options field is full.) Now, we send the time in the gmt_unix_time field if the appropriate one of these mode options is set, but randomize the field if the flag is not set.

nmathewson · 2013-10-09T14:40:42Z

Ben suggested that the behavior here should be controlled by a runtime option. I've adjusted it to work that way.

nmathewson · 2013-10-09T14:52:31Z

Implementation nodes: I made the compatibility flag use the SSL_MODE_* flags, since the SSL_OP_* bitfield seems to be full, except for 0x400, which apparently has history. But I'd be happy to use 0x400 in ssl->options if that's a better choice.

The other choice would be to add a bunch of machinery to widen the options field, but the most obvious ways to do that would probably break binary compatibility and make people sad.

(I seem to recall a similar chain of logic playing out when I added SSL_MODE_RELEASE_BUFFERS.)

Other suggestions are welcome!

benlaurie · 2013-10-19T11:52:01Z

Merged at 833a896.

This allows us to point to whatever libcrypto we want. Reviewed-by: Matt Caswell <matt@openssl.org> (Merged from openssl/openssl#23)

fix some compilation issue with 2.066.0

This happens usually if an template object is created and there is an out of memory error before the ASN1_OP_NEW_POST method is called, but asn1_item_embed_free calls now the ASN1_OP_FREE_POST which may crash because the object is not properly initialized. Apparently that is only an issue with the ASN1_OP_FREE_POST handling of crypot/x509/x_crl.c, which ought to be tolerant to incomplete initialized objects. The error can be reproduced with the reproducible error injection patch: $ ERROR_INJECT=1652890550 ../util/shlib_wrap.sh ./asn1-test ./corpora/asn1/0ff17293911f54d1538b9896563a4048d67d9ee4 #0 0x7faae9dbeeba in __sanitizer_print_stack_trace ../../../../gcc-trunk/libsanitizer/asan/asan_stack.cpp:87 openssl#1 0x408dc4 in my_malloc fuzz/test-corpus.c:114 openssl#2 0x7faae99f2430 in CRYPTO_zalloc crypto/mem.c:230 openssl#3 0x7faae97f09e5 in ASN1_STRING_type_new crypto/asn1/asn1_lib.c:341 openssl#4 0x7faae98118f7 in asn1_primitive_new crypto/asn1/tasn_new.c:318 openssl#5 0x7faae9812401 in asn1_item_embed_new crypto/asn1/tasn_new.c:78 openssl#6 0x7faae9812401 in asn1_template_new crypto/asn1/tasn_new.c:240 openssl#7 0x7faae9812315 in asn1_item_embed_new crypto/asn1/tasn_new.c:137 openssl#8 0x7faae9812315 in asn1_template_new crypto/asn1/tasn_new.c:240 openssl#9 0x7faae9812a54 in asn1_item_embed_new crypto/asn1/tasn_new.c:137 openssl#10 0x7faae9812a54 in ASN1_item_ex_new crypto/asn1/tasn_new.c:39 openssl#11 0x7faae980be51 in asn1_item_embed_d2i crypto/asn1/tasn_dec.c:325 openssl#12 0x7faae980c813 in asn1_template_noexp_d2i crypto/asn1/tasn_dec.c:611 openssl#13 0x7faae980d288 in asn1_template_ex_d2i crypto/asn1/tasn_dec.c:518 openssl#14 0x7faae980b9ce in asn1_item_embed_d2i crypto/asn1/tasn_dec.c:382 openssl#15 0x7faae980caf5 in asn1_template_noexp_d2i crypto/asn1/tasn_dec.c:643 openssl#16 0x7faae980d7d3 in asn1_template_ex_d2i crypto/asn1/tasn_dec.c:494 openssl#17 0x7faae980b9ce in asn1_item_embed_d2i crypto/asn1/tasn_dec.c:382 openssl#18 0x7faae980dd1f in ASN1_item_ex_d2i crypto/asn1/tasn_dec.c:124 openssl#19 0x7faae980de35 in ASN1_item_d2i crypto/asn1/tasn_dec.c:114 openssl#20 0x40712c in FuzzerTestOneInput fuzz/asn1.c:301 openssl#21 0x40893b in testfile fuzz/test-corpus.c:182 openssl#22 0x406b86 in main fuzz/test-corpus.c:226 openssl#23 0x7faae8eb1f44 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x21f44) AddressSanitizer:DEADLYSIGNAL ================================================================= ==1194==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000010 (pc 0x7faae9b0625f bp 0x7fffffe41a00 sp 0x7fffffe41920 T0) ==1194==The signal is caused by a READ memory access. ==1194==Hint: address points to the zero page. #0 0x7faae9b0625f in crl_cb crypto/x509/x_crl.c:258 openssl#1 0x7faae9811255 in asn1_item_embed_free crypto/asn1/tasn_fre.c:113 openssl#2 0x7faae9812a65 in asn1_item_embed_new crypto/asn1/tasn_new.c:150 openssl#3 0x7faae9812a65 in ASN1_item_ex_new crypto/asn1/tasn_new.c:39 openssl#4 0x7faae980be51 in asn1_item_embed_d2i crypto/asn1/tasn_dec.c:325 openssl#5 0x7faae980c813 in asn1_template_noexp_d2i crypto/asn1/tasn_dec.c:611 openssl#6 0x7faae980d288 in asn1_template_ex_d2i crypto/asn1/tasn_dec.c:518 openssl#7 0x7faae980b9ce in asn1_item_embed_d2i crypto/asn1/tasn_dec.c:382 openssl#8 0x7faae980caf5 in asn1_template_noexp_d2i crypto/asn1/tasn_dec.c:643 openssl#9 0x7faae980d7d3 in asn1_template_ex_d2i crypto/asn1/tasn_dec.c:494 openssl#10 0x7faae980b9ce in asn1_item_embed_d2i crypto/asn1/tasn_dec.c:382 openssl#11 0x7faae980dd1f in ASN1_item_ex_d2i crypto/asn1/tasn_dec.c:124 openssl#12 0x7faae980de35 in ASN1_item_d2i crypto/asn1/tasn_dec.c:114 openssl#13 0x40712c in FuzzerTestOneInput fuzz/asn1.c:301 openssl#14 0x40893b in testfile fuzz/test-corpus.c:182 openssl#15 0x406b86 in main fuzz/test-corpus.c:226 openssl#16 0x7faae8eb1f44 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x21f44) AddressSanitizer can not provide additional info. SUMMARY: AddressSanitizer: SEGV crypto/x509/x_crl.c:258 in crl_cb ==1194==ABORTING

This can be reproduced with my error injection patch. The test vector has been validated on the 1.1.1 branch but the issue is of course identical in all branches. $ ERROR_INJECT=1653520461 ../util/shlib_wrap.sh ./cms-test ./corpora/cms/3eff1d2f1232bd66d5635db2c3f9e7f23830dfd1 log file: cms-3eff1d2f1232bd66d5635db2c3f9e7f23830dfd1-32454-test.out ERROR_INJECT=1653520461 #0 0x7fd5d8b8eeba in __sanitizer_print_stack_trace ../../../../gcc-trunk/libsanitizer/asan/asan_stack.cpp:87 openssl#1 0x402fc4 in my_realloc fuzz/test-corpus.c:129 openssl#2 0x7fd5d8893c49 in sk_reserve crypto/stack/stack.c:198 openssl#3 0x7fd5d8893c49 in OPENSSL_sk_insert crypto/stack/stack.c:242 openssl#4 0x7fd5d88d6d7f in sk_GENERAL_NAMES_push include/openssl/x509v3.h:168 openssl#5 0x7fd5d88d6d7f in crl_set_issuers crypto/x509/x_crl.c:111 openssl#6 0x7fd5d88d6d7f in crl_cb crypto/x509/x_crl.c:246 openssl#7 0x7fd5d85dc032 in asn1_item_embed_d2i crypto/asn1/tasn_dec.c:432 openssl#8 0x7fd5d85dcaf5 in asn1_template_noexp_d2i crypto/asn1/tasn_dec.c:643 openssl#9 0x7fd5d85dd288 in asn1_template_ex_d2i crypto/asn1/tasn_dec.c:518 openssl#10 0x7fd5d85db2b5 in asn1_item_embed_d2i crypto/asn1/tasn_dec.c:259 openssl#11 0x7fd5d85dc813 in asn1_template_noexp_d2i crypto/asn1/tasn_dec.c:611 openssl#12 0x7fd5d85dd288 in asn1_template_ex_d2i crypto/asn1/tasn_dec.c:518 openssl#13 0x7fd5d85db9ce in asn1_item_embed_d2i crypto/asn1/tasn_dec.c:382 openssl#14 0x7fd5d85dca28 in asn1_template_noexp_d2i crypto/asn1/tasn_dec.c:633 openssl#15 0x7fd5d85dd288 in asn1_template_ex_d2i crypto/asn1/tasn_dec.c:518 openssl#16 0x7fd5d85db9ce in asn1_item_embed_d2i crypto/asn1/tasn_dec.c:382 openssl#17 0x7fd5d85dcaf5 in asn1_template_noexp_d2i crypto/asn1/tasn_dec.c:643 openssl#18 0x7fd5d85dd7d3 in asn1_template_ex_d2i crypto/asn1/tasn_dec.c:494 openssl#19 0x7fd5d85db9ce in asn1_item_embed_d2i crypto/asn1/tasn_dec.c:382 openssl#20 0x7fd5d85ddd1f in ASN1_item_ex_d2i crypto/asn1/tasn_dec.c:124 openssl#21 0x7fd5d85dde35 in ASN1_item_d2i crypto/asn1/tasn_dec.c:114 openssl#22 0x7fd5d85a77e0 in ASN1_item_d2i_bio crypto/asn1/a_d2i_fp.c:69 openssl#23 0x402845 in FuzzerTestOneInput fuzz/cms.c:43 openssl#24 0x402bbb in testfile fuzz/test-corpus.c:182 openssl#25 0x402626 in main fuzz/test-corpus.c:226 openssl#26 0x7fd5d7c81f44 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x21f44) openssl#27 0x402706 (/home/ed/OPC/openssl/fuzz/cms-test+0x402706) ================================================================= ==29625==ERROR: LeakSanitizer: detected memory leaks Direct leak of 32 byte(s) in 1 object(s) allocated from: #0 0x7fd5d8b8309f in __interceptor_malloc ../../../../gcc-trunk/libsanitizer/asan/asan_malloc_linux.cpp:69 openssl#1 0x7fd5d87c2430 in CRYPTO_zalloc crypto/mem.c:230 openssl#2 0x7fd5d889501f in OPENSSL_sk_new_reserve crypto/stack/stack.c:209 openssl#3 0x7fd5d85dcbc3 in sk_ASN1_VALUE_new_null include/openssl/asn1t.h:928 openssl#4 0x7fd5d85dcbc3 in asn1_template_noexp_d2i crypto/asn1/tasn_dec.c:577 openssl#5 0x7fd5d85dd288 in asn1_template_ex_d2i crypto/asn1/tasn_dec.c:518 openssl#6 0x7fd5d85db104 in asn1_item_embed_d2i crypto/asn1/tasn_dec.c:178 openssl#7 0x7fd5d85ddd1f in ASN1_item_ex_d2i crypto/asn1/tasn_dec.c:124 openssl#8 0x7fd5d85dde35 in ASN1_item_d2i crypto/asn1/tasn_dec.c:114 openssl#9 0x7fd5d88f86d9 in X509V3_EXT_d2i crypto/x509v3/v3_lib.c:142 openssl#10 0x7fd5d88d6d3c in crl_set_issuers crypto/x509/x_crl.c:97 openssl#11 0x7fd5d88d6d3c in crl_cb crypto/x509/x_crl.c:246 openssl#12 0x7fd5d85dc032 in asn1_item_embed_d2i crypto/asn1/tasn_dec.c:432 openssl#13 0x7fd5d85dcaf5 in asn1_template_noexp_d2i crypto/asn1/tasn_dec.c:643 openssl#14 0x7fd5d85dd288 in asn1_template_ex_d2i crypto/asn1/tasn_dec.c:518 openssl#15 0x7fd5d85db2b5 in asn1_item_embed_d2i crypto/asn1/tasn_dec.c:259 openssl#16 0x7fd5d85dc813 in asn1_template_noexp_d2i crypto/asn1/tasn_dec.c:611 openssl#17 0x7fd5d85dd288 in asn1_template_ex_d2i crypto/asn1/tasn_dec.c:518 openssl#18 0x7fd5d85db9ce in asn1_item_embed_d2i crypto/asn1/tasn_dec.c:382 openssl#19 0x7fd5d85dca28 in asn1_template_noexp_d2i crypto/asn1/tasn_dec.c:633 openssl#20 0x7fd5d85dd288 in asn1_template_ex_d2i crypto/asn1/tasn_dec.c:518 openssl#21 0x7fd5d85db9ce in asn1_item_embed_d2i crypto/asn1/tasn_dec.c:382 openssl#22 0x7fd5d85dcaf5 in asn1_template_noexp_d2i crypto/asn1/tasn_dec.c:643 openssl#23 0x7fd5d85dd7d3 in asn1_template_ex_d2i crypto/asn1/tasn_dec.c:494 openssl#24 0x7fd5d85db9ce in asn1_item_embed_d2i crypto/asn1/tasn_dec.c:382 openssl#25 0x7fd5d85ddd1f in ASN1_item_ex_d2i crypto/asn1/tasn_dec.c:124 openssl#26 0x7fd5d85dde35 in ASN1_item_d2i crypto/asn1/tasn_dec.c:114 openssl#27 0x7fd5d85a77e0 in ASN1_item_d2i_bio crypto/asn1/a_d2i_fp.c:69 openssl#28 0x402845 in FuzzerTestOneInput fuzz/cms.c:43 openssl#29 0x402bbb in testfile fuzz/test-corpus.c:182 openssl#30 0x402626 in main fuzz/test-corpus.c:226 openssl#31 0x7fd5d7c81f44 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x21f44) SUMMARY: AddressSanitizer: 32 byte(s) leaked in 1 allocation(s).

This happens usually if an template object is created and there is an out of memory error before the ASN1_OP_NEW_POST method is called, but asn1_item_embed_free calls now the ASN1_OP_FREE_POST which may crash because the object is not properly initialized. Apparently that is only an issue with the ASN1_OP_FREE_POST handling of crypot/x509/x_crl.c, which ought to be tolerant to incomplete initialized objects. The error can be reproduced with the reproducible error injection patch: $ ERROR_INJECT=1652890550 ../util/shlib_wrap.sh ./asn1-test ./corpora/asn1/0ff17293911f54d1538b9896563a4048d67d9ee4 #0 0x7faae9dbeeba in __sanitizer_print_stack_trace ../../../../gcc-trunk/libsanitizer/asan/asan_stack.cpp:87 #1 0x408dc4 in my_malloc fuzz/test-corpus.c:114 #2 0x7faae99f2430 in CRYPTO_zalloc crypto/mem.c:230 #3 0x7faae97f09e5 in ASN1_STRING_type_new crypto/asn1/asn1_lib.c:341 #4 0x7faae98118f7 in asn1_primitive_new crypto/asn1/tasn_new.c:318 #5 0x7faae9812401 in asn1_item_embed_new crypto/asn1/tasn_new.c:78 #6 0x7faae9812401 in asn1_template_new crypto/asn1/tasn_new.c:240 #7 0x7faae9812315 in asn1_item_embed_new crypto/asn1/tasn_new.c:137 #8 0x7faae9812315 in asn1_template_new crypto/asn1/tasn_new.c:240 #9 0x7faae9812a54 in asn1_item_embed_new crypto/asn1/tasn_new.c:137 #10 0x7faae9812a54 in ASN1_item_ex_new crypto/asn1/tasn_new.c:39 #11 0x7faae980be51 in asn1_item_embed_d2i crypto/asn1/tasn_dec.c:325 #12 0x7faae980c813 in asn1_template_noexp_d2i crypto/asn1/tasn_dec.c:611 #13 0x7faae980d288 in asn1_template_ex_d2i crypto/asn1/tasn_dec.c:518 #14 0x7faae980b9ce in asn1_item_embed_d2i crypto/asn1/tasn_dec.c:382 #15 0x7faae980caf5 in asn1_template_noexp_d2i crypto/asn1/tasn_dec.c:643 #16 0x7faae980d7d3 in asn1_template_ex_d2i crypto/asn1/tasn_dec.c:494 #17 0x7faae980b9ce in asn1_item_embed_d2i crypto/asn1/tasn_dec.c:382 #18 0x7faae980dd1f in ASN1_item_ex_d2i crypto/asn1/tasn_dec.c:124 #19 0x7faae980de35 in ASN1_item_d2i crypto/asn1/tasn_dec.c:114 #20 0x40712c in FuzzerTestOneInput fuzz/asn1.c:301 #21 0x40893b in testfile fuzz/test-corpus.c:182 #22 0x406b86 in main fuzz/test-corpus.c:226 #23 0x7faae8eb1f44 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x21f44) AddressSanitizer:DEADLYSIGNAL ================================================================= ==1194==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000010 (pc 0x7faae9b0625f bp 0x7fffffe41a00 sp 0x7fffffe41920 T0) ==1194==The signal is caused by a READ memory access. ==1194==Hint: address points to the zero page. #0 0x7faae9b0625f in crl_cb crypto/x509/x_crl.c:258 #1 0x7faae9811255 in asn1_item_embed_free crypto/asn1/tasn_fre.c:113 #2 0x7faae9812a65 in asn1_item_embed_new crypto/asn1/tasn_new.c:150 #3 0x7faae9812a65 in ASN1_item_ex_new crypto/asn1/tasn_new.c:39 #4 0x7faae980be51 in asn1_item_embed_d2i crypto/asn1/tasn_dec.c:325 #5 0x7faae980c813 in asn1_template_noexp_d2i crypto/asn1/tasn_dec.c:611 #6 0x7faae980d288 in asn1_template_ex_d2i crypto/asn1/tasn_dec.c:518 #7 0x7faae980b9ce in asn1_item_embed_d2i crypto/asn1/tasn_dec.c:382 #8 0x7faae980caf5 in asn1_template_noexp_d2i crypto/asn1/tasn_dec.c:643 #9 0x7faae980d7d3 in asn1_template_ex_d2i crypto/asn1/tasn_dec.c:494 #10 0x7faae980b9ce in asn1_item_embed_d2i crypto/asn1/tasn_dec.c:382 #11 0x7faae980dd1f in ASN1_item_ex_d2i crypto/asn1/tasn_dec.c:124 #12 0x7faae980de35 in ASN1_item_d2i crypto/asn1/tasn_dec.c:114 #13 0x40712c in FuzzerTestOneInput fuzz/asn1.c:301 #14 0x40893b in testfile fuzz/test-corpus.c:182 #15 0x406b86 in main fuzz/test-corpus.c:226 #16 0x7faae8eb1f44 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x21f44) AddressSanitizer can not provide additional info. SUMMARY: AddressSanitizer: SEGV crypto/x509/x_crl.c:258 in crl_cb ==1194==ABORTING Reviewed-by: Matt Caswell <matt@openssl.org> Reviewed-by: Tomas Mraz <tomas@openssl.org> (Merged from #18360) (cherry picked from commit 557825a)

The following issue was found in automatic tests with thread sanitizer builds in ClickHouse (which uses OpenSSL 3.2.1) [0]. The first stack [1] does proper locking (function 'x509_store_add', x509_lu.c) but in the second stack [2], function 'get_cert_by_subject_ex' (by_dir.b) forgets to lock when calling 'sk_X509_OBJECT_is_sorted'. [0] ClickHouse/ClickHouse#63049 [1] WARNING: ThreadSanitizer: data race (pid=1870) Write of size 4 at 0x7b08003d6810 by thread T552 (mutexes: write M0, write M1, write M2, write M3): #0 OPENSSL_sk_insert build_docker/./contrib/openssl/crypto/stack/stack.c:280:16 (clickhouse+0x203ad7e4) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) #1 OPENSSL_sk_push build_docker/./contrib/openssl/crypto/stack/stack.c:401:12 (clickhouse+0x203ad7e4) #2 x509_store_add build_docker/./contrib/openssl/crypto/x509/x509_lu.c:419:17 (clickhouse+0x203d4a52) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) #3 X509_STORE_add_cert build_docker/./contrib/openssl/crypto/x509/x509_lu.c:432:10 (clickhouse+0x203d48a2) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) #4 X509_load_cert_file_ex build_docker/./contrib/openssl/crypto/x509/by_file.c:127:18 (clickhouse+0x203b74e6) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) #5 get_cert_by_subject_ex build_docker/./contrib/openssl/crypto/x509/by_dir.c:333:22 (clickhouse+0x203b684c) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) #6 X509_LOOKUP_by_subject_ex build_docker/./contrib/openssl/crypto/x509/x509_lu.c:105:16 (clickhouse+0x203d46ec) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) #7 ossl_x509_store_ctx_get_by_subject build_docker/./contrib/openssl/crypto/x509/x509_lu.c:360:17 (clickhouse+0x203d46ec) #8 X509_STORE_CTX_get1_issuer build_docker/./contrib/openssl/crypto/x509/x509_lu.c:782:10 (clickhouse+0x203d56cb) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) #9 get1_trusted_issuer build_docker/./contrib/openssl/crypto/x509/x509_vfy.c:3194:10 (clickhouse+0x203db4a9) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) #10 build_chain build_docker/./contrib/openssl/crypto/x509/x509_vfy.c:3324:40 (clickhouse+0x203db4a9) #11 verify_chain build_docker/./contrib/openssl/crypto/x509/x509_vfy.c:240:15 (clickhouse+0x203dbe27) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) #12 x509_verify_x509 build_docker/./contrib/openssl/crypto/x509/x509_vfy.c:358 (clickhouse+0x203d7fd8) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) #13 X509_verify_cert build_docker/./contrib/openssl/crypto/x509/x509_vfy.c:293:56 (clickhouse+0x203d8215) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) #14 ssl_verify_internal build_docker/./contrib/openssl/ssl/ssl_cert.c:496:13 (clickhouse+0x2019a2a4) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) #15 ssl_verify_cert_chain build_docker/./contrib/openssl/ssl/ssl_cert.c:543:12 (clickhouse+0x2019a402) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) #16 tls_post_process_server_certificate build_docker/./contrib/openssl/ssl/statem/statem_clnt.c:2072:9 (clickhouse+0x20227658) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) #17 ossl_statem_client_post_process_message build_docker/./contrib/openssl/ssl/statem/statem_clnt.c:1159:16 (clickhouse+0x202272ee) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) #18 read_state_machine build_docker/./contrib/openssl/ssl/statem/statem.c:712:35 (clickhouse+0x2021e96d) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) #19 state_machine build_docker/./contrib/openssl/ssl/statem/statem.c:478:21 (clickhouse+0x2021e96d) #20 ossl_statem_connect build_docker/./contrib/openssl/ssl/statem/statem.c:297:12 (clickhouse+0x2021ddce) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) #21 SSL_do_handshake build_docker/./contrib/openssl/ssl/ssl_lib.c:4746:19 (clickhouse+0x201a5781) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) #22 SSL_connect build_docker/./contrib/openssl/ssl/ssl_lib.c:2208:12 (clickhouse+0x201a5893) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) #23 Poco::Net::SecureSocketImpl::connectSSL(bool) build_docker/./base/poco/NetSSL_OpenSSL/src/SecureSocketImpl.cpp:206:11 (clickhouse+0x1d179567) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) [2] Previous read of size 4 at 0x7b08003d6810 by thread T553 (mutexes: write M4, write M5, write M6): #0 OPENSSL_sk_is_sorted build_docker/./contrib/openssl/crypto/stack/stack.c:490:33 (clickhouse+0x203adcff) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) #1 get_cert_by_subject_ex build_docker/./contrib/openssl/crypto/x509/by_dir.c:423:10 (clickhouse+0x203b6d8f) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) #2 X509_LOOKUP_by_subject_ex build_docker/./contrib/openssl/crypto/x509/x509_lu.c:105:16 (clickhouse+0x203d46ec) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) #3 ossl_x509_store_ctx_get_by_subject build_docker/./contrib/openssl/crypto/x509/x509_lu.c:360:17 (clickhouse+0x203d46ec) #4 X509_STORE_CTX_get1_issuer build_docker/./contrib/openssl/crypto/x509/x509_lu.c:782:10 (clickhouse+0x203d56cb) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) #5 get1_trusted_issuer build_docker/./contrib/openssl/crypto/x509/x509_vfy.c:3194:10 (clickhouse+0x203db4a9) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) #6 build_chain build_docker/./contrib/openssl/crypto/x509/x509_vfy.c:3324:40 (clickhouse+0x203db4a9) #7 verify_chain build_docker/./contrib/openssl/crypto/x509/x509_vfy.c:240:15 (clickhouse+0x203dbe27) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) #8 x509_verify_x509 build_docker/./contrib/openssl/crypto/x509/x509_vfy.c:358 (clickhouse+0x203d7fd8) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) #9 X509_verify_cert build_docker/./contrib/openssl/crypto/x509/x509_vfy.c:293:56 (clickhouse+0x203d8215) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) #10 ssl_verify_internal build_docker/./contrib/openssl/ssl/ssl_cert.c:496:13 (clickhouse+0x2019a2a4) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) #11 ssl_verify_cert_chain build_docker/./contrib/openssl/ssl/ssl_cert.c:543:12 (clickhouse+0x2019a402) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) #12 tls_post_process_server_certificate build_docker/./contrib/openssl/ssl/statem/statem_clnt.c:2072:9 (clickhouse+0x20227658) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) #13 ossl_statem_client_post_process_message build_docker/./contrib/openssl/ssl/statem/statem_clnt.c:1159:16 (clickhouse+0x202272ee) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) #14 read_state_machine build_docker/./contrib/openssl/ssl/statem/statem.c:712:35 (clickhouse+0x2021e96d) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) #15 state_machine build_docker/./contrib/openssl/ssl/statem/statem.c:478:21 (clickhouse+0x2021e96d) #16 ossl_statem_connect build_docker/./contrib/openssl/ssl/statem/statem.c:297:12 (clickhouse+0x2021ddce) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) #17 SSL_do_handshake build_docker/./contrib/openssl/ssl/ssl_lib.c:4746:19 (clickhouse+0x201a5781) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) #18 SSL_connect build_docker/./contrib/openssl/ssl/ssl_lib.c:2208:12 (clickhouse+0x201a5893) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) #19 Poco::Net::SecureSocketImpl::connectSSL(bool) build_docker/./base/poco/NetSSL_OpenSSL/src/SecureSocketImpl.cpp:206:11 (clickhouse+0x1d179567) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) CLA: trivial Reviewed-by: Todd Short <todd.short@me.com> Reviewed-by: Tomas Mraz <tomas@openssl.org> (Merged from #24295) (cherry picked from commit af75373)

The following issue was found in automatic tests with thread sanitizer builds in ClickHouse (which uses OpenSSL 3.2.1) [0]. The first stack [1] does proper locking (function 'x509_store_add', x509_lu.c) but in the second stack [2], function 'get_cert_by_subject_ex' (by_dir.b) forgets to lock when calling 'sk_X509_OBJECT_is_sorted'. [0] ClickHouse/ClickHouse#63049 [1] WARNING: ThreadSanitizer: data race (pid=1870) Write of size 4 at 0x7b08003d6810 by thread T552 (mutexes: write M0, write M1, write M2, write M3): #0 OPENSSL_sk_insert build_docker/./contrib/openssl/crypto/stack/stack.c:280:16 (clickhouse+0x203ad7e4) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) #1 OPENSSL_sk_push build_docker/./contrib/openssl/crypto/stack/stack.c:401:12 (clickhouse+0x203ad7e4) #2 x509_store_add build_docker/./contrib/openssl/crypto/x509/x509_lu.c:419:17 (clickhouse+0x203d4a52) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) #3 X509_STORE_add_cert build_docker/./contrib/openssl/crypto/x509/x509_lu.c:432:10 (clickhouse+0x203d48a2) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) #4 X509_load_cert_file_ex build_docker/./contrib/openssl/crypto/x509/by_file.c:127:18 (clickhouse+0x203b74e6) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) #5 get_cert_by_subject_ex build_docker/./contrib/openssl/crypto/x509/by_dir.c:333:22 (clickhouse+0x203b684c) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) #6 X509_LOOKUP_by_subject_ex build_docker/./contrib/openssl/crypto/x509/x509_lu.c:105:16 (clickhouse+0x203d46ec) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) #7 ossl_x509_store_ctx_get_by_subject build_docker/./contrib/openssl/crypto/x509/x509_lu.c:360:17 (clickhouse+0x203d46ec) #8 X509_STORE_CTX_get1_issuer build_docker/./contrib/openssl/crypto/x509/x509_lu.c:782:10 (clickhouse+0x203d56cb) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) #9 get1_trusted_issuer build_docker/./contrib/openssl/crypto/x509/x509_vfy.c:3194:10 (clickhouse+0x203db4a9) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) #10 build_chain build_docker/./contrib/openssl/crypto/x509/x509_vfy.c:3324:40 (clickhouse+0x203db4a9) #11 verify_chain build_docker/./contrib/openssl/crypto/x509/x509_vfy.c:240:15 (clickhouse+0x203dbe27) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) #12 x509_verify_x509 build_docker/./contrib/openssl/crypto/x509/x509_vfy.c:358 (clickhouse+0x203d7fd8) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) #13 X509_verify_cert build_docker/./contrib/openssl/crypto/x509/x509_vfy.c:293:56 (clickhouse+0x203d8215) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) #14 ssl_verify_internal build_docker/./contrib/openssl/ssl/ssl_cert.c:496:13 (clickhouse+0x2019a2a4) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) #15 ssl_verify_cert_chain build_docker/./contrib/openssl/ssl/ssl_cert.c:543:12 (clickhouse+0x2019a402) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) #16 tls_post_process_server_certificate build_docker/./contrib/openssl/ssl/statem/statem_clnt.c:2072:9 (clickhouse+0x20227658) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) #17 ossl_statem_client_post_process_message build_docker/./contrib/openssl/ssl/statem/statem_clnt.c:1159:16 (clickhouse+0x202272ee) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) #18 read_state_machine build_docker/./contrib/openssl/ssl/statem/statem.c:712:35 (clickhouse+0x2021e96d) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) #19 state_machine build_docker/./contrib/openssl/ssl/statem/statem.c:478:21 (clickhouse+0x2021e96d) #20 ossl_statem_connect build_docker/./contrib/openssl/ssl/statem/statem.c:297:12 (clickhouse+0x2021ddce) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) #21 SSL_do_handshake build_docker/./contrib/openssl/ssl/ssl_lib.c:4746:19 (clickhouse+0x201a5781) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) #22 SSL_connect build_docker/./contrib/openssl/ssl/ssl_lib.c:2208:12 (clickhouse+0x201a5893) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) #23 Poco::Net::SecureSocketImpl::connectSSL(bool) build_docker/./base/poco/NetSSL_OpenSSL/src/SecureSocketImpl.cpp:206:11 (clickhouse+0x1d179567) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) [2] Previous read of size 4 at 0x7b08003d6810 by thread T553 (mutexes: write M4, write M5, write M6): #0 OPENSSL_sk_is_sorted build_docker/./contrib/openssl/crypto/stack/stack.c:490:33 (clickhouse+0x203adcff) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) #1 get_cert_by_subject_ex build_docker/./contrib/openssl/crypto/x509/by_dir.c:423:10 (clickhouse+0x203b6d8f) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) #2 X509_LOOKUP_by_subject_ex build_docker/./contrib/openssl/crypto/x509/x509_lu.c:105:16 (clickhouse+0x203d46ec) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) #3 ossl_x509_store_ctx_get_by_subject build_docker/./contrib/openssl/crypto/x509/x509_lu.c:360:17 (clickhouse+0x203d46ec) #4 X509_STORE_CTX_get1_issuer build_docker/./contrib/openssl/crypto/x509/x509_lu.c:782:10 (clickhouse+0x203d56cb) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) #5 get1_trusted_issuer build_docker/./contrib/openssl/crypto/x509/x509_vfy.c:3194:10 (clickhouse+0x203db4a9) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) #6 build_chain build_docker/./contrib/openssl/crypto/x509/x509_vfy.c:3324:40 (clickhouse+0x203db4a9) #7 verify_chain build_docker/./contrib/openssl/crypto/x509/x509_vfy.c:240:15 (clickhouse+0x203dbe27) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) #8 x509_verify_x509 build_docker/./contrib/openssl/crypto/x509/x509_vfy.c:358 (clickhouse+0x203d7fd8) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) #9 X509_verify_cert build_docker/./contrib/openssl/crypto/x509/x509_vfy.c:293:56 (clickhouse+0x203d8215) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) #10 ssl_verify_internal build_docker/./contrib/openssl/ssl/ssl_cert.c:496:13 (clickhouse+0x2019a2a4) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) #11 ssl_verify_cert_chain build_docker/./contrib/openssl/ssl/ssl_cert.c:543:12 (clickhouse+0x2019a402) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) #12 tls_post_process_server_certificate build_docker/./contrib/openssl/ssl/statem/statem_clnt.c:2072:9 (clickhouse+0x20227658) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) #13 ossl_statem_client_post_process_message build_docker/./contrib/openssl/ssl/statem/statem_clnt.c:1159:16 (clickhouse+0x202272ee) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) #14 read_state_machine build_docker/./contrib/openssl/ssl/statem/statem.c:712:35 (clickhouse+0x2021e96d) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) #15 state_machine build_docker/./contrib/openssl/ssl/statem/statem.c:478:21 (clickhouse+0x2021e96d) #16 ossl_statem_connect build_docker/./contrib/openssl/ssl/statem/statem.c:297:12 (clickhouse+0x2021ddce) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) #17 SSL_do_handshake build_docker/./contrib/openssl/ssl/ssl_lib.c:4746:19 (clickhouse+0x201a5781) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) #18 SSL_connect build_docker/./contrib/openssl/ssl/ssl_lib.c:2208:12 (clickhouse+0x201a5893) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) #19 Poco::Net::SecureSocketImpl::connectSSL(bool) build_docker/./base/poco/NetSSL_OpenSSL/src/SecureSocketImpl.cpp:206:11 (clickhouse+0x1d179567) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) CLA: trivial Reviewed-by: Todd Short <todd.short@me.com> Reviewed-by: Tomas Mraz <tomas@openssl.org> (Merged from #24295)

The following issue was found in automatic tests with thread sanitizer builds in ClickHouse (which uses OpenSSL 3.2.1) [0]. The first stack [1] does proper locking (function 'x509_store_add', x509_lu.c) but in the second stack [2], function 'get_cert_by_subject_ex' (by_dir.b) forgets to lock when calling 'sk_X509_OBJECT_is_sorted'. [0] ClickHouse/ClickHouse#63049 [1] WARNING: ThreadSanitizer: data race (pid=1870) Write of size 4 at 0x7b08003d6810 by thread T552 (mutexes: write M0, write M1, write M2, write M3): #0 OPENSSL_sk_insert build_docker/./contrib/openssl/crypto/stack/stack.c:280:16 (clickhouse+0x203ad7e4) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) openssl#1 OPENSSL_sk_push build_docker/./contrib/openssl/crypto/stack/stack.c:401:12 (clickhouse+0x203ad7e4) openssl#2 x509_store_add build_docker/./contrib/openssl/crypto/x509/x509_lu.c:419:17 (clickhouse+0x203d4a52) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) openssl#3 X509_STORE_add_cert build_docker/./contrib/openssl/crypto/x509/x509_lu.c:432:10 (clickhouse+0x203d48a2) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) openssl#4 X509_load_cert_file_ex build_docker/./contrib/openssl/crypto/x509/by_file.c:127:18 (clickhouse+0x203b74e6) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) openssl#5 get_cert_by_subject_ex build_docker/./contrib/openssl/crypto/x509/by_dir.c:333:22 (clickhouse+0x203b684c) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) openssl#6 X509_LOOKUP_by_subject_ex build_docker/./contrib/openssl/crypto/x509/x509_lu.c:105:16 (clickhouse+0x203d46ec) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) openssl#7 ossl_x509_store_ctx_get_by_subject build_docker/./contrib/openssl/crypto/x509/x509_lu.c:360:17 (clickhouse+0x203d46ec) openssl#8 X509_STORE_CTX_get1_issuer build_docker/./contrib/openssl/crypto/x509/x509_lu.c:782:10 (clickhouse+0x203d56cb) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) openssl#9 get1_trusted_issuer build_docker/./contrib/openssl/crypto/x509/x509_vfy.c:3194:10 (clickhouse+0x203db4a9) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) openssl#10 build_chain build_docker/./contrib/openssl/crypto/x509/x509_vfy.c:3324:40 (clickhouse+0x203db4a9) openssl#11 verify_chain build_docker/./contrib/openssl/crypto/x509/x509_vfy.c:240:15 (clickhouse+0x203dbe27) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) openssl#12 x509_verify_x509 build_docker/./contrib/openssl/crypto/x509/x509_vfy.c:358 (clickhouse+0x203d7fd8) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) openssl#13 X509_verify_cert build_docker/./contrib/openssl/crypto/x509/x509_vfy.c:293:56 (clickhouse+0x203d8215) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) openssl#14 ssl_verify_internal build_docker/./contrib/openssl/ssl/ssl_cert.c:496:13 (clickhouse+0x2019a2a4) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) openssl#15 ssl_verify_cert_chain build_docker/./contrib/openssl/ssl/ssl_cert.c:543:12 (clickhouse+0x2019a402) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) openssl#16 tls_post_process_server_certificate build_docker/./contrib/openssl/ssl/statem/statem_clnt.c:2072:9 (clickhouse+0x20227658) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) openssl#17 ossl_statem_client_post_process_message build_docker/./contrib/openssl/ssl/statem/statem_clnt.c:1159:16 (clickhouse+0x202272ee) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) openssl#18 read_state_machine build_docker/./contrib/openssl/ssl/statem/statem.c:712:35 (clickhouse+0x2021e96d) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) openssl#19 state_machine build_docker/./contrib/openssl/ssl/statem/statem.c:478:21 (clickhouse+0x2021e96d) openssl#20 ossl_statem_connect build_docker/./contrib/openssl/ssl/statem/statem.c:297:12 (clickhouse+0x2021ddce) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) openssl#21 SSL_do_handshake build_docker/./contrib/openssl/ssl/ssl_lib.c:4746:19 (clickhouse+0x201a5781) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) openssl#22 SSL_connect build_docker/./contrib/openssl/ssl/ssl_lib.c:2208:12 (clickhouse+0x201a5893) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) openssl#23 Poco::Net::SecureSocketImpl::connectSSL(bool) build_docker/./base/poco/NetSSL_OpenSSL/src/SecureSocketImpl.cpp:206:11 (clickhouse+0x1d179567) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) [2] Previous read of size 4 at 0x7b08003d6810 by thread T553 (mutexes: write M4, write M5, write M6): #0 OPENSSL_sk_is_sorted build_docker/./contrib/openssl/crypto/stack/stack.c:490:33 (clickhouse+0x203adcff) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) openssl#1 get_cert_by_subject_ex build_docker/./contrib/openssl/crypto/x509/by_dir.c:423:10 (clickhouse+0x203b6d8f) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) openssl#2 X509_LOOKUP_by_subject_ex build_docker/./contrib/openssl/crypto/x509/x509_lu.c:105:16 (clickhouse+0x203d46ec) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) openssl#3 ossl_x509_store_ctx_get_by_subject build_docker/./contrib/openssl/crypto/x509/x509_lu.c:360:17 (clickhouse+0x203d46ec) openssl#4 X509_STORE_CTX_get1_issuer build_docker/./contrib/openssl/crypto/x509/x509_lu.c:782:10 (clickhouse+0x203d56cb) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) openssl#5 get1_trusted_issuer build_docker/./contrib/openssl/crypto/x509/x509_vfy.c:3194:10 (clickhouse+0x203db4a9) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) openssl#6 build_chain build_docker/./contrib/openssl/crypto/x509/x509_vfy.c:3324:40 (clickhouse+0x203db4a9) openssl#7 verify_chain build_docker/./contrib/openssl/crypto/x509/x509_vfy.c:240:15 (clickhouse+0x203dbe27) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) openssl#8 x509_verify_x509 build_docker/./contrib/openssl/crypto/x509/x509_vfy.c:358 (clickhouse+0x203d7fd8) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) openssl#9 X509_verify_cert build_docker/./contrib/openssl/crypto/x509/x509_vfy.c:293:56 (clickhouse+0x203d8215) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) openssl#10 ssl_verify_internal build_docker/./contrib/openssl/ssl/ssl_cert.c:496:13 (clickhouse+0x2019a2a4) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) openssl#11 ssl_verify_cert_chain build_docker/./contrib/openssl/ssl/ssl_cert.c:543:12 (clickhouse+0x2019a402) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) openssl#12 tls_post_process_server_certificate build_docker/./contrib/openssl/ssl/statem/statem_clnt.c:2072:9 (clickhouse+0x20227658) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) openssl#13 ossl_statem_client_post_process_message build_docker/./contrib/openssl/ssl/statem/statem_clnt.c:1159:16 (clickhouse+0x202272ee) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) openssl#14 read_state_machine build_docker/./contrib/openssl/ssl/statem/statem.c:712:35 (clickhouse+0x2021e96d) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) openssl#15 state_machine build_docker/./contrib/openssl/ssl/statem/statem.c:478:21 (clickhouse+0x2021e96d) openssl#16 ossl_statem_connect build_docker/./contrib/openssl/ssl/statem/statem.c:297:12 (clickhouse+0x2021ddce) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) openssl#17 SSL_do_handshake build_docker/./contrib/openssl/ssl/ssl_lib.c:4746:19 (clickhouse+0x201a5781) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) openssl#18 SSL_connect build_docker/./contrib/openssl/ssl/ssl_lib.c:2208:12 (clickhouse+0x201a5893) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) openssl#19 Poco::Net::SecureSocketImpl::connectSSL(bool) build_docker/./base/poco/NetSSL_OpenSSL/src/SecureSocketImpl.cpp:206:11 (clickhouse+0x1d179567) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) CLA: trivial Reviewed-by: Todd Short <todd.short@me.com> Reviewed-by: Tomas Mraz <tomas@openssl.org> (Merged from openssl#24295)

During my testing (*), I got below stack about leaked memory. - at database startup, Azure is registered as an object storage (--> registerAzureObjectStorage) - this calls into Azure, then into curl, and then into OpenSSL - curl asks OpenSSL for a bunch of random numbers (--> function 'ossl_random' in curl's OpenSSL wrapper) - OpenSSL initializes the random number generator and stores it in some random number generator context object (--> *RAND_get0_public) - this object is registered via pthread_key_create and pthread_setspecific registered in TLS - if registerAzureObjectStorage was the only place which initializes the RNG, we could argue that the leaked memory does not matter anyways as it is released after shutdown - RAND_get0_public also registers a free handler (rand_delete_thread_state) that runs in TLS, so the memory is released also if registration is called from arbitrary other threads. In sum: this is a false positive. (*) ClickHouse/clickhouse-private#10107 (comment) ``` ================================================================= ==2757181==ERROR: LeakSanitizer: detected memory leaks Direct leak of 48 byte(s) in 1 object(s) allocated from: #0 0x5672f4a48083 in malloc (/data/ch4/build_asan/programs/clickhouse+0xa2b7083) (BuildId: 4d868624feaa8b5598d9d0b327749c877bfb6cfd) #1 0x567321d05c7e in CRYPTO_malloc build_asan/./contrib/openssl/crypto/mem.c:202:11 #2 0x567321d05c7e in CRYPTO_zalloc build_asan/./contrib/openssl/crypto/mem.c:222:11 #3 0x567321d5f9cf in EVP_RAND_CTX_new build_asan/./contrib/openssl/crypto/evp/evp_rand.c:353:11 #4 0x567321d627db in rand_new_drbg build_asan/./contrib/openssl/crypto/rand/rand_lib.c:658:11 #5 0x567321d61e7b in RAND_get0_public build_asan/./contrib/openssl/crypto/rand/rand_lib.c:777:16 #6 0x567321d61d4f in RAND_bytes_ex build_asan/./contrib/openssl/crypto/rand/rand_lib.c:378:12 #7 0x56731cfe9267 in ossl_random openssl.c #8 0x56731cf534ae in Curl_rand (/data/ch4/build_asan/programs/clickhouse+0x327c24ae) (BuildId: 4d868624feaa8b5598d9d0b327749c877bfb6cfd) openssl#9 0x56731cf757dc in Curl_socketpair (/data/ch4/build_asan/programs/clickhouse+0x327e47dc) (BuildId: 4d868624feaa8b5598d9d0b327749c877bfb6cfd) openssl#10 0x56731cf40bfe in Curl_multi_handle (/data/ch4/build_asan/programs/clickhouse+0x327afbfe) (BuildId: 4d868624feaa8b5598d9d0b327749c877bfb6cfd) openssl#11 0x56731cecb260 in curl_easy_perform (/data/ch4/build_asan/programs/clickhouse+0x3273a260) (BuildId: 4d868624feaa8b5598d9d0b327749c877bfb6cfd) openssl#12 0x56731cd4a351 in Azure::Core::Http::CurlConnection::CurlConnection(Azure::Core::Http::Request&, Azure::Core::Http::CurlTransportOptions const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&) build_asan/./contrib/azure/sdk/core/azure-core/src/http/curl/curl.cpp:2441:24 openssl#13 0x56731cd3176b in std::__1::__unique_if<Azure::Core::Http::CurlConnection>::__unique_single std::__1::make_unique[abi:v15000]<Azure::Core::Http::CurlConnection, Azure::Core::Http::Request&, Azure::Core::Http::CurlTransportOptions const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&>(Azure::Core::Http::Request&, Azure::Core::H ttp::CurlTransportOptions const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&) build_asan/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714:32 openssl#14 0x56731cd3176b in Azure::Core::Http::_detail::CurlConnectionPool::ExtractOrCreateCurlConnection(Azure::Core::Http::Request&, Azure::Core::Http::CurlTransportOptions const&, bool) build_asan/./contrib/azure/sdk/core/azure-core/src/http/curl/curl.cpp:2126:10 openssl#15 0x56731cd2f891 in Azure::Core::Http::CurlTransport::Send(Azure::Core::Http::Request&, Azure::Core::Context const&) build_asan/./contrib/azure/sdk/core/azure-core/src/http/curl/curl.cpp:351:48 openssl#16 0x56731cdd66db in Azure::Core::Http::Policies::_internal::TransportPolicy::Send(Azure::Core::Http::Request&, Azure::Core::Http::Policies::NextHttpPolicy, Azure::Core::Context const&) const build_asan/./contrib/azure/sdk/core/azure-core/src/http/transport_policy.cpp:121:40 openssl#17 0x56731cd7c75b in Azure::Core::Http::Policies::_internal::LogPolicy::Send(Azure::Core::Http::Request&, Azure::Core::Http::Policies::NextHttpPolicy, Azure::Core::Context const&) const build_asan/./contrib/azure/sdk/core/azure-core/src/http/log_policy.cpp:114:23 openssl#18 0x56731cdd0218 in Azure::Core::Http::Policies::_internal::RequestActivityPolicy::Send(Azure::Core::Http::Request&, Azure::Core::Http::Policies::NextHttpPolicy, Azure::Core::Context const&) const build_asan/./contrib/azure/sdk/core/azure-core/src/http/request_activity_policy.cpp:110:23 openssl#19 0x56731cec5f18 in Azure::Storage::_internal::StoragePerRetryPolicy::Send(Azure::Core::Http::Request&, Azure::Core::Http::Policies::NextHttpPolicy, Azure::Core::Context const&) const build_asan/./contrib/azure/sdk/storage/azure-storage-common/src/storage_per_retry_policy.cpp:57:23 openssl#20 0x56731cec6d0b in Azure::Storage::_internal::StorageSwitchToSecondaryPolicy::Send(Azure::Core::Http::Request&, Azure::Core::Http::Policies::NextHttpPolicy, Azure::Core::Context const&) const build_asan/./contrib/azure/sdk/storage/azure-storage-common/src/storage_switch_to_secondary_policy.cpp:36:32 openssl#21 0x56731cdcd019 in Azure::Core::Http::Policies::_internal::RetryPolicy::Send(Azure::Core::Http::Request&, Azure::Core::Http::Policies::NextHttpPolicy, Azure::Core::Context const&) const build_asan/./contrib/azure/sdk/core/azure-core/src/http/retry_policy.cpp:146:34 openssl#22 0x56731cdd6e17 in Azure::Core::Http::Policies::_internal::TelemetryPolicy::Send(Azure::Core::Http::Request&, Azure::Core::Http::Policies::NextHttpPolicy, Azure::Core::Context const&) const build_asan/./contrib/azure/sdk/core/azure-core/src/http/telemetry_policy.cpp:23:21 openssl#23 0x56731cdb0ef5 in Azure::Core::Http::Policies::_internal::RequestIdPolicy::Send(Azure::Core::Http::Request&, Azure::Core::Http::Policies::NextHttpPolicy, Azure::Core::Context const&) const build_asan/./contrib/azure/sdk/core/azure-core/inc/azure/core/http/policies/policy.hpp:453:27 openssl#24 0x56731cdfc53e in Azure::Storage::_internal::StorageServiceVersionPolicy::Send(Azure::Core::Http::Request&, Azure::Core::Http::Policies::NextHttpPolicy, Azure::Core::Context const&) const build_asan/./contrib/azure/sdk/storage/azure-storage-common/inc/azure/storage/common/internal/storage_service_version_policy.hpp:34:25 openssl#25 0x56731ce4bff1 in Azure::Core::Http::_internal::HttpPipeline::Send(Azure::Core::Http::Request&, Azure::Core::Context const&) const build_asan/./contrib/azure/sdk/core/azure-core/inc/azure/core/internal/http/pipeline.hpp:230:29 openssl#26 0x56731ce4bff1 in Azure::Storage::Blobs::_detail::BlobContainerClient::Create(Azure::Core::Http::_internal::HttpPipeline&, Azure::Core::Url const&, Azure::Storage::Blobs::_detail::BlobContainerClient::CreateBlobContainerOptions const&, Azure::Core::Context const&) build_asan/./contrib/azure/sdk/storage/azure-storage-blobs/src/rest_client.cpp:1415:36 openssl#27 0x56731ce0bdd8 in Azure::Storage::Blobs::BlobContainerClient::Create(Azure::Storage::Blobs::CreateBlobContainerOptions const&, Azure::Core::Context const&) const build_asan/./contrib/azure/sdk/storage/azure-storage-blobs/src/blob_container_client.cpp:258:12 openssl#28 0x56731ce396dc in Azure::Storage::Blobs::BlobServiceClient::CreateBlobContainer(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, Azure::Storage::Blobs::CreateBlobContainerOptions const&, Azure::Core::Context const&) const build_asan/./contrib/azure/sdk/storage/azure-storage-blobs/src/blob_service_client.cpp:264:41 openssl#29 0x56731018fcff in DB::getAzureBlobContainerClient(Poco::Util::AbstractConfiguration const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&) build_asan/./src/Disks/ObjectStorages/AzureBlobStorage/AzureBlobStorageAuth.cpp:236:75 openssl#30 0x5673115a7352 in DB::registerAzureObjectStorage(DB::ObjectStorageFactory&)::$_0::operator()(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, Poco::Util::AbstractConfiguration const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::shared_ptr<DB::Context const> const&, bool) const build_asan/./src/Disks/ObjectStorages/ObjectStorageFactory.cpp:315:13 openssl#31 0x5673115a7352 in decltype(std::declval<DB::registerAzureObjectStorage(DB::ObjectStorageFactory&)::$_0&>()(std::declval<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&>(), std::declval<Poco::Util::AbstractConfiguration const&>(), std::declval<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&>(), std::declval<std::__1::shared_ptr<DB::Context const> const&>(), std::declval<bool>() )) std::__1::__invoke[abi:v15000]<DB::registerAzureObjectStorage(DB::ObjectStorageFactory&)::$_0&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, Poco::Util::AbstractConfiguration const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::shared_ptr<DB::Context const> const&, bool>(DB::registerAzureObjectStorage(DB::ObjectStorageFactory&)::$_0&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, Poco::Util::AbstractConfiguration const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::shared_ptr<DB::Context const> const&, bool&&) build_asan/./contrib/llvm-project/libcxx/include/__functional/invoke.h:394:23 openssl#32 0x5673115a7352 in std::__1::shared_ptr<DB::IObjectStorage> std::__1::__invoke_void_return_wrapper<std::__1::shared_ptr<DB::IObjectStorage>, false>::__call<DB::registerAzureObjectStorage(DB::ObjectStorageFactory&)::$_0&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, Poco::Util::AbstractConfiguration const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::shared_pt r<DB::Context const> const&, bool>(DB::registerAzureObjectStorage(DB::ObjectStorageFactory&)::$_0&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, Poco::Util::AbstractConfiguration const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::shared_ptr<DB::Context const> const&, bool&&) build_asan/./contrib/llvm-project/libcxx/include/__functional/invoke.h:470:16 openssl#33 0x5673115a7352 in std::__1::__function::__default_alloc_func<DB::registerAzureObjectStorage(DB::ObjectStorageFactory&)::$_0, std::__1::shared_ptr<DB::IObjectStorage> (std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, Poco::Util::AbstractConfiguration const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::shared_ptr<DB::Context const> const&, bool)>::operator()[abi:v 15000](std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, Poco::Util::AbstractConfiguration const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::shared_ptr<DB::Context const> const&, bool&&) build_asan/./contrib/llvm-project/libcxx/include/__functional/function.h:235:12 openssl#34 0x5673115a7352 in std::__1::shared_ptr<DB::IObjectStorage> std::__1::__function::__policy_invoker<std::__1::shared_ptr<DB::IObjectStorage> (std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, Poco::Util::AbstractConfiguration const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::shared_ptr<DB::Context const> const&, bool)>::__call_impl<std::__1::__function::__default _alloc_func<DB::registerAzureObjectStorage(DB::ObjectStorageFactory&)::$_0, std::__1::shared_ptr<DB::IObjectStorage> (std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, Poco::Util::AbstractConfiguration const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::shared_ptr<DB::Context const> const&, bool)>>(std::__1::__function::__policy_storage const*, std::__1::basic_string<cha r, std::__1::char_traits<char>, std::__1::allocator<char>> const&, Poco::Util::AbstractConfiguration const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::shared_ptr<DB::Context const> const&, bool) build_asan/./contrib/llvm-project/libcxx/include/__functional/function.h:716:16 openssl#35 0x5673115a0780 in std::__1::__function::__policy_func<std::__1::shared_ptr<DB::IObjectStorage> (std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, Poco::Util::AbstractConfiguration const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::shared_ptr<DB::Context const> const&, bool)>::operator()[abi:v15000](std::__1::basic_string<char, std::__1::char_traits<char>, std::_ _1::allocator<char>> const&, Poco::Util::AbstractConfiguration const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::shared_ptr<DB::Context const> const&, bool&&) const build_asan/./contrib/llvm-project/libcxx/include/__functional/function.h:848:16 openssl#36 0x5673115a0780 in std::__1::function<std::__1::shared_ptr<DB::IObjectStorage> (std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, Poco::Util::AbstractConfiguration const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::shared_ptr<DB::Context const> const&, bool)>::operator()(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, Poco::Util::AbstractConfiguration const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::shared_ptr<DB::Context const> const&, bool) const build_asan/./contrib/llvm-project/libcxx/include/__functional/function.h:1187:12 openssl#37 0x5673115a0780 in DB::ObjectStorageFactory::create(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, Poco::Util::AbstractConfiguration const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::shared_ptr<DB::Context const> const&, bool) const build_asan/./src/Disks/ObjectStorages/ObjectStorageFactory.cpp:135:12 openssl#38 0x56731159ef45 in DB::registerDiskObjectStorage(DB::DiskFactory&, bool)::$_0::operator()(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, Poco::Util::AbstractConfiguration const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::shared_ptr<DB::Context const>, std::__1::map<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, std::__1: :shared_ptr<DB::IDisk>, std::__1::less<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>, std::__1::allocator<std::__1::pair<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const, std::__1::shared_ptr<DB::IDisk>>>> const&, bool, bool) const build_asan/./src/Disks/ObjectStorages/RegisterDiskObjectStorage.cpp:27:64 openssl#39 0x56731159ef45 in decltype(std::declval<DB::registerDiskObjectStorage(DB::DiskFactory&, bool)::$_0&>()(std::declval<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&>(), std::declval<Poco::Util::AbstractConfiguration const&>(), std::declval<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&>(), std::declval<std::__1::shared_ptr<DB::Context const>>(), std::declval<std::__1::map<std: :__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, std::__1::shared_ptr<DB::IDisk>, std::__1::less<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>, std::__1::allocator<std::__1::pair<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const, std::__1::shared_ptr<DB::IDisk>>>> const&>(), std::declval<bool>(), std::declval<bool>())) std::__1::__invoke[abi:v15000]<DB::regis terDiskObjectStorage(DB::DiskFactory&, bool)::$_0&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, Poco::Util::AbstractConfiguration const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::shared_ptr<DB::Context const>, std::__1::map<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, std::__1::shared_ptr<DB::IDisk>, std::__1::less<std::_ _1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>, std::__1::allocator<std::__1::pair<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const, std::__1::shared_ptr<DB::IDisk>>>> const&, bool, bool>(DB::registerDiskObjectStorage(DB::DiskFactory&, bool)::$_0&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, Poco::Util::AbstractConfiguration const&, std::__1::basic_ string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::shared_ptr<DB::Context const>&&, std::__1::map<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, std::__1::shared_ptr<DB::IDisk>, std::__1::less<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>, std::__1::allocator<std::__1::pair<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const, std::__1::shared_ptr<DB::IDisk>>>> const&, bool&&, bool&&) build_asan/./contrib/llvm-project/libcxx/include/__functional/invoke.h:394:23 openssl#40 0x56731159ef45 in std::__1::shared_ptr<DB::IDisk> std::__1::__invoke_void_return_wrapper<std::__1::shared_ptr<DB::IDisk>, false>::__call<DB::registerDiskObjectStorage(DB::DiskFactory&, bool)::$_0&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, Poco::Util::AbstractConfiguration const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::shared_ptr<DB::Context const>, std::__1::map<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, std::__1::shared_ptr<DB::IDisk>, std::__1::less<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>, std::__1::allocator<std::__1::pair<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const, std::__1::shared_ptr<DB::IDisk>>>> const&, bool, bool>(DB::registerDiskObjectStorage(DB::DiskFactory&, bool)::$_0 &, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, Poco::Util::AbstractConfiguration const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::shared_ptr<DB::Context const>&&, std::__1::map<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, std::__1::shared_ptr<DB::IDisk>, std::__1::less<std::__1::basic_string<char, std::__1::char_traits<ch ar>, std::__1::allocator<char>>>, std::__1::allocator<std::__1::pair<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const, std::__1::shared_ptr<DB::IDisk>>>> const&, bool&&, bool&&) build_asan/./contrib/llvm-project/libcxx/include/__functional/invoke.h:470:16 openssl#41 0x56731159ef45 in std::__1::__function::__default_alloc_func<DB::registerDiskObjectStorage(DB::DiskFactory&, bool)::$_0, std::__1::shared_ptr<DB::IDisk> (std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, Poco::Util::AbstractConfiguration const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::shared_ptr<DB::Context const>, std::__1::map<std::__1::basic_string<char, s td::__1::char_traits<char>, std::__1::allocator<char>>, std::__1::shared_ptr<DB::IDisk>, std::__1::less<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>, std::__1::allocator<std::__1::pair<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const, std::__1::shared_ptr<DB::IDisk>>>> const&, bool, bool)>::operator()[abi:v15000](std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator <char>> const&, Poco::Util::AbstractConfiguration const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::shared_ptr<DB::Context const>&&, std::__1::map<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, std::__1::shared_ptr<DB::IDisk>, std::__1::less<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>, std::__1::allocator<std::__1::pair<std::__1::b asic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const, std::__1::shared_ptr<DB::IDisk>>>> const&, bool&&, bool&&) build_asan/./contrib/llvm-project/libcxx/include/__functional/function.h:235:12 ```

ClickHouse issue ClickHouse/clickhouse-private#10107 (comment) reports this memory leak: (note how it is very similar to ClickHouse/clickhouse-private#10107 (comment)) ``` Direct leak of 528 byte(s) in 11 object(s) allocated from: #0 0x561369af24cf in malloc (/usr/bin/clickhouse+0xa6cf4cf) (BuildId: 22880fad595a96b17eb9add20e7a01f8ded54c49) #1 0x561397b86a7e in CRYPTO_malloc build_docker/./contrib/openssl/crypto/mem.c:202:11 #2 0x561397b86a7e in CRYPTO_zalloc build_docker/./contrib/openssl/crypto/mem.c:222:11 #3 0x561397be07cf in EVP_RAND_CTX_new build_docker/./contrib/openssl/crypto/evp/evp_rand.c:353:11 #4 0x561397be35fb in rand_new_drbg build_docker/./contrib/openssl/crypto/rand/rand_lib.c:665:11 #5 0x561397be2a9b in RAND_get0_private build_docker/./contrib/openssl/crypto/rand/rand_lib.c:827:16 #6 0x561397be296f in RAND_priv_bytes_ex build_docker/./contrib/openssl/crypto/rand/rand_lib.c:356:12 #7 0x5613978d7cd8 in SSL_CTX_new_ex build_docker/./contrib/openssl/ssl/ssl_lib.c:4016:13 #8 0x561392d1eabd in ossl_connect_common openssl.c openssl#9 0x561392d11e23 in ssl_cf_connect vtls.c openssl#10 0x561392c13a1f in cf_setup_connect connect.c openssl#11 0x561392c1b0e5 in cf_hc_connect cf-https-connect.c openssl#12 0x561392c0a332 in Curl_conn_connect (/usr/bin/clickhouse+0x337e7332) (BuildId: 22880fad595a96b17eb9add20e7a01f8ded54c49) openssl#13 0x561392c7a28a in multi_runsingle multi.c openssl#14 0x561392c78f6d in curl_multi_perform (/usr/bin/clickhouse+0x33855f6d) (BuildId: 22880fad595a96b17eb9add20e7a01f8ded54c49) openssl#15 0x561392bfd53e in curl_easy_perform (/usr/bin/clickhouse+0x337da53e) (BuildId: 22880fad595a96b17eb9add20e7a01f8ded54c49) openssl#16 0x561392a7c411 in Azure::Core::Http::CurlConnection::CurlConnection(Azure::Core::Http::Request&, Azure::Core::Http::CurlTransportOptions const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&) build_docker/./contrib/azure/sdk/core/azure-core/src/http/curl/curl.cpp:2441:24 openssl#17 0x561392a6382b in std::__1::__unique_if<Azure::Core::Http::CurlConnection>::__unique_single std::__1::make_unique[abi:v15000]<Azure::Core::Http::CurlConnection, Azure::Core::Http::Request&, Azure::Core::Http::CurlTransportOptions const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&>(Azure::Core::Http::Request&, Azure::Core::Http::CurlTransportOptions const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&) build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714:32 openssl#18 0x561392a6382b in Azure::Core::Http::_detail::CurlConnectionPool::ExtractOrCreateCurlConnection(Azure::Core::Http::Request&, Azure::Core::Http::CurlTransportOptions const&, bool) build_docker/./contrib/azure/sdk/core/azure-core/src/http/curl/curl.cpp:2126:10 openssl#19 0x561392a61951 in Azure::Core::Http::CurlTransport::Send(Azure::Core::Http::Request&, Azure::Core::Context const&) build_docker/./contrib/azure/sdk/core/azure-core/src/http/curl/curl.cpp:351:48 openssl#20 0x561392b0879b in Azure::Core::Http::Policies::_internal::TransportPolicy::Send(Azure::Core::Http::Request&, Azure::Core::Http::Policies::NextHttpPolicy, Azure::Core::Context const&) const build_docker/./contrib/azure/sdk/core/azure-core/src/http/transport_policy.cpp:121:40 openssl#21 0x561392aae81b in Azure::Core::Http::Policies::_internal::LogPolicy::Send(Azure::Core::Http::Request&, Azure::Core::Http::Policies::NextHttpPolicy, Azure::Core::Context const&) const build_docker/./contrib/azure/sdk/core/azure-core/src/http/log_policy.cpp:114:23 openssl#22 0x561392b022d8 in Azure::Core::Http::Policies::_internal::RequestActivityPolicy::Send(Azure::Core::Http::Request&, Azure::Core::Http::Policies::NextHttpPolicy, Azure::Core::Context const&) const build_docker/./contrib/azure/sdk/core/azure-core/src/http/request_activity_policy.cpp:110:23 openssl#23 0x561392b2e02a in Azure::Storage::_internal::SharedKeyPolicy::Send(Azure::Core::Http::Request&, Azure::Core::Http::Policies::NextHttpPolicy, Azure::Core::Context const&) const build_docker/./contrib/azure/sdk/storage/azure-storage-common/inc/azure/storage/common/internal/shared_key_policy.hpp:36:25 openssl#24 0x561392bf7fd8 in Azure::Storage::_internal::StoragePerRetryPolicy::Send(Azure::Core::Http::Request&, Azure::Core::Http::Policies::NextHttpPolicy, Azure::Core::Context const&) const build_docker/./contrib/azure/sdk/storage/azure-storage-common/src/storage_per_retry_policy.cpp:57:23 openssl#25 0x561392bf8dcb in Azure::Storage::_internal::StorageSwitchToSecondaryPolicy::Send(Azure::Core::Http::Request&, Azure::Core::Http::Policies::NextHttpPolicy, Azure::Core::Context const&) const build_docker/./contrib/azure/sdk/storage/azure-storage-common/src/storage_switch_to_secondary_policy.cpp:36:32 openssl#26 0x561392aff0d9 in Azure::Core::Http::Policies::_internal::RetryPolicy::Send(Azure::Core::Http::Request&, Azure::Core::Http::Policies::NextHttpPolicy, Azure::Core::Context const&) const build_docker/./contrib/azure/sdk/core/azure-core/src/http/retry_policy.cpp:146:34 openssl#27 0x561392b08ed7 in Azure::Core::Http::Policies::_internal::TelemetryPolicy::Send(Azure::Core::Http::Request&, Azure::Core::Http::Policies::NextHttpPolicy, Azure::Core::Context const&) const build_docker/./contrib/azure/sdk/core/azure-core/src/http/telemetry_policy.cpp:23:21 openssl#28 0x561392ae2fb5 in Azure::Core::Http::Policies::_internal::RequestIdPolicy::Send(Azure::Core::Http::Request&, Azure::Core::Http::Policies::NextHttpPolicy, Azure::Core::Context const&) const build_docker/./contrib/azure/sdk/core/azure-core/inc/azure/core/http/policies/policy.hpp:453:27 openssl#29 0x561392b2e5fe in Azure::Storage::_internal::StorageServiceVersionPolicy::Send(Azure::Core::Http::Request&, Azure::Core::Http::Policies::NextHttpPolicy, Azure::Core::Context const&) const build_docker/./contrib/azure/sdk/storage/azure-storage-common/inc/azure/storage/common/internal/storage_service_version_policy.hpp:34:25 openssl#30 0x561392b7e0b1 in Azure::Core::Http::_internal::HttpPipeline::Send(Azure::Core::Http::Request&, Azure::Core::Context const&) const build_docker/./contrib/azure/sdk/core/azure-core/inc/azure/core/internal/http/pipeline.hpp:230:29 openssl#31 0x561392b7e0b1 in Azure::Storage::Blobs::_detail::BlobContainerClient::Create(Azure::Core::Http::_internal::HttpPipeline&, Azure::Core::Url const&, Azure::Storage::Blobs::_detail::BlobContainerClient::CreateBlobContainerOptions const&, Azure::Core::Context const&) build_docker/./contrib/azure/sdk/storage/azure-storage-blobs/src/rest_client.cpp:1415:36 openssl#32 0x561392b3de98 in Azure::Storage::Blobs::BlobContainerClient::Create(Azure::Storage::Blobs::CreateBlobContainerOptions const&, Azure::Core::Context const&) const build_docker/./contrib/azure/sdk/storage/azure-storage-blobs/src/blob_container_client.cpp:258:12 ``` - at database startup, Azure is registered as an object storage (--> registerAzureObjectStorage) - this calls into Azure, then into curl, and then into OpenSSL - curl asks OpenSSL for a bunch of random numbers (--> function 'ossl_random' in curl's OpenSSL wrapper) - OpenSSL initializes the random number generator and stores it in some random number generator context object (--> *RAND_get0_private) - this object is registered via pthread_key_create and pthread_setspecific registered in TLS - if registerAzureObjectStorage was the only place which initializes the RNG, we could argue that the leaked memory does not matter anyways as it is released after shutdown - RAND_get0_public also registers a free handler (rand_delete_thread_state) that runs in TLS, so the memory is released also if registration is called from arbitrary other threads. In sum: this is a false positive. As a result, the fix is similar to earlier fix 5c4b034

----- E Exception: Sanitizer assert found for instance ================== E WARNING: ThreadSanitizer: data race (pid=1) E Write of size 8 at 0x7b2800025d30 by thread T2 (mutexes: write M0, write M1): E #0 free <null> (clickhouse+0x709a3e5) (BuildId: 706d92b17db171493f293d517643f726ee1b7b1e) E #1 CRYPTO_free build_docker/./contrib/openssl/crypto/mem.c:282:5 (clickhouse+0x2015f8ea) (BuildId: 706d92b17db171493f293d517643f726ee1b7b1e) E #2 EVP_PKEY_free build_docker/./contrib/openssl/crypto/evp/p_lib.c:1809:5 (clickhouse+0x2012a751) (BuildId: 706d92b17db171493f293d517643f726ee1b7b1e) E #3 Poco::Crypto::EVPPKey::~EVPPKey() build_docker/./base/poco/Crypto/src/EVPPKey.cpp:121:17 (clickhouse+0x1d00ffa9) (BuildId: 706d92b17db171493f293d517643f726ee1b7b1e) E #4 DB::CertificateReloader::Data::~Data() build_docker/./src/Server/CertificateReloader.h:71:12 (clickhouse+0x194fb42d) (BuildId: 706d92b17db171493f293d517643f726ee1b7b1e) E #5 std::__1::default_delete<DB::CertificateReloader::Data const>::operator()[abi:v15000](DB::CertificateReloader::Data const*) const build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48:5 (clickhouse+0x194fb42d) E #6 std::__1::__shared_ptr_pointer<DB::CertificateReloader::Data const*, std::__1::default_delete<DB::CertificateReloader::Data const>, std::__1::allocator<DB::CertificateReloader::Data const>>::__on_zero_shared() build_docker/./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:263:5 (clickhouse+0x194fb42d) E #7 std::__1::__shared_count::__release_shared[abi:v15000]() build_docker/./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:174:9 (clickhouse+0x194fade0) (BuildId: 706d92b17db171493f293d517643f726ee1b7b1e) E #8 std::__1::__shared_weak_count::__release_shared[abi:v15000]() build_docker/./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:215:27 (clickhouse+0x194fade0) E openssl#9 std::__1::shared_ptr<DB::CertificateReloader::Data const>::~shared_ptr[abi:v15000]() build_docker/./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:702:23 (clickhouse+0x194fade0) E openssl#10 std::__1::shared_ptr<DB::CertificateReloader::Data const>::operator=[abi:v15000](std::__1::shared_ptr<DB::CertificateReloader::Data const>&&) build_docker/./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723:9 (clickhouse+0x194fade0) E openssl#11 MultiVersion<DB::CertificateReloader::Data>::set(std::__1::unique_ptr<DB::CertificateReloader::Data const, std::__1::default_delete<DB::CertificateReloader::Data const>>&&) build_docker/./src/Common/MultiVersion.h:76:25 (clickhouse+0x194fade0) E openssl#12 DB::CertificateReloader::tryLoad(Poco::Util::AbstractConfiguration const&) build_docker/./src/Server/CertificateReloader.cpp:83:18 (clickhouse+0x194f94ca) (BuildId: 706d92b17db171493f293d517643f726ee1b7b1e) E openssl#13 DB::Server::main(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>> const&)::$_6::operator()(Poco::AutoPtr<Poco::Util::AbstractConfiguration>, bool) const build_docker/./programs/server/Server.cpp:1546:45 (clickhouse+0xf384df7) (BuildId: 706d92b17db171493f293d517643f726ee1b7b1e) E openssl#14 decltype(std::declval<DB::Server::main(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>> const&)::$_6&>()(std::declval<Poco::AutoPtr<Poco::Util::AbstractConfiguration>>(), std::declval<bool>())) std::__1::__invoke[abi:v15000]<DB::Server::main(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>> const&)::$_6&, Poco::AutoPtr<Poco::Util::AbstractConfiguration>, bool>(DB::Server::main(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>> const&)::$_6&, Poco::AutoPtr<Poco::Util::AbstractConfiguration>&&, bool&&) build_docker/./contrib/llvm-project/libcxx/include/__functional/invoke.h:394:23 (clickhouse+0xf3827a9) (BuildId: 706d92b17db171493f293d517643f726ee1b7b1e) E openssl#15 void std::__1::__invoke_void_return_wrapper<void, true>::__call<DB::Server::main(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>> const&)::$_6&, Poco::AutoPtr<Poco::Util::AbstractConfiguration>, bool>(DB::Server::main(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>> const&)::$_6&, Poco::AutoPtr<Poco::Util::AbstractConfiguration>&&, bool&&) build_docker/./contrib/llvm-project/libcxx/include/__functional/invoke.h:479:9 (clickhouse+0xf3827a9) E openssl#16 std::__1::__function::__default_alloc_func<DB::Server::main(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>> const&)::$_6, void (Poco::AutoPtr<Poco::Util::AbstractConfiguration>, bool)>::operator()[abi:v15000](Poco::AutoPtr<Poco::Util::AbstractConfiguration>&&, bool&&) build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:235:12 (clickhouse+0xf3827a9) E openssl#17 void std::__1::__function::__policy_invoker<void (Poco::AutoPtr<Poco::Util::AbstractConfiguration>, bool)>::__call_impl<std::__1::__function::__default_alloc_func<DB::Server::main(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>> const&)::$_6, void (Poco::AutoPtr<Poco::Util::AbstractConfiguration>, bool)>>(std::__1::__function::__policy_storage const*, Poco::AutoPtr<Poco::Util::AbstractConfiguration>&&, bool) build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:716:16 (clickhouse+0xf3827a9) E openssl#18 std::__1::__function::__policy_func<void (Poco::AutoPtr<Poco::Util::AbstractConfiguration>, bool)>::operator()[abi:v15000](Poco::AutoPtr<Poco::Util::AbstractConfiguration>&&, bool&&) const build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:848:16 (clickhouse+0x19fd2cbe) (BuildId: 706d92b17db171493f293d517643f726ee1b7b1e) E openssl#19 std::__1::function<void (Poco::AutoPtr<Poco::Util::AbstractConfiguration>, bool)>::operator()(Poco::AutoPtr<Poco::Util::AbstractConfiguration>, bool) const build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:1187:12 (clickhouse+0x19fd2cbe) E openssl#20 DB::ConfigReloader::reloadIfNewer(bool, bool, bool, bool) build_docker/./src/Common/Config/ConfigReloader.cpp:150:13 (clickhouse+0x19fd2cbe) E openssl#21 DB::ConfigReloader::reload() build_docker/./src/Common/Config/ConfigReloader.h:51:21 (clickhouse+0xf38767c) (BuildId: 706d92b17db171493f293d517643f726ee1b7b1e) E openssl#22 DB::Server::main(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>> const&)::$_13::operator()() const build_docker/./programs/server/Server.cpp:1731:31 (clickhouse+0xf38767c) E openssl#23 decltype(std::declval<DB::Server::main(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>> const&)::$_13&>()()) std::__1::__invoke[abi:v15000]<DB::Server::main(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>> const&)::$_13&>(DB::Server::main(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>> const&)::$_13&) build_docker/./contrib/llvm-project/libcxx/include/__functional/invoke.h:394:23 (clickhouse+0xf38767c) E openssl#24 void std::__1::__invoke_void_return_wrapper<void, true>::__call<DB::Server::main(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>> const&)::$_13&>(DB::Server::main(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>> const&)::$_13&) build_docker/./contrib/llvm-project/libcxx/include/__functional/invoke.h:479:9 (clickhouse+0xf38767c) E openssl#25 std::__1::__function::__default_alloc_func<DB::Server::main(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>> const&)::$_13, void ()>::operator()[abi:v15000]() build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:235:12 (clickhouse+0xf38767c) E openssl#26 void std::__1::__function::__policy_invoker<void ()>::__call_impl<std::__1::__function::__default_alloc_func<DB::Server::main(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>> const&)::$_13, void ()>>(std::__1::__function::__policy_storage const*) build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:716:16 (clickhouse+0xf38767c) E openssl#27 std::__1::__function::__policy_func<void ()>::operator()[abi:v15000]() const build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:848:16 (clickhouse+0x16907aa0) (BuildId: 706d92b17db171493f293d517643f726ee1b7b1e) E openssl#28 std::__1::function<void ()>::operator()() const build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:1187:12 (clickhouse+0x16907aa0) E openssl#29 DB::Context::reloadConfig() const build_docker/./src/Interpreters/Context.cpp:4357:5 (clickhouse+0x16907aa0) E openssl#30 DB::InterpreterSystemQuery::execute() build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:577:29 (clickhouse+0x17e78c19) (BuildId: 706d92b17db171493f293d517643f726ee1b7b1e) E openssl#31 DB::executeQueryImpl(char const*, char const*, std::__1::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) build_docker/./src/Interpreters/executeQuery.cpp:1195:40 (clickhouse+0x17e3e462) (BuildId: 706d92b17db171493f293d517643f726ee1b7b1e) E openssl#32 DB::executeQuery(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) build_docker/./src/Interpreters/executeQuery.cpp:1374:26 (clickhouse+0x17e39837) (BuildId: 706d92b17db171493f293d517643f726ee1b7b1e) E openssl#33 DB::TCPHandler::runImpl() build_docker/./src/Server/TCPHandler.cpp:518:54 (clickhouse+0x195cc651) (BuildId: 706d92b17db171493f293d517643f726ee1b7b1e) E openssl#34 DB::TCPHandler::run() build_docker/./src/Server/TCPHandler.cpp:2329:9 (clickhouse+0x195e8707) (BuildId: 706d92b17db171493f293d517643f726ee1b7b1e) E openssl#35 Poco::Net::TCPServerConnection::start() build_docker/./base/poco/Net/src/TCPServerConnection.cpp:43:3 (clickhouse+0x1d00d942) (BuildId: 706d92b17db171493f293d517643f726ee1b7b1e) E openssl#36 Poco::Net::TCPServerDispatcher::run() build_docker/./base/poco/Net/src/TCPServerDispatcher.cpp:115:20 (clickhouse+0x1d00e1b1) (BuildId: 706d92b17db171493f293d517643f726ee1b7b1e) E openssl#37 Poco::PooledThread::run() build_docker/./base/poco/Foundation/src/ThreadPool.cpp:188:14 (clickhouse+0x1d20f2e6) (BuildId: 706d92b17db171493f293d517643f726ee1b7b1e) E openssl#38 Poco::(anonymous namespace)::RunnableHolder::run() build_docker/./base/poco/Foundation/src/Thread.cpp:45:11 (clickhouse+0x1d20d5af) (BuildId: 706d92b17db171493f293d517643f726ee1b7b1e) E openssl#39 Poco::ThreadImpl::runnableEntry(void*) build_docker/./base/poco/Foundation/src/Thread_POSIX.cpp:335:27 (clickhouse+0x1d20ba69) (BuildId: 706d92b17db171493f293d517643f726ee1b7b1e) E E Previous atomic write of size 4 at 0x7b2800025d30 by thread T3 (mutexes: write M2): E #0 CRYPTO_DOWN_REF build_docker/./contrib/openssl/include/internal/refcount.h:51:12 (clickhouse+0x2012a6e6) (BuildId: 706d92b17db171493f293d517643f726ee1b7b1e) E #1 EVP_PKEY_free build_docker/./contrib/openssl/crypto/evp/p_lib.c:1795:5 (clickhouse+0x2012a6e6) E #2 ssl_cert_clear_certs build_docker/./contrib/openssl/ssl/ssl_cert.c:246:9 (clickhouse+0x1ffafd37) (BuildId: 706d92b17db171493f293d517643f726ee1b7b1e) E #3 ssl_cert_free build_docker/./contrib/openssl/ssl/ssl_cert.c:277:5 (clickhouse+0x1ffafd37) E #4 ossl_ssl_connection_free build_docker/./contrib/openssl/ssl/ssl_lib.c:1458:5 (clickhouse+0x1ffba6af) (BuildId: 706d92b17db171493f293d517643f726ee1b7b1e) E #5 SSL_free build_docker/./contrib/openssl/ssl/ssl_lib.c:1417:9 (clickhouse+0x1ffb920e) (BuildId: 706d92b17db171493f293d517643f726ee1b7b1e) E #6 Poco::Net::SecureSocketImpl::reset() build_docker/./base/poco/NetSSL_OpenSSL/src/SecureSocketImpl.cpp:583:3 (clickhouse+0x1cfaac60) (BuildId: 706d92b17db171493f293d517643f726ee1b7b1e) E #7 Poco::Net::SecureSocketImpl::~SecureSocketImpl() build_docker/./base/poco/NetSSL_OpenSSL/src/SecureSocketImpl.cpp:80:3 (clickhouse+0x1cfaac60) E #8 Poco::Net::SecureStreamSocketImpl::~SecureStreamSocketImpl() build_docker/./base/poco/NetSSL_OpenSSL/src/SecureStreamSocketImpl.cpp:52:1 (clickhouse+0x1cfb15dd) (BuildId: 706d92b17db171493f293d517643f726ee1b7b1e) E openssl#9 Poco::Net::SecureStreamSocketImpl::~SecureStreamSocketImpl() build_docker/./base/poco/NetSSL_OpenSSL/src/SecureStreamSocketImpl.cpp:43:1 (clickhouse+0x1cfb15dd) E openssl#10 Poco::RefCountedObject::release() const build_docker/./base/poco/Foundation/include/Poco/RefCountedObject.h:86:13 (clickhouse+0x1cffc81e) (BuildId: 706d92b17db171493f293d517643f726ee1b7b1e) E openssl#11 Poco::Net::Socket::~Socket() build_docker/./base/poco/Net/src/Socket.cpp:68:10 (clickhouse+0x1cffc81e) E openssl#12 Poco::Net::StreamSocket::~StreamSocket() build_docker/./base/poco/Net/src/StreamSocket.cpp:63:1 (clickhouse+0x1d009c39) (BuildId: 706d92b17db171493f293d517643f726ee1b7b1e) E openssl#13 Poco::Net::TCPConnectionNotification::~TCPConnectionNotification() build_docker/./base/poco/Net/src/TCPServerDispatcher.cpp:43:2 (clickhouse+0x1d00ef50) (BuildId: 706d92b17db171493f293d517643f726ee1b7b1e) E openssl#14 Poco::Net::TCPConnectionNotification::~TCPConnectionNotification() build_docker/./base/poco/Net/src/TCPServerDispatcher.cpp:42:2 (clickhouse+0x1d00ef50) E openssl#15 Poco::RefCountedObject::release() const build_docker/./base/poco/Foundation/include/Poco/RefCountedObject.h:86:13 (clickhouse+0x1d00e203) (BuildId: 706d92b17db171493f293d517643f726ee1b7b1e) E openssl#16 Poco::AutoPtr<Poco::Notification>::~AutoPtr() build_docker/./base/poco/Foundation/include/Poco/AutoPtr.h:91:19 (clickhouse+0x1d00e203) E openssl#17 Poco::Net::TCPServerDispatcher::run() build_docker/./base/poco/Net/src/TCPServerDispatcher.cpp:122:3 (clickhouse+0x1d00e203) E openssl#18 Poco::PooledThread::run() build_docker/./base/poco/Foundation/src/ThreadPool.cpp:188:14 (clickhouse+0x1d20f2e6) (BuildId: 706d92b17db171493f293d517643f726ee1b7b1e) E openssl#19 Poco::(anonymous namespace)::RunnableHolder::run() build_docker/./base/poco/Foundation/src/Thread.cpp:45:11 (clickhouse+0x1d20d5af) (BuildId: 706d92b17db171493f293d517643f726ee1b7b1e) E openssl#20 Poco::ThreadImpl::runnableEntry(void*) build_docker/./base/poco/Foundation/src/Thread_POSIX.cpp:335:27 (clickhouse+0x1d20ba69) (BuildId: 706d92b17db171493f293d517643f726ee1b7b1e)

During my testing (*), I got below stack about leaked memory. - at database startup, Azure is registered as an object storage (--> registerAzureObjectStorage) - this calls into Azure, then into curl, and then into OpenSSL - curl asks OpenSSL for a bunch of random numbers (--> function 'ossl_random' in curl's OpenSSL wrapper) - OpenSSL initializes the random number generator and stores it in some random number generator context object (--> *RAND_get0_public) - this object is registered via pthread_key_create and pthread_setspecific registered in TLS - if registerAzureObjectStorage was the only place which initializes the RNG, we could argue that the leaked memory does not matter anyways as it is released after shutdown - RAND_get0_public also registers a free handler (rand_delete_thread_state) that runs in TLS, so the memory is released also if registration is called from arbitrary other threads. In sum: this is a false positive. (*) ClickHouse/clickhouse-private#10107 (comment) ``` ================================================================= ==2757181==ERROR: LeakSanitizer: detected memory leaks Direct leak of 48 byte(s) in 1 object(s) allocated from: #0 0x5672f4a48083 in malloc (/data/ch4/build_asan/programs/clickhouse+0xa2b7083) (BuildId: 4d868624feaa8b5598d9d0b327749c877bfb6cfd) #1 0x567321d05c7e in CRYPTO_malloc build_asan/./contrib/openssl/crypto/mem.c:202:11 #2 0x567321d05c7e in CRYPTO_zalloc build_asan/./contrib/openssl/crypto/mem.c:222:11 #3 0x567321d5f9cf in EVP_RAND_CTX_new build_asan/./contrib/openssl/crypto/evp/evp_rand.c:353:11 #4 0x567321d627db in rand_new_drbg build_asan/./contrib/openssl/crypto/rand/rand_lib.c:658:11 #5 0x567321d61e7b in RAND_get0_public build_asan/./contrib/openssl/crypto/rand/rand_lib.c:777:16 #6 0x567321d61d4f in RAND_bytes_ex build_asan/./contrib/openssl/crypto/rand/rand_lib.c:378:12 #7 0x56731cfe9267 in ossl_random openssl.c #8 0x56731cf534ae in Curl_rand (/data/ch4/build_asan/programs/clickhouse+0x327c24ae) (BuildId: 4d868624feaa8b5598d9d0b327749c877bfb6cfd) openssl#9 0x56731cf757dc in Curl_socketpair (/data/ch4/build_asan/programs/clickhouse+0x327e47dc) (BuildId: 4d868624feaa8b5598d9d0b327749c877bfb6cfd) openssl#10 0x56731cf40bfe in Curl_multi_handle (/data/ch4/build_asan/programs/clickhouse+0x327afbfe) (BuildId: 4d868624feaa8b5598d9d0b327749c877bfb6cfd) openssl#11 0x56731cecb260 in curl_easy_perform (/data/ch4/build_asan/programs/clickhouse+0x3273a260) (BuildId: 4d868624feaa8b5598d9d0b327749c877bfb6cfd) openssl#12 0x56731cd4a351 in Azure::Core::Http::CurlConnection::CurlConnection(Azure::Core::Http::Request&, Azure::Core::Http::CurlTransportOptions const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&) build_asan/./contrib/azure/sdk/core/azure-core/src/http/curl/curl.cpp:2441:24 openssl#13 0x56731cd3176b in std::__1::__unique_if<Azure::Core::Http::CurlConnection>::__unique_single std::__1::make_unique[abi:v15000]<Azure::Core::Http::CurlConnection, Azure::Core::Http::Request&, Azure::Core::Http::CurlTransportOptions const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&>(Azure::Core::Http::Request&, Azure::Core::H ttp::CurlTransportOptions const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&) build_asan/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714:32 openssl#14 0x56731cd3176b in Azure::Core::Http::_detail::CurlConnectionPool::ExtractOrCreateCurlConnection(Azure::Core::Http::Request&, Azure::Core::Http::CurlTransportOptions const&, bool) build_asan/./contrib/azure/sdk/core/azure-core/src/http/curl/curl.cpp:2126:10 openssl#15 0x56731cd2f891 in Azure::Core::Http::CurlTransport::Send(Azure::Core::Http::Request&, Azure::Core::Context const&) build_asan/./contrib/azure/sdk/core/azure-core/src/http/curl/curl.cpp:351:48 openssl#16 0x56731cdd66db in Azure::Core::Http::Policies::_internal::TransportPolicy::Send(Azure::Core::Http::Request&, Azure::Core::Http::Policies::NextHttpPolicy, Azure::Core::Context const&) const build_asan/./contrib/azure/sdk/core/azure-core/src/http/transport_policy.cpp:121:40 openssl#17 0x56731cd7c75b in Azure::Core::Http::Policies::_internal::LogPolicy::Send(Azure::Core::Http::Request&, Azure::Core::Http::Policies::NextHttpPolicy, Azure::Core::Context const&) const build_asan/./contrib/azure/sdk/core/azure-core/src/http/log_policy.cpp:114:23 openssl#18 0x56731cdd0218 in Azure::Core::Http::Policies::_internal::RequestActivityPolicy::Send(Azure::Core::Http::Request&, Azure::Core::Http::Policies::NextHttpPolicy, Azure::Core::Context const&) const build_asan/./contrib/azure/sdk/core/azure-core/src/http/request_activity_policy.cpp:110:23 openssl#19 0x56731cec5f18 in Azure::Storage::_internal::StoragePerRetryPolicy::Send(Azure::Core::Http::Request&, Azure::Core::Http::Policies::NextHttpPolicy, Azure::Core::Context const&) const build_asan/./contrib/azure/sdk/storage/azure-storage-common/src/storage_per_retry_policy.cpp:57:23 openssl#20 0x56731cec6d0b in Azure::Storage::_internal::StorageSwitchToSecondaryPolicy::Send(Azure::Core::Http::Request&, Azure::Core::Http::Policies::NextHttpPolicy, Azure::Core::Context const&) const build_asan/./contrib/azure/sdk/storage/azure-storage-common/src/storage_switch_to_secondary_policy.cpp:36:32 openssl#21 0x56731cdcd019 in Azure::Core::Http::Policies::_internal::RetryPolicy::Send(Azure::Core::Http::Request&, Azure::Core::Http::Policies::NextHttpPolicy, Azure::Core::Context const&) const build_asan/./contrib/azure/sdk/core/azure-core/src/http/retry_policy.cpp:146:34 openssl#22 0x56731cdd6e17 in Azure::Core::Http::Policies::_internal::TelemetryPolicy::Send(Azure::Core::Http::Request&, Azure::Core::Http::Policies::NextHttpPolicy, Azure::Core::Context const&) const build_asan/./contrib/azure/sdk/core/azure-core/src/http/telemetry_policy.cpp:23:21 openssl#23 0x56731cdb0ef5 in Azure::Core::Http::Policies::_internal::RequestIdPolicy::Send(Azure::Core::Http::Request&, Azure::Core::Http::Policies::NextHttpPolicy, Azure::Core::Context const&) const build_asan/./contrib/azure/sdk/core/azure-core/inc/azure/core/http/policies/policy.hpp:453:27 openssl#24 0x56731cdfc53e in Azure::Storage::_internal::StorageServiceVersionPolicy::Send(Azure::Core::Http::Request&, Azure::Core::Http::Policies::NextHttpPolicy, Azure::Core::Context const&) const build_asan/./contrib/azure/sdk/storage/azure-storage-common/inc/azure/storage/common/internal/storage_service_version_policy.hpp:34:25 openssl#25 0x56731ce4bff1 in Azure::Core::Http::_internal::HttpPipeline::Send(Azure::Core::Http::Request&, Azure::Core::Context const&) const build_asan/./contrib/azure/sdk/core/azure-core/inc/azure/core/internal/http/pipeline.hpp:230:29 openssl#26 0x56731ce4bff1 in Azure::Storage::Blobs::_detail::BlobContainerClient::Create(Azure::Core::Http::_internal::HttpPipeline&, Azure::Core::Url const&, Azure::Storage::Blobs::_detail::BlobContainerClient::CreateBlobContainerOptions const&, Azure::Core::Context const&) build_asan/./contrib/azure/sdk/storage/azure-storage-blobs/src/rest_client.cpp:1415:36 openssl#27 0x56731ce0bdd8 in Azure::Storage::Blobs::BlobContainerClient::Create(Azure::Storage::Blobs::CreateBlobContainerOptions const&, Azure::Core::Context const&) const build_asan/./contrib/azure/sdk/storage/azure-storage-blobs/src/blob_container_client.cpp:258:12 openssl#28 0x56731ce396dc in Azure::Storage::Blobs::BlobServiceClient::CreateBlobContainer(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, Azure::Storage::Blobs::CreateBlobContainerOptions const&, Azure::Core::Context const&) const build_asan/./contrib/azure/sdk/storage/azure-storage-blobs/src/blob_service_client.cpp:264:41 openssl#29 0x56731018fcff in DB::getAzureBlobContainerClient(Poco::Util::AbstractConfiguration const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&) build_asan/./src/Disks/ObjectStorages/AzureBlobStorage/AzureBlobStorageAuth.cpp:236:75 openssl#30 0x5673115a7352 in DB::registerAzureObjectStorage(DB::ObjectStorageFactory&)::$_0::operator()(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, Poco::Util::AbstractConfiguration const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::shared_ptr<DB::Context const> const&, bool) const build_asan/./src/Disks/ObjectStorages/ObjectStorageFactory.cpp:315:13 openssl#31 0x5673115a7352 in decltype(std::declval<DB::registerAzureObjectStorage(DB::ObjectStorageFactory&)::$_0&>()(std::declval<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&>(), std::declval<Poco::Util::AbstractConfiguration const&>(), std::declval<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&>(), std::declval<std::__1::shared_ptr<DB::Context const> const&>(), std::declval<bool>() )) std::__1::__invoke[abi:v15000]<DB::registerAzureObjectStorage(DB::ObjectStorageFactory&)::$_0&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, Poco::Util::AbstractConfiguration const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::shared_ptr<DB::Context const> const&, bool>(DB::registerAzureObjectStorage(DB::ObjectStorageFactory&)::$_0&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, Poco::Util::AbstractConfiguration const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::shared_ptr<DB::Context const> const&, bool&&) build_asan/./contrib/llvm-project/libcxx/include/__functional/invoke.h:394:23 openssl#32 0x5673115a7352 in std::__1::shared_ptr<DB::IObjectStorage> std::__1::__invoke_void_return_wrapper<std::__1::shared_ptr<DB::IObjectStorage>, false>::__call<DB::registerAzureObjectStorage(DB::ObjectStorageFactory&)::$_0&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, Poco::Util::AbstractConfiguration const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::shared_pt r<DB::Context const> const&, bool>(DB::registerAzureObjectStorage(DB::ObjectStorageFactory&)::$_0&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, Poco::Util::AbstractConfiguration const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::shared_ptr<DB::Context const> const&, bool&&) build_asan/./contrib/llvm-project/libcxx/include/__functional/invoke.h:470:16 openssl#33 0x5673115a7352 in std::__1::__function::__default_alloc_func<DB::registerAzureObjectStorage(DB::ObjectStorageFactory&)::$_0, std::__1::shared_ptr<DB::IObjectStorage> (std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, Poco::Util::AbstractConfiguration const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::shared_ptr<DB::Context const> const&, bool)>::operator()[abi:v 15000](std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, Poco::Util::AbstractConfiguration const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::shared_ptr<DB::Context const> const&, bool&&) build_asan/./contrib/llvm-project/libcxx/include/__functional/function.h:235:12 openssl#34 0x5673115a7352 in std::__1::shared_ptr<DB::IObjectStorage> std::__1::__function::__policy_invoker<std::__1::shared_ptr<DB::IObjectStorage> (std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, Poco::Util::AbstractConfiguration const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::shared_ptr<DB::Context const> const&, bool)>::__call_impl<std::__1::__function::__default _alloc_func<DB::registerAzureObjectStorage(DB::ObjectStorageFactory&)::$_0, std::__1::shared_ptr<DB::IObjectStorage> (std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, Poco::Util::AbstractConfiguration const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::shared_ptr<DB::Context const> const&, bool)>>(std::__1::__function::__policy_storage const*, std::__1::basic_string<cha r, std::__1::char_traits<char>, std::__1::allocator<char>> const&, Poco::Util::AbstractConfiguration const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::shared_ptr<DB::Context const> const&, bool) build_asan/./contrib/llvm-project/libcxx/include/__functional/function.h:716:16 openssl#35 0x5673115a0780 in std::__1::__function::__policy_func<std::__1::shared_ptr<DB::IObjectStorage> (std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, Poco::Util::AbstractConfiguration const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::shared_ptr<DB::Context const> const&, bool)>::operator()[abi:v15000](std::__1::basic_string<char, std::__1::char_traits<char>, std::_ _1::allocator<char>> const&, Poco::Util::AbstractConfiguration const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::shared_ptr<DB::Context const> const&, bool&&) const build_asan/./contrib/llvm-project/libcxx/include/__functional/function.h:848:16 openssl#36 0x5673115a0780 in std::__1::function<std::__1::shared_ptr<DB::IObjectStorage> (std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, Poco::Util::AbstractConfiguration const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::shared_ptr<DB::Context const> const&, bool)>::operator()(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, Poco::Util::AbstractConfiguration const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::shared_ptr<DB::Context const> const&, bool) const build_asan/./contrib/llvm-project/libcxx/include/__functional/function.h:1187:12 openssl#37 0x5673115a0780 in DB::ObjectStorageFactory::create(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, Poco::Util::AbstractConfiguration const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::shared_ptr<DB::Context const> const&, bool) const build_asan/./src/Disks/ObjectStorages/ObjectStorageFactory.cpp:135:12 openssl#38 0x56731159ef45 in DB::registerDiskObjectStorage(DB::DiskFactory&, bool)::$_0::operator()(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, Poco::Util::AbstractConfiguration const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::shared_ptr<DB::Context const>, std::__1::map<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, std::__1: :shared_ptr<DB::IDisk>, std::__1::less<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>, std::__1::allocator<std::__1::pair<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const, std::__1::shared_ptr<DB::IDisk>>>> const&, bool, bool) const build_asan/./src/Disks/ObjectStorages/RegisterDiskObjectStorage.cpp:27:64 openssl#39 0x56731159ef45 in decltype(std::declval<DB::registerDiskObjectStorage(DB::DiskFactory&, bool)::$_0&>()(std::declval<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&>(), std::declval<Poco::Util::AbstractConfiguration const&>(), std::declval<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&>(), std::declval<std::__1::shared_ptr<DB::Context const>>(), std::declval<std::__1::map<std: :__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, std::__1::shared_ptr<DB::IDisk>, std::__1::less<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>, std::__1::allocator<std::__1::pair<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const, std::__1::shared_ptr<DB::IDisk>>>> const&>(), std::declval<bool>(), std::declval<bool>())) std::__1::__invoke[abi:v15000]<DB::regis terDiskObjectStorage(DB::DiskFactory&, bool)::$_0&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, Poco::Util::AbstractConfiguration const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::shared_ptr<DB::Context const>, std::__1::map<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, std::__1::shared_ptr<DB::IDisk>, std::__1::less<std::_ _1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>, std::__1::allocator<std::__1::pair<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const, std::__1::shared_ptr<DB::IDisk>>>> const&, bool, bool>(DB::registerDiskObjectStorage(DB::DiskFactory&, bool)::$_0&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, Poco::Util::AbstractConfiguration const&, std::__1::basic_ string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::shared_ptr<DB::Context const>&&, std::__1::map<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, std::__1::shared_ptr<DB::IDisk>, std::__1::less<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>, std::__1::allocator<std::__1::pair<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const, std::__1::shared_ptr<DB::IDisk>>>> const&, bool&&, bool&&) build_asan/./contrib/llvm-project/libcxx/include/__functional/invoke.h:394:23 openssl#40 0x56731159ef45 in std::__1::shared_ptr<DB::IDisk> std::__1::__invoke_void_return_wrapper<std::__1::shared_ptr<DB::IDisk>, false>::__call<DB::registerDiskObjectStorage(DB::DiskFactory&, bool)::$_0&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, Poco::Util::AbstractConfiguration const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::shared_ptr<DB::Context const>, std::__1::map<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, std::__1::shared_ptr<DB::IDisk>, std::__1::less<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>, std::__1::allocator<std::__1::pair<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const, std::__1::shared_ptr<DB::IDisk>>>> const&, bool, bool>(DB::registerDiskObjectStorage(DB::DiskFactory&, bool)::$_0 &, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, Poco::Util::AbstractConfiguration const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::shared_ptr<DB::Context const>&&, std::__1::map<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, std::__1::shared_ptr<DB::IDisk>, std::__1::less<std::__1::basic_string<char, std::__1::char_traits<ch ar>, std::__1::allocator<char>>>, std::__1::allocator<std::__1::pair<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const, std::__1::shared_ptr<DB::IDisk>>>> const&, bool&&, bool&&) build_asan/./contrib/llvm-project/libcxx/include/__functional/invoke.h:470:16 openssl#41 0x56731159ef45 in std::__1::__function::__default_alloc_func<DB::registerDiskObjectStorage(DB::DiskFactory&, bool)::$_0, std::__1::shared_ptr<DB::IDisk> (std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, Poco::Util::AbstractConfiguration const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::shared_ptr<DB::Context const>, std::__1::map<std::__1::basic_string<char, s td::__1::char_traits<char>, std::__1::allocator<char>>, std::__1::shared_ptr<DB::IDisk>, std::__1::less<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>, std::__1::allocator<std::__1::pair<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const, std::__1::shared_ptr<DB::IDisk>>>> const&, bool, bool)>::operator()[abi:v15000](std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator <char>> const&, Poco::Util::AbstractConfiguration const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::shared_ptr<DB::Context const>&&, std::__1::map<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, std::__1::shared_ptr<DB::IDisk>, std::__1::less<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>, std::__1::allocator<std::__1::pair<std::__1::b asic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const, std::__1::shared_ptr<DB::IDisk>>>> const&, bool&&, bool&&) build_asan/./contrib/llvm-project/libcxx/include/__functional/function.h:235:12 ```

ClickHouse issue ClickHouse/clickhouse-private#10107 (comment) reports this memory leak: (note how it is very similar to ClickHouse/clickhouse-private#10107 (comment)) ``` Direct leak of 528 byte(s) in 11 object(s) allocated from: #0 0x561369af24cf in malloc (/usr/bin/clickhouse+0xa6cf4cf) (BuildId: 22880fad595a96b17eb9add20e7a01f8ded54c49) #1 0x561397b86a7e in CRYPTO_malloc build_docker/./contrib/openssl/crypto/mem.c:202:11 #2 0x561397b86a7e in CRYPTO_zalloc build_docker/./contrib/openssl/crypto/mem.c:222:11 #3 0x561397be07cf in EVP_RAND_CTX_new build_docker/./contrib/openssl/crypto/evp/evp_rand.c:353:11 #4 0x561397be35fb in rand_new_drbg build_docker/./contrib/openssl/crypto/rand/rand_lib.c:665:11 #5 0x561397be2a9b in RAND_get0_private build_docker/./contrib/openssl/crypto/rand/rand_lib.c:827:16 #6 0x561397be296f in RAND_priv_bytes_ex build_docker/./contrib/openssl/crypto/rand/rand_lib.c:356:12 #7 0x5613978d7cd8 in SSL_CTX_new_ex build_docker/./contrib/openssl/ssl/ssl_lib.c:4016:13 #8 0x561392d1eabd in ossl_connect_common openssl.c openssl#9 0x561392d11e23 in ssl_cf_connect vtls.c openssl#10 0x561392c13a1f in cf_setup_connect connect.c openssl#11 0x561392c1b0e5 in cf_hc_connect cf-https-connect.c openssl#12 0x561392c0a332 in Curl_conn_connect (/usr/bin/clickhouse+0x337e7332) (BuildId: 22880fad595a96b17eb9add20e7a01f8ded54c49) openssl#13 0x561392c7a28a in multi_runsingle multi.c openssl#14 0x561392c78f6d in curl_multi_perform (/usr/bin/clickhouse+0x33855f6d) (BuildId: 22880fad595a96b17eb9add20e7a01f8ded54c49) openssl#15 0x561392bfd53e in curl_easy_perform (/usr/bin/clickhouse+0x337da53e) (BuildId: 22880fad595a96b17eb9add20e7a01f8ded54c49) openssl#16 0x561392a7c411 in Azure::Core::Http::CurlConnection::CurlConnection(Azure::Core::Http::Request&, Azure::Core::Http::CurlTransportOptions const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&) build_docker/./contrib/azure/sdk/core/azure-core/src/http/curl/curl.cpp:2441:24 openssl#17 0x561392a6382b in std::__1::__unique_if<Azure::Core::Http::CurlConnection>::__unique_single std::__1::make_unique[abi:v15000]<Azure::Core::Http::CurlConnection, Azure::Core::Http::Request&, Azure::Core::Http::CurlTransportOptions const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&>(Azure::Core::Http::Request&, Azure::Core::Http::CurlTransportOptions const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&) build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714:32 openssl#18 0x561392a6382b in Azure::Core::Http::_detail::CurlConnectionPool::ExtractOrCreateCurlConnection(Azure::Core::Http::Request&, Azure::Core::Http::CurlTransportOptions const&, bool) build_docker/./contrib/azure/sdk/core/azure-core/src/http/curl/curl.cpp:2126:10 openssl#19 0x561392a61951 in Azure::Core::Http::CurlTransport::Send(Azure::Core::Http::Request&, Azure::Core::Context const&) build_docker/./contrib/azure/sdk/core/azure-core/src/http/curl/curl.cpp:351:48 openssl#20 0x561392b0879b in Azure::Core::Http::Policies::_internal::TransportPolicy::Send(Azure::Core::Http::Request&, Azure::Core::Http::Policies::NextHttpPolicy, Azure::Core::Context const&) const build_docker/./contrib/azure/sdk/core/azure-core/src/http/transport_policy.cpp:121:40 openssl#21 0x561392aae81b in Azure::Core::Http::Policies::_internal::LogPolicy::Send(Azure::Core::Http::Request&, Azure::Core::Http::Policies::NextHttpPolicy, Azure::Core::Context const&) const build_docker/./contrib/azure/sdk/core/azure-core/src/http/log_policy.cpp:114:23 openssl#22 0x561392b022d8 in Azure::Core::Http::Policies::_internal::RequestActivityPolicy::Send(Azure::Core::Http::Request&, Azure::Core::Http::Policies::NextHttpPolicy, Azure::Core::Context const&) const build_docker/./contrib/azure/sdk/core/azure-core/src/http/request_activity_policy.cpp:110:23 openssl#23 0x561392b2e02a in Azure::Storage::_internal::SharedKeyPolicy::Send(Azure::Core::Http::Request&, Azure::Core::Http::Policies::NextHttpPolicy, Azure::Core::Context const&) const build_docker/./contrib/azure/sdk/storage/azure-storage-common/inc/azure/storage/common/internal/shared_key_policy.hpp:36:25 openssl#24 0x561392bf7fd8 in Azure::Storage::_internal::StoragePerRetryPolicy::Send(Azure::Core::Http::Request&, Azure::Core::Http::Policies::NextHttpPolicy, Azure::Core::Context const&) const build_docker/./contrib/azure/sdk/storage/azure-storage-common/src/storage_per_retry_policy.cpp:57:23 openssl#25 0x561392bf8dcb in Azure::Storage::_internal::StorageSwitchToSecondaryPolicy::Send(Azure::Core::Http::Request&, Azure::Core::Http::Policies::NextHttpPolicy, Azure::Core::Context const&) const build_docker/./contrib/azure/sdk/storage/azure-storage-common/src/storage_switch_to_secondary_policy.cpp:36:32 openssl#26 0x561392aff0d9 in Azure::Core::Http::Policies::_internal::RetryPolicy::Send(Azure::Core::Http::Request&, Azure::Core::Http::Policies::NextHttpPolicy, Azure::Core::Context const&) const build_docker/./contrib/azure/sdk/core/azure-core/src/http/retry_policy.cpp:146:34 openssl#27 0x561392b08ed7 in Azure::Core::Http::Policies::_internal::TelemetryPolicy::Send(Azure::Core::Http::Request&, Azure::Core::Http::Policies::NextHttpPolicy, Azure::Core::Context const&) const build_docker/./contrib/azure/sdk/core/azure-core/src/http/telemetry_policy.cpp:23:21 openssl#28 0x561392ae2fb5 in Azure::Core::Http::Policies::_internal::RequestIdPolicy::Send(Azure::Core::Http::Request&, Azure::Core::Http::Policies::NextHttpPolicy, Azure::Core::Context const&) const build_docker/./contrib/azure/sdk/core/azure-core/inc/azure/core/http/policies/policy.hpp:453:27 openssl#29 0x561392b2e5fe in Azure::Storage::_internal::StorageServiceVersionPolicy::Send(Azure::Core::Http::Request&, Azure::Core::Http::Policies::NextHttpPolicy, Azure::Core::Context const&) const build_docker/./contrib/azure/sdk/storage/azure-storage-common/inc/azure/storage/common/internal/storage_service_version_policy.hpp:34:25 openssl#30 0x561392b7e0b1 in Azure::Core::Http::_internal::HttpPipeline::Send(Azure::Core::Http::Request&, Azure::Core::Context const&) const build_docker/./contrib/azure/sdk/core/azure-core/inc/azure/core/internal/http/pipeline.hpp:230:29 openssl#31 0x561392b7e0b1 in Azure::Storage::Blobs::_detail::BlobContainerClient::Create(Azure::Core::Http::_internal::HttpPipeline&, Azure::Core::Url const&, Azure::Storage::Blobs::_detail::BlobContainerClient::CreateBlobContainerOptions const&, Azure::Core::Context const&) build_docker/./contrib/azure/sdk/storage/azure-storage-blobs/src/rest_client.cpp:1415:36 openssl#32 0x561392b3de98 in Azure::Storage::Blobs::BlobContainerClient::Create(Azure::Storage::Blobs::CreateBlobContainerOptions const&, Azure::Core::Context const&) const build_docker/./contrib/azure/sdk/storage/azure-storage-blobs/src/blob_container_client.cpp:258:12 ``` - at database startup, Azure is registered as an object storage (--> registerAzureObjectStorage) - this calls into Azure, then into curl, and then into OpenSSL - curl asks OpenSSL for a bunch of random numbers (--> function 'ossl_random' in curl's OpenSSL wrapper) - OpenSSL initializes the random number generator and stores it in some random number generator context object (--> *RAND_get0_private) - this object is registered via pthread_key_create and pthread_setspecific registered in TLS - if registerAzureObjectStorage was the only place which initializes the RNG, we could argue that the leaked memory does not matter anyways as it is released after shutdown - RAND_get0_public also registers a free handler (rand_delete_thread_state) that runs in TLS, so the memory is released also if registration is called from arbitrary other threads. In sum: this is a false positive. As a result, the fix is similar to earlier fix 5c4b034

----- E Exception: Sanitizer assert found for instance ================== E WARNING: ThreadSanitizer: data race (pid=1) E Write of size 8 at 0x7b2800025d30 by thread T2 (mutexes: write M0, write M1): E #0 free <null> (clickhouse+0x709a3e5) (BuildId: 706d92b17db171493f293d517643f726ee1b7b1e) E #1 CRYPTO_free build_docker/./contrib/openssl/crypto/mem.c:282:5 (clickhouse+0x2015f8ea) (BuildId: 706d92b17db171493f293d517643f726ee1b7b1e) E #2 EVP_PKEY_free build_docker/./contrib/openssl/crypto/evp/p_lib.c:1809:5 (clickhouse+0x2012a751) (BuildId: 706d92b17db171493f293d517643f726ee1b7b1e) E #3 Poco::Crypto::EVPPKey::~EVPPKey() build_docker/./base/poco/Crypto/src/EVPPKey.cpp:121:17 (clickhouse+0x1d00ffa9) (BuildId: 706d92b17db171493f293d517643f726ee1b7b1e) E #4 DB::CertificateReloader::Data::~Data() build_docker/./src/Server/CertificateReloader.h:71:12 (clickhouse+0x194fb42d) (BuildId: 706d92b17db171493f293d517643f726ee1b7b1e) E #5 std::__1::default_delete<DB::CertificateReloader::Data const>::operator()[abi:v15000](DB::CertificateReloader::Data const*) const build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48:5 (clickhouse+0x194fb42d) E #6 std::__1::__shared_ptr_pointer<DB::CertificateReloader::Data const*, std::__1::default_delete<DB::CertificateReloader::Data const>, std::__1::allocator<DB::CertificateReloader::Data const>>::__on_zero_shared() build_docker/./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:263:5 (clickhouse+0x194fb42d) E #7 std::__1::__shared_count::__release_shared[abi:v15000]() build_docker/./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:174:9 (clickhouse+0x194fade0) (BuildId: 706d92b17db171493f293d517643f726ee1b7b1e) E #8 std::__1::__shared_weak_count::__release_shared[abi:v15000]() build_docker/./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:215:27 (clickhouse+0x194fade0) E openssl#9 std::__1::shared_ptr<DB::CertificateReloader::Data const>::~shared_ptr[abi:v15000]() build_docker/./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:702:23 (clickhouse+0x194fade0) E openssl#10 std::__1::shared_ptr<DB::CertificateReloader::Data const>::operator=[abi:v15000](std::__1::shared_ptr<DB::CertificateReloader::Data const>&&) build_docker/./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:723:9 (clickhouse+0x194fade0) E openssl#11 MultiVersion<DB::CertificateReloader::Data>::set(std::__1::unique_ptr<DB::CertificateReloader::Data const, std::__1::default_delete<DB::CertificateReloader::Data const>>&&) build_docker/./src/Common/MultiVersion.h:76:25 (clickhouse+0x194fade0) E openssl#12 DB::CertificateReloader::tryLoad(Poco::Util::AbstractConfiguration const&) build_docker/./src/Server/CertificateReloader.cpp:83:18 (clickhouse+0x194f94ca) (BuildId: 706d92b17db171493f293d517643f726ee1b7b1e) E openssl#13 DB::Server::main(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>> const&)::$_6::operator()(Poco::AutoPtr<Poco::Util::AbstractConfiguration>, bool) const build_docker/./programs/server/Server.cpp:1546:45 (clickhouse+0xf384df7) (BuildId: 706d92b17db171493f293d517643f726ee1b7b1e) E openssl#14 decltype(std::declval<DB::Server::main(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>> const&)::$_6&>()(std::declval<Poco::AutoPtr<Poco::Util::AbstractConfiguration>>(), std::declval<bool>())) std::__1::__invoke[abi:v15000]<DB::Server::main(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>> const&)::$_6&, Poco::AutoPtr<Poco::Util::AbstractConfiguration>, bool>(DB::Server::main(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>> const&)::$_6&, Poco::AutoPtr<Poco::Util::AbstractConfiguration>&&, bool&&) build_docker/./contrib/llvm-project/libcxx/include/__functional/invoke.h:394:23 (clickhouse+0xf3827a9) (BuildId: 706d92b17db171493f293d517643f726ee1b7b1e) E openssl#15 void std::__1::__invoke_void_return_wrapper<void, true>::__call<DB::Server::main(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>> const&)::$_6&, Poco::AutoPtr<Poco::Util::AbstractConfiguration>, bool>(DB::Server::main(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>> const&)::$_6&, Poco::AutoPtr<Poco::Util::AbstractConfiguration>&&, bool&&) build_docker/./contrib/llvm-project/libcxx/include/__functional/invoke.h:479:9 (clickhouse+0xf3827a9) E openssl#16 std::__1::__function::__default_alloc_func<DB::Server::main(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>> const&)::$_6, void (Poco::AutoPtr<Poco::Util::AbstractConfiguration>, bool)>::operator()[abi:v15000](Poco::AutoPtr<Poco::Util::AbstractConfiguration>&&, bool&&) build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:235:12 (clickhouse+0xf3827a9) E openssl#17 void std::__1::__function::__policy_invoker<void (Poco::AutoPtr<Poco::Util::AbstractConfiguration>, bool)>::__call_impl<std::__1::__function::__default_alloc_func<DB::Server::main(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>> const&)::$_6, void (Poco::AutoPtr<Poco::Util::AbstractConfiguration>, bool)>>(std::__1::__function::__policy_storage const*, Poco::AutoPtr<Poco::Util::AbstractConfiguration>&&, bool) build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:716:16 (clickhouse+0xf3827a9) E openssl#18 std::__1::__function::__policy_func<void (Poco::AutoPtr<Poco::Util::AbstractConfiguration>, bool)>::operator()[abi:v15000](Poco::AutoPtr<Poco::Util::AbstractConfiguration>&&, bool&&) const build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:848:16 (clickhouse+0x19fd2cbe) (BuildId: 706d92b17db171493f293d517643f726ee1b7b1e) E openssl#19 std::__1::function<void (Poco::AutoPtr<Poco::Util::AbstractConfiguration>, bool)>::operator()(Poco::AutoPtr<Poco::Util::AbstractConfiguration>, bool) const build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:1187:12 (clickhouse+0x19fd2cbe) E openssl#20 DB::ConfigReloader::reloadIfNewer(bool, bool, bool, bool) build_docker/./src/Common/Config/ConfigReloader.cpp:150:13 (clickhouse+0x19fd2cbe) E openssl#21 DB::ConfigReloader::reload() build_docker/./src/Common/Config/ConfigReloader.h:51:21 (clickhouse+0xf38767c) (BuildId: 706d92b17db171493f293d517643f726ee1b7b1e) E openssl#22 DB::Server::main(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>> const&)::$_13::operator()() const build_docker/./programs/server/Server.cpp:1731:31 (clickhouse+0xf38767c) E openssl#23 decltype(std::declval<DB::Server::main(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>> const&)::$_13&>()()) std::__1::__invoke[abi:v15000]<DB::Server::main(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>> const&)::$_13&>(DB::Server::main(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>> const&)::$_13&) build_docker/./contrib/llvm-project/libcxx/include/__functional/invoke.h:394:23 (clickhouse+0xf38767c) E openssl#24 void std::__1::__invoke_void_return_wrapper<void, true>::__call<DB::Server::main(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>> const&)::$_13&>(DB::Server::main(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>> const&)::$_13&) build_docker/./contrib/llvm-project/libcxx/include/__functional/invoke.h:479:9 (clickhouse+0xf38767c) E openssl#25 std::__1::__function::__default_alloc_func<DB::Server::main(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>> const&)::$_13, void ()>::operator()[abi:v15000]() build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:235:12 (clickhouse+0xf38767c) E openssl#26 void std::__1::__function::__policy_invoker<void ()>::__call_impl<std::__1::__function::__default_alloc_func<DB::Server::main(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>> const&)::$_13, void ()>>(std::__1::__function::__policy_storage const*) build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:716:16 (clickhouse+0xf38767c) E openssl#27 std::__1::__function::__policy_func<void ()>::operator()[abi:v15000]() const build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:848:16 (clickhouse+0x16907aa0) (BuildId: 706d92b17db171493f293d517643f726ee1b7b1e) E openssl#28 std::__1::function<void ()>::operator()() const build_docker/./contrib/llvm-project/libcxx/include/__functional/function.h:1187:12 (clickhouse+0x16907aa0) E openssl#29 DB::Context::reloadConfig() const build_docker/./src/Interpreters/Context.cpp:4357:5 (clickhouse+0x16907aa0) E openssl#30 DB::InterpreterSystemQuery::execute() build_docker/./src/Interpreters/InterpreterSystemQuery.cpp:577:29 (clickhouse+0x17e78c19) (BuildId: 706d92b17db171493f293d517643f726ee1b7b1e) E openssl#31 DB::executeQueryImpl(char const*, char const*, std::__1::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) build_docker/./src/Interpreters/executeQuery.cpp:1195:40 (clickhouse+0x17e3e462) (BuildId: 706d92b17db171493f293d517643f726ee1b7b1e) E openssl#32 DB::executeQuery(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) build_docker/./src/Interpreters/executeQuery.cpp:1374:26 (clickhouse+0x17e39837) (BuildId: 706d92b17db171493f293d517643f726ee1b7b1e) E openssl#33 DB::TCPHandler::runImpl() build_docker/./src/Server/TCPHandler.cpp:518:54 (clickhouse+0x195cc651) (BuildId: 706d92b17db171493f293d517643f726ee1b7b1e) E openssl#34 DB::TCPHandler::run() build_docker/./src/Server/TCPHandler.cpp:2329:9 (clickhouse+0x195e8707) (BuildId: 706d92b17db171493f293d517643f726ee1b7b1e) E openssl#35 Poco::Net::TCPServerConnection::start() build_docker/./base/poco/Net/src/TCPServerConnection.cpp:43:3 (clickhouse+0x1d00d942) (BuildId: 706d92b17db171493f293d517643f726ee1b7b1e) E openssl#36 Poco::Net::TCPServerDispatcher::run() build_docker/./base/poco/Net/src/TCPServerDispatcher.cpp:115:20 (clickhouse+0x1d00e1b1) (BuildId: 706d92b17db171493f293d517643f726ee1b7b1e) E openssl#37 Poco::PooledThread::run() build_docker/./base/poco/Foundation/src/ThreadPool.cpp:188:14 (clickhouse+0x1d20f2e6) (BuildId: 706d92b17db171493f293d517643f726ee1b7b1e) E openssl#38 Poco::(anonymous namespace)::RunnableHolder::run() build_docker/./base/poco/Foundation/src/Thread.cpp:45:11 (clickhouse+0x1d20d5af) (BuildId: 706d92b17db171493f293d517643f726ee1b7b1e) E openssl#39 Poco::ThreadImpl::runnableEntry(void*) build_docker/./base/poco/Foundation/src/Thread_POSIX.cpp:335:27 (clickhouse+0x1d20ba69) (BuildId: 706d92b17db171493f293d517643f726ee1b7b1e) E E Previous atomic write of size 4 at 0x7b2800025d30 by thread T3 (mutexes: write M2): E #0 CRYPTO_DOWN_REF build_docker/./contrib/openssl/include/internal/refcount.h:51:12 (clickhouse+0x2012a6e6) (BuildId: 706d92b17db171493f293d517643f726ee1b7b1e) E #1 EVP_PKEY_free build_docker/./contrib/openssl/crypto/evp/p_lib.c:1795:5 (clickhouse+0x2012a6e6) E #2 ssl_cert_clear_certs build_docker/./contrib/openssl/ssl/ssl_cert.c:246:9 (clickhouse+0x1ffafd37) (BuildId: 706d92b17db171493f293d517643f726ee1b7b1e) E #3 ssl_cert_free build_docker/./contrib/openssl/ssl/ssl_cert.c:277:5 (clickhouse+0x1ffafd37) E #4 ossl_ssl_connection_free build_docker/./contrib/openssl/ssl/ssl_lib.c:1458:5 (clickhouse+0x1ffba6af) (BuildId: 706d92b17db171493f293d517643f726ee1b7b1e) E #5 SSL_free build_docker/./contrib/openssl/ssl/ssl_lib.c:1417:9 (clickhouse+0x1ffb920e) (BuildId: 706d92b17db171493f293d517643f726ee1b7b1e) E #6 Poco::Net::SecureSocketImpl::reset() build_docker/./base/poco/NetSSL_OpenSSL/src/SecureSocketImpl.cpp:583:3 (clickhouse+0x1cfaac60) (BuildId: 706d92b17db171493f293d517643f726ee1b7b1e) E #7 Poco::Net::SecureSocketImpl::~SecureSocketImpl() build_docker/./base/poco/NetSSL_OpenSSL/src/SecureSocketImpl.cpp:80:3 (clickhouse+0x1cfaac60) E #8 Poco::Net::SecureStreamSocketImpl::~SecureStreamSocketImpl() build_docker/./base/poco/NetSSL_OpenSSL/src/SecureStreamSocketImpl.cpp:52:1 (clickhouse+0x1cfb15dd) (BuildId: 706d92b17db171493f293d517643f726ee1b7b1e) E openssl#9 Poco::Net::SecureStreamSocketImpl::~SecureStreamSocketImpl() build_docker/./base/poco/NetSSL_OpenSSL/src/SecureStreamSocketImpl.cpp:43:1 (clickhouse+0x1cfb15dd) E openssl#10 Poco::RefCountedObject::release() const build_docker/./base/poco/Foundation/include/Poco/RefCountedObject.h:86:13 (clickhouse+0x1cffc81e) (BuildId: 706d92b17db171493f293d517643f726ee1b7b1e) E openssl#11 Poco::Net::Socket::~Socket() build_docker/./base/poco/Net/src/Socket.cpp:68:10 (clickhouse+0x1cffc81e) E openssl#12 Poco::Net::StreamSocket::~StreamSocket() build_docker/./base/poco/Net/src/StreamSocket.cpp:63:1 (clickhouse+0x1d009c39) (BuildId: 706d92b17db171493f293d517643f726ee1b7b1e) E openssl#13 Poco::Net::TCPConnectionNotification::~TCPConnectionNotification() build_docker/./base/poco/Net/src/TCPServerDispatcher.cpp:43:2 (clickhouse+0x1d00ef50) (BuildId: 706d92b17db171493f293d517643f726ee1b7b1e) E openssl#14 Poco::Net::TCPConnectionNotification::~TCPConnectionNotification() build_docker/./base/poco/Net/src/TCPServerDispatcher.cpp:42:2 (clickhouse+0x1d00ef50) E openssl#15 Poco::RefCountedObject::release() const build_docker/./base/poco/Foundation/include/Poco/RefCountedObject.h:86:13 (clickhouse+0x1d00e203) (BuildId: 706d92b17db171493f293d517643f726ee1b7b1e) E openssl#16 Poco::AutoPtr<Poco::Notification>::~AutoPtr() build_docker/./base/poco/Foundation/include/Poco/AutoPtr.h:91:19 (clickhouse+0x1d00e203) E openssl#17 Poco::Net::TCPServerDispatcher::run() build_docker/./base/poco/Net/src/TCPServerDispatcher.cpp:122:3 (clickhouse+0x1d00e203) E openssl#18 Poco::PooledThread::run() build_docker/./base/poco/Foundation/src/ThreadPool.cpp:188:14 (clickhouse+0x1d20f2e6) (BuildId: 706d92b17db171493f293d517643f726ee1b7b1e) E openssl#19 Poco::(anonymous namespace)::RunnableHolder::run() build_docker/./base/poco/Foundation/src/Thread.cpp:45:11 (clickhouse+0x1d20d5af) (BuildId: 706d92b17db171493f293d517643f726ee1b7b1e) E openssl#20 Poco::ThreadImpl::runnableEntry(void*) build_docker/./base/poco/Foundation/src/Thread_POSIX.cpp:335:27 (clickhouse+0x1d20ba69) (BuildId: 706d92b17db171493f293d517643f726ee1b7b1e)

During my testing (*), I got below stack about leaked memory. - at database startup, Azure is registered as an object storage (--> registerAzureObjectStorage) - this calls into Azure, then into curl, and then into OpenSSL - curl asks OpenSSL for a bunch of random numbers (--> function 'ossl_random' in curl's OpenSSL wrapper) - OpenSSL initializes the random number generator and stores it in some random number generator context object (--> *RAND_get0_public) - this object is registered via pthread_key_create and pthread_setspecific registered in TLS - if registerAzureObjectStorage was the only place which initializes the RNG, we could argue that the leaked memory does not matter anyways as it is released after shutdown - RAND_get0_public also registers a free handler (rand_delete_thread_state) that runs in TLS, so the memory is released also if registration is called from arbitrary other threads. In sum: this is a false positive. (*) ClickHouse/clickhouse-private#10107 (comment) ``` ================================================================= ==2757181==ERROR: LeakSanitizer: detected memory leaks Direct leak of 48 byte(s) in 1 object(s) allocated from: #0 0x5672f4a48083 in malloc (/data/ch4/build_asan/programs/clickhouse+0xa2b7083) (BuildId: 4d868624feaa8b5598d9d0b327749c877bfb6cfd) #1 0x567321d05c7e in CRYPTO_malloc build_asan/./contrib/openssl/crypto/mem.c:202:11 #2 0x567321d05c7e in CRYPTO_zalloc build_asan/./contrib/openssl/crypto/mem.c:222:11 #3 0x567321d5f9cf in EVP_RAND_CTX_new build_asan/./contrib/openssl/crypto/evp/evp_rand.c:353:11 #4 0x567321d627db in rand_new_drbg build_asan/./contrib/openssl/crypto/rand/rand_lib.c:658:11 #5 0x567321d61e7b in RAND_get0_public build_asan/./contrib/openssl/crypto/rand/rand_lib.c:777:16 #6 0x567321d61d4f in RAND_bytes_ex build_asan/./contrib/openssl/crypto/rand/rand_lib.c:378:12 #7 0x56731cfe9267 in ossl_random openssl.c #8 0x56731cf534ae in Curl_rand (/data/ch4/build_asan/programs/clickhouse+0x327c24ae) (BuildId: 4d868624feaa8b5598d9d0b327749c877bfb6cfd) openssl#9 0x56731cf757dc in Curl_socketpair (/data/ch4/build_asan/programs/clickhouse+0x327e47dc) (BuildId: 4d868624feaa8b5598d9d0b327749c877bfb6cfd) openssl#10 0x56731cf40bfe in Curl_multi_handle (/data/ch4/build_asan/programs/clickhouse+0x327afbfe) (BuildId: 4d868624feaa8b5598d9d0b327749c877bfb6cfd) openssl#11 0x56731cecb260 in curl_easy_perform (/data/ch4/build_asan/programs/clickhouse+0x3273a260) (BuildId: 4d868624feaa8b5598d9d0b327749c877bfb6cfd) openssl#12 0x56731cd4a351 in Azure::Core::Http::CurlConnection::CurlConnection(Azure::Core::Http::Request&, Azure::Core::Http::CurlTransportOptions const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&) build_asan/./contrib/azure/sdk/core/azure-core/src/http/curl/curl.cpp:2441:24 openssl#13 0x56731cd3176b in std::__1::__unique_if<Azure::Core::Http::CurlConnection>::__unique_single std::__1::make_unique[abi:v15000]<Azure::Core::Http::CurlConnection, Azure::Core::Http::Request&, Azure::Core::Http::CurlTransportOptions const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&>(Azure::Core::Http::Request&, Azure::Core::H ttp::CurlTransportOptions const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&) build_asan/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714:32 openssl#14 0x56731cd3176b in Azure::Core::Http::_detail::CurlConnectionPool::ExtractOrCreateCurlConnection(Azure::Core::Http::Request&, Azure::Core::Http::CurlTransportOptions const&, bool) build_asan/./contrib/azure/sdk/core/azure-core/src/http/curl/curl.cpp:2126:10 openssl#15 0x56731cd2f891 in Azure::Core::Http::CurlTransport::Send(Azure::Core::Http::Request&, Azure::Core::Context const&) build_asan/./contrib/azure/sdk/core/azure-core/src/http/curl/curl.cpp:351:48 openssl#16 0x56731cdd66db in Azure::Core::Http::Policies::_internal::TransportPolicy::Send(Azure::Core::Http::Request&, Azure::Core::Http::Policies::NextHttpPolicy, Azure::Core::Context const&) const build_asan/./contrib/azure/sdk/core/azure-core/src/http/transport_policy.cpp:121:40 openssl#17 0x56731cd7c75b in Azure::Core::Http::Policies::_internal::LogPolicy::Send(Azure::Core::Http::Request&, Azure::Core::Http::Policies::NextHttpPolicy, Azure::Core::Context const&) const build_asan/./contrib/azure/sdk/core/azure-core/src/http/log_policy.cpp:114:23 openssl#18 0x56731cdd0218 in Azure::Core::Http::Policies::_internal::RequestActivityPolicy::Send(Azure::Core::Http::Request&, Azure::Core::Http::Policies::NextHttpPolicy, Azure::Core::Context const&) const build_asan/./contrib/azure/sdk/core/azure-core/src/http/request_activity_policy.cpp:110:23 openssl#19 0x56731cec5f18 in Azure::Storage::_internal::StoragePerRetryPolicy::Send(Azure::Core::Http::Request&, Azure::Core::Http::Policies::NextHttpPolicy, Azure::Core::Context const&) const build_asan/./contrib/azure/sdk/storage/azure-storage-common/src/storage_per_retry_policy.cpp:57:23 openssl#20 0x56731cec6d0b in Azure::Storage::_internal::StorageSwitchToSecondaryPolicy::Send(Azure::Core::Http::Request&, Azure::Core::Http::Policies::NextHttpPolicy, Azure::Core::Context const&) const build_asan/./contrib/azure/sdk/storage/azure-storage-common/src/storage_switch_to_secondary_policy.cpp:36:32 openssl#21 0x56731cdcd019 in Azure::Core::Http::Policies::_internal::RetryPolicy::Send(Azure::Core::Http::Request&, Azure::Core::Http::Policies::NextHttpPolicy, Azure::Core::Context const&) const build_asan/./contrib/azure/sdk/core/azure-core/src/http/retry_policy.cpp:146:34 openssl#22 0x56731cdd6e17 in Azure::Core::Http::Policies::_internal::TelemetryPolicy::Send(Azure::Core::Http::Request&, Azure::Core::Http::Policies::NextHttpPolicy, Azure::Core::Context const&) const build_asan/./contrib/azure/sdk/core/azure-core/src/http/telemetry_policy.cpp:23:21 openssl#23 0x56731cdb0ef5 in Azure::Core::Http::Policies::_internal::RequestIdPolicy::Send(Azure::Core::Http::Request&, Azure::Core::Http::Policies::NextHttpPolicy, Azure::Core::Context const&) const build_asan/./contrib/azure/sdk/core/azure-core/inc/azure/core/http/policies/policy.hpp:453:27 openssl#24 0x56731cdfc53e in Azure::Storage::_internal::StorageServiceVersionPolicy::Send(Azure::Core::Http::Request&, Azure::Core::Http::Policies::NextHttpPolicy, Azure::Core::Context const&) const build_asan/./contrib/azure/sdk/storage/azure-storage-common/inc/azure/storage/common/internal/storage_service_version_policy.hpp:34:25 openssl#25 0x56731ce4bff1 in Azure::Core::Http::_internal::HttpPipeline::Send(Azure::Core::Http::Request&, Azure::Core::Context const&) const build_asan/./contrib/azure/sdk/core/azure-core/inc/azure/core/internal/http/pipeline.hpp:230:29 openssl#26 0x56731ce4bff1 in Azure::Storage::Blobs::_detail::BlobContainerClient::Create(Azure::Core::Http::_internal::HttpPipeline&, Azure::Core::Url const&, Azure::Storage::Blobs::_detail::BlobContainerClient::CreateBlobContainerOptions const&, Azure::Core::Context const&) build_asan/./contrib/azure/sdk/storage/azure-storage-blobs/src/rest_client.cpp:1415:36 openssl#27 0x56731ce0bdd8 in Azure::Storage::Blobs::BlobContainerClient::Create(Azure::Storage::Blobs::CreateBlobContainerOptions const&, Azure::Core::Context const&) const build_asan/./contrib/azure/sdk/storage/azure-storage-blobs/src/blob_container_client.cpp:258:12 openssl#28 0x56731ce396dc in Azure::Storage::Blobs::BlobServiceClient::CreateBlobContainer(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, Azure::Storage::Blobs::CreateBlobContainerOptions const&, Azure::Core::Context const&) const build_asan/./contrib/azure/sdk/storage/azure-storage-blobs/src/blob_service_client.cpp:264:41 openssl#29 0x56731018fcff in DB::getAzureBlobContainerClient(Poco::Util::AbstractConfiguration const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&) build_asan/./src/Disks/ObjectStorages/AzureBlobStorage/AzureBlobStorageAuth.cpp:236:75 openssl#30 0x5673115a7352 in DB::registerAzureObjectStorage(DB::ObjectStorageFactory&)::$_0::operator()(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, Poco::Util::AbstractConfiguration const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::shared_ptr<DB::Context const> const&, bool) const build_asan/./src/Disks/ObjectStorages/ObjectStorageFactory.cpp:315:13 openssl#31 0x5673115a7352 in decltype(std::declval<DB::registerAzureObjectStorage(DB::ObjectStorageFactory&)::$_0&>()(std::declval<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&>(), std::declval<Poco::Util::AbstractConfiguration const&>(), std::declval<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&>(), std::declval<std::__1::shared_ptr<DB::Context const> const&>(), std::declval<bool>() )) std::__1::__invoke[abi:v15000]<DB::registerAzureObjectStorage(DB::ObjectStorageFactory&)::$_0&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, Poco::Util::AbstractConfiguration const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::shared_ptr<DB::Context const> const&, bool>(DB::registerAzureObjectStorage(DB::ObjectStorageFactory&)::$_0&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, Poco::Util::AbstractConfiguration const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::shared_ptr<DB::Context const> const&, bool&&) build_asan/./contrib/llvm-project/libcxx/include/__functional/invoke.h:394:23 openssl#32 0x5673115a7352 in std::__1::shared_ptr<DB::IObjectStorage> std::__1::__invoke_void_return_wrapper<std::__1::shared_ptr<DB::IObjectStorage>, false>::__call<DB::registerAzureObjectStorage(DB::ObjectStorageFactory&)::$_0&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, Poco::Util::AbstractConfiguration const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::shared_pt r<DB::Context const> const&, bool>(DB::registerAzureObjectStorage(DB::ObjectStorageFactory&)::$_0&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, Poco::Util::AbstractConfiguration const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::shared_ptr<DB::Context const> const&, bool&&) build_asan/./contrib/llvm-project/libcxx/include/__functional/invoke.h:470:16 openssl#33 0x5673115a7352 in std::__1::__function::__default_alloc_func<DB::registerAzureObjectStorage(DB::ObjectStorageFactory&)::$_0, std::__1::shared_ptr<DB::IObjectStorage> (std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, Poco::Util::AbstractConfiguration const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::shared_ptr<DB::Context const> const&, bool)>::operator()[abi:v 15000](std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, Poco::Util::AbstractConfiguration const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::shared_ptr<DB::Context const> const&, bool&&) build_asan/./contrib/llvm-project/libcxx/include/__functional/function.h:235:12 openssl#34 0x5673115a7352 in std::__1::shared_ptr<DB::IObjectStorage> std::__1::__function::__policy_invoker<std::__1::shared_ptr<DB::IObjectStorage> (std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, Poco::Util::AbstractConfiguration const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::shared_ptr<DB::Context const> const&, bool)>::__call_impl<std::__1::__function::__default _alloc_func<DB::registerAzureObjectStorage(DB::ObjectStorageFactory&)::$_0, std::__1::shared_ptr<DB::IObjectStorage> (std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, Poco::Util::AbstractConfiguration const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::shared_ptr<DB::Context const> const&, bool)>>(std::__1::__function::__policy_storage const*, std::__1::basic_string<cha r, std::__1::char_traits<char>, std::__1::allocator<char>> const&, Poco::Util::AbstractConfiguration const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::shared_ptr<DB::Context const> const&, bool) build_asan/./contrib/llvm-project/libcxx/include/__functional/function.h:716:16 openssl#35 0x5673115a0780 in std::__1::__function::__policy_func<std::__1::shared_ptr<DB::IObjectStorage> (std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, Poco::Util::AbstractConfiguration const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::shared_ptr<DB::Context const> const&, bool)>::operator()[abi:v15000](std::__1::basic_string<char, std::__1::char_traits<char>, std::_ _1::allocator<char>> const&, Poco::Util::AbstractConfiguration const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::shared_ptr<DB::Context const> const&, bool&&) const build_asan/./contrib/llvm-project/libcxx/include/__functional/function.h:848:16 openssl#36 0x5673115a0780 in std::__1::function<std::__1::shared_ptr<DB::IObjectStorage> (std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, Poco::Util::AbstractConfiguration const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::shared_ptr<DB::Context const> const&, bool)>::operator()(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, Poco::Util::AbstractConfiguration const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::shared_ptr<DB::Context const> const&, bool) const build_asan/./contrib/llvm-project/libcxx/include/__functional/function.h:1187:12 openssl#37 0x5673115a0780 in DB::ObjectStorageFactory::create(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, Poco::Util::AbstractConfiguration const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::shared_ptr<DB::Context const> const&, bool) const build_asan/./src/Disks/ObjectStorages/ObjectStorageFactory.cpp:135:12 openssl#38 0x56731159ef45 in DB::registerDiskObjectStorage(DB::DiskFactory&, bool)::$_0::operator()(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, Poco::Util::AbstractConfiguration const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::shared_ptr<DB::Context const>, std::__1::map<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, std::__1: :shared_ptr<DB::IDisk>, std::__1::less<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>, std::__1::allocator<std::__1::pair<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const, std::__1::shared_ptr<DB::IDisk>>>> const&, bool, bool) const build_asan/./src/Disks/ObjectStorages/RegisterDiskObjectStorage.cpp:27:64 openssl#39 0x56731159ef45 in decltype(std::declval<DB::registerDiskObjectStorage(DB::DiskFactory&, bool)::$_0&>()(std::declval<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&>(), std::declval<Poco::Util::AbstractConfiguration const&>(), std::declval<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&>(), std::declval<std::__1::shared_ptr<DB::Context const>>(), std::declval<std::__1::map<std: :__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, std::__1::shared_ptr<DB::IDisk>, std::__1::less<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>, std::__1::allocator<std::__1::pair<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const, std::__1::shared_ptr<DB::IDisk>>>> const&>(), std::declval<bool>(), std::declval<bool>())) std::__1::__invoke[abi:v15000]<DB::regis terDiskObjectStorage(DB::DiskFactory&, bool)::$_0&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, Poco::Util::AbstractConfiguration const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::shared_ptr<DB::Context const>, std::__1::map<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, std::__1::shared_ptr<DB::IDisk>, std::__1::less<std::_ _1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>, std::__1::allocator<std::__1::pair<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const, std::__1::shared_ptr<DB::IDisk>>>> const&, bool, bool>(DB::registerDiskObjectStorage(DB::DiskFactory&, bool)::$_0&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, Poco::Util::AbstractConfiguration const&, std::__1::basic_ string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::shared_ptr<DB::Context const>&&, std::__1::map<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, std::__1::shared_ptr<DB::IDisk>, std::__1::less<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>, std::__1::allocator<std::__1::pair<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const, std::__1::shared_ptr<DB::IDisk>>>> const&, bool&&, bool&&) build_asan/./contrib/llvm-project/libcxx/include/__functional/invoke.h:394:23 openssl#40 0x56731159ef45 in std::__1::shared_ptr<DB::IDisk> std::__1::__invoke_void_return_wrapper<std::__1::shared_ptr<DB::IDisk>, false>::__call<DB::registerDiskObjectStorage(DB::DiskFactory&, bool)::$_0&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, Poco::Util::AbstractConfiguration const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::shared_ptr<DB::Context const>, std::__1::map<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, std::__1::shared_ptr<DB::IDisk>, std::__1::less<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>, std::__1::allocator<std::__1::pair<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const, std::__1::shared_ptr<DB::IDisk>>>> const&, bool, bool>(DB::registerDiskObjectStorage(DB::DiskFactory&, bool)::$_0 &, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, Poco::Util::AbstractConfiguration const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::shared_ptr<DB::Context const>&&, std::__1::map<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, std::__1::shared_ptr<DB::IDisk>, std::__1::less<std::__1::basic_string<char, std::__1::char_traits<ch ar>, std::__1::allocator<char>>>, std::__1::allocator<std::__1::pair<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const, std::__1::shared_ptr<DB::IDisk>>>> const&, bool&&, bool&&) build_asan/./contrib/llvm-project/libcxx/include/__functional/invoke.h:470:16 openssl#41 0x56731159ef45 in std::__1::__function::__default_alloc_func<DB::registerDiskObjectStorage(DB::DiskFactory&, bool)::$_0, std::__1::shared_ptr<DB::IDisk> (std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, Poco::Util::AbstractConfiguration const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::shared_ptr<DB::Context const>, std::__1::map<std::__1::basic_string<char, s td::__1::char_traits<char>, std::__1::allocator<char>>, std::__1::shared_ptr<DB::IDisk>, std::__1::less<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>, std::__1::allocator<std::__1::pair<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const, std::__1::shared_ptr<DB::IDisk>>>> const&, bool, bool)>::operator()[abi:v15000](std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator <char>> const&, Poco::Util::AbstractConfiguration const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::shared_ptr<DB::Context const>&&, std::__1::map<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, std::__1::shared_ptr<DB::IDisk>, std::__1::less<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>, std::__1::allocator<std::__1::pair<std::__1::b asic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const, std::__1::shared_ptr<DB::IDisk>>>> const&, bool&&, bool&&) build_asan/./contrib/llvm-project/libcxx/include/__functional/function.h:235:12 ```

ClickHouse issue ClickHouse/clickhouse-private#10107 (comment) reports this memory leak: (note how it is very similar to ClickHouse/clickhouse-private#10107 (comment)) ``` Direct leak of 528 byte(s) in 11 object(s) allocated from: #0 0x561369af24cf in malloc (/usr/bin/clickhouse+0xa6cf4cf) (BuildId: 22880fad595a96b17eb9add20e7a01f8ded54c49) #1 0x561397b86a7e in CRYPTO_malloc build_docker/./contrib/openssl/crypto/mem.c:202:11 #2 0x561397b86a7e in CRYPTO_zalloc build_docker/./contrib/openssl/crypto/mem.c:222:11 #3 0x561397be07cf in EVP_RAND_CTX_new build_docker/./contrib/openssl/crypto/evp/evp_rand.c:353:11 #4 0x561397be35fb in rand_new_drbg build_docker/./contrib/openssl/crypto/rand/rand_lib.c:665:11 #5 0x561397be2a9b in RAND_get0_private build_docker/./contrib/openssl/crypto/rand/rand_lib.c:827:16 #6 0x561397be296f in RAND_priv_bytes_ex build_docker/./contrib/openssl/crypto/rand/rand_lib.c:356:12 #7 0x5613978d7cd8 in SSL_CTX_new_ex build_docker/./contrib/openssl/ssl/ssl_lib.c:4016:13 #8 0x561392d1eabd in ossl_connect_common openssl.c openssl#9 0x561392d11e23 in ssl_cf_connect vtls.c openssl#10 0x561392c13a1f in cf_setup_connect connect.c openssl#11 0x561392c1b0e5 in cf_hc_connect cf-https-connect.c openssl#12 0x561392c0a332 in Curl_conn_connect (/usr/bin/clickhouse+0x337e7332) (BuildId: 22880fad595a96b17eb9add20e7a01f8ded54c49) openssl#13 0x561392c7a28a in multi_runsingle multi.c openssl#14 0x561392c78f6d in curl_multi_perform (/usr/bin/clickhouse+0x33855f6d) (BuildId: 22880fad595a96b17eb9add20e7a01f8ded54c49) openssl#15 0x561392bfd53e in curl_easy_perform (/usr/bin/clickhouse+0x337da53e) (BuildId: 22880fad595a96b17eb9add20e7a01f8ded54c49) openssl#16 0x561392a7c411 in Azure::Core::Http::CurlConnection::CurlConnection(Azure::Core::Http::Request&, Azure::Core::Http::CurlTransportOptions const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&) build_docker/./contrib/azure/sdk/core/azure-core/src/http/curl/curl.cpp:2441:24 openssl#17 0x561392a6382b in std::__1::__unique_if<Azure::Core::Http::CurlConnection>::__unique_single std::__1::make_unique[abi:v15000]<Azure::Core::Http::CurlConnection, Azure::Core::Http::Request&, Azure::Core::Http::CurlTransportOptions const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&>(Azure::Core::Http::Request&, Azure::Core::Http::CurlTransportOptions const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&) build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714:32 openssl#18 0x561392a6382b in Azure::Core::Http::_detail::CurlConnectionPool::ExtractOrCreateCurlConnection(Azure::Core::Http::Request&, Azure::Core::Http::CurlTransportOptions const&, bool) build_docker/./contrib/azure/sdk/core/azure-core/src/http/curl/curl.cpp:2126:10 openssl#19 0x561392a61951 in Azure::Core::Http::CurlTransport::Send(Azure::Core::Http::Request&, Azure::Core::Context const&) build_docker/./contrib/azure/sdk/core/azure-core/src/http/curl/curl.cpp:351:48 openssl#20 0x561392b0879b in Azure::Core::Http::Policies::_internal::TransportPolicy::Send(Azure::Core::Http::Request&, Azure::Core::Http::Policies::NextHttpPolicy, Azure::Core::Context const&) const build_docker/./contrib/azure/sdk/core/azure-core/src/http/transport_policy.cpp:121:40 openssl#21 0x561392aae81b in Azure::Core::Http::Policies::_internal::LogPolicy::Send(Azure::Core::Http::Request&, Azure::Core::Http::Policies::NextHttpPolicy, Azure::Core::Context const&) const build_docker/./contrib/azure/sdk/core/azure-core/src/http/log_policy.cpp:114:23 openssl#22 0x561392b022d8 in Azure::Core::Http::Policies::_internal::RequestActivityPolicy::Send(Azure::Core::Http::Request&, Azure::Core::Http::Policies::NextHttpPolicy, Azure::Core::Context const&) const build_docker/./contrib/azure/sdk/core/azure-core/src/http/request_activity_policy.cpp:110:23 openssl#23 0x561392b2e02a in Azure::Storage::_internal::SharedKeyPolicy::Send(Azure::Core::Http::Request&, Azure::Core::Http::Policies::NextHttpPolicy, Azure::Core::Context const&) const build_docker/./contrib/azure/sdk/storage/azure-storage-common/inc/azure/storage/common/internal/shared_key_policy.hpp:36:25 openssl#24 0x561392bf7fd8 in Azure::Storage::_internal::StoragePerRetryPolicy::Send(Azure::Core::Http::Request&, Azure::Core::Http::Policies::NextHttpPolicy, Azure::Core::Context const&) const build_docker/./contrib/azure/sdk/storage/azure-storage-common/src/storage_per_retry_policy.cpp:57:23 openssl#25 0x561392bf8dcb in Azure::Storage::_internal::StorageSwitchToSecondaryPolicy::Send(Azure::Core::Http::Request&, Azure::Core::Http::Policies::NextHttpPolicy, Azure::Core::Context const&) const build_docker/./contrib/azure/sdk/storage/azure-storage-common/src/storage_switch_to_secondary_policy.cpp:36:32 openssl#26 0x561392aff0d9 in Azure::Core::Http::Policies::_internal::RetryPolicy::Send(Azure::Core::Http::Request&, Azure::Core::Http::Policies::NextHttpPolicy, Azure::Core::Context const&) const build_docker/./contrib/azure/sdk/core/azure-core/src/http/retry_policy.cpp:146:34 openssl#27 0x561392b08ed7 in Azure::Core::Http::Policies::_internal::TelemetryPolicy::Send(Azure::Core::Http::Request&, Azure::Core::Http::Policies::NextHttpPolicy, Azure::Core::Context const&) const build_docker/./contrib/azure/sdk/core/azure-core/src/http/telemetry_policy.cpp:23:21 openssl#28 0x561392ae2fb5 in Azure::Core::Http::Policies::_internal::RequestIdPolicy::Send(Azure::Core::Http::Request&, Azure::Core::Http::Policies::NextHttpPolicy, Azure::Core::Context const&) const build_docker/./contrib/azure/sdk/core/azure-core/inc/azure/core/http/policies/policy.hpp:453:27 openssl#29 0x561392b2e5fe in Azure::Storage::_internal::StorageServiceVersionPolicy::Send(Azure::Core::Http::Request&, Azure::Core::Http::Policies::NextHttpPolicy, Azure::Core::Context const&) const build_docker/./contrib/azure/sdk/storage/azure-storage-common/inc/azure/storage/common/internal/storage_service_version_policy.hpp:34:25 openssl#30 0x561392b7e0b1 in Azure::Core::Http::_internal::HttpPipeline::Send(Azure::Core::Http::Request&, Azure::Core::Context const&) const build_docker/./contrib/azure/sdk/core/azure-core/inc/azure/core/internal/http/pipeline.hpp:230:29 openssl#31 0x561392b7e0b1 in Azure::Storage::Blobs::_detail::BlobContainerClient::Create(Azure::Core::Http::_internal::HttpPipeline&, Azure::Core::Url const&, Azure::Storage::Blobs::_detail::BlobContainerClient::CreateBlobContainerOptions const&, Azure::Core::Context const&) build_docker/./contrib/azure/sdk/storage/azure-storage-blobs/src/rest_client.cpp:1415:36 openssl#32 0x561392b3de98 in Azure::Storage::Blobs::BlobContainerClient::Create(Azure::Storage::Blobs::CreateBlobContainerOptions const&, Azure::Core::Context const&) const build_docker/./contrib/azure/sdk/storage/azure-storage-blobs/src/blob_container_client.cpp:258:12 ``` - at database startup, Azure is registered as an object storage (--> registerAzureObjectStorage) - this calls into Azure, then into curl, and then into OpenSSL - curl asks OpenSSL for a bunch of random numbers (--> function 'ossl_random' in curl's OpenSSL wrapper) - OpenSSL initializes the random number generator and stores it in some random number generator context object (--> *RAND_get0_private) - this object is registered via pthread_key_create and pthread_setspecific registered in TLS - if registerAzureObjectStorage was the only place which initializes the RNG, we could argue that the leaked memory does not matter anyways as it is released after shutdown - RAND_get0_public also registers a free handler (rand_delete_thread_state) that runs in TLS, so the memory is released also if registration is called from arbitrary other threads. In sum: this is a false positive. As a result, the fix is similar to earlier fix 5c4b034

This happens usually if an template object is created and there is an out of memory error before the ASN1_OP_NEW_POST method is called, but asn1_item_embed_free calls now the ASN1_OP_FREE_POST which may crash because the object is not properly initialized. Apparently that is only an issue with the ASN1_OP_FREE_POST handling of crypot/x509/x_crl.c, which ought to be tolerant to incomplete initialized objects. The error can be reproduced with the reproducible error injection patch: $ ERROR_INJECT=1652890550 ../util/shlib_wrap.sh ./asn1-test ./corpora/asn1/0ff17293911f54d1538b9896563a4048d67d9ee4 #0 0x7faae9dbeeba in __sanitizer_print_stack_trace ../../../../gcc-trunk/libsanitizer/asan/asan_stack.cpp:87 rogii-com#1 0x408dc4 in my_malloc fuzz/test-corpus.c:114 rogii-com#2 0x7faae99f2430 in CRYPTO_zalloc crypto/mem.c:230 rogii-com#3 0x7faae97f09e5 in ASN1_STRING_type_new crypto/asn1/asn1_lib.c:341 rogii-com#4 0x7faae98118f7 in asn1_primitive_new crypto/asn1/tasn_new.c:318 rogii-com#5 0x7faae9812401 in asn1_item_embed_new crypto/asn1/tasn_new.c:78 openssl#6 0x7faae9812401 in asn1_template_new crypto/asn1/tasn_new.c:240 openssl#7 0x7faae9812315 in asn1_item_embed_new crypto/asn1/tasn_new.c:137 openssl#8 0x7faae9812315 in asn1_template_new crypto/asn1/tasn_new.c:240 openssl#9 0x7faae9812a54 in asn1_item_embed_new crypto/asn1/tasn_new.c:137 openssl#10 0x7faae9812a54 in ASN1_item_ex_new crypto/asn1/tasn_new.c:39 openssl#11 0x7faae980be51 in asn1_item_embed_d2i crypto/asn1/tasn_dec.c:325 openssl#12 0x7faae980c813 in asn1_template_noexp_d2i crypto/asn1/tasn_dec.c:611 openssl#13 0x7faae980d288 in asn1_template_ex_d2i crypto/asn1/tasn_dec.c:518 openssl#14 0x7faae980b9ce in asn1_item_embed_d2i crypto/asn1/tasn_dec.c:382 openssl#15 0x7faae980caf5 in asn1_template_noexp_d2i crypto/asn1/tasn_dec.c:643 openssl#16 0x7faae980d7d3 in asn1_template_ex_d2i crypto/asn1/tasn_dec.c:494 openssl#17 0x7faae980b9ce in asn1_item_embed_d2i crypto/asn1/tasn_dec.c:382 openssl#18 0x7faae980dd1f in ASN1_item_ex_d2i crypto/asn1/tasn_dec.c:124 openssl#19 0x7faae980de35 in ASN1_item_d2i crypto/asn1/tasn_dec.c:114 openssl#20 0x40712c in FuzzerTestOneInput fuzz/asn1.c:301 openssl#21 0x40893b in testfile fuzz/test-corpus.c:182 openssl#22 0x406b86 in main fuzz/test-corpus.c:226 openssl#23 0x7faae8eb1f44 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x21f44) AddressSanitizer:DEADLYSIGNAL ================================================================= ==1194==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000010 (pc 0x7faae9b0625f bp 0x7fffffe41a00 sp 0x7fffffe41920 T0) ==1194==The signal is caused by a READ memory access. ==1194==Hint: address points to the zero page. #0 0x7faae9b0625f in crl_cb crypto/x509/x_crl.c:258 rogii-com#1 0x7faae9811255 in asn1_item_embed_free crypto/asn1/tasn_fre.c:113 rogii-com#2 0x7faae9812a65 in asn1_item_embed_new crypto/asn1/tasn_new.c:150 rogii-com#3 0x7faae9812a65 in ASN1_item_ex_new crypto/asn1/tasn_new.c:39 rogii-com#4 0x7faae980be51 in asn1_item_embed_d2i crypto/asn1/tasn_dec.c:325 rogii-com#5 0x7faae980c813 in asn1_template_noexp_d2i crypto/asn1/tasn_dec.c:611 openssl#6 0x7faae980d288 in asn1_template_ex_d2i crypto/asn1/tasn_dec.c:518 openssl#7 0x7faae980b9ce in asn1_item_embed_d2i crypto/asn1/tasn_dec.c:382 openssl#8 0x7faae980caf5 in asn1_template_noexp_d2i crypto/asn1/tasn_dec.c:643 openssl#9 0x7faae980d7d3 in asn1_template_ex_d2i crypto/asn1/tasn_dec.c:494 openssl#10 0x7faae980b9ce in asn1_item_embed_d2i crypto/asn1/tasn_dec.c:382 openssl#11 0x7faae980dd1f in ASN1_item_ex_d2i crypto/asn1/tasn_dec.c:124 openssl#12 0x7faae980de35 in ASN1_item_d2i crypto/asn1/tasn_dec.c:114 openssl#13 0x40712c in FuzzerTestOneInput fuzz/asn1.c:301 openssl#14 0x40893b in testfile fuzz/test-corpus.c:182 openssl#15 0x406b86 in main fuzz/test-corpus.c:226 openssl#16 0x7faae8eb1f44 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x21f44) AddressSanitizer can not provide additional info. SUMMARY: AddressSanitizer: SEGV crypto/x509/x_crl.c:258 in crl_cb ==1194==ABORTING Reviewed-by: Matt Caswell <matt@openssl.org> Reviewed-by: Tomas Mraz <tomas@openssl.org> (Merged from openssl#18360) (cherry picked from commit 557825a)

This can be reproduced with my error injection patch. The test vector has been validated on the 1.1.1 branch but the issue is of course identical in all branches. $ ERROR_INJECT=1653520461 ../util/shlib_wrap.sh ./cms-test ./corpora/cms/3eff1d2f1232bd66d5635db2c3f9e7f23830dfd1 log file: cms-3eff1d2f1232bd66d5635db2c3f9e7f23830dfd1-32454-test.out ERROR_INJECT=1653520461 #0 0x7fd5d8b8eeba in __sanitizer_print_stack_trace ../../../../gcc-trunk/libsanitizer/asan/asan_stack.cpp:87 rogii-com#1 0x402fc4 in my_realloc fuzz/test-corpus.c:129 rogii-com#2 0x7fd5d8893c49 in sk_reserve crypto/stack/stack.c:198 rogii-com#3 0x7fd5d8893c49 in OPENSSL_sk_insert crypto/stack/stack.c:242 rogii-com#4 0x7fd5d88d6d7f in sk_GENERAL_NAMES_push include/openssl/x509v3.h:168 rogii-com#5 0x7fd5d88d6d7f in crl_set_issuers crypto/x509/x_crl.c:111 openssl#6 0x7fd5d88d6d7f in crl_cb crypto/x509/x_crl.c:246 openssl#7 0x7fd5d85dc032 in asn1_item_embed_d2i crypto/asn1/tasn_dec.c:432 openssl#8 0x7fd5d85dcaf5 in asn1_template_noexp_d2i crypto/asn1/tasn_dec.c:643 openssl#9 0x7fd5d85dd288 in asn1_template_ex_d2i crypto/asn1/tasn_dec.c:518 openssl#10 0x7fd5d85db2b5 in asn1_item_embed_d2i crypto/asn1/tasn_dec.c:259 openssl#11 0x7fd5d85dc813 in asn1_template_noexp_d2i crypto/asn1/tasn_dec.c:611 openssl#12 0x7fd5d85dd288 in asn1_template_ex_d2i crypto/asn1/tasn_dec.c:518 openssl#13 0x7fd5d85db9ce in asn1_item_embed_d2i crypto/asn1/tasn_dec.c:382 openssl#14 0x7fd5d85dca28 in asn1_template_noexp_d2i crypto/asn1/tasn_dec.c:633 openssl#15 0x7fd5d85dd288 in asn1_template_ex_d2i crypto/asn1/tasn_dec.c:518 openssl#16 0x7fd5d85db9ce in asn1_item_embed_d2i crypto/asn1/tasn_dec.c:382 openssl#17 0x7fd5d85dcaf5 in asn1_template_noexp_d2i crypto/asn1/tasn_dec.c:643 openssl#18 0x7fd5d85dd7d3 in asn1_template_ex_d2i crypto/asn1/tasn_dec.c:494 openssl#19 0x7fd5d85db9ce in asn1_item_embed_d2i crypto/asn1/tasn_dec.c:382 openssl#20 0x7fd5d85ddd1f in ASN1_item_ex_d2i crypto/asn1/tasn_dec.c:124 openssl#21 0x7fd5d85dde35 in ASN1_item_d2i crypto/asn1/tasn_dec.c:114 openssl#22 0x7fd5d85a77e0 in ASN1_item_d2i_bio crypto/asn1/a_d2i_fp.c:69 openssl#23 0x402845 in FuzzerTestOneInput fuzz/cms.c:43 openssl#24 0x402bbb in testfile fuzz/test-corpus.c:182 openssl#25 0x402626 in main fuzz/test-corpus.c:226 openssl#26 0x7fd5d7c81f44 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x21f44) openssl#27 0x402706 (/home/ed/OPC/openssl/fuzz/cms-test+0x402706) ================================================================= ==29625==ERROR: LeakSanitizer: detected memory leaks Direct leak of 32 byte(s) in 1 object(s) allocated from: #0 0x7fd5d8b8309f in __interceptor_malloc ../../../../gcc-trunk/libsanitizer/asan/asan_malloc_linux.cpp:69 rogii-com#1 0x7fd5d87c2430 in CRYPTO_zalloc crypto/mem.c:230 rogii-com#2 0x7fd5d889501f in OPENSSL_sk_new_reserve crypto/stack/stack.c:209 rogii-com#3 0x7fd5d85dcbc3 in sk_ASN1_VALUE_new_null include/openssl/asn1t.h:928 rogii-com#4 0x7fd5d85dcbc3 in asn1_template_noexp_d2i crypto/asn1/tasn_dec.c:577 rogii-com#5 0x7fd5d85dd288 in asn1_template_ex_d2i crypto/asn1/tasn_dec.c:518 openssl#6 0x7fd5d85db104 in asn1_item_embed_d2i crypto/asn1/tasn_dec.c:178 openssl#7 0x7fd5d85ddd1f in ASN1_item_ex_d2i crypto/asn1/tasn_dec.c:124 openssl#8 0x7fd5d85dde35 in ASN1_item_d2i crypto/asn1/tasn_dec.c:114 openssl#9 0x7fd5d88f86d9 in X509V3_EXT_d2i crypto/x509v3/v3_lib.c:142 openssl#10 0x7fd5d88d6d3c in crl_set_issuers crypto/x509/x_crl.c:97 openssl#11 0x7fd5d88d6d3c in crl_cb crypto/x509/x_crl.c:246 openssl#12 0x7fd5d85dc032 in asn1_item_embed_d2i crypto/asn1/tasn_dec.c:432 openssl#13 0x7fd5d85dcaf5 in asn1_template_noexp_d2i crypto/asn1/tasn_dec.c:643 openssl#14 0x7fd5d85dd288 in asn1_template_ex_d2i crypto/asn1/tasn_dec.c:518 openssl#15 0x7fd5d85db2b5 in asn1_item_embed_d2i crypto/asn1/tasn_dec.c:259 openssl#16 0x7fd5d85dc813 in asn1_template_noexp_d2i crypto/asn1/tasn_dec.c:611 openssl#17 0x7fd5d85dd288 in asn1_template_ex_d2i crypto/asn1/tasn_dec.c:518 openssl#18 0x7fd5d85db9ce in asn1_item_embed_d2i crypto/asn1/tasn_dec.c:382 openssl#19 0x7fd5d85dca28 in asn1_template_noexp_d2i crypto/asn1/tasn_dec.c:633 openssl#20 0x7fd5d85dd288 in asn1_template_ex_d2i crypto/asn1/tasn_dec.c:518 openssl#21 0x7fd5d85db9ce in asn1_item_embed_d2i crypto/asn1/tasn_dec.c:382 openssl#22 0x7fd5d85dcaf5 in asn1_template_noexp_d2i crypto/asn1/tasn_dec.c:643 openssl#23 0x7fd5d85dd7d3 in asn1_template_ex_d2i crypto/asn1/tasn_dec.c:494 openssl#24 0x7fd5d85db9ce in asn1_item_embed_d2i crypto/asn1/tasn_dec.c:382 openssl#25 0x7fd5d85ddd1f in ASN1_item_ex_d2i crypto/asn1/tasn_dec.c:124 openssl#26 0x7fd5d85dde35 in ASN1_item_d2i crypto/asn1/tasn_dec.c:114 openssl#27 0x7fd5d85a77e0 in ASN1_item_d2i_bio crypto/asn1/a_d2i_fp.c:69 openssl#28 0x402845 in FuzzerTestOneInput fuzz/cms.c:43 openssl#29 0x402bbb in testfile fuzz/test-corpus.c:182 openssl#30 0x402626 in main fuzz/test-corpus.c:226 openssl#31 0x7fd5d7c81f44 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x21f44) SUMMARY: AddressSanitizer: 32 byte(s) leaked in 1 allocation(s). Reviewed-by: Paul Dale <pauli@openssl.org> Reviewed-by: Tomas Mraz <tomas@openssl.org> (Merged from openssl#18391) (cherry picked from commit e9007e0)

This can be reproduced with my error injection patch. The test vector has been validated on the 1.1.1 branch but the issue is of course identical in all branches. $ ERROR_INJECT=1656112173 ../util/shlib_wrap.sh ./x509-test ./corpora/x509/fe543a8d7e09109a9a08114323eefec802ad79e2 #0 0x7fb61945eeba in __sanitizer_print_stack_trace ../../../../gcc-trunk/libsanitizer/asan/asan_stack.cpp:87 rogii-com#1 0x402f84 in my_malloc fuzz/test-corpus.c:114 rogii-com#2 0x7fb619092430 in CRYPTO_zalloc crypto/mem.c:230 rogii-com#3 0x7fb618ef7561 in bn_expand_internal crypto/bn/bn_lib.c:280 rogii-com#4 0x7fb618ef7561 in bn_expand2 crypto/bn/bn_lib.c:304 rogii-com#5 0x7fb618ef819d in BN_bin2bn crypto/bn/bn_lib.c:454 openssl#6 0x7fb618e7aa13 in asn1_string_to_bn crypto/asn1/a_int.c:503 openssl#7 0x7fb618e7aa13 in ASN1_INTEGER_to_BN crypto/asn1/a_int.c:559 openssl#8 0x7fb618fd8e79 in EC_GROUP_new_from_ecparameters crypto/ec/ec_asn1.c:814 openssl#9 0x7fb618fd98e8 in EC_GROUP_new_from_ecpkparameters crypto/ec/ec_asn1.c:935 openssl#10 0x7fb618fd9aec in d2i_ECPKParameters crypto/ec/ec_asn1.c:966 openssl#11 0x7fb618fdace9 in d2i_ECParameters crypto/ec/ec_asn1.c:1184 openssl#12 0x7fb618fd1fc7 in eckey_type2param crypto/ec/ec_ameth.c:119 openssl#13 0x7fb618fd57b4 in eckey_pub_decode crypto/ec/ec_ameth.c:165 openssl#14 0x7fb6191a9c62 in x509_pubkey_decode crypto/x509/x_pubkey.c:124 openssl#15 0x7fb6191a9e42 in pubkey_cb crypto/x509/x_pubkey.c:46 openssl#16 0x7fb618eac032 in asn1_item_embed_d2i crypto/asn1/tasn_dec.c:432 openssl#17 0x7fb618eacaf5 in asn1_template_noexp_d2i crypto/asn1/tasn_dec.c:643 openssl#18 0x7fb618ead288 in asn1_template_ex_d2i crypto/asn1/tasn_dec.c:518 openssl#19 0x7fb618eab9ce in asn1_item_embed_d2i crypto/asn1/tasn_dec.c:382 openssl#20 0x7fb618eacaf5 in asn1_template_noexp_d2i crypto/asn1/tasn_dec.c:643 openssl#21 0x7fb618ead288 in asn1_template_ex_d2i crypto/asn1/tasn_dec.c:518 openssl#22 0x7fb618eab9ce in asn1_item_embed_d2i crypto/asn1/tasn_dec.c:382 openssl#23 0x7fb618eadd1f in ASN1_item_ex_d2i crypto/asn1/tasn_dec.c:124 openssl#24 0x7fb618eade35 in ASN1_item_d2i crypto/asn1/tasn_dec.c:114 openssl#25 0x40310c in FuzzerTestOneInput fuzz/x509.c:33 openssl#26 0x402afb in testfile fuzz/test-corpus.c:182 openssl#27 0x402656 in main fuzz/test-corpus.c:226 openssl#28 0x7fb618551f44 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x21f44) openssl#29 0x402756 (/home/ed/OPC/openssl/fuzz/x509-test+0x402756) ================================================================= ==12221==ERROR: LeakSanitizer: detected memory leaks Direct leak of 24 byte(s) in 1 object(s) allocated from: #0 0x7fb61945309f in __interceptor_malloc ../../../../gcc-trunk/libsanitizer/asan/asan_malloc_linux.cpp:69 rogii-com#1 0x7fb619092430 in CRYPTO_zalloc crypto/mem.c:230 rogii-com#2 0x7fb618ef5f11 in BN_new crypto/bn/bn_lib.c:246 rogii-com#3 0x7fb618ef82f4 in BN_bin2bn crypto/bn/bn_lib.c:440 rogii-com#4 0x7fb618fd8933 in EC_GROUP_new_from_ecparameters crypto/ec/ec_asn1.c:618 rogii-com#5 0x7fb618fd98e8 in EC_GROUP_new_from_ecpkparameters crypto/ec/ec_asn1.c:935 openssl#6 0x7fb618fd9aec in d2i_ECPKParameters crypto/ec/ec_asn1.c:966 openssl#7 0x7fb618fdace9 in d2i_ECParameters crypto/ec/ec_asn1.c:1184 openssl#8 0x7fb618fd1fc7 in eckey_type2param crypto/ec/ec_ameth.c:119 openssl#9 0x7fb618fd57b4 in eckey_pub_decode crypto/ec/ec_ameth.c:165 openssl#10 0x7fb6191a9c62 in x509_pubkey_decode crypto/x509/x_pubkey.c:124 openssl#11 0x7fb6191a9e42 in pubkey_cb crypto/x509/x_pubkey.c:46 openssl#12 0x7fb618eac032 in asn1_item_embed_d2i crypto/asn1/tasn_dec.c:432 openssl#13 0x7fb618eacaf5 in asn1_template_noexp_d2i crypto/asn1/tasn_dec.c:643 openssl#14 0x7fb618ead288 in asn1_template_ex_d2i crypto/asn1/tasn_dec.c:518 openssl#15 0x7fb618eab9ce in asn1_item_embed_d2i crypto/asn1/tasn_dec.c:382 openssl#16 0x7fb618eacaf5 in asn1_template_noexp_d2i crypto/asn1/tasn_dec.c:643 openssl#17 0x7fb618ead288 in asn1_template_ex_d2i crypto/asn1/tasn_dec.c:518 openssl#18 0x7fb618eab9ce in asn1_item_embed_d2i crypto/asn1/tasn_dec.c:382 openssl#19 0x7fb618eadd1f in ASN1_item_ex_d2i crypto/asn1/tasn_dec.c:124 openssl#20 0x7fb618eade35 in ASN1_item_d2i crypto/asn1/tasn_dec.c:114 openssl#21 0x40310c in FuzzerTestOneInput fuzz/x509.c:33 openssl#22 0x402afb in testfile fuzz/test-corpus.c:182 openssl#23 0x402656 in main fuzz/test-corpus.c:226 openssl#24 0x7fb618551f44 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x21f44) Indirect leak of 56 byte(s) in 1 object(s) allocated from: #0 0x7fb61945309f in __interceptor_malloc ../../../../gcc-trunk/libsanitizer/asan/asan_malloc_linux.cpp:69 rogii-com#1 0x7fb619092430 in CRYPTO_zalloc crypto/mem.c:230 rogii-com#2 0x7fb618ef7561 in bn_expand_internal crypto/bn/bn_lib.c:280 rogii-com#3 0x7fb618ef7561 in bn_expand2 crypto/bn/bn_lib.c:304 rogii-com#4 0x7fb618ef819d in BN_bin2bn crypto/bn/bn_lib.c:454 rogii-com#5 0x7fb618fd8933 in EC_GROUP_new_from_ecparameters crypto/ec/ec_asn1.c:618 openssl#6 0x7fb618fd98e8 in EC_GROUP_new_from_ecpkparameters crypto/ec/ec_asn1.c:935 openssl#7 0x7fb618fd9aec in d2i_ECPKParameters crypto/ec/ec_asn1.c:966 openssl#8 0x7fb618fdace9 in d2i_ECParameters crypto/ec/ec_asn1.c:1184 openssl#9 0x7fb618fd1fc7 in eckey_type2param crypto/ec/ec_ameth.c:119 openssl#10 0x7fb618fd57b4 in eckey_pub_decode crypto/ec/ec_ameth.c:165 openssl#11 0x7fb6191a9c62 in x509_pubkey_decode crypto/x509/x_pubkey.c:124 openssl#12 0x7fb6191a9e42 in pubkey_cb crypto/x509/x_pubkey.c:46 openssl#13 0x7fb618eac032 in asn1_item_embed_d2i crypto/asn1/tasn_dec.c:432 openssl#14 0x7fb618eacaf5 in asn1_template_noexp_d2i crypto/asn1/tasn_dec.c:643 openssl#15 0x7fb618ead288 in asn1_template_ex_d2i crypto/asn1/tasn_dec.c:518 openssl#16 0x7fb618eab9ce in asn1_item_embed_d2i crypto/asn1/tasn_dec.c:382 openssl#17 0x7fb618eacaf5 in asn1_template_noexp_d2i crypto/asn1/tasn_dec.c:643 openssl#18 0x7fb618ead288 in asn1_template_ex_d2i crypto/asn1/tasn_dec.c:518 openssl#19 0x7fb618eab9ce in asn1_item_embed_d2i crypto/asn1/tasn_dec.c:382 openssl#20 0x7fb618eadd1f in ASN1_item_ex_d2i crypto/asn1/tasn_dec.c:124 openssl#21 0x7fb618eade35 in ASN1_item_d2i crypto/asn1/tasn_dec.c:114 openssl#22 0x40310c in FuzzerTestOneInput fuzz/x509.c:33 openssl#23 0x402afb in testfile fuzz/test-corpus.c:182 openssl#24 0x402656 in main fuzz/test-corpus.c:226 openssl#25 0x7fb618551f44 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x21f44) SUMMARY: AddressSanitizer: 80 byte(s) leaked in 2 allocation(s). Reviewed-by: Tomas Mraz <tomas@openssl.org> Reviewed-by: Kurt Roeckx <kurt@roeckx.be> (Merged from openssl#18632)