Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

first cut removing default algs, biker2, adding biker3 #26

Merged
merged 4 commits into from
Jul 10, 2021
Merged

first cut removing default algs, biker2, adding biker3 #26

merged 4 commits into from
Jul 10, 2021

Conversation

baentsch
Copy link
Member

@baentsch baentsch commented Jul 8, 2021 •
edited
Loading

Aims to fix #27

@baentsch baentsch mentioned this pull request Jul 8, 2021
Closed
@baentsch baentsch marked this pull request as ready for review July 8, 2021 13:00
@baentsch baentsch requested a review from bhess July 8, 2021 13:01
@dstebila
Copy link
Member

dstebila commented Jul 8, 2021

Are the NIDs in generate_extras.yml manually synced with the NIDs table in the main OQS-OpenSSL repo?

@baentsch
Copy link
Member Author

baentsch commented Jul 8, 2021

Are the NIDs in generate_extras.yml manually synced with the NIDs table in the main OQS-OpenSSL repo?

No. See the discussion in #27. They now simply moved 0x40 up (and away from the "official" code points -- my mistake to call these NIDs).

@baentsch baentsch requested a review from dstebila July 8, 2021 14:52
bhess
bhess reviewed Jul 8, 2021
View reviewed changes
Copy link
Member

@bhess bhess left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are the NIDs in generate_extras.yml manually synced with the NIDs table in the main OQS-OpenSSL repo?

The code points in generate_extras.yml were for hybrid combinations with x25519/x448, which OQS-OpenSSL111 didn't support, so there wasn't any syncing necessary. Once OQS-OpenSSL111 supports all the extra combinations, the generate_extras.yml will become obsolete. In the meanwhile I think it's best to keep the generate_extras.yml.

The cleanest way could be to fetch the available x25519/x448 code points from the OSSL111 yml and only provide extra ones in the local generate_extras.yml.

oqs-template/generate-extras.yml Outdated Show resolved Hide resolved
bhess
bhess reviewed Jul 8, 2021
View reviewed changes
Copy link
Member

@bhess bhess left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

#26 (comment)
...
yes, the nid_ecx_hybrid fields for kyber512, sikep434 and bike are already defined in OQS-OSSL111 generate.yml and yes, it would be cleaner to drop them from the generate-extras.yml. I will commit to this branch tomorrow with a proposed update.

oqs-template/generate-extras.yml Outdated Show resolved Hide resolved
oqs-template/generate-extras.yml Outdated Show resolved Hide resolved
@bhess
Updates generate.py to resolve and avoid conflicting ecx-hybrid pairs.
a02569b 
Removes sikep434 and kyber512 ecx pairs from generate-extras.yml (they are defined in the OQS-OpenSSL generate.yml).
@bhess
Copy link
Member

bhess commented Jul 9, 2021

a02569b changes generate.py so that it gives priority to the extra code points for ecx from OQS-OpenSSL generate.yml. Also removes the obsolete ones for sikep434 and kyber512 ecx-hybrids from generate-extras.yml.
If there are duplicate/conflicting entries in generate-extras.yml, a warning is printed during a generate.py run.

running the interop tests at the moment...

@bhess
Copy link
Member

bhess commented Jul 9, 2021

running the interop tests at the moment...

The BIKE and Kyber interop tests against test.openquantumsafe.org still fail. The server might not yet accept the new code points.

@baentsch
Copy link
Member Author

baentsch commented Jul 9, 2021

The server might not yet accept the new code points.

It doesn't: "This corresponds to the OQS release version 0.6.0-rc3 ". Let me upgrade it now that oqs-demos seems to mostly build again (except oqs-provider....)

@baentsch
Copy link
Member Author

baentsch commented Jul 9, 2021

@bhess test.openquantumsafe.org now running 0.7.0-dev code: Interop tests should pass now. If so, please approve PR so that also oqs-demos can pass in toto.

@bhess bhess self-requested a review July 9, 2021 14:23
bhess
bhess approved these changes Jul 9, 2021
View reviewed changes
Copy link
Member

@bhess bhess left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, looks good to me.
Interop tests seem to work fine. Just the assignments.json entries seem to be off by a few ports (http://test.openquantumsafe.org/assignments.json) E.g. frodo640aes is listed under port 6003, but it is actually port 6001. However, I tried a few ports manually and it looks to work fine with oqs-provider.

@baentsch
Copy link
Member Author

assignments.json entries seem to be off by a few ports

Thanks for the report: For some reason, the new JSON didn't get moved into the download folder. Now fixed. Thanks also for the review, fix and approval. Now settled.

@baentsch baentsch merged commit 040a10d into main Jul 10, 2021
@baentsch baentsch deleted the mb-060update branch July 10, 2021 05:05
@baentsch
Copy link
Member Author

@bhess -- in implementing #111 I noticed "surprising" ID choices for BIKE x-hybrids in https://github.com/open-quantum-safe/oqs-provider/blob/main/oqs-template/generate.yml. How did you make those choices? Second question: Can't we completely do away with https://github.com/open-quantum-safe/oqs-provider/blob/main/oqs-template/generate-extras.yml?

@bhess
Copy link
Member

bhess commented Feb 13, 2023

I think the BIKE x-hybrids were added in open-quantum-safe/openssl#313 but I don't know how the ID/code point choice was made.

The code points I added were the extra ones for the provider in https://github.com/open-quantum-safe/oqs-provider/blob/main/oqs-template/generate-extras.yml. The idea was to express the ones separately that are only supported in oqs-provider. It seems you now detached the generate.yml from oqs-openssl, so I also think that generate.yml and generate-extras.yml can be merged.

@baentsch
Copy link
Member Author

baentsch commented Feb 13, 2023 •
edited
Loading

Thanks for the pointer to open-quantum-safe/openssl#313: I didn't find that. Indeed, one generate.yml would be good. We can do a YML-cross-comparator if there is still an interest to maintain oqs-openssl111 (and keep it in sync with oqs-provider). I'll remove generate-extras.yml then when resolving #111. Edit/Add: Tagging @dstebila FYI (and asking how the X-hybrid IDs were chosen).

thb-sb added a commit to thb-sb/oqs-provider that referenced this pull request Sep 5, 2023
@thb-sb
Fix various memory leaks.
70faa7d 
This commit fixes various memory leaks that have been found using ASan.


1. Memory leak in `oqs_test_signatures`.
===

In `oqs_test_signatures`, [`OPENSSL_free`] was used with `EVP_PKEY_CTX`. However,
the right destructor function for `EVP_PKEY_CTX` is [`EVP_PKEY_CTX_free`].

This commmit fixes this minor bug.

[`OPENSSL_free`](https://www.openssl.org/docs/man3.0/man3/OPENSSL_free.html)
[`EVP_PKEY_CTX_free`](https://www.openssl.org/docs/man3.0/man3/EVP_PKEY_CTX_free.html)


2. a selftest key that wasn't freed.
===

In `oqsprov/oqsprov_keys.c`, `d2i_PrivateKey` is used to verify that we the
key we've just generated is sound.
However, the returned object wasn't freed.
ASan trace that helped me finding this one:

```
Indirect leak of 96 byte(s) in 4 object(s) allocated from:
    #0 0x5651eef1428e in malloc (/home/pawn/work/oqs-provider/build/test/oqs_test_signatures+0xb928e) (BuildId: 6cc7ebd4bc73a0c78b2ab8d7b9dcb8b165d88e74)
    open-quantum-safe#1 0x7fb7a23a946b in CRYPTO_malloc /home/pawn/work/openssl/crypto/mem.c:190:12
    open-quantum-safe#2 0x7fb7a23a94a2 in CRYPTO_zalloc /home/pawn/work/openssl/crypto/mem.c:197:11
    open-quantum-safe#3 0x7fb7a209cdcd in BN_new /home/pawn/work/openssl/crypto/bn/bn_lib.c:247:16
    open-quantum-safe#4 0x7fb7a2224a00 in ossl_ec_GFp_mont_group_set_curve /home/pawn/work/openssl/crypto/ec/ecp_mont.c:169:11
    open-quantum-safe#5 0x7fb7a220f5fa in EC_GROUP_set_curve /home/pawn/work/openssl/crypto/ec/ec_lib.c:562:12
    open-quantum-safe#6 0x7fb7a2202d75 in EC_GROUP_new_curve_GFp /home/pawn/work/openssl/crypto/ec/ec_cvt.c:61:10
    open-quantum-safe#7 0x7fb7a220184d in ec_group_new_from_data /home/pawn/work/openssl/crypto/ec/ec_curve.c:3179:22
    open-quantum-safe#8 0x7fb7a2200ff9 in EC_GROUP_new_by_curve_name_ex /home/pawn/work/openssl/crypto/ec/ec_curve.c:3287:19
    open-quantum-safe#9 0x7fb7a2201ed9 in EC_GROUP_new_by_curve_name /home/pawn/work/openssl/crypto/ec/ec_curve.c:3303:12
    open-quantum-safe#10 0x7fb7a21f84e2 in EC_GROUP_new_from_ecpkparameters /home/pawn/work/openssl/crypto/ec/ec_asn1.c:863:20
    open-quantum-safe#11 0x7fb7a21f8f28 in d2i_ECPrivateKey /home/pawn/work/openssl/crypto/ec/ec_asn1.c:956:22
    open-quantum-safe#12 0x7fb7a270b4b5 in der2key_decode /home/pawn/work/openssl/providers/implementations/encode_decode/decode_der2key.c:220:19
    open-quantum-safe#13 0x7fb7a226eae8 in decoder_process /home/pawn/work/openssl/crypto/encode_decode/decoder_lib.c:971:14
    open-quantum-safe#14 0x7fb7a226d106 in OSSL_DECODER_from_bio /home/pawn/work/openssl/crypto/encode_decode/decoder_lib.c:82:10
    open-quantum-safe#15 0x7fb7a226f007 in OSSL_DECODER_from_data /home/pawn/work/openssl/crypto/encode_decode/decoder_lib.c:157:9
    open-quantum-safe#16 0x7fb7a200e123 in d2i_PrivateKey_decoder /home/pawn/work/openssl/crypto/asn1/d2i_pr.c:74:11
    open-quantum-safe#17 0x7fb7a200dbba in d2i_PrivateKey_ex /home/pawn/work/openssl/crypto/asn1/d2i_pr.c:162:11
    open-quantum-safe#18 0x7fb7a200e3d2 in d2i_PrivateKey /home/pawn/work/openssl/crypto/asn1/d2i_pr.c:172:12
    open-quantum-safe#19 0x7fb79dc6a22e in oqsx_key_gen_evp_key /home/pawn/work/oqs-provider/oqsprov/oqsprov_keys.c:1036:25
    open-quantum-safe#20 0x7fb79dc68f3f in oqsx_key_gen /home/pawn/work/oqs-provider/oqsprov/oqsprov_keys.c:1077:16
    open-quantum-safe#21 0x7fb79dc7164a in oqsx_genkey /home/pawn/work/oqs-provider/oqsprov/oqs_kmgmt.c:496:9
    open-quantum-safe#22 0x7fb79dc6d9f8 in oqsx_gen /home/pawn/work/oqs-provider/oqsprov/oqs_kmgmt.c:509:12
    open-quantum-safe#23 0x7fb7a23373c7 in evp_keymgmt_gen /home/pawn/work/openssl/crypto/evp/keymgmt_meth.c:366:12
    open-quantum-safe#24 0x7fb7a2334b58 in evp_keymgmt_util_gen /home/pawn/work/openssl/crypto/evp/keymgmt_lib.c:517:20
    open-quantum-safe#25 0x7fb7a235a775 in EVP_PKEY_generate /home/pawn/work/openssl/crypto/evp/pmeth_gn.c:189:13
    open-quantum-safe#26 0x5651eef4f2bd in test_oqs_signatures /home/pawn/work/oqs-provider/test/oqs_test_signatures.c:39:48
    open-quantum-safe#27 0x5651eef4ee3d in main /home/pawn/work/oqs-provider/test/oqs_test_signatures.c:114:17
    open-quantum-safe#28 0x7fb7a1b671c9 in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
```

3. a `EVP_PKEY_CTX` that wasn't freed.
===

In `oqsx_key_free`, `key->oqsx_provider_ctx.oqsx_evp_ctx->ctx` is freed if
the keytype is `ECP_HYB_KEM` or `ECX_HYB_KEM`. However,
`key->oqsx_provider_ctx.oqsx_evp_ctx->ctx` is also used for signatures.
In that case, a call to `EVP_PKEY_CTX_free` was missing.
ASan trace that helped me finding this one:

```
Indirect leak of 7 byte(s) in 2 object(s) allocated from:
    #0 0x55b6fc8d427e in malloc (/home/pawn/work/oqs-provider/build/test/oqs_test_signatures+0xb927e) (BuildId: 3a8bff15d3315ba7ac1746fdc7b67eb464c607a1)
    open-quantum-safe#1 0x7ff34dfa946b in CRYPTO_malloc /home/pawn/work/openssl/crypto/mem.c:190:12
    open-quantum-safe#2 0x7ff34dfac50a in CRYPTO_strndup /home/pawn/work/openssl/crypto/o_str.c:43:11
    open-quantum-safe#3 0x7ff34df9d152 in ossl_algorithm_get1_first_name /home/pawn/work/openssl/crypto/core_algorithm.c:195:11
    open-quantum-safe#4 0x7ff34df353a1 in keymgmt_from_algorithm /home/pawn/work/openssl/crypto/evp/keymgmt_meth.c:50:31
    open-quantum-safe#5 0x7ff34df15e44 in construct_evp_method /home/pawn/work/openssl/crypto/evp/evp_fetch.c:219:14
    open-quantum-safe#6 0x7ff34df9e3ce in ossl_method_construct_this /home/pawn/work/openssl/crypto/core_fetch.c:109:19
    open-quantum-safe#7 0x7ff34df9d740 in algorithm_do_map /home/pawn/work/openssl/crypto/core_algorithm.c:77:13
    open-quantum-safe#8 0x7ff34df9ce88 in algorithm_do_this /home/pawn/work/openssl/crypto/core_algorithm.c:122:15
    open-quantum-safe#9 0x7ff34dfcce3e in ossl_provider_doall_activated /home/pawn/work/openssl/crypto/provider_core.c:1423:14
    open-quantum-safe#10 0x7ff34df9ca64 in ossl_algorithm_do_all /home/pawn/work/openssl/crypto/core_algorithm.c:162:9
    open-quantum-safe#11 0x7ff34df9dd64 in ossl_method_construct /home/pawn/work/openssl/crypto/core_fetch.c:153:5
    open-quantum-safe#12 0x7ff34df13b6f in inner_evp_generic_fetch /home/pawn/work/openssl/crypto/evp/evp_fetch.c:312:23
    open-quantum-safe#13 0x7ff34df13226 in evp_generic_fetch /home/pawn/work/openssl/crypto/evp/evp_fetch.c:364:14
    open-quantum-safe#14 0x7ff34df36a42 in EVP_KEYMGMT_fetch /home/pawn/work/openssl/crypto/evp/keymgmt_meth.c:220:12
    open-quantum-safe#15 0x7ff34df5d6e4 in int_ctx_new /home/pawn/work/openssl/crypto/evp/pmeth_lib.c:280:23
    open-quantum-safe#16 0x7ff34df5f0a9 in EVP_PKEY_CTX_new_id /home/pawn/work/openssl/crypto/evp/pmeth_lib.c:469:12
    open-quantum-safe#17 0x7ff349865bcb in oqsx_hybsig_init /home/pawn/work/oqs-provider/oqsprov/oqsprov_keys.c:578:20
    open-quantum-safe#18 0x7ff349865157 in oqsx_key_new /home/pawn/work/oqs-provider/oqsprov/oqsprov_keys.c:791:16
    open-quantum-safe#19 0x7ff3498715a3 in oqsx_genkey /home/pawn/work/oqs-provider/oqsprov/oqs_kmgmt.c:487:16
    open-quantum-safe#20 0x7ff34986da08 in oqsx_gen /home/pawn/work/oqs-provider/oqsprov/oqs_kmgmt.c:509:12
    open-quantum-safe#21 0x7ff34df373c7 in evp_keymgmt_gen /home/pawn/work/openssl/crypto/evp/keymgmt_meth.c:366:12
    open-quantum-safe#22 0x7ff34df34b58 in evp_keymgmt_util_gen /home/pawn/work/openssl/crypto/evp/keymgmt_lib.c:517:20
    open-quantum-safe#23 0x7ff34df5a775 in EVP_PKEY_generate /home/pawn/work/openssl/crypto/evp/pmeth_gn.c:189:13
    open-quantum-safe#24 0x55b6fc90f2ad in test_oqs_signatures /home/pawn/work/oqs-provider/test/oqs_test_signatures.c:39:48
    open-quantum-safe#25 0x55b6fc90ee2d in main /home/pawn/work/oqs-provider/test/oqs_test_signatures.c:114:17
    open-quantum-safe#26 0x7ff34d8461c9 in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
```

4. Missing `EVP_PKEY_CTX_free` in case of error.
===

In `oqsx_hybsig_init`, a `EVP_PKEY_CTX` is initialized using `EVP_PKEY_CTX_new_id`.
However, later in the code, some extra work is done, and in case of error, the
`EVP_PKEY_CTX` wasn't freed.

(I couldn't recover the ASan message for this one…)


5. `OPENSSL_free` instead of `EVP_PKEY_free`.
===

In `oqsx_key_gen`, when a classical private key is generated, the field
`classical_pkey` of a `OQSX_KEY` is set with a value of type `EVP_PKEY`.

However, in `oqsx_key_free`, `OPENSSL_free` was used on `classical_pkey`
instead of `EVP_PKEY_free`.

ASan trace that helped me finding this one:

```
   #0 0x555a6b1b728e in malloc (/home/pawn/work/oqs-provider/build/test/oqs_test_signatures+0xb928e) (BuildId: 6cc7ebd4bc73a0c78b2ab8d7b9dcb8b165d88e74)
    open-quantum-safe#1 0x7f59a93a946b in CRYPTO_malloc /home/pawn/work/openssl/crypto/mem.c:190:12
    open-quantum-safe#2 0x7f59a93a94a2 in CRYPTO_zalloc /home/pawn/work/openssl/crypto/mem.c:197:11
    open-quantum-safe#3 0x7f59a909cdcd in BN_new /home/pawn/work/openssl/crypto/bn/bn_lib.c:247:16
    open-quantum-safe#4 0x7f59a920b955 in ossl_ec_group_new_ex /home/pawn/work/openssl/crypto/ec/ec_lib.c:61:25
    open-quantum-safe#5 0x7f59a920dd55 in EC_GROUP_dup /home/pawn/work/openssl/crypto/ec/ec_lib.c:273:14
    open-quantum-safe#6 0x7f59a9207774 in EC_KEY_set_group /home/pawn/work/openssl/crypto/ec/ec_key.c:733:18
    open-quantum-safe#7 0x7f59a975b3b5 in ec_gen_assign_group /home/pawn/work/openssl/providers/implementations/keymgmt/ec_kmgmt.c:1236:12
    open-quantum-safe#8 0x7f59a9758bdb in ec_gen /home/pawn/work/openssl/providers/implementations/keymgmt/ec_kmgmt.c:1274:11
    open-quantum-safe#9 0x7f59a93373c7 in evp_keymgmt_gen /home/pawn/work/openssl/crypto/evp/keymgmt_meth.c:366:12
    open-quantum-safe#10 0x7f59a9334b58 in evp_keymgmt_util_gen /home/pawn/work/openssl/crypto/evp/keymgmt_lib.c:517:20
    open-quantum-safe#11 0x7f59a935a775 in EVP_PKEY_generate /home/pawn/work/openssl/crypto/evp/pmeth_gn.c:189:13
    open-quantum-safe#12 0x7f59a935b404 in EVP_PKEY_keygen /home/pawn/work/openssl/crypto/evp/pmeth_gn.c:274:12
    open-quantum-safe#13 0x7f59a4c69a51 in oqsx_key_gen_evp_key /home/pawn/work/oqs-provider/oqsprov/oqsprov_keys.c:1012:12
    open-quantum-safe#14 0x7f59a4c68faf in oqsx_key_gen /home/pawn/work/oqs-provider/oqsprov/oqsprov_keys.c:1084:16
    open-quantum-safe#15 0x7f59a4c716ca in oqsx_genkey /home/pawn/work/oqs-provider/oqsprov/oqs_kmgmt.c:496:9
    open-quantum-safe#16 0x7f59a4c6da78 in oqsx_gen /home/pawn/work/oqs-provider/oqsprov/oqs_kmgmt.c:509:12
    open-quantum-safe#17 0x7f59a93373c7 in evp_keymgmt_gen /home/pawn/work/openssl/crypto/evp/keymgmt_meth.c:366:12
    open-quantum-safe#18 0x7f59a9334b58 in evp_keymgmt_util_gen /home/pawn/work/openssl/crypto/evp/keymgmt_lib.c:517:20
    open-quantum-safe#19 0x7f59a935a775 in EVP_PKEY_generate /home/pawn/work/openssl/crypto/evp/pmeth_gn.c:189:13
    open-quantum-safe#20 0x555a6b1f2890 in test_oqs_signatures /home/pawn/work/oqs-provider/test/oqs_test_signatures.c:68:44
    open-quantum-safe#21 0x555a6b1f1e3d in main /home/pawn/work/oqs-provider/test/oqs_test_signatures.c:114:17
    open-quantum-safe#22 0x7f59a8c461c9 in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
```

6. Bonus point: fix a potential double-free.
===

In `oqsx_key_gen`, when the keytype isn't `KEY_TYPE_HYB_SIG`, the
generated `pkey` is freed, and the flow goes down to the `err` label:

https://github.com/open-quantum-safe/oqs-provider/blob/cc3895f208730abfe1efc15f7c3ff4358ac6e4c0/oqsprov/oqsprov_keys.c#L1089-L1104

There is a potential double-free bug here: if `oqsx_key_gen_oqs` fails, then
`EVP_PKEY_free` will be called twice on the same `pkey` pointer.

Note that it is very unlikely that `oqsx_key_gen_oqs` fails, since it only
calls `OQS_KEM_keypair`/`OQS_SIG_keypair`.
thb-sb added a commit to thb-sb/oqs-provider that referenced this pull request Sep 5, 2023
@thb-sb
Fix various memory leaks.
3fc853a 
This commit fixes various memory leaks that have been found using ASan.

Note: I will soon add a CI job for testing oqs-provider against memory leaks
with ASan.


1. Memory leak in `oqs_test_signatures`.
===

In `oqs_test_signatures`, [`OPENSSL_free`] was used with `EVP_PKEY_CTX`. However,
the right destructor function for `EVP_PKEY_CTX` is [`EVP_PKEY_CTX_free`].

This commmit fixes this minor bug.

[`OPENSSL_free`](https://www.openssl.org/docs/man3.0/man3/OPENSSL_free.html)
[`EVP_PKEY_CTX_free`](https://www.openssl.org/docs/man3.0/man3/EVP_PKEY_CTX_free.html)


2. a selftest key that wasn't freed.
===

In `oqsprov/oqsprov_keys.c`, `d2i_PrivateKey` is used to verify that we the
key we've just generated is sound.
However, the returned object wasn't freed.
ASan trace that helped me finding this one:

```
Indirect leak of 96 byte(s) in 4 object(s) allocated from:
    #0 0x5651eef1428e in malloc (/home/pawn/work/oqs-provider/build/test/oqs_test_signatures+0xb928e) (BuildId: 6cc7ebd4bc73a0c78b2ab8d7b9dcb8b165d88e74)
    open-quantum-safe#1 0x7fb7a23a946b in CRYPTO_malloc /home/pawn/work/openssl/crypto/mem.c:190:12
    open-quantum-safe#2 0x7fb7a23a94a2 in CRYPTO_zalloc /home/pawn/work/openssl/crypto/mem.c:197:11
    open-quantum-safe#3 0x7fb7a209cdcd in BN_new /home/pawn/work/openssl/crypto/bn/bn_lib.c:247:16
    open-quantum-safe#4 0x7fb7a2224a00 in ossl_ec_GFp_mont_group_set_curve /home/pawn/work/openssl/crypto/ec/ecp_mont.c:169:11
    open-quantum-safe#5 0x7fb7a220f5fa in EC_GROUP_set_curve /home/pawn/work/openssl/crypto/ec/ec_lib.c:562:12
    open-quantum-safe#6 0x7fb7a2202d75 in EC_GROUP_new_curve_GFp /home/pawn/work/openssl/crypto/ec/ec_cvt.c:61:10
    open-quantum-safe#7 0x7fb7a220184d in ec_group_new_from_data /home/pawn/work/openssl/crypto/ec/ec_curve.c:3179:22
    open-quantum-safe#8 0x7fb7a2200ff9 in EC_GROUP_new_by_curve_name_ex /home/pawn/work/openssl/crypto/ec/ec_curve.c:3287:19
    open-quantum-safe#9 0x7fb7a2201ed9 in EC_GROUP_new_by_curve_name /home/pawn/work/openssl/crypto/ec/ec_curve.c:3303:12
    open-quantum-safe#10 0x7fb7a21f84e2 in EC_GROUP_new_from_ecpkparameters /home/pawn/work/openssl/crypto/ec/ec_asn1.c:863:20
    open-quantum-safe#11 0x7fb7a21f8f28 in d2i_ECPrivateKey /home/pawn/work/openssl/crypto/ec/ec_asn1.c:956:22
    open-quantum-safe#12 0x7fb7a270b4b5 in der2key_decode /home/pawn/work/openssl/providers/implementations/encode_decode/decode_der2key.c:220:19
    open-quantum-safe#13 0x7fb7a226eae8 in decoder_process /home/pawn/work/openssl/crypto/encode_decode/decoder_lib.c:971:14
    open-quantum-safe#14 0x7fb7a226d106 in OSSL_DECODER_from_bio /home/pawn/work/openssl/crypto/encode_decode/decoder_lib.c:82:10
    open-quantum-safe#15 0x7fb7a226f007 in OSSL_DECODER_from_data /home/pawn/work/openssl/crypto/encode_decode/decoder_lib.c:157:9
    open-quantum-safe#16 0x7fb7a200e123 in d2i_PrivateKey_decoder /home/pawn/work/openssl/crypto/asn1/d2i_pr.c:74:11
    open-quantum-safe#17 0x7fb7a200dbba in d2i_PrivateKey_ex /home/pawn/work/openssl/crypto/asn1/d2i_pr.c:162:11
    open-quantum-safe#18 0x7fb7a200e3d2 in d2i_PrivateKey /home/pawn/work/openssl/crypto/asn1/d2i_pr.c:172:12
    open-quantum-safe#19 0x7fb79dc6a22e in oqsx_key_gen_evp_key /home/pawn/work/oqs-provider/oqsprov/oqsprov_keys.c:1036:25
    open-quantum-safe#20 0x7fb79dc68f3f in oqsx_key_gen /home/pawn/work/oqs-provider/oqsprov/oqsprov_keys.c:1077:16
    open-quantum-safe#21 0x7fb79dc7164a in oqsx_genkey /home/pawn/work/oqs-provider/oqsprov/oqs_kmgmt.c:496:9
    open-quantum-safe#22 0x7fb79dc6d9f8 in oqsx_gen /home/pawn/work/oqs-provider/oqsprov/oqs_kmgmt.c:509:12
    open-quantum-safe#23 0x7fb7a23373c7 in evp_keymgmt_gen /home/pawn/work/openssl/crypto/evp/keymgmt_meth.c:366:12
    open-quantum-safe#24 0x7fb7a2334b58 in evp_keymgmt_util_gen /home/pawn/work/openssl/crypto/evp/keymgmt_lib.c:517:20
    open-quantum-safe#25 0x7fb7a235a775 in EVP_PKEY_generate /home/pawn/work/openssl/crypto/evp/pmeth_gn.c:189:13
    open-quantum-safe#26 0x5651eef4f2bd in test_oqs_signatures /home/pawn/work/oqs-provider/test/oqs_test_signatures.c:39:48
    open-quantum-safe#27 0x5651eef4ee3d in main /home/pawn/work/oqs-provider/test/oqs_test_signatures.c:114:17
    open-quantum-safe#28 0x7fb7a1b671c9 in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
```

3. a `EVP_PKEY_CTX` that wasn't freed.
===

In `oqsx_key_free`, `key->oqsx_provider_ctx.oqsx_evp_ctx->ctx` is freed if
the keytype is `ECP_HYB_KEM` or `ECX_HYB_KEM`. However,
`key->oqsx_provider_ctx.oqsx_evp_ctx->ctx` is also used for signatures.
In that case, a call to `EVP_PKEY_CTX_free` was missing.
ASan trace that helped me finding this one:

```
Indirect leak of 7 byte(s) in 2 object(s) allocated from:
    #0 0x55b6fc8d427e in malloc (/home/pawn/work/oqs-provider/build/test/oqs_test_signatures+0xb927e) (BuildId: 3a8bff15d3315ba7ac1746fdc7b67eb464c607a1)
    open-quantum-safe#1 0x7ff34dfa946b in CRYPTO_malloc /home/pawn/work/openssl/crypto/mem.c:190:12
    open-quantum-safe#2 0x7ff34dfac50a in CRYPTO_strndup /home/pawn/work/openssl/crypto/o_str.c:43:11
    open-quantum-safe#3 0x7ff34df9d152 in ossl_algorithm_get1_first_name /home/pawn/work/openssl/crypto/core_algorithm.c:195:11
    open-quantum-safe#4 0x7ff34df353a1 in keymgmt_from_algorithm /home/pawn/work/openssl/crypto/evp/keymgmt_meth.c:50:31
    open-quantum-safe#5 0x7ff34df15e44 in construct_evp_method /home/pawn/work/openssl/crypto/evp/evp_fetch.c:219:14
    open-quantum-safe#6 0x7ff34df9e3ce in ossl_method_construct_this /home/pawn/work/openssl/crypto/core_fetch.c:109:19
    open-quantum-safe#7 0x7ff34df9d740 in algorithm_do_map /home/pawn/work/openssl/crypto/core_algorithm.c:77:13
    open-quantum-safe#8 0x7ff34df9ce88 in algorithm_do_this /home/pawn/work/openssl/crypto/core_algorithm.c:122:15
    open-quantum-safe#9 0x7ff34dfcce3e in ossl_provider_doall_activated /home/pawn/work/openssl/crypto/provider_core.c:1423:14
    open-quantum-safe#10 0x7ff34df9ca64 in ossl_algorithm_do_all /home/pawn/work/openssl/crypto/core_algorithm.c:162:9
    open-quantum-safe#11 0x7ff34df9dd64 in ossl_method_construct /home/pawn/work/openssl/crypto/core_fetch.c:153:5
    open-quantum-safe#12 0x7ff34df13b6f in inner_evp_generic_fetch /home/pawn/work/openssl/crypto/evp/evp_fetch.c:312:23
    open-quantum-safe#13 0x7ff34df13226 in evp_generic_fetch /home/pawn/work/openssl/crypto/evp/evp_fetch.c:364:14
    open-quantum-safe#14 0x7ff34df36a42 in EVP_KEYMGMT_fetch /home/pawn/work/openssl/crypto/evp/keymgmt_meth.c:220:12
    open-quantum-safe#15 0x7ff34df5d6e4 in int_ctx_new /home/pawn/work/openssl/crypto/evp/pmeth_lib.c:280:23
    open-quantum-safe#16 0x7ff34df5f0a9 in EVP_PKEY_CTX_new_id /home/pawn/work/openssl/crypto/evp/pmeth_lib.c:469:12
    open-quantum-safe#17 0x7ff349865bcb in oqsx_hybsig_init /home/pawn/work/oqs-provider/oqsprov/oqsprov_keys.c:578:20
    open-quantum-safe#18 0x7ff349865157 in oqsx_key_new /home/pawn/work/oqs-provider/oqsprov/oqsprov_keys.c:791:16
    open-quantum-safe#19 0x7ff3498715a3 in oqsx_genkey /home/pawn/work/oqs-provider/oqsprov/oqs_kmgmt.c:487:16
    open-quantum-safe#20 0x7ff34986da08 in oqsx_gen /home/pawn/work/oqs-provider/oqsprov/oqs_kmgmt.c:509:12
    open-quantum-safe#21 0x7ff34df373c7 in evp_keymgmt_gen /home/pawn/work/openssl/crypto/evp/keymgmt_meth.c:366:12
    open-quantum-safe#22 0x7ff34df34b58 in evp_keymgmt_util_gen /home/pawn/work/openssl/crypto/evp/keymgmt_lib.c:517:20
    open-quantum-safe#23 0x7ff34df5a775 in EVP_PKEY_generate /home/pawn/work/openssl/crypto/evp/pmeth_gn.c:189:13
    open-quantum-safe#24 0x55b6fc90f2ad in test_oqs_signatures /home/pawn/work/oqs-provider/test/oqs_test_signatures.c:39:48
    open-quantum-safe#25 0x55b6fc90ee2d in main /home/pawn/work/oqs-provider/test/oqs_test_signatures.c:114:17
    open-quantum-safe#26 0x7ff34d8461c9 in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
```

4. Missing `EVP_PKEY_CTX_free` in case of error.
===

In `oqsx_hybsig_init`, a `EVP_PKEY_CTX` is initialized using `EVP_PKEY_CTX_new_id`.
However, later in the code, some extra work is done, and in case of error, the
`EVP_PKEY_CTX` wasn't freed.

(I couldn't recover the ASan message for this one…)


5. `OPENSSL_free` instead of `EVP_PKEY_free`.
===

In `oqsx_key_gen`, when a classical private key is generated, the field
`classical_pkey` of a `OQSX_KEY` is set with a value of type `EVP_PKEY`.

However, in `oqsx_key_free`, `OPENSSL_free` was used on `classical_pkey`
instead of `EVP_PKEY_free`.

ASan trace that helped me finding this one:

```
   #0 0x555a6b1b728e in malloc (/home/pawn/work/oqs-provider/build/test/oqs_test_signatures+0xb928e) (BuildId: 6cc7ebd4bc73a0c78b2ab8d7b9dcb8b165d88e74)
    open-quantum-safe#1 0x7f59a93a946b in CRYPTO_malloc /home/pawn/work/openssl/crypto/mem.c:190:12
    open-quantum-safe#2 0x7f59a93a94a2 in CRYPTO_zalloc /home/pawn/work/openssl/crypto/mem.c:197:11
    open-quantum-safe#3 0x7f59a909cdcd in BN_new /home/pawn/work/openssl/crypto/bn/bn_lib.c:247:16
    open-quantum-safe#4 0x7f59a920b955 in ossl_ec_group_new_ex /home/pawn/work/openssl/crypto/ec/ec_lib.c:61:25
    open-quantum-safe#5 0x7f59a920dd55 in EC_GROUP_dup /home/pawn/work/openssl/crypto/ec/ec_lib.c:273:14
    open-quantum-safe#6 0x7f59a9207774 in EC_KEY_set_group /home/pawn/work/openssl/crypto/ec/ec_key.c:733:18
    open-quantum-safe#7 0x7f59a975b3b5 in ec_gen_assign_group /home/pawn/work/openssl/providers/implementations/keymgmt/ec_kmgmt.c:1236:12
    open-quantum-safe#8 0x7f59a9758bdb in ec_gen /home/pawn/work/openssl/providers/implementations/keymgmt/ec_kmgmt.c:1274:11
    open-quantum-safe#9 0x7f59a93373c7 in evp_keymgmt_gen /home/pawn/work/openssl/crypto/evp/keymgmt_meth.c:366:12
    open-quantum-safe#10 0x7f59a9334b58 in evp_keymgmt_util_gen /home/pawn/work/openssl/crypto/evp/keymgmt_lib.c:517:20
    open-quantum-safe#11 0x7f59a935a775 in EVP_PKEY_generate /home/pawn/work/openssl/crypto/evp/pmeth_gn.c:189:13
    open-quantum-safe#12 0x7f59a935b404 in EVP_PKEY_keygen /home/pawn/work/openssl/crypto/evp/pmeth_gn.c:274:12
    open-quantum-safe#13 0x7f59a4c69a51 in oqsx_key_gen_evp_key /home/pawn/work/oqs-provider/oqsprov/oqsprov_keys.c:1012:12
    open-quantum-safe#14 0x7f59a4c68faf in oqsx_key_gen /home/pawn/work/oqs-provider/oqsprov/oqsprov_keys.c:1084:16
    open-quantum-safe#15 0x7f59a4c716ca in oqsx_genkey /home/pawn/work/oqs-provider/oqsprov/oqs_kmgmt.c:496:9
    open-quantum-safe#16 0x7f59a4c6da78 in oqsx_gen /home/pawn/work/oqs-provider/oqsprov/oqs_kmgmt.c:509:12
    open-quantum-safe#17 0x7f59a93373c7 in evp_keymgmt_gen /home/pawn/work/openssl/crypto/evp/keymgmt_meth.c:366:12
    open-quantum-safe#18 0x7f59a9334b58 in evp_keymgmt_util_gen /home/pawn/work/openssl/crypto/evp/keymgmt_lib.c:517:20
    open-quantum-safe#19 0x7f59a935a775 in EVP_PKEY_generate /home/pawn/work/openssl/crypto/evp/pmeth_gn.c:189:13
    open-quantum-safe#20 0x555a6b1f2890 in test_oqs_signatures /home/pawn/work/oqs-provider/test/oqs_test_signatures.c:68:44
    open-quantum-safe#21 0x555a6b1f1e3d in main /home/pawn/work/oqs-provider/test/oqs_test_signatures.c:114:17
    open-quantum-safe#22 0x7f59a8c461c9 in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
```

6. Bonus point: fix a potential double-free.
===

In `oqsx_key_gen`, when the keytype isn't `KEY_TYPE_HYB_SIG`, the
generated `pkey` is freed, and the flow goes down to the `err` label:

https://github.com/open-quantum-safe/oqs-provider/blob/cc3895f208730abfe1efc15f7c3ff4358ac6e4c0/oqsprov/oqsprov_keys.c#L1089-L1104

There is a potential double-free bug here: if `oqsx_key_gen_oqs` fails, then
`EVP_PKEY_free` will be called twice on the same `pkey` pointer.

Note that it is very unlikely that `oqsx_key_gen_oqs` fails, since it only
calls `OQS_KEM_keypair`/`OQS_SIG_keypair`.
thb-sb added a commit to thb-sb/oqs-provider that referenced this pull request Sep 5, 2023
@thb-sb
Fix various memory leaks.
120abc3 
This commit fixes various memory leaks that have been found using ASan.

Note: I will soon add a CI job for testing oqs-provider against memory leaks
with ASan.


1. Memory leak in `oqs_test_signatures`.
===

In `oqs_test_signatures`, [`OPENSSL_free`] was used with `EVP_PKEY_CTX`. However,
the right destructor function for `EVP_PKEY_CTX` is [`EVP_PKEY_CTX_free`].

This commmit fixes this minor bug.

[`OPENSSL_free`](https://www.openssl.org/docs/man3.0/man3/OPENSSL_free.html)
[`EVP_PKEY_CTX_free`](https://www.openssl.org/docs/man3.0/man3/EVP_PKEY_CTX_free.html)


2. a selftest key that wasn't freed.
===

In `oqsprov/oqsprov_keys.c`, `d2i_PrivateKey` is used to verify that we the
key we've just generated is sound.
However, the returned object wasn't freed.
ASan trace that helped me finding this one:

```
Indirect leak of 96 byte(s) in 4 object(s) allocated from:
    #0 0x5651eef1428e in malloc (/home/pawn/work/oqs-provider/build/test/oqs_test_signatures+0xb928e) (BuildId: 6cc7ebd4bc73a0c78b2ab8d7b9dcb8b165d88e74)
    open-quantum-safe#1 0x7fb7a23a946b in CRYPTO_malloc /home/pawn/work/openssl/crypto/mem.c:190:12
    open-quantum-safe#2 0x7fb7a23a94a2 in CRYPTO_zalloc /home/pawn/work/openssl/crypto/mem.c:197:11
    open-quantum-safe#3 0x7fb7a209cdcd in BN_new /home/pawn/work/openssl/crypto/bn/bn_lib.c:247:16
    open-quantum-safe#4 0x7fb7a2224a00 in ossl_ec_GFp_mont_group_set_curve /home/pawn/work/openssl/crypto/ec/ecp_mont.c:169:11
    open-quantum-safe#5 0x7fb7a220f5fa in EC_GROUP_set_curve /home/pawn/work/openssl/crypto/ec/ec_lib.c:562:12
    open-quantum-safe#6 0x7fb7a2202d75 in EC_GROUP_new_curve_GFp /home/pawn/work/openssl/crypto/ec/ec_cvt.c:61:10
    open-quantum-safe#7 0x7fb7a220184d in ec_group_new_from_data /home/pawn/work/openssl/crypto/ec/ec_curve.c:3179:22
    open-quantum-safe#8 0x7fb7a2200ff9 in EC_GROUP_new_by_curve_name_ex /home/pawn/work/openssl/crypto/ec/ec_curve.c:3287:19
    open-quantum-safe#9 0x7fb7a2201ed9 in EC_GROUP_new_by_curve_name /home/pawn/work/openssl/crypto/ec/ec_curve.c:3303:12
    open-quantum-safe#10 0x7fb7a21f84e2 in EC_GROUP_new_from_ecpkparameters /home/pawn/work/openssl/crypto/ec/ec_asn1.c:863:20
    open-quantum-safe#11 0x7fb7a21f8f28 in d2i_ECPrivateKey /home/pawn/work/openssl/crypto/ec/ec_asn1.c:956:22
    open-quantum-safe#12 0x7fb7a270b4b5 in der2key_decode /home/pawn/work/openssl/providers/implementations/encode_decode/decode_der2key.c:220:19
    open-quantum-safe#13 0x7fb7a226eae8 in decoder_process /home/pawn/work/openssl/crypto/encode_decode/decoder_lib.c:971:14
    open-quantum-safe#14 0x7fb7a226d106 in OSSL_DECODER_from_bio /home/pawn/work/openssl/crypto/encode_decode/decoder_lib.c:82:10
    open-quantum-safe#15 0x7fb7a226f007 in OSSL_DECODER_from_data /home/pawn/work/openssl/crypto/encode_decode/decoder_lib.c:157:9
    open-quantum-safe#16 0x7fb7a200e123 in d2i_PrivateKey_decoder /home/pawn/work/openssl/crypto/asn1/d2i_pr.c:74:11
    open-quantum-safe#17 0x7fb7a200dbba in d2i_PrivateKey_ex /home/pawn/work/openssl/crypto/asn1/d2i_pr.c:162:11
    open-quantum-safe#18 0x7fb7a200e3d2 in d2i_PrivateKey /home/pawn/work/openssl/crypto/asn1/d2i_pr.c:172:12
    open-quantum-safe#19 0x7fb79dc6a22e in oqsx_key_gen_evp_key /home/pawn/work/oqs-provider/oqsprov/oqsprov_keys.c:1036:25
    open-quantum-safe#20 0x7fb79dc68f3f in oqsx_key_gen /home/pawn/work/oqs-provider/oqsprov/oqsprov_keys.c:1077:16
    open-quantum-safe#21 0x7fb79dc7164a in oqsx_genkey /home/pawn/work/oqs-provider/oqsprov/oqs_kmgmt.c:496:9
    open-quantum-safe#22 0x7fb79dc6d9f8 in oqsx_gen /home/pawn/work/oqs-provider/oqsprov/oqs_kmgmt.c:509:12
    open-quantum-safe#23 0x7fb7a23373c7 in evp_keymgmt_gen /home/pawn/work/openssl/crypto/evp/keymgmt_meth.c:366:12
    open-quantum-safe#24 0x7fb7a2334b58 in evp_keymgmt_util_gen /home/pawn/work/openssl/crypto/evp/keymgmt_lib.c:517:20
    open-quantum-safe#25 0x7fb7a235a775 in EVP_PKEY_generate /home/pawn/work/openssl/crypto/evp/pmeth_gn.c:189:13
    open-quantum-safe#26 0x5651eef4f2bd in test_oqs_signatures /home/pawn/work/oqs-provider/test/oqs_test_signatures.c:39:48
    open-quantum-safe#27 0x5651eef4ee3d in main /home/pawn/work/oqs-provider/test/oqs_test_signatures.c:114:17
    open-quantum-safe#28 0x7fb7a1b671c9 in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
```

3. a `EVP_PKEY_CTX` that wasn't freed.
===

In `oqsx_key_free`, `key->oqsx_provider_ctx.oqsx_evp_ctx->ctx` is freed if
the keytype is `ECP_HYB_KEM` or `ECX_HYB_KEM`. However,
`key->oqsx_provider_ctx.oqsx_evp_ctx->ctx` is also used for signatures.
In that case, a call to `EVP_PKEY_CTX_free` was missing.
ASan trace that helped me finding this one:

```
Indirect leak of 7 byte(s) in 2 object(s) allocated from:
    #0 0x55b6fc8d427e in malloc (/home/pawn/work/oqs-provider/build/test/oqs_test_signatures+0xb927e) (BuildId: 3a8bff15d3315ba7ac1746fdc7b67eb464c607a1)
    open-quantum-safe#1 0x7ff34dfa946b in CRYPTO_malloc /home/pawn/work/openssl/crypto/mem.c:190:12
    open-quantum-safe#2 0x7ff34dfac50a in CRYPTO_strndup /home/pawn/work/openssl/crypto/o_str.c:43:11
    open-quantum-safe#3 0x7ff34df9d152 in ossl_algorithm_get1_first_name /home/pawn/work/openssl/crypto/core_algorithm.c:195:11
    open-quantum-safe#4 0x7ff34df353a1 in keymgmt_from_algorithm /home/pawn/work/openssl/crypto/evp/keymgmt_meth.c:50:31
    open-quantum-safe#5 0x7ff34df15e44 in construct_evp_method /home/pawn/work/openssl/crypto/evp/evp_fetch.c:219:14
    open-quantum-safe#6 0x7ff34df9e3ce in ossl_method_construct_this /home/pawn/work/openssl/crypto/core_fetch.c:109:19
    open-quantum-safe#7 0x7ff34df9d740 in algorithm_do_map /home/pawn/work/openssl/crypto/core_algorithm.c:77:13
    open-quantum-safe#8 0x7ff34df9ce88 in algorithm_do_this /home/pawn/work/openssl/crypto/core_algorithm.c:122:15
    open-quantum-safe#9 0x7ff34dfcce3e in ossl_provider_doall_activated /home/pawn/work/openssl/crypto/provider_core.c:1423:14
    open-quantum-safe#10 0x7ff34df9ca64 in ossl_algorithm_do_all /home/pawn/work/openssl/crypto/core_algorithm.c:162:9
    open-quantum-safe#11 0x7ff34df9dd64 in ossl_method_construct /home/pawn/work/openssl/crypto/core_fetch.c:153:5
    open-quantum-safe#12 0x7ff34df13b6f in inner_evp_generic_fetch /home/pawn/work/openssl/crypto/evp/evp_fetch.c:312:23
    open-quantum-safe#13 0x7ff34df13226 in evp_generic_fetch /home/pawn/work/openssl/crypto/evp/evp_fetch.c:364:14
    open-quantum-safe#14 0x7ff34df36a42 in EVP_KEYMGMT_fetch /home/pawn/work/openssl/crypto/evp/keymgmt_meth.c:220:12
    open-quantum-safe#15 0x7ff34df5d6e4 in int_ctx_new /home/pawn/work/openssl/crypto/evp/pmeth_lib.c:280:23
    open-quantum-safe#16 0x7ff34df5f0a9 in EVP_PKEY_CTX_new_id /home/pawn/work/openssl/crypto/evp/pmeth_lib.c:469:12
    open-quantum-safe#17 0x7ff349865bcb in oqsx_hybsig_init /home/pawn/work/oqs-provider/oqsprov/oqsprov_keys.c:578:20
    open-quantum-safe#18 0x7ff349865157 in oqsx_key_new /home/pawn/work/oqs-provider/oqsprov/oqsprov_keys.c:791:16
    open-quantum-safe#19 0x7ff3498715a3 in oqsx_genkey /home/pawn/work/oqs-provider/oqsprov/oqs_kmgmt.c:487:16
    open-quantum-safe#20 0x7ff34986da08 in oqsx_gen /home/pawn/work/oqs-provider/oqsprov/oqs_kmgmt.c:509:12
    open-quantum-safe#21 0x7ff34df373c7 in evp_keymgmt_gen /home/pawn/work/openssl/crypto/evp/keymgmt_meth.c:366:12
    open-quantum-safe#22 0x7ff34df34b58 in evp_keymgmt_util_gen /home/pawn/work/openssl/crypto/evp/keymgmt_lib.c:517:20
    open-quantum-safe#23 0x7ff34df5a775 in EVP_PKEY_generate /home/pawn/work/openssl/crypto/evp/pmeth_gn.c:189:13
    open-quantum-safe#24 0x55b6fc90f2ad in test_oqs_signatures /home/pawn/work/oqs-provider/test/oqs_test_signatures.c:39:48
    open-quantum-safe#25 0x55b6fc90ee2d in main /home/pawn/work/oqs-provider/test/oqs_test_signatures.c:114:17
    open-quantum-safe#26 0x7ff34d8461c9 in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
```

4. Missing `EVP_PKEY_CTX_free` in case of error.
===

In `oqsx_hybsig_init`, a `EVP_PKEY_CTX` is initialized using `EVP_PKEY_CTX_new_id`.
However, later in the code, some extra work is done, and in case of error, the
`EVP_PKEY_CTX` wasn't freed.

(I couldn't recover the ASan message for this one…)


5. `OPENSSL_free` instead of `EVP_PKEY_free`.
===

In `oqsx_key_gen`, when a classical private key is generated, the field
`classical_pkey` of a `OQSX_KEY` is set with a value of type `EVP_PKEY`.

However, in `oqsx_key_free`, `OPENSSL_free` was used on `classical_pkey`
instead of `EVP_PKEY_free`.

ASan trace that helped me finding this one:

```
   #0 0x555a6b1b728e in malloc (/home/pawn/work/oqs-provider/build/test/oqs_test_signatures+0xb928e) (BuildId: 6cc7ebd4bc73a0c78b2ab8d7b9dcb8b165d88e74)
    open-quantum-safe#1 0x7f59a93a946b in CRYPTO_malloc /home/pawn/work/openssl/crypto/mem.c:190:12
    open-quantum-safe#2 0x7f59a93a94a2 in CRYPTO_zalloc /home/pawn/work/openssl/crypto/mem.c:197:11
    open-quantum-safe#3 0x7f59a909cdcd in BN_new /home/pawn/work/openssl/crypto/bn/bn_lib.c:247:16
    open-quantum-safe#4 0x7f59a920b955 in ossl_ec_group_new_ex /home/pawn/work/openssl/crypto/ec/ec_lib.c:61:25
    open-quantum-safe#5 0x7f59a920dd55 in EC_GROUP_dup /home/pawn/work/openssl/crypto/ec/ec_lib.c:273:14
    open-quantum-safe#6 0x7f59a9207774 in EC_KEY_set_group /home/pawn/work/openssl/crypto/ec/ec_key.c:733:18
    open-quantum-safe#7 0x7f59a975b3b5 in ec_gen_assign_group /home/pawn/work/openssl/providers/implementations/keymgmt/ec_kmgmt.c:1236:12
    open-quantum-safe#8 0x7f59a9758bdb in ec_gen /home/pawn/work/openssl/providers/implementations/keymgmt/ec_kmgmt.c:1274:11
    open-quantum-safe#9 0x7f59a93373c7 in evp_keymgmt_gen /home/pawn/work/openssl/crypto/evp/keymgmt_meth.c:366:12
    open-quantum-safe#10 0x7f59a9334b58 in evp_keymgmt_util_gen /home/pawn/work/openssl/crypto/evp/keymgmt_lib.c:517:20
    open-quantum-safe#11 0x7f59a935a775 in EVP_PKEY_generate /home/pawn/work/openssl/crypto/evp/pmeth_gn.c:189:13
    open-quantum-safe#12 0x7f59a935b404 in EVP_PKEY_keygen /home/pawn/work/openssl/crypto/evp/pmeth_gn.c:274:12
    open-quantum-safe#13 0x7f59a4c69a51 in oqsx_key_gen_evp_key /home/pawn/work/oqs-provider/oqsprov/oqsprov_keys.c:1012:12
    open-quantum-safe#14 0x7f59a4c68faf in oqsx_key_gen /home/pawn/work/oqs-provider/oqsprov/oqsprov_keys.c:1084:16
    open-quantum-safe#15 0x7f59a4c716ca in oqsx_genkey /home/pawn/work/oqs-provider/oqsprov/oqs_kmgmt.c:496:9
    open-quantum-safe#16 0x7f59a4c6da78 in oqsx_gen /home/pawn/work/oqs-provider/oqsprov/oqs_kmgmt.c:509:12
    open-quantum-safe#17 0x7f59a93373c7 in evp_keymgmt_gen /home/pawn/work/openssl/crypto/evp/keymgmt_meth.c:366:12
    open-quantum-safe#18 0x7f59a9334b58 in evp_keymgmt_util_gen /home/pawn/work/openssl/crypto/evp/keymgmt_lib.c:517:20
    open-quantum-safe#19 0x7f59a935a775 in EVP_PKEY_generate /home/pawn/work/openssl/crypto/evp/pmeth_gn.c:189:13
    open-quantum-safe#20 0x555a6b1f2890 in test_oqs_signatures /home/pawn/work/oqs-provider/test/oqs_test_signatures.c:68:44
    open-quantum-safe#21 0x555a6b1f1e3d in main /home/pawn/work/oqs-provider/test/oqs_test_signatures.c:114:17
    open-quantum-safe#22 0x7f59a8c461c9 in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
```

6. Bonus point: fix a potential double-free.
===

In `oqsx_key_gen`, when the keytype isn't `KEY_TYPE_HYB_SIG`, the
generated `pkey` is freed, and the flow goes down to the `err` label:

https://github.com/open-quantum-safe/oqs-provider/blob/cc3895f208730abfe1efc15f7c3ff4358ac6e4c0/oqsprov/oqsprov_keys.c#L1089-L1104

There is a potential double-free bug here: if `oqsx_key_gen_oqs` fails, then
`EVP_PKEY_free` will be called twice on the same `pkey` pointer.

Note that it is very unlikely that `oqsx_key_gen_oqs` fails, since it only
calls `OQS_KEM_keypair`/`OQS_SIG_keypair`.
thb-sb added a commit to thb-sb/oqs-provider that referenced this pull request Sep 5, 2023
@thb-sb
Fix various memory leaks.
129cd68 
This commit fixes various memory leaks that have been found using ASan.

Note: I will soon add a CI job for testing oqs-provider against memory leaks
with ASan.


# 1. Memory leak in `oqs_test_signatures`.

In `oqs_test_signatures`, [`OPENSSL_free`] was used with `EVP_PKEY_CTX`. However,
the right destructor function for `EVP_PKEY_CTX` is [`EVP_PKEY_CTX_free`].

This commmit fixes this minor bug.

[`OPENSSL_free`](https://www.openssl.org/docs/man3.0/man3/OPENSSL_free.html)
[`EVP_PKEY_CTX_free`](https://www.openssl.org/docs/man3.0/man3/EVP_PKEY_CTX_free.html)


## 2. a selftest key that wasn't freed.

In `oqsprov/oqsprov_keys.c`, `d2i_PrivateKey` is used to verify that we the
key we've just generated is sound.
However, the returned object wasn't freed.
ASan trace that helped me finding this one:

```
Indirect leak of 96 byte(s) in 4 object(s) allocated from:
    #0 0x5651eef1428e in malloc (/home/pawn/work/oqs-provider/build/test/oqs_test_signatures+0xb928e) (BuildId: 6cc7ebd4bc73a0c78b2ab8d7b9dcb8b165d88e74)
    open-quantum-safe#1 0x7fb7a23a946b in CRYPTO_malloc /home/pawn/work/openssl/crypto/mem.c:190:12
    open-quantum-safe#2 0x7fb7a23a94a2 in CRYPTO_zalloc /home/pawn/work/openssl/crypto/mem.c:197:11
    open-quantum-safe#3 0x7fb7a209cdcd in BN_new /home/pawn/work/openssl/crypto/bn/bn_lib.c:247:16
    open-quantum-safe#4 0x7fb7a2224a00 in ossl_ec_GFp_mont_group_set_curve /home/pawn/work/openssl/crypto/ec/ecp_mont.c:169:11
    open-quantum-safe#5 0x7fb7a220f5fa in EC_GROUP_set_curve /home/pawn/work/openssl/crypto/ec/ec_lib.c:562:12
    open-quantum-safe#6 0x7fb7a2202d75 in EC_GROUP_new_curve_GFp /home/pawn/work/openssl/crypto/ec/ec_cvt.c:61:10
    open-quantum-safe#7 0x7fb7a220184d in ec_group_new_from_data /home/pawn/work/openssl/crypto/ec/ec_curve.c:3179:22
    open-quantum-safe#8 0x7fb7a2200ff9 in EC_GROUP_new_by_curve_name_ex /home/pawn/work/openssl/crypto/ec/ec_curve.c:3287:19
    open-quantum-safe#9 0x7fb7a2201ed9 in EC_GROUP_new_by_curve_name /home/pawn/work/openssl/crypto/ec/ec_curve.c:3303:12
    open-quantum-safe#10 0x7fb7a21f84e2 in EC_GROUP_new_from_ecpkparameters /home/pawn/work/openssl/crypto/ec/ec_asn1.c:863:20
    open-quantum-safe#11 0x7fb7a21f8f28 in d2i_ECPrivateKey /home/pawn/work/openssl/crypto/ec/ec_asn1.c:956:22
    open-quantum-safe#12 0x7fb7a270b4b5 in der2key_decode /home/pawn/work/openssl/providers/implementations/encode_decode/decode_der2key.c:220:19
    open-quantum-safe#13 0x7fb7a226eae8 in decoder_process /home/pawn/work/openssl/crypto/encode_decode/decoder_lib.c:971:14
    open-quantum-safe#14 0x7fb7a226d106 in OSSL_DECODER_from_bio /home/pawn/work/openssl/crypto/encode_decode/decoder_lib.c:82:10
    open-quantum-safe#15 0x7fb7a226f007 in OSSL_DECODER_from_data /home/pawn/work/openssl/crypto/encode_decode/decoder_lib.c:157:9
    open-quantum-safe#16 0x7fb7a200e123 in d2i_PrivateKey_decoder /home/pawn/work/openssl/crypto/asn1/d2i_pr.c:74:11
    open-quantum-safe#17 0x7fb7a200dbba in d2i_PrivateKey_ex /home/pawn/work/openssl/crypto/asn1/d2i_pr.c:162:11
    open-quantum-safe#18 0x7fb7a200e3d2 in d2i_PrivateKey /home/pawn/work/openssl/crypto/asn1/d2i_pr.c:172:12
    open-quantum-safe#19 0x7fb79dc6a22e in oqsx_key_gen_evp_key /home/pawn/work/oqs-provider/oqsprov/oqsprov_keys.c:1036:25
    open-quantum-safe#20 0x7fb79dc68f3f in oqsx_key_gen /home/pawn/work/oqs-provider/oqsprov/oqsprov_keys.c:1077:16
    open-quantum-safe#21 0x7fb79dc7164a in oqsx_genkey /home/pawn/work/oqs-provider/oqsprov/oqs_kmgmt.c:496:9
    open-quantum-safe#22 0x7fb79dc6d9f8 in oqsx_gen /home/pawn/work/oqs-provider/oqsprov/oqs_kmgmt.c:509:12
    open-quantum-safe#23 0x7fb7a23373c7 in evp_keymgmt_gen /home/pawn/work/openssl/crypto/evp/keymgmt_meth.c:366:12
    open-quantum-safe#24 0x7fb7a2334b58 in evp_keymgmt_util_gen /home/pawn/work/openssl/crypto/evp/keymgmt_lib.c:517:20
    open-quantum-safe#25 0x7fb7a235a775 in EVP_PKEY_generate /home/pawn/work/openssl/crypto/evp/pmeth_gn.c:189:13
    open-quantum-safe#26 0x5651eef4f2bd in test_oqs_signatures /home/pawn/work/oqs-provider/test/oqs_test_signatures.c:39:48
    open-quantum-safe#27 0x5651eef4ee3d in main /home/pawn/work/oqs-provider/test/oqs_test_signatures.c:114:17
    open-quantum-safe#28 0x7fb7a1b671c9 in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
```

## 3. a `EVP_PKEY_CTX` that wasn't freed.

In `oqsx_key_free`, `key->oqsx_provider_ctx.oqsx_evp_ctx->ctx` is freed if
the keytype is `ECP_HYB_KEM` or `ECX_HYB_KEM`. However,
`key->oqsx_provider_ctx.oqsx_evp_ctx->ctx` is also used for signatures.
In that case, a call to `EVP_PKEY_CTX_free` was missing.
ASan trace that helped me finding this one:

```
Indirect leak of 7 byte(s) in 2 object(s) allocated from:
    #0 0x55b6fc8d427e in malloc (/home/pawn/work/oqs-provider/build/test/oqs_test_signatures+0xb927e) (BuildId: 3a8bff15d3315ba7ac1746fdc7b67eb464c607a1)
    open-quantum-safe#1 0x7ff34dfa946b in CRYPTO_malloc /home/pawn/work/openssl/crypto/mem.c:190:12
    open-quantum-safe#2 0x7ff34dfac50a in CRYPTO_strndup /home/pawn/work/openssl/crypto/o_str.c:43:11
    open-quantum-safe#3 0x7ff34df9d152 in ossl_algorithm_get1_first_name /home/pawn/work/openssl/crypto/core_algorithm.c:195:11
    open-quantum-safe#4 0x7ff34df353a1 in keymgmt_from_algorithm /home/pawn/work/openssl/crypto/evp/keymgmt_meth.c:50:31
    open-quantum-safe#5 0x7ff34df15e44 in construct_evp_method /home/pawn/work/openssl/crypto/evp/evp_fetch.c:219:14
    open-quantum-safe#6 0x7ff34df9e3ce in ossl_method_construct_this /home/pawn/work/openssl/crypto/core_fetch.c:109:19
    open-quantum-safe#7 0x7ff34df9d740 in algorithm_do_map /home/pawn/work/openssl/crypto/core_algorithm.c:77:13
    open-quantum-safe#8 0x7ff34df9ce88 in algorithm_do_this /home/pawn/work/openssl/crypto/core_algorithm.c:122:15
    open-quantum-safe#9 0x7ff34dfcce3e in ossl_provider_doall_activated /home/pawn/work/openssl/crypto/provider_core.c:1423:14
    open-quantum-safe#10 0x7ff34df9ca64 in ossl_algorithm_do_all /home/pawn/work/openssl/crypto/core_algorithm.c:162:9
    open-quantum-safe#11 0x7ff34df9dd64 in ossl_method_construct /home/pawn/work/openssl/crypto/core_fetch.c:153:5
    open-quantum-safe#12 0x7ff34df13b6f in inner_evp_generic_fetch /home/pawn/work/openssl/crypto/evp/evp_fetch.c:312:23
    open-quantum-safe#13 0x7ff34df13226 in evp_generic_fetch /home/pawn/work/openssl/crypto/evp/evp_fetch.c:364:14
    open-quantum-safe#14 0x7ff34df36a42 in EVP_KEYMGMT_fetch /home/pawn/work/openssl/crypto/evp/keymgmt_meth.c:220:12
    open-quantum-safe#15 0x7ff34df5d6e4 in int_ctx_new /home/pawn/work/openssl/crypto/evp/pmeth_lib.c:280:23
    open-quantum-safe#16 0x7ff34df5f0a9 in EVP_PKEY_CTX_new_id /home/pawn/work/openssl/crypto/evp/pmeth_lib.c:469:12
    open-quantum-safe#17 0x7ff349865bcb in oqsx_hybsig_init /home/pawn/work/oqs-provider/oqsprov/oqsprov_keys.c:578:20
    open-quantum-safe#18 0x7ff349865157 in oqsx_key_new /home/pawn/work/oqs-provider/oqsprov/oqsprov_keys.c:791:16
    open-quantum-safe#19 0x7ff3498715a3 in oqsx_genkey /home/pawn/work/oqs-provider/oqsprov/oqs_kmgmt.c:487:16
    open-quantum-safe#20 0x7ff34986da08 in oqsx_gen /home/pawn/work/oqs-provider/oqsprov/oqs_kmgmt.c:509:12
    open-quantum-safe#21 0x7ff34df373c7 in evp_keymgmt_gen /home/pawn/work/openssl/crypto/evp/keymgmt_meth.c:366:12
    open-quantum-safe#22 0x7ff34df34b58 in evp_keymgmt_util_gen /home/pawn/work/openssl/crypto/evp/keymgmt_lib.c:517:20
    open-quantum-safe#23 0x7ff34df5a775 in EVP_PKEY_generate /home/pawn/work/openssl/crypto/evp/pmeth_gn.c:189:13
    open-quantum-safe#24 0x55b6fc90f2ad in test_oqs_signatures /home/pawn/work/oqs-provider/test/oqs_test_signatures.c:39:48
    open-quantum-safe#25 0x55b6fc90ee2d in main /home/pawn/work/oqs-provider/test/oqs_test_signatures.c:114:17
    open-quantum-safe#26 0x7ff34d8461c9 in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
```

## 4. Missing `EVP_PKEY_CTX_free` in case of error.

In `oqsx_hybsig_init`, a `EVP_PKEY_CTX` is initialized using `EVP_PKEY_CTX_new_id`.
However, later in the code, some extra work is done, and in case of error, the
`EVP_PKEY_CTX` wasn't freed.

(I couldn't recover the ASan message for this one…)


## 5. `OPENSSL_free` instead of `EVP_PKEY_free`.

In `oqsx_key_gen`, when a classical private key is generated, the field
`classical_pkey` of a `OQSX_KEY` is set with a value of type `EVP_PKEY`.

However, in `oqsx_key_free`, `OPENSSL_free` was used on `classical_pkey`
instead of `EVP_PKEY_free`.

ASan trace that helped me finding this one:

```
   #0 0x555a6b1b728e in malloc (/home/pawn/work/oqs-provider/build/test/oqs_test_signatures+0xb928e) (BuildId: 6cc7ebd4bc73a0c78b2ab8d7b9dcb8b165d88e74)
    open-quantum-safe#1 0x7f59a93a946b in CRYPTO_malloc /home/pawn/work/openssl/crypto/mem.c:190:12
    open-quantum-safe#2 0x7f59a93a94a2 in CRYPTO_zalloc /home/pawn/work/openssl/crypto/mem.c:197:11
    open-quantum-safe#3 0x7f59a909cdcd in BN_new /home/pawn/work/openssl/crypto/bn/bn_lib.c:247:16
    open-quantum-safe#4 0x7f59a920b955 in ossl_ec_group_new_ex /home/pawn/work/openssl/crypto/ec/ec_lib.c:61:25
    open-quantum-safe#5 0x7f59a920dd55 in EC_GROUP_dup /home/pawn/work/openssl/crypto/ec/ec_lib.c:273:14
    open-quantum-safe#6 0x7f59a9207774 in EC_KEY_set_group /home/pawn/work/openssl/crypto/ec/ec_key.c:733:18
    open-quantum-safe#7 0x7f59a975b3b5 in ec_gen_assign_group /home/pawn/work/openssl/providers/implementations/keymgmt/ec_kmgmt.c:1236:12
    open-quantum-safe#8 0x7f59a9758bdb in ec_gen /home/pawn/work/openssl/providers/implementations/keymgmt/ec_kmgmt.c:1274:11
    open-quantum-safe#9 0x7f59a93373c7 in evp_keymgmt_gen /home/pawn/work/openssl/crypto/evp/keymgmt_meth.c:366:12
    open-quantum-safe#10 0x7f59a9334b58 in evp_keymgmt_util_gen /home/pawn/work/openssl/crypto/evp/keymgmt_lib.c:517:20
    open-quantum-safe#11 0x7f59a935a775 in EVP_PKEY_generate /home/pawn/work/openssl/crypto/evp/pmeth_gn.c:189:13
    open-quantum-safe#12 0x7f59a935b404 in EVP_PKEY_keygen /home/pawn/work/openssl/crypto/evp/pmeth_gn.c:274:12
    open-quantum-safe#13 0x7f59a4c69a51 in oqsx_key_gen_evp_key /home/pawn/work/oqs-provider/oqsprov/oqsprov_keys.c:1012:12
    open-quantum-safe#14 0x7f59a4c68faf in oqsx_key_gen /home/pawn/work/oqs-provider/oqsprov/oqsprov_keys.c:1084:16
    open-quantum-safe#15 0x7f59a4c716ca in oqsx_genkey /home/pawn/work/oqs-provider/oqsprov/oqs_kmgmt.c:496:9
    open-quantum-safe#16 0x7f59a4c6da78 in oqsx_gen /home/pawn/work/oqs-provider/oqsprov/oqs_kmgmt.c:509:12
    open-quantum-safe#17 0x7f59a93373c7 in evp_keymgmt_gen /home/pawn/work/openssl/crypto/evp/keymgmt_meth.c:366:12
    open-quantum-safe#18 0x7f59a9334b58 in evp_keymgmt_util_gen /home/pawn/work/openssl/crypto/evp/keymgmt_lib.c:517:20
    open-quantum-safe#19 0x7f59a935a775 in EVP_PKEY_generate /home/pawn/work/openssl/crypto/evp/pmeth_gn.c:189:13
    open-quantum-safe#20 0x555a6b1f2890 in test_oqs_signatures /home/pawn/work/oqs-provider/test/oqs_test_signatures.c:68:44
    open-quantum-safe#21 0x555a6b1f1e3d in main /home/pawn/work/oqs-provider/test/oqs_test_signatures.c:114:17
    open-quantum-safe#22 0x7f59a8c461c9 in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
```

## 6. Some `OPENSSL_free` were missing in `oqs_test_kems.c`

```
==569479==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 346035 byte(s) in 42 object(s) allocated from:
    #0 0x559c76ef524e in malloc (/home/thomas_bailleux/work/oqs-provider/build/test/oqs_test_kems+0xb924e) (BuildId: 76a98f5859ba3c90334cef4286e89aa76436b1ae)
    open-quantum-safe#1 0x7f04407a946b in CRYPTO_malloc /home/thomas_bailleux/work/openssl/crypto/mem.c:190:12
    open-quantum-safe#2 0x559c76f30389 in test_oqs_kems /home/thomas_bailleux/work/oqs-provider/test/oqs_test_kems.c:44:26
    open-quantum-safe#3 0x559c76f2fdfd in main /home/thomas_bailleux/work/oqs-provider/test/oqs_test_kems.c:93:17
    open-quantum-safe#4 0x7f04400461c9 in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16

Direct leak of 2690 byte(s) in 42 object(s) allocated from:
    #0 0x559c76ef524e in malloc (/home/thomas_bailleux/work/oqs-provider/build/test/oqs_test_kems+0xb924e) (BuildId: 76a98f5859ba3c90334cef4286e89aa76436b1ae)
    open-quantum-safe#1 0x7f04407a946b in CRYPTO_malloc /home/thomas_bailleux/work/openssl/crypto/mem.c:190:12
    open-quantum-safe#2 0x559c76f304a5 in test_oqs_kems /home/thomas_bailleux/work/oqs-provider/test/oqs_test_kems.c:47:29
    open-quantum-safe#3 0x559c76f2fdfd in main /home/thomas_bailleux/work/oqs-provider/test/oqs_test_kems.c:93:17
    open-quantum-safe#4 0x7f04400461c9 in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16

Direct leak of 2690 byte(s) in 42 object(s) allocated from:
    #0 0x559c76ef524e in malloc (/home/thomas_bailleux/work/oqs-provider/build/test/oqs_test_kems+0xb924e) (BuildId: 76a98f5859ba3c90334cef4286e89aa76436b1ae)
    open-quantum-safe#1 0x7f04407a946b in CRYPTO_malloc /home/thomas_bailleux/work/openssl/crypto/mem.c:190:12
    open-quantum-safe#2 0x559c76f303e4 in test_oqs_kems /home/thomas_bailleux/work/oqs-provider/test/oqs_test_kems.c:45:29
    open-quantum-safe#3 0x559c76f2fdfd in main /home/thomas_bailleux/work/oqs-provider/test/oqs_test_kems.c:93:17
    open-quantum-safe#4 0x7f04400461c9 in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
```


## 7. Bonus point: fix a potential double-free.

In `oqsx_key_gen`, when the keytype isn't `KEY_TYPE_HYB_SIG`, the
generated `pkey` is freed, and the flow goes down to the `err` label:

https://github.com/open-quantum-safe/oqs-provider/blob/cc3895f208730abfe1efc15f7c3ff4358ac6e4c0/oqsprov/oqsprov_keys.c#L1089-L1104

There is a potential double-free bug here: if `oqsx_key_gen_oqs` fails, then
`EVP_PKEY_free` will be called twice on the same `pkey` pointer.

Note that it is very unlikely that `oqsx_key_gen_oqs` fails, since it only
calls `OQS_KEM_keypair`/`OQS_SIG_keypair`.
thb-sb added a commit to thb-sb/oqs-provider that referenced this pull request Sep 5, 2023
@thb-sb
Fix various memory leaks.
c13b211 
This commit fixes various memory leaks that have been found using ASan.

Note: I will soon add a CI job for testing oqs-provider against memory leaks
with ASan.


# 1. Memory leak in `oqs_test_signatures`.

In `oqs_test_signatures`, [`OPENSSL_free`] was used with `EVP_PKEY_CTX`. However,
the right destructor function for `EVP_PKEY_CTX` is [`EVP_PKEY_CTX_free`].

This commmit fixes this minor bug.

[`OPENSSL_free`](https://www.openssl.org/docs/man3.0/man3/OPENSSL_free.html)
[`EVP_PKEY_CTX_free`](https://www.openssl.org/docs/man3.0/man3/EVP_PKEY_CTX_free.html)


## 2. a selftest key that wasn't freed.

In `oqsprov/oqsprov_keys.c`, `d2i_PrivateKey` is used to verify that we the
key we've just generated is sound.
However, the returned object wasn't freed.
ASan trace that helped me finding this one:

```
Indirect leak of 96 byte(s) in 4 object(s) allocated from:
    #0 0x5651eef1428e in malloc (/home/pawn/work/oqs-provider/build/test/oqs_test_signatures+0xb928e) (BuildId: 6cc7ebd4bc73a0c78b2ab8d7b9dcb8b165d88e74)
    open-quantum-safe#1 0x7fb7a23a946b in CRYPTO_malloc /home/pawn/work/openssl/crypto/mem.c:190:12
    open-quantum-safe#2 0x7fb7a23a94a2 in CRYPTO_zalloc /home/pawn/work/openssl/crypto/mem.c:197:11
    open-quantum-safe#3 0x7fb7a209cdcd in BN_new /home/pawn/work/openssl/crypto/bn/bn_lib.c:247:16
    open-quantum-safe#4 0x7fb7a2224a00 in ossl_ec_GFp_mont_group_set_curve /home/pawn/work/openssl/crypto/ec/ecp_mont.c:169:11
    open-quantum-safe#5 0x7fb7a220f5fa in EC_GROUP_set_curve /home/pawn/work/openssl/crypto/ec/ec_lib.c:562:12
    open-quantum-safe#6 0x7fb7a2202d75 in EC_GROUP_new_curve_GFp /home/pawn/work/openssl/crypto/ec/ec_cvt.c:61:10
    open-quantum-safe#7 0x7fb7a220184d in ec_group_new_from_data /home/pawn/work/openssl/crypto/ec/ec_curve.c:3179:22
    open-quantum-safe#8 0x7fb7a2200ff9 in EC_GROUP_new_by_curve_name_ex /home/pawn/work/openssl/crypto/ec/ec_curve.c:3287:19
    open-quantum-safe#9 0x7fb7a2201ed9 in EC_GROUP_new_by_curve_name /home/pawn/work/openssl/crypto/ec/ec_curve.c:3303:12
    open-quantum-safe#10 0x7fb7a21f84e2 in EC_GROUP_new_from_ecpkparameters /home/pawn/work/openssl/crypto/ec/ec_asn1.c:863:20
    open-quantum-safe#11 0x7fb7a21f8f28 in d2i_ECPrivateKey /home/pawn/work/openssl/crypto/ec/ec_asn1.c:956:22
    open-quantum-safe#12 0x7fb7a270b4b5 in der2key_decode /home/pawn/work/openssl/providers/implementations/encode_decode/decode_der2key.c:220:19
    open-quantum-safe#13 0x7fb7a226eae8 in decoder_process /home/pawn/work/openssl/crypto/encode_decode/decoder_lib.c:971:14
    open-quantum-safe#14 0x7fb7a226d106 in OSSL_DECODER_from_bio /home/pawn/work/openssl/crypto/encode_decode/decoder_lib.c:82:10
    open-quantum-safe#15 0x7fb7a226f007 in OSSL_DECODER_from_data /home/pawn/work/openssl/crypto/encode_decode/decoder_lib.c:157:9
    open-quantum-safe#16 0x7fb7a200e123 in d2i_PrivateKey_decoder /home/pawn/work/openssl/crypto/asn1/d2i_pr.c:74:11
    open-quantum-safe#17 0x7fb7a200dbba in d2i_PrivateKey_ex /home/pawn/work/openssl/crypto/asn1/d2i_pr.c:162:11
    open-quantum-safe#18 0x7fb7a200e3d2 in d2i_PrivateKey /home/pawn/work/openssl/crypto/asn1/d2i_pr.c:172:12
    open-quantum-safe#19 0x7fb79dc6a22e in oqsx_key_gen_evp_key /home/pawn/work/oqs-provider/oqsprov/oqsprov_keys.c:1036:25
    open-quantum-safe#20 0x7fb79dc68f3f in oqsx_key_gen /home/pawn/work/oqs-provider/oqsprov/oqsprov_keys.c:1077:16
    open-quantum-safe#21 0x7fb79dc7164a in oqsx_genkey /home/pawn/work/oqs-provider/oqsprov/oqs_kmgmt.c:496:9
    open-quantum-safe#22 0x7fb79dc6d9f8 in oqsx_gen /home/pawn/work/oqs-provider/oqsprov/oqs_kmgmt.c:509:12
    open-quantum-safe#23 0x7fb7a23373c7 in evp_keymgmt_gen /home/pawn/work/openssl/crypto/evp/keymgmt_meth.c:366:12
    open-quantum-safe#24 0x7fb7a2334b58 in evp_keymgmt_util_gen /home/pawn/work/openssl/crypto/evp/keymgmt_lib.c:517:20
    open-quantum-safe#25 0x7fb7a235a775 in EVP_PKEY_generate /home/pawn/work/openssl/crypto/evp/pmeth_gn.c:189:13
    open-quantum-safe#26 0x5651eef4f2bd in test_oqs_signatures /home/pawn/work/oqs-provider/test/oqs_test_signatures.c:39:48
    open-quantum-safe#27 0x5651eef4ee3d in main /home/pawn/work/oqs-provider/test/oqs_test_signatures.c:114:17
    open-quantum-safe#28 0x7fb7a1b671c9 in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
```

## 3. a `EVP_PKEY_CTX` that wasn't freed.

In `oqsx_key_free`, `key->oqsx_provider_ctx.oqsx_evp_ctx->ctx` is freed if
the keytype is `ECP_HYB_KEM` or `ECX_HYB_KEM`. However,
`key->oqsx_provider_ctx.oqsx_evp_ctx->ctx` is also used for signatures.
In that case, a call to `EVP_PKEY_CTX_free` was missing.
ASan trace that helped me finding this one:

```
Indirect leak of 7 byte(s) in 2 object(s) allocated from:
    #0 0x55b6fc8d427e in malloc (/home/pawn/work/oqs-provider/build/test/oqs_test_signatures+0xb927e) (BuildId: 3a8bff15d3315ba7ac1746fdc7b67eb464c607a1)
    open-quantum-safe#1 0x7ff34dfa946b in CRYPTO_malloc /home/pawn/work/openssl/crypto/mem.c:190:12
    open-quantum-safe#2 0x7ff34dfac50a in CRYPTO_strndup /home/pawn/work/openssl/crypto/o_str.c:43:11
    open-quantum-safe#3 0x7ff34df9d152 in ossl_algorithm_get1_first_name /home/pawn/work/openssl/crypto/core_algorithm.c:195:11
    open-quantum-safe#4 0x7ff34df353a1 in keymgmt_from_algorithm /home/pawn/work/openssl/crypto/evp/keymgmt_meth.c:50:31
    open-quantum-safe#5 0x7ff34df15e44 in construct_evp_method /home/pawn/work/openssl/crypto/evp/evp_fetch.c:219:14
    open-quantum-safe#6 0x7ff34df9e3ce in ossl_method_construct_this /home/pawn/work/openssl/crypto/core_fetch.c:109:19
    open-quantum-safe#7 0x7ff34df9d740 in algorithm_do_map /home/pawn/work/openssl/crypto/core_algorithm.c:77:13
    open-quantum-safe#8 0x7ff34df9ce88 in algorithm_do_this /home/pawn/work/openssl/crypto/core_algorithm.c:122:15
    open-quantum-safe#9 0x7ff34dfcce3e in ossl_provider_doall_activated /home/pawn/work/openssl/crypto/provider_core.c:1423:14
    open-quantum-safe#10 0x7ff34df9ca64 in ossl_algorithm_do_all /home/pawn/work/openssl/crypto/core_algorithm.c:162:9
    open-quantum-safe#11 0x7ff34df9dd64 in ossl_method_construct /home/pawn/work/openssl/crypto/core_fetch.c:153:5
    open-quantum-safe#12 0x7ff34df13b6f in inner_evp_generic_fetch /home/pawn/work/openssl/crypto/evp/evp_fetch.c:312:23
    open-quantum-safe#13 0x7ff34df13226 in evp_generic_fetch /home/pawn/work/openssl/crypto/evp/evp_fetch.c:364:14
    open-quantum-safe#14 0x7ff34df36a42 in EVP_KEYMGMT_fetch /home/pawn/work/openssl/crypto/evp/keymgmt_meth.c:220:12
    open-quantum-safe#15 0x7ff34df5d6e4 in int_ctx_new /home/pawn/work/openssl/crypto/evp/pmeth_lib.c:280:23
    open-quantum-safe#16 0x7ff34df5f0a9 in EVP_PKEY_CTX_new_id /home/pawn/work/openssl/crypto/evp/pmeth_lib.c:469:12
    open-quantum-safe#17 0x7ff349865bcb in oqsx_hybsig_init /home/pawn/work/oqs-provider/oqsprov/oqsprov_keys.c:578:20
    open-quantum-safe#18 0x7ff349865157 in oqsx_key_new /home/pawn/work/oqs-provider/oqsprov/oqsprov_keys.c:791:16
    open-quantum-safe#19 0x7ff3498715a3 in oqsx_genkey /home/pawn/work/oqs-provider/oqsprov/oqs_kmgmt.c:487:16
    open-quantum-safe#20 0x7ff34986da08 in oqsx_gen /home/pawn/work/oqs-provider/oqsprov/oqs_kmgmt.c:509:12
    open-quantum-safe#21 0x7ff34df373c7 in evp_keymgmt_gen /home/pawn/work/openssl/crypto/evp/keymgmt_meth.c:366:12
    open-quantum-safe#22 0x7ff34df34b58 in evp_keymgmt_util_gen /home/pawn/work/openssl/crypto/evp/keymgmt_lib.c:517:20
    open-quantum-safe#23 0x7ff34df5a775 in EVP_PKEY_generate /home/pawn/work/openssl/crypto/evp/pmeth_gn.c:189:13
    open-quantum-safe#24 0x55b6fc90f2ad in test_oqs_signatures /home/pawn/work/oqs-provider/test/oqs_test_signatures.c:39:48
    open-quantum-safe#25 0x55b6fc90ee2d in main /home/pawn/work/oqs-provider/test/oqs_test_signatures.c:114:17
    open-quantum-safe#26 0x7ff34d8461c9 in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
```

## 4. Missing `EVP_PKEY_CTX_free` in case of error.

In `oqsx_hybsig_init`, a `EVP_PKEY_CTX` is initialized using `EVP_PKEY_CTX_new_id`.
However, later in the code, some extra work is done, and in case of error, the
`EVP_PKEY_CTX` wasn't freed.

(I couldn't recover the ASan message for this one…)


## 5. `OPENSSL_free` instead of `EVP_PKEY_free`.

In `oqsx_key_gen`, when a classical private key is generated, the field
`classical_pkey` of a `OQSX_KEY` is set with a value of type `EVP_PKEY`.

However, in `oqsx_key_free`, `OPENSSL_free` was used on `classical_pkey`
instead of `EVP_PKEY_free`.

ASan trace that helped me finding this one:

```
   #0 0x555a6b1b728e in malloc (/home/pawn/work/oqs-provider/build/test/oqs_test_signatures+0xb928e) (BuildId: 6cc7ebd4bc73a0c78b2ab8d7b9dcb8b165d88e74)
    open-quantum-safe#1 0x7f59a93a946b in CRYPTO_malloc /home/pawn/work/openssl/crypto/mem.c:190:12
    open-quantum-safe#2 0x7f59a93a94a2 in CRYPTO_zalloc /home/pawn/work/openssl/crypto/mem.c:197:11
    open-quantum-safe#3 0x7f59a909cdcd in BN_new /home/pawn/work/openssl/crypto/bn/bn_lib.c:247:16
    open-quantum-safe#4 0x7f59a920b955 in ossl_ec_group_new_ex /home/pawn/work/openssl/crypto/ec/ec_lib.c:61:25
    open-quantum-safe#5 0x7f59a920dd55 in EC_GROUP_dup /home/pawn/work/openssl/crypto/ec/ec_lib.c:273:14
    open-quantum-safe#6 0x7f59a9207774 in EC_KEY_set_group /home/pawn/work/openssl/crypto/ec/ec_key.c:733:18
    open-quantum-safe#7 0x7f59a975b3b5 in ec_gen_assign_group /home/pawn/work/openssl/providers/implementations/keymgmt/ec_kmgmt.c:1236:12
    open-quantum-safe#8 0x7f59a9758bdb in ec_gen /home/pawn/work/openssl/providers/implementations/keymgmt/ec_kmgmt.c:1274:11
    open-quantum-safe#9 0x7f59a93373c7 in evp_keymgmt_gen /home/pawn/work/openssl/crypto/evp/keymgmt_meth.c:366:12
    open-quantum-safe#10 0x7f59a9334b58 in evp_keymgmt_util_gen /home/pawn/work/openssl/crypto/evp/keymgmt_lib.c:517:20
    open-quantum-safe#11 0x7f59a935a775 in EVP_PKEY_generate /home/pawn/work/openssl/crypto/evp/pmeth_gn.c:189:13
    open-quantum-safe#12 0x7f59a935b404 in EVP_PKEY_keygen /home/pawn/work/openssl/crypto/evp/pmeth_gn.c:274:12
    open-quantum-safe#13 0x7f59a4c69a51 in oqsx_key_gen_evp_key /home/pawn/work/oqs-provider/oqsprov/oqsprov_keys.c:1012:12
    open-quantum-safe#14 0x7f59a4c68faf in oqsx_key_gen /home/pawn/work/oqs-provider/oqsprov/oqsprov_keys.c:1084:16
    open-quantum-safe#15 0x7f59a4c716ca in oqsx_genkey /home/pawn/work/oqs-provider/oqsprov/oqs_kmgmt.c:496:9
    open-quantum-safe#16 0x7f59a4c6da78 in oqsx_gen /home/pawn/work/oqs-provider/oqsprov/oqs_kmgmt.c:509:12
    open-quantum-safe#17 0x7f59a93373c7 in evp_keymgmt_gen /home/pawn/work/openssl/crypto/evp/keymgmt_meth.c:366:12
    open-quantum-safe#18 0x7f59a9334b58 in evp_keymgmt_util_gen /home/pawn/work/openssl/crypto/evp/keymgmt_lib.c:517:20
    open-quantum-safe#19 0x7f59a935a775 in EVP_PKEY_generate /home/pawn/work/openssl/crypto/evp/pmeth_gn.c:189:13
    open-quantum-safe#20 0x555a6b1f2890 in test_oqs_signatures /home/pawn/work/oqs-provider/test/oqs_test_signatures.c:68:44
    open-quantum-safe#21 0x555a6b1f1e3d in main /home/pawn/work/oqs-provider/test/oqs_test_signatures.c:114:17
    open-quantum-safe#22 0x7f59a8c461c9 in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
```

## 6. Some `OPENSSL_free` were missing in `oqs_test_kems.c`

```
==569479==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 346035 byte(s) in 42 object(s) allocated from:
    #0 0x559c76ef524e in malloc (/home/thomas_bailleux/work/oqs-provider/build/test/oqs_test_kems+0xb924e) (BuildId: 76a98f5859ba3c90334cef4286e89aa76436b1ae)
    open-quantum-safe#1 0x7f04407a946b in CRYPTO_malloc /home/thomas_bailleux/work/openssl/crypto/mem.c:190:12
    open-quantum-safe#2 0x559c76f30389 in test_oqs_kems /home/thomas_bailleux/work/oqs-provider/test/oqs_test_kems.c:44:26
    open-quantum-safe#3 0x559c76f2fdfd in main /home/thomas_bailleux/work/oqs-provider/test/oqs_test_kems.c:93:17
    open-quantum-safe#4 0x7f04400461c9 in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16

Direct leak of 2690 byte(s) in 42 object(s) allocated from:
    #0 0x559c76ef524e in malloc (/home/thomas_bailleux/work/oqs-provider/build/test/oqs_test_kems+0xb924e) (BuildId: 76a98f5859ba3c90334cef4286e89aa76436b1ae)
    open-quantum-safe#1 0x7f04407a946b in CRYPTO_malloc /home/thomas_bailleux/work/openssl/crypto/mem.c:190:12
    open-quantum-safe#2 0x559c76f304a5 in test_oqs_kems /home/thomas_bailleux/work/oqs-provider/test/oqs_test_kems.c:47:29
    open-quantum-safe#3 0x559c76f2fdfd in main /home/thomas_bailleux/work/oqs-provider/test/oqs_test_kems.c:93:17
    open-quantum-safe#4 0x7f04400461c9 in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16

Direct leak of 2690 byte(s) in 42 object(s) allocated from:
    #0 0x559c76ef524e in malloc (/home/thomas_bailleux/work/oqs-provider/build/test/oqs_test_kems+0xb924e) (BuildId: 76a98f5859ba3c90334cef4286e89aa76436b1ae)
    open-quantum-safe#1 0x7f04407a946b in CRYPTO_malloc /home/thomas_bailleux/work/openssl/crypto/mem.c:190:12
    open-quantum-safe#2 0x559c76f303e4 in test_oqs_kems /home/thomas_bailleux/work/oqs-provider/test/oqs_test_kems.c:45:29
    open-quantum-safe#3 0x559c76f2fdfd in main /home/thomas_bailleux/work/oqs-provider/test/oqs_test_kems.c:93:17
    open-quantum-safe#4 0x7f04400461c9 in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
```


## 7. Bonus point: fix a potential double-free.

In `oqsx_key_gen`, when the keytype isn't `KEY_TYPE_HYB_SIG`, the
generated `pkey` is freed, and the flow goes down to the `err` label:

https://github.com/open-quantum-safe/oqs-provider/blob/cc3895f208730abfe1efc15f7c3ff4358ac6e4c0/oqsprov/oqsprov_keys.c#L1089-L1104

There is a potential double-free bug here: if `oqsx_key_gen_oqs` fails, then
`EVP_PKEY_free` will be called twice on the same `pkey` pointer.

Note that it is very unlikely that `oqsx_key_gen_oqs` fails, since it only
calls `OQS_KEM_keypair`/`OQS_SIG_keypair`.
thb-sb added a commit to thb-sb/oqs-provider that referenced this pull request Sep 5, 2023
@thb-sb
Fix various memory leaks.
6c9e1ce 
This commit fixes various memory leaks that have been found using ASan.

Note: I will soon add a CI job for testing oqs-provider against memory leaks
with ASan.


# 1. Memory leak in `oqs_test_signatures`.

In `oqs_test_signatures`, [`OPENSSL_free`] was used with `EVP_PKEY_CTX`. However,
the right destructor function for `EVP_PKEY_CTX` is [`EVP_PKEY_CTX_free`].

This commmit fixes this minor bug.

[`OPENSSL_free`](https://www.openssl.org/docs/man3.0/man3/OPENSSL_free.html)
[`EVP_PKEY_CTX_free`](https://www.openssl.org/docs/man3.0/man3/EVP_PKEY_CTX_free.html)


## 2. a selftest key that wasn't freed.

In `oqsprov/oqsprov_keys.c`, `d2i_PrivateKey` is used to verify that we the
key we've just generated is sound.
However, the returned object wasn't freed.
ASan trace that helped me finding this one:

```
Indirect leak of 96 byte(s) in 4 object(s) allocated from:
    #0 0x5651eef1428e in malloc (/home/pawn/work/oqs-provider/build/test/oqs_test_signatures+0xb928e) (BuildId: 6cc7ebd4bc73a0c78b2ab8d7b9dcb8b165d88e74)
    open-quantum-safe#1 0x7fb7a23a946b in CRYPTO_malloc /home/pawn/work/openssl/crypto/mem.c:190:12
    open-quantum-safe#2 0x7fb7a23a94a2 in CRYPTO_zalloc /home/pawn/work/openssl/crypto/mem.c:197:11
    open-quantum-safe#3 0x7fb7a209cdcd in BN_new /home/pawn/work/openssl/crypto/bn/bn_lib.c:247:16
    open-quantum-safe#4 0x7fb7a2224a00 in ossl_ec_GFp_mont_group_set_curve /home/pawn/work/openssl/crypto/ec/ecp_mont.c:169:11
    open-quantum-safe#5 0x7fb7a220f5fa in EC_GROUP_set_curve /home/pawn/work/openssl/crypto/ec/ec_lib.c:562:12
    open-quantum-safe#6 0x7fb7a2202d75 in EC_GROUP_new_curve_GFp /home/pawn/work/openssl/crypto/ec/ec_cvt.c:61:10
    open-quantum-safe#7 0x7fb7a220184d in ec_group_new_from_data /home/pawn/work/openssl/crypto/ec/ec_curve.c:3179:22
    open-quantum-safe#8 0x7fb7a2200ff9 in EC_GROUP_new_by_curve_name_ex /home/pawn/work/openssl/crypto/ec/ec_curve.c:3287:19
    open-quantum-safe#9 0x7fb7a2201ed9 in EC_GROUP_new_by_curve_name /home/pawn/work/openssl/crypto/ec/ec_curve.c:3303:12
    open-quantum-safe#10 0x7fb7a21f84e2 in EC_GROUP_new_from_ecpkparameters /home/pawn/work/openssl/crypto/ec/ec_asn1.c:863:20
    open-quantum-safe#11 0x7fb7a21f8f28 in d2i_ECPrivateKey /home/pawn/work/openssl/crypto/ec/ec_asn1.c:956:22
    open-quantum-safe#12 0x7fb7a270b4b5 in der2key_decode /home/pawn/work/openssl/providers/implementations/encode_decode/decode_der2key.c:220:19
    open-quantum-safe#13 0x7fb7a226eae8 in decoder_process /home/pawn/work/openssl/crypto/encode_decode/decoder_lib.c:971:14
    open-quantum-safe#14 0x7fb7a226d106 in OSSL_DECODER_from_bio /home/pawn/work/openssl/crypto/encode_decode/decoder_lib.c:82:10
    open-quantum-safe#15 0x7fb7a226f007 in OSSL_DECODER_from_data /home/pawn/work/openssl/crypto/encode_decode/decoder_lib.c:157:9
    open-quantum-safe#16 0x7fb7a200e123 in d2i_PrivateKey_decoder /home/pawn/work/openssl/crypto/asn1/d2i_pr.c:74:11
    open-quantum-safe#17 0x7fb7a200dbba in d2i_PrivateKey_ex /home/pawn/work/openssl/crypto/asn1/d2i_pr.c:162:11
    open-quantum-safe#18 0x7fb7a200e3d2 in d2i_PrivateKey /home/pawn/work/openssl/crypto/asn1/d2i_pr.c:172:12
    open-quantum-safe#19 0x7fb79dc6a22e in oqsx_key_gen_evp_key /home/pawn/work/oqs-provider/oqsprov/oqsprov_keys.c:1036:25
    open-quantum-safe#20 0x7fb79dc68f3f in oqsx_key_gen /home/pawn/work/oqs-provider/oqsprov/oqsprov_keys.c:1077:16
    open-quantum-safe#21 0x7fb79dc7164a in oqsx_genkey /home/pawn/work/oqs-provider/oqsprov/oqs_kmgmt.c:496:9
    open-quantum-safe#22 0x7fb79dc6d9f8 in oqsx_gen /home/pawn/work/oqs-provider/oqsprov/oqs_kmgmt.c:509:12
    open-quantum-safe#23 0x7fb7a23373c7 in evp_keymgmt_gen /home/pawn/work/openssl/crypto/evp/keymgmt_meth.c:366:12
    open-quantum-safe#24 0x7fb7a2334b58 in evp_keymgmt_util_gen /home/pawn/work/openssl/crypto/evp/keymgmt_lib.c:517:20
    open-quantum-safe#25 0x7fb7a235a775 in EVP_PKEY_generate /home/pawn/work/openssl/crypto/evp/pmeth_gn.c:189:13
    open-quantum-safe#26 0x5651eef4f2bd in test_oqs_signatures /home/pawn/work/oqs-provider/test/oqs_test_signatures.c:39:48
    open-quantum-safe#27 0x5651eef4ee3d in main /home/pawn/work/oqs-provider/test/oqs_test_signatures.c:114:17
    open-quantum-safe#28 0x7fb7a1b671c9 in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
```

## 3. a `EVP_PKEY_CTX` that wasn't freed.

In `oqsx_key_free`, `key->oqsx_provider_ctx.oqsx_evp_ctx->ctx` is freed if
the keytype is `ECP_HYB_KEM` or `ECX_HYB_KEM`. However,
`key->oqsx_provider_ctx.oqsx_evp_ctx->ctx` is also used for signatures.
In that case, a call to `EVP_PKEY_CTX_free` was missing.
ASan trace that helped me finding this one:

```
Indirect leak of 7 byte(s) in 2 object(s) allocated from:
    #0 0x55b6fc8d427e in malloc (/home/pawn/work/oqs-provider/build/test/oqs_test_signatures+0xb927e) (BuildId: 3a8bff15d3315ba7ac1746fdc7b67eb464c607a1)
    open-quantum-safe#1 0x7ff34dfa946b in CRYPTO_malloc /home/pawn/work/openssl/crypto/mem.c:190:12
    open-quantum-safe#2 0x7ff34dfac50a in CRYPTO_strndup /home/pawn/work/openssl/crypto/o_str.c:43:11
    open-quantum-safe#3 0x7ff34df9d152 in ossl_algorithm_get1_first_name /home/pawn/work/openssl/crypto/core_algorithm.c:195:11
    open-quantum-safe#4 0x7ff34df353a1 in keymgmt_from_algorithm /home/pawn/work/openssl/crypto/evp/keymgmt_meth.c:50:31
    open-quantum-safe#5 0x7ff34df15e44 in construct_evp_method /home/pawn/work/openssl/crypto/evp/evp_fetch.c:219:14
    open-quantum-safe#6 0x7ff34df9e3ce in ossl_method_construct_this /home/pawn/work/openssl/crypto/core_fetch.c:109:19
    open-quantum-safe#7 0x7ff34df9d740 in algorithm_do_map /home/pawn/work/openssl/crypto/core_algorithm.c:77:13
    open-quantum-safe#8 0x7ff34df9ce88 in algorithm_do_this /home/pawn/work/openssl/crypto/core_algorithm.c:122:15
    open-quantum-safe#9 0x7ff34dfcce3e in ossl_provider_doall_activated /home/pawn/work/openssl/crypto/provider_core.c:1423:14
    open-quantum-safe#10 0x7ff34df9ca64 in ossl_algorithm_do_all /home/pawn/work/openssl/crypto/core_algorithm.c:162:9
    open-quantum-safe#11 0x7ff34df9dd64 in ossl_method_construct /home/pawn/work/openssl/crypto/core_fetch.c:153:5
    open-quantum-safe#12 0x7ff34df13b6f in inner_evp_generic_fetch /home/pawn/work/openssl/crypto/evp/evp_fetch.c:312:23
    open-quantum-safe#13 0x7ff34df13226 in evp_generic_fetch /home/pawn/work/openssl/crypto/evp/evp_fetch.c:364:14
    open-quantum-safe#14 0x7ff34df36a42 in EVP_KEYMGMT_fetch /home/pawn/work/openssl/crypto/evp/keymgmt_meth.c:220:12
    open-quantum-safe#15 0x7ff34df5d6e4 in int_ctx_new /home/pawn/work/openssl/crypto/evp/pmeth_lib.c:280:23
    open-quantum-safe#16 0x7ff34df5f0a9 in EVP_PKEY_CTX_new_id /home/pawn/work/openssl/crypto/evp/pmeth_lib.c:469:12
    open-quantum-safe#17 0x7ff349865bcb in oqsx_hybsig_init /home/pawn/work/oqs-provider/oqsprov/oqsprov_keys.c:578:20
    open-quantum-safe#18 0x7ff349865157 in oqsx_key_new /home/pawn/work/oqs-provider/oqsprov/oqsprov_keys.c:791:16
    open-quantum-safe#19 0x7ff3498715a3 in oqsx_genkey /home/pawn/work/oqs-provider/oqsprov/oqs_kmgmt.c:487:16
    open-quantum-safe#20 0x7ff34986da08 in oqsx_gen /home/pawn/work/oqs-provider/oqsprov/oqs_kmgmt.c:509:12
    open-quantum-safe#21 0x7ff34df373c7 in evp_keymgmt_gen /home/pawn/work/openssl/crypto/evp/keymgmt_meth.c:366:12
    open-quantum-safe#22 0x7ff34df34b58 in evp_keymgmt_util_gen /home/pawn/work/openssl/crypto/evp/keymgmt_lib.c:517:20
    open-quantum-safe#23 0x7ff34df5a775 in EVP_PKEY_generate /home/pawn/work/openssl/crypto/evp/pmeth_gn.c:189:13
    open-quantum-safe#24 0x55b6fc90f2ad in test_oqs_signatures /home/pawn/work/oqs-provider/test/oqs_test_signatures.c:39:48
    open-quantum-safe#25 0x55b6fc90ee2d in main /home/pawn/work/oqs-provider/test/oqs_test_signatures.c:114:17
    open-quantum-safe#26 0x7ff34d8461c9 in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
```

## 4. Missing `EVP_PKEY_CTX_free` in case of error.

In `oqsx_hybsig_init`, a `EVP_PKEY_CTX` is initialized using `EVP_PKEY_CTX_new_id`.
However, later in the code, some extra work is done, and in case of error, the
`EVP_PKEY_CTX` wasn't freed.

(I couldn't recover the ASan message for this one…)


## 5. `OPENSSL_free` instead of `EVP_PKEY_free`.

In `oqsx_key_gen`, when a classical private key is generated, the field
`classical_pkey` of a `OQSX_KEY` is set with a value of type `EVP_PKEY`.

However, in `oqsx_key_free`, `OPENSSL_free` was used on `classical_pkey`
instead of `EVP_PKEY_free`.

ASan trace that helped me finding this one:

```
   #0 0x555a6b1b728e in malloc (/home/pawn/work/oqs-provider/build/test/oqs_test_signatures+0xb928e) (BuildId: 6cc7ebd4bc73a0c78b2ab8d7b9dcb8b165d88e74)
    open-quantum-safe#1 0x7f59a93a946b in CRYPTO_malloc /home/pawn/work/openssl/crypto/mem.c:190:12
    open-quantum-safe#2 0x7f59a93a94a2 in CRYPTO_zalloc /home/pawn/work/openssl/crypto/mem.c:197:11
    open-quantum-safe#3 0x7f59a909cdcd in BN_new /home/pawn/work/openssl/crypto/bn/bn_lib.c:247:16
    open-quantum-safe#4 0x7f59a920b955 in ossl_ec_group_new_ex /home/pawn/work/openssl/crypto/ec/ec_lib.c:61:25
    open-quantum-safe#5 0x7f59a920dd55 in EC_GROUP_dup /home/pawn/work/openssl/crypto/ec/ec_lib.c:273:14
    open-quantum-safe#6 0x7f59a9207774 in EC_KEY_set_group /home/pawn/work/openssl/crypto/ec/ec_key.c:733:18
    open-quantum-safe#7 0x7f59a975b3b5 in ec_gen_assign_group /home/pawn/work/openssl/providers/implementations/keymgmt/ec_kmgmt.c:1236:12
    open-quantum-safe#8 0x7f59a9758bdb in ec_gen /home/pawn/work/openssl/providers/implementations/keymgmt/ec_kmgmt.c:1274:11
    open-quantum-safe#9 0x7f59a93373c7 in evp_keymgmt_gen /home/pawn/work/openssl/crypto/evp/keymgmt_meth.c:366:12
    open-quantum-safe#10 0x7f59a9334b58 in evp_keymgmt_util_gen /home/pawn/work/openssl/crypto/evp/keymgmt_lib.c:517:20
    open-quantum-safe#11 0x7f59a935a775 in EVP_PKEY_generate /home/pawn/work/openssl/crypto/evp/pmeth_gn.c:189:13
    open-quantum-safe#12 0x7f59a935b404 in EVP_PKEY_keygen /home/pawn/work/openssl/crypto/evp/pmeth_gn.c:274:12
    open-quantum-safe#13 0x7f59a4c69a51 in oqsx_key_gen_evp_key /home/pawn/work/oqs-provider/oqsprov/oqsprov_keys.c:1012:12
    open-quantum-safe#14 0x7f59a4c68faf in oqsx_key_gen /home/pawn/work/oqs-provider/oqsprov/oqsprov_keys.c:1084:16
    open-quantum-safe#15 0x7f59a4c716ca in oqsx_genkey /home/pawn/work/oqs-provider/oqsprov/oqs_kmgmt.c:496:9
    open-quantum-safe#16 0x7f59a4c6da78 in oqsx_gen /home/pawn/work/oqs-provider/oqsprov/oqs_kmgmt.c:509:12
    open-quantum-safe#17 0x7f59a93373c7 in evp_keymgmt_gen /home/pawn/work/openssl/crypto/evp/keymgmt_meth.c:366:12
    open-quantum-safe#18 0x7f59a9334b58 in evp_keymgmt_util_gen /home/pawn/work/openssl/crypto/evp/keymgmt_lib.c:517:20
    open-quantum-safe#19 0x7f59a935a775 in EVP_PKEY_generate /home/pawn/work/openssl/crypto/evp/pmeth_gn.c:189:13
    open-quantum-safe#20 0x555a6b1f2890 in test_oqs_signatures /home/pawn/work/oqs-provider/test/oqs_test_signatures.c:68:44
    open-quantum-safe#21 0x555a6b1f1e3d in main /home/pawn/work/oqs-provider/test/oqs_test_signatures.c:114:17
    open-quantum-safe#22 0x7f59a8c461c9 in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
```

## 6. Some `OPENSSL_free` were missing in `oqs_test_kems.c`

```
==569479==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 346035 byte(s) in 42 object(s) allocated from:
    #0 0x559c76ef524e in malloc (/home/pawn/work/oqs-provider/build/test/oqs_test_kems+0xb924e) (BuildId: 76a98f5859ba3c90334cef4286e89aa76436b1ae)
    open-quantum-safe#1 0x7f04407a946b in CRYPTO_malloc /home/pawn/work/openssl/crypto/mem.c:190:12
    open-quantum-safe#2 0x559c76f30389 in test_oqs_kems /home/pawn/work/oqs-provider/test/oqs_test_kems.c:44:26
    open-quantum-safe#3 0x559c76f2fdfd in main /home/pawn/work/oqs-provider/test/oqs_test_kems.c:93:17
    open-quantum-safe#4 0x7f04400461c9 in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16

Direct leak of 2690 byte(s) in 42 object(s) allocated from:
    #0 0x559c76ef524e in malloc (/home/pawn/work/oqs-provider/build/test/oqs_test_kems+0xb924e) (BuildId: 76a98f5859ba3c90334cef4286e89aa76436b1ae)
    open-quantum-safe#1 0x7f04407a946b in CRYPTO_malloc /home/pawn/work/openssl/crypto/mem.c:190:12
    open-quantum-safe#2 0x559c76f304a5 in test_oqs_kems /home/pawn/work/oqs-provider/test/oqs_test_kems.c:47:29
    open-quantum-safe#3 0x559c76f2fdfd in main /home/pawn/work/oqs-provider/test/oqs_test_kems.c:93:17
    open-quantum-safe#4 0x7f04400461c9 in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16

Direct leak of 2690 byte(s) in 42 object(s) allocated from:
    #0 0x559c76ef524e in malloc (/home/pawn/work/oqs-provider/build/test/oqs_test_kems+0xb924e) (BuildId: 76a98f5859ba3c90334cef4286e89aa76436b1ae)
    open-quantum-safe#1 0x7f04407a946b in CRYPTO_malloc /home/pawn/work/openssl/crypto/mem.c:190:12
    open-quantum-safe#2 0x559c76f303e4 in test_oqs_kems /home/pawn/work/oqs-provider/test/oqs_test_kems.c:45:29
    open-quantum-safe#3 0x559c76f2fdfd in main /home/pawn/work/oqs-provider/test/oqs_test_kems.c:93:17
    open-quantum-safe#4 0x7f04400461c9 in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
```


## 7. Bonus point: fix a potential double-free.

In `oqsx_key_gen`, when the keytype isn't `KEY_TYPE_HYB_SIG`, the
generated `pkey` is freed, and the flow goes down to the `err` label:

https://github.com/open-quantum-safe/oqs-provider/blob/cc3895f208730abfe1efc15f7c3ff4358ac6e4c0/oqsprov/oqsprov_keys.c#L1089-L1104

There is a potential double-free bug here: if `oqsx_key_gen_oqs` fails, then
`EVP_PKEY_free` will be called twice on the same `pkey` pointer.

Note that it is very unlikely that `oqsx_key_gen_oqs` fails, since it only
calls `OQS_KEM_keypair`/`OQS_SIG_keypair`.
thb-sb added a commit to thb-sb/oqs-provider that referenced this pull request Sep 5, 2023
@thb-sb
Fix various memory leaks.
e9c402c 
This commit fixes various memory leaks that have been found using ASan.

Note: I will soon add a CI job for testing oqs-provider against memory leaks
with ASan.


# 1. Memory leak in `oqs_test_signatures`.

In `oqs_test_signatures`, [`OPENSSL_free`] was used with `EVP_PKEY_CTX`. However,
the right destructor function for `EVP_PKEY_CTX` is [`EVP_PKEY_CTX_free`].

This commmit fixes this minor bug.

[`OPENSSL_free`](https://www.openssl.org/docs/man3.0/man3/OPENSSL_free.html)
[`EVP_PKEY_CTX_free`](https://www.openssl.org/docs/man3.0/man3/EVP_PKEY_CTX_free.html)


## 2. a selftest key that wasn't freed.

In `oqsprov/oqsprov_keys.c`, `d2i_PrivateKey` is used to verify that we the
key we've just generated is sound.
However, the returned object wasn't freed.
ASan trace that helped me finding this one:

```
Indirect leak of 96 byte(s) in 4 object(s) allocated from:
    #0 0x5651eef1428e in malloc (/home/pawn/work/oqs-provider/build/test/oqs_test_signatures+0xb928e) (BuildId: 6cc7ebd4bc73a0c78b2ab8d7b9dcb8b165d88e74)
    open-quantum-safe#1 0x7fb7a23a946b in CRYPTO_malloc /home/pawn/work/openssl/crypto/mem.c:190:12
    open-quantum-safe#2 0x7fb7a23a94a2 in CRYPTO_zalloc /home/pawn/work/openssl/crypto/mem.c:197:11
    open-quantum-safe#3 0x7fb7a209cdcd in BN_new /home/pawn/work/openssl/crypto/bn/bn_lib.c:247:16
    open-quantum-safe#4 0x7fb7a2224a00 in ossl_ec_GFp_mont_group_set_curve /home/pawn/work/openssl/crypto/ec/ecp_mont.c:169:11
    open-quantum-safe#5 0x7fb7a220f5fa in EC_GROUP_set_curve /home/pawn/work/openssl/crypto/ec/ec_lib.c:562:12
    open-quantum-safe#6 0x7fb7a2202d75 in EC_GROUP_new_curve_GFp /home/pawn/work/openssl/crypto/ec/ec_cvt.c:61:10
    open-quantum-safe#7 0x7fb7a220184d in ec_group_new_from_data /home/pawn/work/openssl/crypto/ec/ec_curve.c:3179:22
    open-quantum-safe#8 0x7fb7a2200ff9 in EC_GROUP_new_by_curve_name_ex /home/pawn/work/openssl/crypto/ec/ec_curve.c:3287:19
    open-quantum-safe#9 0x7fb7a2201ed9 in EC_GROUP_new_by_curve_name /home/pawn/work/openssl/crypto/ec/ec_curve.c:3303:12
    open-quantum-safe#10 0x7fb7a21f84e2 in EC_GROUP_new_from_ecpkparameters /home/pawn/work/openssl/crypto/ec/ec_asn1.c:863:20
    open-quantum-safe#11 0x7fb7a21f8f28 in d2i_ECPrivateKey /home/pawn/work/openssl/crypto/ec/ec_asn1.c:956:22
    open-quantum-safe#12 0x7fb7a270b4b5 in der2key_decode /home/pawn/work/openssl/providers/implementations/encode_decode/decode_der2key.c:220:19
    open-quantum-safe#13 0x7fb7a226eae8 in decoder_process /home/pawn/work/openssl/crypto/encode_decode/decoder_lib.c:971:14
    open-quantum-safe#14 0x7fb7a226d106 in OSSL_DECODER_from_bio /home/pawn/work/openssl/crypto/encode_decode/decoder_lib.c:82:10
    open-quantum-safe#15 0x7fb7a226f007 in OSSL_DECODER_from_data /home/pawn/work/openssl/crypto/encode_decode/decoder_lib.c:157:9
    open-quantum-safe#16 0x7fb7a200e123 in d2i_PrivateKey_decoder /home/pawn/work/openssl/crypto/asn1/d2i_pr.c:74:11
    open-quantum-safe#17 0x7fb7a200dbba in d2i_PrivateKey_ex /home/pawn/work/openssl/crypto/asn1/d2i_pr.c:162:11
    open-quantum-safe#18 0x7fb7a200e3d2 in d2i_PrivateKey /home/pawn/work/openssl/crypto/asn1/d2i_pr.c:172:12
    open-quantum-safe#19 0x7fb79dc6a22e in oqsx_key_gen_evp_key /home/pawn/work/oqs-provider/oqsprov/oqsprov_keys.c:1036:25
    open-quantum-safe#20 0x7fb79dc68f3f in oqsx_key_gen /home/pawn/work/oqs-provider/oqsprov/oqsprov_keys.c:1077:16
    open-quantum-safe#21 0x7fb79dc7164a in oqsx_genkey /home/pawn/work/oqs-provider/oqsprov/oqs_kmgmt.c:496:9
    open-quantum-safe#22 0x7fb79dc6d9f8 in oqsx_gen /home/pawn/work/oqs-provider/oqsprov/oqs_kmgmt.c:509:12
    open-quantum-safe#23 0x7fb7a23373c7 in evp_keymgmt_gen /home/pawn/work/openssl/crypto/evp/keymgmt_meth.c:366:12
    open-quantum-safe#24 0x7fb7a2334b58 in evp_keymgmt_util_gen /home/pawn/work/openssl/crypto/evp/keymgmt_lib.c:517:20
    open-quantum-safe#25 0x7fb7a235a775 in EVP_PKEY_generate /home/pawn/work/openssl/crypto/evp/pmeth_gn.c:189:13
    open-quantum-safe#26 0x5651eef4f2bd in test_oqs_signatures /home/pawn/work/oqs-provider/test/oqs_test_signatures.c:39:48
    open-quantum-safe#27 0x5651eef4ee3d in main /home/pawn/work/oqs-provider/test/oqs_test_signatures.c:114:17
    open-quantum-safe#28 0x7fb7a1b671c9 in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
```

## 3. a `EVP_PKEY_CTX` that wasn't freed.

In `oqsx_key_free`, `key->oqsx_provider_ctx.oqsx_evp_ctx->ctx` is freed if
the keytype is `ECP_HYB_KEM` or `ECX_HYB_KEM`. However,
`key->oqsx_provider_ctx.oqsx_evp_ctx->ctx` is also used for signatures.
In that case, a call to `EVP_PKEY_CTX_free` was missing.
ASan trace that helped me finding this one:

```
Indirect leak of 7 byte(s) in 2 object(s) allocated from:
    #0 0x55b6fc8d427e in malloc (/home/pawn/work/oqs-provider/build/test/oqs_test_signatures+0xb927e) (BuildId: 3a8bff15d3315ba7ac1746fdc7b67eb464c607a1)
    open-quantum-safe#1 0x7ff34dfa946b in CRYPTO_malloc /home/pawn/work/openssl/crypto/mem.c:190:12
    open-quantum-safe#2 0x7ff34dfac50a in CRYPTO_strndup /home/pawn/work/openssl/crypto/o_str.c:43:11
    open-quantum-safe#3 0x7ff34df9d152 in ossl_algorithm_get1_first_name /home/pawn/work/openssl/crypto/core_algorithm.c:195:11
    open-quantum-safe#4 0x7ff34df353a1 in keymgmt_from_algorithm /home/pawn/work/openssl/crypto/evp/keymgmt_meth.c:50:31
    open-quantum-safe#5 0x7ff34df15e44 in construct_evp_method /home/pawn/work/openssl/crypto/evp/evp_fetch.c:219:14
    open-quantum-safe#6 0x7ff34df9e3ce in ossl_method_construct_this /home/pawn/work/openssl/crypto/core_fetch.c:109:19
    open-quantum-safe#7 0x7ff34df9d740 in algorithm_do_map /home/pawn/work/openssl/crypto/core_algorithm.c:77:13
    open-quantum-safe#8 0x7ff34df9ce88 in algorithm_do_this /home/pawn/work/openssl/crypto/core_algorithm.c:122:15
    open-quantum-safe#9 0x7ff34dfcce3e in ossl_provider_doall_activated /home/pawn/work/openssl/crypto/provider_core.c:1423:14
    open-quantum-safe#10 0x7ff34df9ca64 in ossl_algorithm_do_all /home/pawn/work/openssl/crypto/core_algorithm.c:162:9
    open-quantum-safe#11 0x7ff34df9dd64 in ossl_method_construct /home/pawn/work/openssl/crypto/core_fetch.c:153:5
    open-quantum-safe#12 0x7ff34df13b6f in inner_evp_generic_fetch /home/pawn/work/openssl/crypto/evp/evp_fetch.c:312:23
    open-quantum-safe#13 0x7ff34df13226 in evp_generic_fetch /home/pawn/work/openssl/crypto/evp/evp_fetch.c:364:14
    open-quantum-safe#14 0x7ff34df36a42 in EVP_KEYMGMT_fetch /home/pawn/work/openssl/crypto/evp/keymgmt_meth.c:220:12
    open-quantum-safe#15 0x7ff34df5d6e4 in int_ctx_new /home/pawn/work/openssl/crypto/evp/pmeth_lib.c:280:23
    open-quantum-safe#16 0x7ff34df5f0a9 in EVP_PKEY_CTX_new_id /home/pawn/work/openssl/crypto/evp/pmeth_lib.c:469:12
    open-quantum-safe#17 0x7ff349865bcb in oqsx_hybsig_init /home/pawn/work/oqs-provider/oqsprov/oqsprov_keys.c:578:20
    open-quantum-safe#18 0x7ff349865157 in oqsx_key_new /home/pawn/work/oqs-provider/oqsprov/oqsprov_keys.c:791:16
    open-quantum-safe#19 0x7ff3498715a3 in oqsx_genkey /home/pawn/work/oqs-provider/oqsprov/oqs_kmgmt.c:487:16
    open-quantum-safe#20 0x7ff34986da08 in oqsx_gen /home/pawn/work/oqs-provider/oqsprov/oqs_kmgmt.c:509:12
    open-quantum-safe#21 0x7ff34df373c7 in evp_keymgmt_gen /home/pawn/work/openssl/crypto/evp/keymgmt_meth.c:366:12
    open-quantum-safe#22 0x7ff34df34b58 in evp_keymgmt_util_gen /home/pawn/work/openssl/crypto/evp/keymgmt_lib.c:517:20
    open-quantum-safe#23 0x7ff34df5a775 in EVP_PKEY_generate /home/pawn/work/openssl/crypto/evp/pmeth_gn.c:189:13
    open-quantum-safe#24 0x55b6fc90f2ad in test_oqs_signatures /home/pawn/work/oqs-provider/test/oqs_test_signatures.c:39:48
    open-quantum-safe#25 0x55b6fc90ee2d in main /home/pawn/work/oqs-provider/test/oqs_test_signatures.c:114:17
    open-quantum-safe#26 0x7ff34d8461c9 in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
```

## 4. Missing `EVP_PKEY_CTX_free` in case of error.

In `oqsx_hybsig_init`, a `EVP_PKEY_CTX` is initialized using `EVP_PKEY_CTX_new_id`.
However, later in the code, some extra work is done, and in case of error, the
`EVP_PKEY_CTX` wasn't freed.

(I couldn't recover the ASan message for this one…)


## 5. `OPENSSL_free` instead of `EVP_PKEY_free`.

In `oqsx_key_gen`, when a classical private key is generated, the field
`classical_pkey` of a `OQSX_KEY` is set with a value of type `EVP_PKEY`.

However, in `oqsx_key_free`, `OPENSSL_free` was used on `classical_pkey`
instead of `EVP_PKEY_free`.

ASan trace that helped me finding this one:

```
   #0 0x555a6b1b728e in malloc (/home/pawn/work/oqs-provider/build/test/oqs_test_signatures+0xb928e) (BuildId: 6cc7ebd4bc73a0c78b2ab8d7b9dcb8b165d88e74)
    open-quantum-safe#1 0x7f59a93a946b in CRYPTO_malloc /home/pawn/work/openssl/crypto/mem.c:190:12
    open-quantum-safe#2 0x7f59a93a94a2 in CRYPTO_zalloc /home/pawn/work/openssl/crypto/mem.c:197:11
    open-quantum-safe#3 0x7f59a909cdcd in BN_new /home/pawn/work/openssl/crypto/bn/bn_lib.c:247:16
    open-quantum-safe#4 0x7f59a920b955 in ossl_ec_group_new_ex /home/pawn/work/openssl/crypto/ec/ec_lib.c:61:25
    open-quantum-safe#5 0x7f59a920dd55 in EC_GROUP_dup /home/pawn/work/openssl/crypto/ec/ec_lib.c:273:14
    open-quantum-safe#6 0x7f59a9207774 in EC_KEY_set_group /home/pawn/work/openssl/crypto/ec/ec_key.c:733:18
    open-quantum-safe#7 0x7f59a975b3b5 in ec_gen_assign_group /home/pawn/work/openssl/providers/implementations/keymgmt/ec_kmgmt.c:1236:12
    open-quantum-safe#8 0x7f59a9758bdb in ec_gen /home/pawn/work/openssl/providers/implementations/keymgmt/ec_kmgmt.c:1274:11
    open-quantum-safe#9 0x7f59a93373c7 in evp_keymgmt_gen /home/pawn/work/openssl/crypto/evp/keymgmt_meth.c:366:12
    open-quantum-safe#10 0x7f59a9334b58 in evp_keymgmt_util_gen /home/pawn/work/openssl/crypto/evp/keymgmt_lib.c:517:20
    open-quantum-safe#11 0x7f59a935a775 in EVP_PKEY_generate /home/pawn/work/openssl/crypto/evp/pmeth_gn.c:189:13
    open-quantum-safe#12 0x7f59a935b404 in EVP_PKEY_keygen /home/pawn/work/openssl/crypto/evp/pmeth_gn.c:274:12
    open-quantum-safe#13 0x7f59a4c69a51 in oqsx_key_gen_evp_key /home/pawn/work/oqs-provider/oqsprov/oqsprov_keys.c:1012:12
    open-quantum-safe#14 0x7f59a4c68faf in oqsx_key_gen /home/pawn/work/oqs-provider/oqsprov/oqsprov_keys.c:1084:16
    open-quantum-safe#15 0x7f59a4c716ca in oqsx_genkey /home/pawn/work/oqs-provider/oqsprov/oqs_kmgmt.c:496:9
    open-quantum-safe#16 0x7f59a4c6da78 in oqsx_gen /home/pawn/work/oqs-provider/oqsprov/oqs_kmgmt.c:509:12
    open-quantum-safe#17 0x7f59a93373c7 in evp_keymgmt_gen /home/pawn/work/openssl/crypto/evp/keymgmt_meth.c:366:12
    open-quantum-safe#18 0x7f59a9334b58 in evp_keymgmt_util_gen /home/pawn/work/openssl/crypto/evp/keymgmt_lib.c:517:20
    open-quantum-safe#19 0x7f59a935a775 in EVP_PKEY_generate /home/pawn/work/openssl/crypto/evp/pmeth_gn.c:189:13
    open-quantum-safe#20 0x555a6b1f2890 in test_oqs_signatures /home/pawn/work/oqs-provider/test/oqs_test_signatures.c:68:44
    open-quantum-safe#21 0x555a6b1f1e3d in main /home/pawn/work/oqs-provider/test/oqs_test_signatures.c:114:17
    open-quantum-safe#22 0x7f59a8c461c9 in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
```

## 6. Some `OPENSSL_free` were missing in `oqs_test_kems.c`

```
==569479==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 346035 byte(s) in 42 object(s) allocated from:
    #0 0x559c76ef524e in malloc (/home/pawn/work/oqs-provider/build/test/oqs_test_kems+0xb924e) (BuildId: 76a98f5859ba3c90334cef4286e89aa76436b1ae)
    open-quantum-safe#1 0x7f04407a946b in CRYPTO_malloc /home/pawn/work/openssl/crypto/mem.c:190:12
    open-quantum-safe#2 0x559c76f30389 in test_oqs_kems /home/pawn/work/oqs-provider/test/oqs_test_kems.c:44:26
    open-quantum-safe#3 0x559c76f2fdfd in main /home/pawn/work/oqs-provider/test/oqs_test_kems.c:93:17
    open-quantum-safe#4 0x7f04400461c9 in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16

Direct leak of 2690 byte(s) in 42 object(s) allocated from:
    #0 0x559c76ef524e in malloc (/home/pawn/work/oqs-provider/build/test/oqs_test_kems+0xb924e) (BuildId: 76a98f5859ba3c90334cef4286e89aa76436b1ae)
    open-quantum-safe#1 0x7f04407a946b in CRYPTO_malloc /home/pawn/work/openssl/crypto/mem.c:190:12
    open-quantum-safe#2 0x559c76f304a5 in test_oqs_kems /home/pawn/work/oqs-provider/test/oqs_test_kems.c:47:29
    open-quantum-safe#3 0x559c76f2fdfd in main /home/pawn/work/oqs-provider/test/oqs_test_kems.c:93:17
    open-quantum-safe#4 0x7f04400461c9 in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16

Direct leak of 2690 byte(s) in 42 object(s) allocated from:
    #0 0x559c76ef524e in malloc (/home/pawn/work/oqs-provider/build/test/oqs_test_kems+0xb924e) (BuildId: 76a98f5859ba3c90334cef4286e89aa76436b1ae)
    open-quantum-safe#1 0x7f04407a946b in CRYPTO_malloc /home/pawn/work/openssl/crypto/mem.c:190:12
    open-quantum-safe#2 0x559c76f303e4 in test_oqs_kems /home/pawn/work/oqs-provider/test/oqs_test_kems.c:45:29
    open-quantum-safe#3 0x559c76f2fdfd in main /home/pawn/work/oqs-provider/test/oqs_test_kems.c:93:17
    open-quantum-safe#4 0x7f04400461c9 in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
```

## 7. Missing `X509_PUBKEY_free` in `oqsx_d2i_PUBKEY`.

This was probably the hardest one for me to track down.

In `oqsx_d2i_PUBKEY`, a call is done to `oqsx_d2i_X509_PUBKEY_INTERNAL`.
However, the returned value (of type `X509_PUBKEY`) wasn't freed.

ASan:

```
Direct leak of 48 byte(s) in 1 object(s) allocated from:
    #0 0x55610d68132e in malloc (/home/pawn/work/oqs-provider/build/test/oqs_test_endecode+0xb932e) (BuildId: 53bfaa0a3b850edc7328f30be8b047ab6675ba3c)
    open-quantum-safe#1 0x7f68999a946b in CRYPTO_malloc /home/pawn/work/openssl/crypto/mem.c:190:12
    open-quantum-safe#2 0x7f68999a94a2 in CRYPTO_zalloc /home/pawn/work/openssl/crypto/mem.c:197:11
    open-quantum-safe#3 0x7f6895295897  (<unknown module>)
    open-quantum-safe#4 0x7f6895295ce4  (<unknown module>)
    open-quantum-safe#5 0x7f6895296a6c  (<unknown module>)
    open-quantum-safe#6 0x7f689986eae8 in decoder_process /home/pawn/work/openssl/crypto/encode_decode/decoder_lib.c:971:14
    open-quantum-safe#7 0x7f6899d13383 in spki2typespki_decode /home/pawn/work/openssl/providers/implementations/encode_decode/decode_spki2typespki.c:111:10
    open-quantum-safe#8 0x7f689986eae8 in decoder_process /home/pawn/work/openssl/crypto/encode_decode/decoder_lib.c:971:14
    open-quantum-safe#9 0x7f6899d114d3 in pem2der_decode /home/pawn/work/openssl/providers/implementations/encode_decode/decode_pem2der.c:204:14
    open-quantum-safe#10 0x7f689986eae8 in decoder_process /home/pawn/work/openssl/crypto/encode_decode/decoder_lib.c:971:14
    open-quantum-safe#11 0x7f689986d106 in OSSL_DECODER_from_bio /home/pawn/work/openssl/crypto/encode_decode/decoder_lib.c:82:10
    open-quantum-safe#12 0x55610d6bd407 in decode_EVP_PKEY_prov /home/pawn/work/oqs-provider/test/oqs_test_endecode.c:151:10
    open-quantum-safe#13 0x55610d6bc75b in test_oqs_encdec /home/pawn/work/oqs-provider/test/oqs_test_endecode.c:188:14
    open-quantum-safe#14 0x55610d6bbf37 in main /home/pawn/work/oqs-provider/test/oqs_test_endecode.c:240:17
    open-quantum-safe#15 0x7f68991671c9 in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16

Indirect leak of 1312 byte(s) in 1 object(s) allocated from:
    #0 0x55610d68132e in malloc (/home/pawn/work/oqs-provider/build/test/oqs_test_endecode+0xb932e) (BuildId: 53bfaa0a3b850edc7328f30be8b047ab6675ba3c)
    open-quantum-safe#1 0x7f68999a946b in CRYPTO_malloc /home/pawn/work/openssl/crypto/mem.c:190:12
    open-quantum-safe#2 0x7f68995d6b99 in ossl_c2i_ASN1_BIT_STRING /home/pawn/work/openssl/crypto/asn1/a_bitstr.c:117:13
    open-quantum-safe#3 0x7f689961f659 in asn1_ex_c2i /home/pawn/work/openssl/crypto/asn1/tasn_dec.c:885:14
    open-quantum-safe#4 0x7f689961bbc9 in asn1_d2i_ex_primitive /home/pawn/work/openssl/crypto/asn1/tasn_dec.c:818:10
    open-quantum-safe#5 0x7f6899618054 in asn1_item_embed_d2i /home/pawn/work/openssl/crypto/asn1/tasn_dec.c:217:16
    open-quantum-safe#6 0x7f689961e02f in asn1_template_noexp_d2i /home/pawn/work/openssl/crypto/asn1/tasn_dec.c:682:15
    open-quantum-safe#7 0x7f689961a7b4 in asn1_template_ex_d2i /home/pawn/work/openssl/crypto/asn1/tasn_dec.c:558:16
    open-quantum-safe#8 0x7f689961986a in asn1_item_embed_d2i /home/pawn/work/openssl/crypto/asn1/tasn_dec.c:422:19
    open-quantum-safe#9 0x7f68996175c8 in asn1_item_ex_d2i_intern /home/pawn/work/openssl/crypto/asn1/tasn_dec.c:118:10
    open-quantum-safe#10 0x7f68996177da in ASN1_item_d2i_ex /home/pawn/work/openssl/crypto/asn1/tasn_dec.c:144:9
    open-quantum-safe#11 0x7f6895295926  (<unknown module>)
    open-quantum-safe#12 0x7f6895295ce4  (<unknown module>)
    open-quantum-safe#13 0x7f6895296a6c  (<unknown module>)
    open-quantum-safe#14 0x7f689986eae8 in decoder_process /home/pawn/work/openssl/crypto/encode_decode/decoder_lib.c:971:14
    open-quantum-safe#15 0x7f6899d13383 in spki2typespki_decode /home/pawn/work/openssl/providers/implementations/encode_decode/decode_spki2typespki.c:111:10
    open-quantum-safe#16 0x7f689986eae8 in decoder_process /home/pawn/work/openssl/crypto/encode_decode/decoder_lib.c:971:14
    open-quantum-safe#17 0x7f6899d114d3 in pem2der_decode /home/pawn/work/openssl/providers/implementations/encode_decode/decode_pem2der.c:204:14
    open-quantum-safe#18 0x7f689986eae8 in decoder_process /home/pawn/work/openssl/crypto/encode_decode/decoder_lib.c:971:14
    open-quantum-safe#19 0x7f689986d106 in OSSL_DECODER_from_bio /home/pawn/work/openssl/crypto/encode_decode/decoder_lib.c:82:10
    open-quantum-safe#20 0x55610d6bd407 in decode_EVP_PKEY_prov /home/pawn/work/oqs-provider/test/oqs_test_endecode.c:151:10
    open-quantum-safe#21 0x55610d6bc75b in test_oqs_encdec /home/pawn/work/oqs-provider/test/oqs_test_endecode.c:188:14
    open-quantum-safe#22 0x55610d6bbf37 in main /home/pawn/work/oqs-provider/test/oqs_test_endecode.c:240:17
    open-quantum-safe#23 0x7f68991671c9 in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16

Indirect leak of 24 byte(s) in 1 object(s) allocated from:
    #0 0x55610d68132e in malloc (/home/pawn/work/oqs-provider/build/test/oqs_test_endecode+0xb932e) (BuildId: 53bfaa0a3b850edc7328f30be8b047ab6675ba3c)
    open-quantum-safe#1 0x7f68999a946b in CRYPTO_malloc /home/pawn/work/openssl/crypto/mem.c:190:12
    open-quantum-safe#2 0x7f68999a94a2 in CRYPTO_zalloc /home/pawn/work/openssl/crypto/mem.c:197:11
    open-quantum-safe#3 0x7f68995fde90 in ASN1_STRING_type_new /home/pawn/work/openssl/crypto/asn1/asn1_lib.c:350:11
    open-quantum-safe#4 0x7f689962a7dd in ASN1_BIT_STRING_new /home/pawn/work/openssl/crypto/asn1/tasn_typ.c:31:1
    open-quantum-safe#5 0x7f68995d6a0f in ossl_c2i_ASN1_BIT_STRING /home/pawn/work/openssl/crypto/asn1/a_bitstr.c:98:20
    open-quantum-safe#6 0x7f689961f659 in asn1_ex_c2i /home/pawn/work/openssl/crypto/asn1/tasn_dec.c:885:14
    open-quantum-safe#7 0x7f689961bbc9 in asn1_d2i_ex_primitive /home/pawn/work/openssl/crypto/asn1/tasn_dec.c:818:10
    open-quantum-safe#8 0x7f6899618054 in asn1_item_embed_d2i /home/pawn/work/openssl/crypto/asn1/tasn_dec.c:217:16
    open-quantum-safe#9 0x7f689961e02f in asn1_template_noexp_d2i /home/pawn/work/openssl/crypto/asn1/tasn_dec.c:682:15
    open-quantum-safe#10 0x7f689961a7b4 in asn1_template_ex_d2i /home/pawn/work/openssl/crypto/asn1/tasn_dec.c:558:16
    open-quantum-safe#11 0x7f689961986a in asn1_item_embed_d2i /home/pawn/work/openssl/crypto/asn1/tasn_dec.c:422:19
    open-quantum-safe#12 0x7f68996175c8 in asn1_item_ex_d2i_intern /home/pawn/work/openssl/crypto/asn1/tasn_dec.c:118:10
    open-quantum-safe#13 0x7f68996177da in ASN1_item_d2i_ex /home/pawn/work/openssl/crypto/asn1/tasn_dec.c:144:9
    open-quantum-safe#14 0x7f6895295926  (<unknown module>)
    open-quantum-safe#15 0x7f6895295ce4  (<unknown module>)
    open-quantum-safe#16 0x7f6895296a6c  (<unknown module>)
    open-quantum-safe#17 0x7f689986eae8 in decoder_process /home/pawn/work/openssl/crypto/encode_decode/decoder_lib.c:971:14
    open-quantum-safe#18 0x7f6899d13383 in spki2typespki_decode /home/pawn/work/openssl/providers/implementations/encode_decode/decode_spki2typespki.c:111:10
    open-quantum-safe#19 0x7f689986eae8 in decoder_process /home/pawn/work/openssl/crypto/encode_decode/decoder_lib.c:971:14
    open-quantum-safe#20 0x7f6899d114d3 in pem2der_decode /home/pawn/work/openssl/providers/implementations/encode_decode/decode_pem2der.c:204:14
    open-quantum-safe#21 0x7f689986eae8 in decoder_process /home/pawn/work/openssl/crypto/encode_decode/decoder_lib.c:971:14
    open-quantum-safe#22 0x7f689986d106 in OSSL_DECODER_from_bio /home/pawn/work/openssl/crypto/encode_decode/decoder_lib.c:82:10
    open-quantum-safe#23 0x55610d6bd407 in decode_EVP_PKEY_prov /home/pawn/work/oqs-provider/test/oqs_test_endecode.c:151:10
    open-quantum-safe#24 0x55610d6bc75b in test_oqs_encdec /home/pawn/work/oqs-provider/test/oqs_test_endecode.c:188:14
    open-quantum-safe#25 0x55610d6bbf37 in main /home/pawn/work/oqs-provider/test/oqs_test_endecode.c:240:17
    open-quantum-safe#26 0x7f68991671c9 in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16

Indirect leak of 16 byte(s) in 1 object(s) allocated from:
    #0 0x55610d68132e in malloc (/home/pawn/work/oqs-provider/build/test/oqs_test_endecode+0xb932e) (BuildId: 53bfaa0a3b850edc7328f30be8b047ab6675ba3c)
    open-quantum-safe#1 0x7f68999a946b in CRYPTO_malloc /home/pawn/work/openssl/crypto/mem.c:190:12
    open-quantum-safe#2 0x7f68999a94a2 in CRYPTO_zalloc /home/pawn/work/openssl/crypto/mem.c:197:11
    open-quantum-safe#3 0x7f6899625ed4 in asn1_item_embed_new /home/pawn/work/openssl/crypto/asn1/tasn_new.c:136:21
    open-quantum-safe#4 0x7f689962620e in ossl_asn1_item_ex_new_intern /home/pawn/work/openssl/crypto/asn1/tasn_new.c:52:12
    open-quantum-safe#5 0x7f68996191a5 in asn1_item_embed_d2i /home/pawn/work/openssl/crypto/asn1/tasn_dec.c:366:21
    open-quantum-safe#6 0x7f689961e02f in asn1_template_noexp_d2i /home/pawn/work/openssl/crypto/asn1/tasn_dec.c:682:15
    open-quantum-safe#7 0x7f689961a7b4 in asn1_template_ex_d2i /home/pawn/work/openssl/crypto/asn1/tasn_dec.c:558:16
    open-quantum-safe#8 0x7f689961986a in asn1_item_embed_d2i /home/pawn/work/openssl/crypto/asn1/tasn_dec.c:422:19
    open-quantum-safe#9 0x7f68996175c8 in asn1_item_ex_d2i_intern /home/pawn/work/openssl/crypto/asn1/tasn_dec.c:118:10
    open-quantum-safe#10 0x7f68996177da in ASN1_item_d2i_ex /home/pawn/work/openssl/crypto/asn1/tasn_dec.c:144:9
    open-quantum-safe#11 0x7f6895295926  (<unknown module>)
    open-quantum-safe#12 0x7f6895295ce4  (<unknown module>)
    open-quantum-safe#13 0x7f6895296a6c  (<unknown module>)
    open-quantum-safe#14 0x7f689986eae8 in decoder_process /home/pawn/work/openssl/crypto/encode_decode/decoder_lib.c:971:14
    open-quantum-safe#15 0x7f6899d13383 in spki2typespki_decode /home/pawn/work/openssl/providers/implementations/encode_decode/decode_spki2typespki.c:111:10
    open-quantum-safe#16 0x7f689986eae8 in decoder_process /home/pawn/work/openssl/crypto/encode_decode/decoder_lib.c:971:14
    open-quantum-safe#17 0x7f6899d114d3 in pem2der_decode /home/pawn/work/openssl/providers/implementations/encode_decode/decode_pem2der.c:204:14
    open-quantum-safe#18 0x7f689986eae8 in decoder_process /home/pawn/work/openssl/crypto/encode_decode/decoder_lib.c:971:14
    open-quantum-safe#19 0x7f689986d106 in OSSL_DECODER_from_bio /home/pawn/work/openssl/crypto/encode_decode/decoder_lib.c:82:10
    open-quantum-safe#20 0x55610d6bd407 in decode_EVP_PKEY_prov /home/pawn/work/oqs-provider/test/oqs_test_endecode.c:151:10
    open-quantum-safe#21 0x55610d6bc75b in test_oqs_encdec /home/pawn/work/oqs-provider/test/oqs_test_endecode.c:188:14
    open-quantum-safe#22 0x55610d6bbf37 in main /home/pawn/work/oqs-provider/test/oqs_test_endecode.c:240:17
    open-quantum-safe#23 0x7f68991671c9 in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16

SUMMARY: AddressSanitizer: 1400 byte(s) leaked in 4 allocation(s).
```


## 8. Bonus point: fix a potential double-free.

In `oqsx_key_gen`, when the keytype isn't `KEY_TYPE_HYB_SIG`, the
generated `pkey` is freed, and the flow goes down to the `err` label:

https://github.com/open-quantum-safe/oqs-provider/blob/cc3895f208730abfe1efc15f7c3ff4358ac6e4c0/oqsprov/oqsprov_keys.c#L1089-L1104

There is a potential double-free bug here: if `oqsx_key_gen_oqs` fails, then
`EVP_PKEY_free` will be called twice on the same `pkey` pointer.

Note that it is very unlikely that `oqsx_key_gen_oqs` fails, since it only
calls `OQS_KEM_keypair`/`OQS_SIG_keypair`.
thb-sb added a commit to thb-sb/oqs-provider that referenced this pull request Sep 5, 2023
@thb-sb
Fix various memory leaks.
828be57 
This commit fixes various memory leaks that have been found using ASan.

Note: I will soon add a CI job for testing oqs-provider against memory leaks
with ASan.


# 1. Memory leak in `oqs_test_signatures`.

In `oqs_test_signatures`, [`OPENSSL_free`] was used with `EVP_PKEY_CTX`. However,
the right destructor function for `EVP_PKEY_CTX` is [`EVP_PKEY_CTX_free`].

This commmit fixes this minor bug.

[`OPENSSL_free`](https://www.openssl.org/docs/man3.0/man3/OPENSSL_free.html)
[`EVP_PKEY_CTX_free`](https://www.openssl.org/docs/man3.0/man3/EVP_PKEY_CTX_free.html)


## 2. a selftest key that wasn't freed.

In `oqsprov/oqsprov_keys.c`, `d2i_PrivateKey` is used to verify that we the
key we've just generated is sound.
However, the returned object wasn't freed.
ASan trace that helped me finding this one:

```
Indirect leak of 96 byte(s) in 4 object(s) allocated from:
    #0 0x5651eef1428e in malloc (/home/pawn/work/oqs-provider/build/test/oqs_test_signatures+0xb928e) (BuildId: 6cc7ebd4bc73a0c78b2ab8d7b9dcb8b165d88e74)
    open-quantum-safe#1 0x7fb7a23a946b in CRYPTO_malloc /home/pawn/work/openssl/crypto/mem.c:190:12
    open-quantum-safe#2 0x7fb7a23a94a2 in CRYPTO_zalloc /home/pawn/work/openssl/crypto/mem.c:197:11
    open-quantum-safe#3 0x7fb7a209cdcd in BN_new /home/pawn/work/openssl/crypto/bn/bn_lib.c:247:16
    open-quantum-safe#4 0x7fb7a2224a00 in ossl_ec_GFp_mont_group_set_curve /home/pawn/work/openssl/crypto/ec/ecp_mont.c:169:11
    open-quantum-safe#5 0x7fb7a220f5fa in EC_GROUP_set_curve /home/pawn/work/openssl/crypto/ec/ec_lib.c:562:12
    open-quantum-safe#6 0x7fb7a2202d75 in EC_GROUP_new_curve_GFp /home/pawn/work/openssl/crypto/ec/ec_cvt.c:61:10
    open-quantum-safe#7 0x7fb7a220184d in ec_group_new_from_data /home/pawn/work/openssl/crypto/ec/ec_curve.c:3179:22
    open-quantum-safe#8 0x7fb7a2200ff9 in EC_GROUP_new_by_curve_name_ex /home/pawn/work/openssl/crypto/ec/ec_curve.c:3287:19
    open-quantum-safe#9 0x7fb7a2201ed9 in EC_GROUP_new_by_curve_name /home/pawn/work/openssl/crypto/ec/ec_curve.c:3303:12
    open-quantum-safe#10 0x7fb7a21f84e2 in EC_GROUP_new_from_ecpkparameters /home/pawn/work/openssl/crypto/ec/ec_asn1.c:863:20
    open-quantum-safe#11 0x7fb7a21f8f28 in d2i_ECPrivateKey /home/pawn/work/openssl/crypto/ec/ec_asn1.c:956:22
    open-quantum-safe#12 0x7fb7a270b4b5 in der2key_decode /home/pawn/work/openssl/providers/implementations/encode_decode/decode_der2key.c:220:19
    open-quantum-safe#13 0x7fb7a226eae8 in decoder_process /home/pawn/work/openssl/crypto/encode_decode/decoder_lib.c:971:14
    open-quantum-safe#14 0x7fb7a226d106 in OSSL_DECODER_from_bio /home/pawn/work/openssl/crypto/encode_decode/decoder_lib.c:82:10
    open-quantum-safe#15 0x7fb7a226f007 in OSSL_DECODER_from_data /home/pawn/work/openssl/crypto/encode_decode/decoder_lib.c:157:9
    open-quantum-safe#16 0x7fb7a200e123 in d2i_PrivateKey_decoder /home/pawn/work/openssl/crypto/asn1/d2i_pr.c:74:11
    open-quantum-safe#17 0x7fb7a200dbba in d2i_PrivateKey_ex /home/pawn/work/openssl/crypto/asn1/d2i_pr.c:162:11
    open-quantum-safe#18 0x7fb7a200e3d2 in d2i_PrivateKey /home/pawn/work/openssl/crypto/asn1/d2i_pr.c:172:12
    open-quantum-safe#19 0x7fb79dc6a22e in oqsx_key_gen_evp_key /home/pawn/work/oqs-provider/oqsprov/oqsprov_keys.c:1036:25
    open-quantum-safe#20 0x7fb79dc68f3f in oqsx_key_gen /home/pawn/work/oqs-provider/oqsprov/oqsprov_keys.c:1077:16
    open-quantum-safe#21 0x7fb79dc7164a in oqsx_genkey /home/pawn/work/oqs-provider/oqsprov/oqs_kmgmt.c:496:9
    open-quantum-safe#22 0x7fb79dc6d9f8 in oqsx_gen /home/pawn/work/oqs-provider/oqsprov/oqs_kmgmt.c:509:12
    open-quantum-safe#23 0x7fb7a23373c7 in evp_keymgmt_gen /home/pawn/work/openssl/crypto/evp/keymgmt_meth.c:366:12
    open-quantum-safe#24 0x7fb7a2334b58 in evp_keymgmt_util_gen /home/pawn/work/openssl/crypto/evp/keymgmt_lib.c:517:20
    open-quantum-safe#25 0x7fb7a235a775 in EVP_PKEY_generate /home/pawn/work/openssl/crypto/evp/pmeth_gn.c:189:13
    open-quantum-safe#26 0x5651eef4f2bd in test_oqs_signatures /home/pawn/work/oqs-provider/test/oqs_test_signatures.c:39:48
    open-quantum-safe#27 0x5651eef4ee3d in main /home/pawn/work/oqs-provider/test/oqs_test_signatures.c:114:17
    open-quantum-safe#28 0x7fb7a1b671c9 in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
```

## 3. a `EVP_PKEY_CTX` that wasn't freed.

In `oqsx_key_free`, `key->oqsx_provider_ctx.oqsx_evp_ctx->ctx` is freed if
the keytype is `ECP_HYB_KEM` or `ECX_HYB_KEM`. However,
`key->oqsx_provider_ctx.oqsx_evp_ctx->ctx` is also used for signatures.
In that case, a call to `EVP_PKEY_CTX_free` was missing.
ASan trace that helped me finding this one:

```
Indirect leak of 7 byte(s) in 2 object(s) allocated from:
    #0 0x55b6fc8d427e in malloc (/home/pawn/work/oqs-provider/build/test/oqs_test_signatures+0xb927e) (BuildId: 3a8bff15d3315ba7ac1746fdc7b67eb464c607a1)
    open-quantum-safe#1 0x7ff34dfa946b in CRYPTO_malloc /home/pawn/work/openssl/crypto/mem.c:190:12
    open-quantum-safe#2 0x7ff34dfac50a in CRYPTO_strndup /home/pawn/work/openssl/crypto/o_str.c:43:11
    open-quantum-safe#3 0x7ff34df9d152 in ossl_algorithm_get1_first_name /home/pawn/work/openssl/crypto/core_algorithm.c:195:11
    open-quantum-safe#4 0x7ff34df353a1 in keymgmt_from_algorithm /home/pawn/work/openssl/crypto/evp/keymgmt_meth.c:50:31
    open-quantum-safe#5 0x7ff34df15e44 in construct_evp_method /home/pawn/work/openssl/crypto/evp/evp_fetch.c:219:14
    open-quantum-safe#6 0x7ff34df9e3ce in ossl_method_construct_this /home/pawn/work/openssl/crypto/core_fetch.c:109:19
    open-quantum-safe#7 0x7ff34df9d740 in algorithm_do_map /home/pawn/work/openssl/crypto/core_algorithm.c:77:13
    open-quantum-safe#8 0x7ff34df9ce88 in algorithm_do_this /home/pawn/work/openssl/crypto/core_algorithm.c:122:15
    open-quantum-safe#9 0x7ff34dfcce3e in ossl_provider_doall_activated /home/pawn/work/openssl/crypto/provider_core.c:1423:14
    open-quantum-safe#10 0x7ff34df9ca64 in ossl_algorithm_do_all /home/pawn/work/openssl/crypto/core_algorithm.c:162:9
    open-quantum-safe#11 0x7ff34df9dd64 in ossl_method_construct /home/pawn/work/openssl/crypto/core_fetch.c:153:5
    open-quantum-safe#12 0x7ff34df13b6f in inner_evp_generic_fetch /home/pawn/work/openssl/crypto/evp/evp_fetch.c:312:23
    open-quantum-safe#13 0x7ff34df13226 in evp_generic_fetch /home/pawn/work/openssl/crypto/evp/evp_fetch.c:364:14
    open-quantum-safe#14 0x7ff34df36a42 in EVP_KEYMGMT_fetch /home/pawn/work/openssl/crypto/evp/keymgmt_meth.c:220:12
    open-quantum-safe#15 0x7ff34df5d6e4 in int_ctx_new /home/pawn/work/openssl/crypto/evp/pmeth_lib.c:280:23
    open-quantum-safe#16 0x7ff34df5f0a9 in EVP_PKEY_CTX_new_id /home/pawn/work/openssl/crypto/evp/pmeth_lib.c:469:12
    open-quantum-safe#17 0x7ff349865bcb in oqsx_hybsig_init /home/pawn/work/oqs-provider/oqsprov/oqsprov_keys.c:578:20
    open-quantum-safe#18 0x7ff349865157 in oqsx_key_new /home/pawn/work/oqs-provider/oqsprov/oqsprov_keys.c:791:16
    open-quantum-safe#19 0x7ff3498715a3 in oqsx_genkey /home/pawn/work/oqs-provider/oqsprov/oqs_kmgmt.c:487:16
    open-quantum-safe#20 0x7ff34986da08 in oqsx_gen /home/pawn/work/oqs-provider/oqsprov/oqs_kmgmt.c:509:12
    open-quantum-safe#21 0x7ff34df373c7 in evp_keymgmt_gen /home/pawn/work/openssl/crypto/evp/keymgmt_meth.c:366:12
    open-quantum-safe#22 0x7ff34df34b58 in evp_keymgmt_util_gen /home/pawn/work/openssl/crypto/evp/keymgmt_lib.c:517:20
    open-quantum-safe#23 0x7ff34df5a775 in EVP_PKEY_generate /home/pawn/work/openssl/crypto/evp/pmeth_gn.c:189:13
    open-quantum-safe#24 0x55b6fc90f2ad in test_oqs_signatures /home/pawn/work/oqs-provider/test/oqs_test_signatures.c:39:48
    open-quantum-safe#25 0x55b6fc90ee2d in main /home/pawn/work/oqs-provider/test/oqs_test_signatures.c:114:17
    open-quantum-safe#26 0x7ff34d8461c9 in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
```

## 4. Missing `EVP_PKEY_CTX_free` in case of error.

In `oqsx_hybsig_init`, a `EVP_PKEY_CTX` is initialized using `EVP_PKEY_CTX_new_id`.
However, later in the code, some extra work is done, and in case of error, the
`EVP_PKEY_CTX` wasn't freed.

(I couldn't recover the ASan message for this one…)


## 5. `OPENSSL_free` instead of `EVP_PKEY_free`.

In `oqsx_key_gen`, when a classical private key is generated, the field
`classical_pkey` of a `OQSX_KEY` is set with a value of type `EVP_PKEY`.

However, in `oqsx_key_free`, `OPENSSL_free` was used on `classical_pkey`
instead of `EVP_PKEY_free`.

ASan trace that helped me finding this one:

```
   #0 0x555a6b1b728e in malloc (/home/pawn/work/oqs-provider/build/test/oqs_test_signatures+0xb928e) (BuildId: 6cc7ebd4bc73a0c78b2ab8d7b9dcb8b165d88e74)
    open-quantum-safe#1 0x7f59a93a946b in CRYPTO_malloc /home/pawn/work/openssl/crypto/mem.c:190:12
    open-quantum-safe#2 0x7f59a93a94a2 in CRYPTO_zalloc /home/pawn/work/openssl/crypto/mem.c:197:11
    open-quantum-safe#3 0x7f59a909cdcd in BN_new /home/pawn/work/openssl/crypto/bn/bn_lib.c:247:16
    open-quantum-safe#4 0x7f59a920b955 in ossl_ec_group_new_ex /home/pawn/work/openssl/crypto/ec/ec_lib.c:61:25
    open-quantum-safe#5 0x7f59a920dd55 in EC_GROUP_dup /home/pawn/work/openssl/crypto/ec/ec_lib.c:273:14
    open-quantum-safe#6 0x7f59a9207774 in EC_KEY_set_group /home/pawn/work/openssl/crypto/ec/ec_key.c:733:18
    open-quantum-safe#7 0x7f59a975b3b5 in ec_gen_assign_group /home/pawn/work/openssl/providers/implementations/keymgmt/ec_kmgmt.c:1236:12
    open-quantum-safe#8 0x7f59a9758bdb in ec_gen /home/pawn/work/openssl/providers/implementations/keymgmt/ec_kmgmt.c:1274:11
    open-quantum-safe#9 0x7f59a93373c7 in evp_keymgmt_gen /home/pawn/work/openssl/crypto/evp/keymgmt_meth.c:366:12
    open-quantum-safe#10 0x7f59a9334b58 in evp_keymgmt_util_gen /home/pawn/work/openssl/crypto/evp/keymgmt_lib.c:517:20
    open-quantum-safe#11 0x7f59a935a775 in EVP_PKEY_generate /home/pawn/work/openssl/crypto/evp/pmeth_gn.c:189:13
    open-quantum-safe#12 0x7f59a935b404 in EVP_PKEY_keygen /home/pawn/work/openssl/crypto/evp/pmeth_gn.c:274:12
    open-quantum-safe#13 0x7f59a4c69a51 in oqsx_key_gen_evp_key /home/pawn/work/oqs-provider/oqsprov/oqsprov_keys.c:1012:12
    open-quantum-safe#14 0x7f59a4c68faf in oqsx_key_gen /home/pawn/work/oqs-provider/oqsprov/oqsprov_keys.c:1084:16
    open-quantum-safe#15 0x7f59a4c716ca in oqsx_genkey /home/pawn/work/oqs-provider/oqsprov/oqs_kmgmt.c:496:9
    open-quantum-safe#16 0x7f59a4c6da78 in oqsx_gen /home/pawn/work/oqs-provider/oqsprov/oqs_kmgmt.c:509:12
    open-quantum-safe#17 0x7f59a93373c7 in evp_keymgmt_gen /home/pawn/work/openssl/crypto/evp/keymgmt_meth.c:366:12
    open-quantum-safe#18 0x7f59a9334b58 in evp_keymgmt_util_gen /home/pawn/work/openssl/crypto/evp/keymgmt_lib.c:517:20
    open-quantum-safe#19 0x7f59a935a775 in EVP_PKEY_generate /home/pawn/work/openssl/crypto/evp/pmeth_gn.c:189:13
    open-quantum-safe#20 0x555a6b1f2890 in test_oqs_signatures /home/pawn/work/oqs-provider/test/oqs_test_signatures.c:68:44
    open-quantum-safe#21 0x555a6b1f1e3d in main /home/pawn/work/oqs-provider/test/oqs_test_signatures.c:114:17
    open-quantum-safe#22 0x7f59a8c461c9 in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
```

## 6. Some `OPENSSL_free` were missing in `oqs_test_kems.c`

```
==569479==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 346035 byte(s) in 42 object(s) allocated from:
    #0 0x559c76ef524e in malloc (/home/pawn/work/oqs-provider/build/test/oqs_test_kems+0xb924e) (BuildId: 76a98f5859ba3c90334cef4286e89aa76436b1ae)
    open-quantum-safe#1 0x7f04407a946b in CRYPTO_malloc /home/pawn/work/openssl/crypto/mem.c:190:12
    open-quantum-safe#2 0x559c76f30389 in test_oqs_kems /home/pawn/work/oqs-provider/test/oqs_test_kems.c:44:26
    open-quantum-safe#3 0x559c76f2fdfd in main /home/pawn/work/oqs-provider/test/oqs_test_kems.c:93:17
    open-quantum-safe#4 0x7f04400461c9 in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16

Direct leak of 2690 byte(s) in 42 object(s) allocated from:
    #0 0x559c76ef524e in malloc (/home/pawn/work/oqs-provider/build/test/oqs_test_kems+0xb924e) (BuildId: 76a98f5859ba3c90334cef4286e89aa76436b1ae)
    open-quantum-safe#1 0x7f04407a946b in CRYPTO_malloc /home/pawn/work/openssl/crypto/mem.c:190:12
    open-quantum-safe#2 0x559c76f304a5 in test_oqs_kems /home/pawn/work/oqs-provider/test/oqs_test_kems.c:47:29
    open-quantum-safe#3 0x559c76f2fdfd in main /home/pawn/work/oqs-provider/test/oqs_test_kems.c:93:17
    open-quantum-safe#4 0x7f04400461c9 in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16

Direct leak of 2690 byte(s) in 42 object(s) allocated from:
    #0 0x559c76ef524e in malloc (/home/pawn/work/oqs-provider/build/test/oqs_test_kems+0xb924e) (BuildId: 76a98f5859ba3c90334cef4286e89aa76436b1ae)
    open-quantum-safe#1 0x7f04407a946b in CRYPTO_malloc /home/pawn/work/openssl/crypto/mem.c:190:12
    open-quantum-safe#2 0x559c76f303e4 in test_oqs_kems /home/pawn/work/oqs-provider/test/oqs_test_kems.c:45:29
    open-quantum-safe#3 0x559c76f2fdfd in main /home/pawn/work/oqs-provider/test/oqs_test_kems.c:93:17
    open-quantum-safe#4 0x7f04400461c9 in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
```

## 7. Missing `X509_PUBKEY_free` in `oqsx_d2i_PUBKEY`.

This was probably the hardest one for me to track down.

In `oqsx_d2i_PUBKEY`, a call is done to `oqsx_d2i_X509_PUBKEY_INTERNAL`.
However, the returned value (of type `X509_PUBKEY`) wasn't freed.

ASan:

```
Direct leak of 48 byte(s) in 1 object(s) allocated from:
    #0 0x55610d68132e in malloc (/home/pawn/work/oqs-provider/build/test/oqs_test_endecode+0xb932e) (BuildId: 53bfaa0a3b850edc7328f30be8b047ab6675ba3c)
    open-quantum-safe#1 0x7f68999a946b in CRYPTO_malloc /home/pawn/work/openssl/crypto/mem.c:190:12
    open-quantum-safe#2 0x7f68999a94a2 in CRYPTO_zalloc /home/pawn/work/openssl/crypto/mem.c:197:11
    open-quantum-safe#3 0x7f6895295897  (<unknown module>)
    open-quantum-safe#4 0x7f6895295ce4  (<unknown module>)
    open-quantum-safe#5 0x7f6895296a6c  (<unknown module>)
    open-quantum-safe#6 0x7f689986eae8 in decoder_process /home/pawn/work/openssl/crypto/encode_decode/decoder_lib.c:971:14
    open-quantum-safe#7 0x7f6899d13383 in spki2typespki_decode /home/pawn/work/openssl/providers/implementations/encode_decode/decode_spki2typespki.c:111:10
    open-quantum-safe#8 0x7f689986eae8 in decoder_process /home/pawn/work/openssl/crypto/encode_decode/decoder_lib.c:971:14
    open-quantum-safe#9 0x7f6899d114d3 in pem2der_decode /home/pawn/work/openssl/providers/implementations/encode_decode/decode_pem2der.c:204:14
    open-quantum-safe#10 0x7f689986eae8 in decoder_process /home/pawn/work/openssl/crypto/encode_decode/decoder_lib.c:971:14
    open-quantum-safe#11 0x7f689986d106 in OSSL_DECODER_from_bio /home/pawn/work/openssl/crypto/encode_decode/decoder_lib.c:82:10
    open-quantum-safe#12 0x55610d6bd407 in decode_EVP_PKEY_prov /home/pawn/work/oqs-provider/test/oqs_test_endecode.c:151:10
    open-quantum-safe#13 0x55610d6bc75b in test_oqs_encdec /home/pawn/work/oqs-provider/test/oqs_test_endecode.c:188:14
    open-quantum-safe#14 0x55610d6bbf37 in main /home/pawn/work/oqs-provider/test/oqs_test_endecode.c:240:17
    open-quantum-safe#15 0x7f68991671c9 in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16

Indirect leak of 1312 byte(s) in 1 object(s) allocated from:
    #0 0x55610d68132e in malloc (/home/pawn/work/oqs-provider/build/test/oqs_test_endecode+0xb932e) (BuildId: 53bfaa0a3b850edc7328f30be8b047ab6675ba3c)
    open-quantum-safe#1 0x7f68999a946b in CRYPTO_malloc /home/pawn/work/openssl/crypto/mem.c:190:12
    open-quantum-safe#2 0x7f68995d6b99 in ossl_c2i_ASN1_BIT_STRING /home/pawn/work/openssl/crypto/asn1/a_bitstr.c:117:13
    open-quantum-safe#3 0x7f689961f659 in asn1_ex_c2i /home/pawn/work/openssl/crypto/asn1/tasn_dec.c:885:14
    open-quantum-safe#4 0x7f689961bbc9 in asn1_d2i_ex_primitive /home/pawn/work/openssl/crypto/asn1/tasn_dec.c:818:10
    open-quantum-safe#5 0x7f6899618054 in asn1_item_embed_d2i /home/pawn/work/openssl/crypto/asn1/tasn_dec.c:217:16
    open-quantum-safe#6 0x7f689961e02f in asn1_template_noexp_d2i /home/pawn/work/openssl/crypto/asn1/tasn_dec.c:682:15
    open-quantum-safe#7 0x7f689961a7b4 in asn1_template_ex_d2i /home/pawn/work/openssl/crypto/asn1/tasn_dec.c:558:16
    open-quantum-safe#8 0x7f689961986a in asn1_item_embed_d2i /home/pawn/work/openssl/crypto/asn1/tasn_dec.c:422:19
    open-quantum-safe#9 0x7f68996175c8 in asn1_item_ex_d2i_intern /home/pawn/work/openssl/crypto/asn1/tasn_dec.c:118:10
    open-quantum-safe#10 0x7f68996177da in ASN1_item_d2i_ex /home/pawn/work/openssl/crypto/asn1/tasn_dec.c:144:9
    open-quantum-safe#11 0x7f6895295926  (<unknown module>)
    open-quantum-safe#12 0x7f6895295ce4  (<unknown module>)
    open-quantum-safe#13 0x7f6895296a6c  (<unknown module>)
    open-quantum-safe#14 0x7f689986eae8 in decoder_process /home/pawn/work/openssl/crypto/encode_decode/decoder_lib.c:971:14
    open-quantum-safe#15 0x7f6899d13383 in spki2typespki_decode /home/pawn/work/openssl/providers/implementations/encode_decode/decode_spki2typespki.c:111:10
    open-quantum-safe#16 0x7f689986eae8 in decoder_process /home/pawn/work/openssl/crypto/encode_decode/decoder_lib.c:971:14
    open-quantum-safe#17 0x7f6899d114d3 in pem2der_decode /home/pawn/work/openssl/providers/implementations/encode_decode/decode_pem2der.c:204:14
    open-quantum-safe#18 0x7f689986eae8 in decoder_process /home/pawn/work/openssl/crypto/encode_decode/decoder_lib.c:971:14
    open-quantum-safe#19 0x7f689986d106 in OSSL_DECODER_from_bio /home/pawn/work/openssl/crypto/encode_decode/decoder_lib.c:82:10
    open-quantum-safe#20 0x55610d6bd407 in decode_EVP_PKEY_prov /home/pawn/work/oqs-provider/test/oqs_test_endecode.c:151:10
    open-quantum-safe#21 0x55610d6bc75b in test_oqs_encdec /home/pawn/work/oqs-provider/test/oqs_test_endecode.c:188:14
    open-quantum-safe#22 0x55610d6bbf37 in main /home/pawn/work/oqs-provider/test/oqs_test_endecode.c:240:17
    open-quantum-safe#23 0x7f68991671c9 in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16

Indirect leak of 24 byte(s) in 1 object(s) allocated from:
    #0 0x55610d68132e in malloc (/home/pawn/work/oqs-provider/build/test/oqs_test_endecode+0xb932e) (BuildId: 53bfaa0a3b850edc7328f30be8b047ab6675ba3c)
    open-quantum-safe#1 0x7f68999a946b in CRYPTO_malloc /home/pawn/work/openssl/crypto/mem.c:190:12
    open-quantum-safe#2 0x7f68999a94a2 in CRYPTO_zalloc /home/pawn/work/openssl/crypto/mem.c:197:11
    open-quantum-safe#3 0x7f68995fde90 in ASN1_STRING_type_new /home/pawn/work/openssl/crypto/asn1/asn1_lib.c:350:11
    open-quantum-safe#4 0x7f689962a7dd in ASN1_BIT_STRING_new /home/pawn/work/openssl/crypto/asn1/tasn_typ.c:31:1
    open-quantum-safe#5 0x7f68995d6a0f in ossl_c2i_ASN1_BIT_STRING /home/pawn/work/openssl/crypto/asn1/a_bitstr.c:98:20
    open-quantum-safe#6 0x7f689961f659 in asn1_ex_c2i /home/pawn/work/openssl/crypto/asn1/tasn_dec.c:885:14
    open-quantum-safe#7 0x7f689961bbc9 in asn1_d2i_ex_primitive /home/pawn/work/openssl/crypto/asn1/tasn_dec.c:818:10
    open-quantum-safe#8 0x7f6899618054 in asn1_item_embed_d2i /home/pawn/work/openssl/crypto/asn1/tasn_dec.c:217:16
    open-quantum-safe#9 0x7f689961e02f in asn1_template_noexp_d2i /home/pawn/work/openssl/crypto/asn1/tasn_dec.c:682:15
    open-quantum-safe#10 0x7f689961a7b4 in asn1_template_ex_d2i /home/pawn/work/openssl/crypto/asn1/tasn_dec.c:558:16
    open-quantum-safe#11 0x7f689961986a in asn1_item_embed_d2i /home/pawn/work/openssl/crypto/asn1/tasn_dec.c:422:19
    open-quantum-safe#12 0x7f68996175c8 in asn1_item_ex_d2i_intern /home/pawn/work/openssl/crypto/asn1/tasn_dec.c:118:10
    open-quantum-safe#13 0x7f68996177da in ASN1_item_d2i_ex /home/pawn/work/openssl/crypto/asn1/tasn_dec.c:144:9
    open-quantum-safe#14 0x7f6895295926  (<unknown module>)
    open-quantum-safe#15 0x7f6895295ce4  (<unknown module>)
    open-quantum-safe#16 0x7f6895296a6c  (<unknown module>)
    open-quantum-safe#17 0x7f689986eae8 in decoder_process /home/pawn/work/openssl/crypto/encode_decode/decoder_lib.c:971:14
    open-quantum-safe#18 0x7f6899d13383 in spki2typespki_decode /home/pawn/work/openssl/providers/implementations/encode_decode/decode_spki2typespki.c:111:10
    open-quantum-safe#19 0x7f689986eae8 in decoder_process /home/pawn/work/openssl/crypto/encode_decode/decoder_lib.c:971:14
    open-quantum-safe#20 0x7f6899d114d3 in pem2der_decode /home/pawn/work/openssl/providers/implementations/encode_decode/decode_pem2der.c:204:14
    open-quantum-safe#21 0x7f689986eae8 in decoder_process /home/pawn/work/openssl/crypto/encode_decode/decoder_lib.c:971:14
    open-quantum-safe#22 0x7f689986d106 in OSSL_DECODER_from_bio /home/pawn/work/openssl/crypto/encode_decode/decoder_lib.c:82:10
    open-quantum-safe#23 0x55610d6bd407 in decode_EVP_PKEY_prov /home/pawn/work/oqs-provider/test/oqs_test_endecode.c:151:10
    open-quantum-safe#24 0x55610d6bc75b in test_oqs_encdec /home/pawn/work/oqs-provider/test/oqs_test_endecode.c:188:14
    open-quantum-safe#25 0x55610d6bbf37 in main /home/pawn/work/oqs-provider/test/oqs_test_endecode.c:240:17
    open-quantum-safe#26 0x7f68991671c9 in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16

Indirect leak of 16 byte(s) in 1 object(s) allocated from:
    #0 0x55610d68132e in malloc (/home/pawn/work/oqs-provider/build/test/oqs_test_endecode+0xb932e) (BuildId: 53bfaa0a3b850edc7328f30be8b047ab6675ba3c)
    open-quantum-safe#1 0x7f68999a946b in CRYPTO_malloc /home/pawn/work/openssl/crypto/mem.c:190:12
    open-quantum-safe#2 0x7f68999a94a2 in CRYPTO_zalloc /home/pawn/work/openssl/crypto/mem.c:197:11
    open-quantum-safe#3 0x7f6899625ed4 in asn1_item_embed_new /home/pawn/work/openssl/crypto/asn1/tasn_new.c:136:21
    open-quantum-safe#4 0x7f689962620e in ossl_asn1_item_ex_new_intern /home/pawn/work/openssl/crypto/asn1/tasn_new.c:52:12
    open-quantum-safe#5 0x7f68996191a5 in asn1_item_embed_d2i /home/pawn/work/openssl/crypto/asn1/tasn_dec.c:366:21
    open-quantum-safe#6 0x7f689961e02f in asn1_template_noexp_d2i /home/pawn/work/openssl/crypto/asn1/tasn_dec.c:682:15
    open-quantum-safe#7 0x7f689961a7b4 in asn1_template_ex_d2i /home/pawn/work/openssl/crypto/asn1/tasn_dec.c:558:16
    open-quantum-safe#8 0x7f689961986a in asn1_item_embed_d2i /home/pawn/work/openssl/crypto/asn1/tasn_dec.c:422:19
    open-quantum-safe#9 0x7f68996175c8 in asn1_item_ex_d2i_intern /home/pawn/work/openssl/crypto/asn1/tasn_dec.c:118:10
    open-quantum-safe#10 0x7f68996177da in ASN1_item_d2i_ex /home/pawn/work/openssl/crypto/asn1/tasn_dec.c:144:9
    open-quantum-safe#11 0x7f6895295926  (<unknown module>)
    open-quantum-safe#12 0x7f6895295ce4  (<unknown module>)
    open-quantum-safe#13 0x7f6895296a6c  (<unknown module>)
    open-quantum-safe#14 0x7f689986eae8 in decoder_process /home/pawn/work/openssl/crypto/encode_decode/decoder_lib.c:971:14
    open-quantum-safe#15 0x7f6899d13383 in spki2typespki_decode /home/pawn/work/openssl/providers/implementations/encode_decode/decode_spki2typespki.c:111:10
    open-quantum-safe#16 0x7f689986eae8 in decoder_process /home/pawn/work/openssl/crypto/encode_decode/decoder_lib.c:971:14
    open-quantum-safe#17 0x7f6899d114d3 in pem2der_decode /home/pawn/work/openssl/providers/implementations/encode_decode/decode_pem2der.c:204:14
    open-quantum-safe#18 0x7f689986eae8 in decoder_process /home/pawn/work/openssl/crypto/encode_decode/decoder_lib.c:971:14
    open-quantum-safe#19 0x7f689986d106 in OSSL_DECODER_from_bio /home/pawn/work/openssl/crypto/encode_decode/decoder_lib.c:82:10
    open-quantum-safe#20 0x55610d6bd407 in decode_EVP_PKEY_prov /home/pawn/work/oqs-provider/test/oqs_test_endecode.c:151:10
    open-quantum-safe#21 0x55610d6bc75b in test_oqs_encdec /home/pawn/work/oqs-provider/test/oqs_test_endecode.c:188:14
    open-quantum-safe#22 0x55610d6bbf37 in main /home/pawn/work/oqs-provider/test/oqs_test_endecode.c:240:17
    open-quantum-safe#23 0x7f68991671c9 in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16

SUMMARY: AddressSanitizer: 1400 byte(s) leaked in 4 allocation(s).
```


## 8. Bonus point: fix a potential double-free.

In `oqsx_key_gen`, when the keytype isn't `KEY_TYPE_HYB_SIG`, the
generated `pkey` is freed, and the flow goes down to the `err` label:

https://github.com/open-quantum-safe/oqs-provider/blob/cc3895f208730abfe1efc15f7c3ff4358ac6e4c0/oqsprov/oqsprov_keys.c#L1089-L1104

There is a potential double-free bug here: if `oqsx_key_gen_oqs` fails, then
`EVP_PKEY_free` will be called twice on the same `pkey` pointer.

Note that it is very unlikely that `oqsx_key_gen_oqs` fails, since it only
calls `OQS_KEM_keypair`/`OQS_SIG_keypair`.
thb-sb added a commit to thb-sb/oqs-provider that referenced this pull request Sep 5, 2023
@thb-sb
Fix various memory leaks.
1e80109 
This commit fixes various memory leaks that have been found using ASan.

Note: I will soon add a CI job for testing oqs-provider against memory leaks
with ASan.


# 1. Memory leak in `oqs_test_signatures`.

In `oqs_test_signatures`, [`OPENSSL_free`] was used with `EVP_PKEY_CTX`. However,
the right destructor function for `EVP_PKEY_CTX` is [`EVP_PKEY_CTX_free`].

This commmit fixes this minor bug.

[`OPENSSL_free`](https://www.openssl.org/docs/man3.0/man3/OPENSSL_free.html)
[`EVP_PKEY_CTX_free`](https://www.openssl.org/docs/man3.0/man3/EVP_PKEY_CTX_free.html)


## 2. a selftest key that wasn't freed.

In `oqsprov/oqsprov_keys.c`, `d2i_PrivateKey` is used to verify that we the
key we've just generated is sound.
However, the returned object wasn't freed.
ASan trace that helped me finding this one:

```
Indirect leak of 96 byte(s) in 4 object(s) allocated from:
    #0 0x5651eef1428e in malloc (/home/pawn/work/oqs-provider/build/test/oqs_test_signatures+0xb928e) (BuildId: 6cc7ebd4bc73a0c78b2ab8d7b9dcb8b165d88e74)
    open-quantum-safe#1 0x7fb7a23a946b in CRYPTO_malloc /home/pawn/work/openssl/crypto/mem.c:190:12
    open-quantum-safe#2 0x7fb7a23a94a2 in CRYPTO_zalloc /home/pawn/work/openssl/crypto/mem.c:197:11
    open-quantum-safe#3 0x7fb7a209cdcd in BN_new /home/pawn/work/openssl/crypto/bn/bn_lib.c:247:16
    open-quantum-safe#4 0x7fb7a2224a00 in ossl_ec_GFp_mont_group_set_curve /home/pawn/work/openssl/crypto/ec/ecp_mont.c:169:11
    open-quantum-safe#5 0x7fb7a220f5fa in EC_GROUP_set_curve /home/pawn/work/openssl/crypto/ec/ec_lib.c:562:12
    open-quantum-safe#6 0x7fb7a2202d75 in EC_GROUP_new_curve_GFp /home/pawn/work/openssl/crypto/ec/ec_cvt.c:61:10
    open-quantum-safe#7 0x7fb7a220184d in ec_group_new_from_data /home/pawn/work/openssl/crypto/ec/ec_curve.c:3179:22
    open-quantum-safe#8 0x7fb7a2200ff9 in EC_GROUP_new_by_curve_name_ex /home/pawn/work/openssl/crypto/ec/ec_curve.c:3287:19
    open-quantum-safe#9 0x7fb7a2201ed9 in EC_GROUP_new_by_curve_name /home/pawn/work/openssl/crypto/ec/ec_curve.c:3303:12
    open-quantum-safe#10 0x7fb7a21f84e2 in EC_GROUP_new_from_ecpkparameters /home/pawn/work/openssl/crypto/ec/ec_asn1.c:863:20
    open-quantum-safe#11 0x7fb7a21f8f28 in d2i_ECPrivateKey /home/pawn/work/openssl/crypto/ec/ec_asn1.c:956:22
    open-quantum-safe#12 0x7fb7a270b4b5 in der2key_decode /home/pawn/work/openssl/providers/implementations/encode_decode/decode_der2key.c:220:19
    open-quantum-safe#13 0x7fb7a226eae8 in decoder_process /home/pawn/work/openssl/crypto/encode_decode/decoder_lib.c:971:14
    open-quantum-safe#14 0x7fb7a226d106 in OSSL_DECODER_from_bio /home/pawn/work/openssl/crypto/encode_decode/decoder_lib.c:82:10
    open-quantum-safe#15 0x7fb7a226f007 in OSSL_DECODER_from_data /home/pawn/work/openssl/crypto/encode_decode/decoder_lib.c:157:9
    open-quantum-safe#16 0x7fb7a200e123 in d2i_PrivateKey_decoder /home/pawn/work/openssl/crypto/asn1/d2i_pr.c:74:11
    open-quantum-safe#17 0x7fb7a200dbba in d2i_PrivateKey_ex /home/pawn/work/openssl/crypto/asn1/d2i_pr.c:162:11
    open-quantum-safe#18 0x7fb7a200e3d2 in d2i_PrivateKey /home/pawn/work/openssl/crypto/asn1/d2i_pr.c:172:12
    open-quantum-safe#19 0x7fb79dc6a22e in oqsx_key_gen_evp_key /home/pawn/work/oqs-provider/oqsprov/oqsprov_keys.c:1036:25
    open-quantum-safe#20 0x7fb79dc68f3f in oqsx_key_gen /home/pawn/work/oqs-provider/oqsprov/oqsprov_keys.c:1077:16
    open-quantum-safe#21 0x7fb79dc7164a in oqsx_genkey /home/pawn/work/oqs-provider/oqsprov/oqs_kmgmt.c:496:9
    open-quantum-safe#22 0x7fb79dc6d9f8 in oqsx_gen /home/pawn/work/oqs-provider/oqsprov/oqs_kmgmt.c:509:12
    open-quantum-safe#23 0x7fb7a23373c7 in evp_keymgmt_gen /home/pawn/work/openssl/crypto/evp/keymgmt_meth.c:366:12
    open-quantum-safe#24 0x7fb7a2334b58 in evp_keymgmt_util_gen /home/pawn/work/openssl/crypto/evp/keymgmt_lib.c:517:20
    open-quantum-safe#25 0x7fb7a235a775 in EVP_PKEY_generate /home/pawn/work/openssl/crypto/evp/pmeth_gn.c:189:13
    open-quantum-safe#26 0x5651eef4f2bd in test_oqs_signatures /home/pawn/work/oqs-provider/test/oqs_test_signatures.c:39:48
    open-quantum-safe#27 0x5651eef4ee3d in main /home/pawn/work/oqs-provider/test/oqs_test_signatures.c:114:17
    open-quantum-safe#28 0x7fb7a1b671c9 in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
```

## 3. a `EVP_PKEY_CTX` that wasn't freed.

In `oqsx_key_free`, `key->oqsx_provider_ctx.oqsx_evp_ctx->ctx` is freed if
the keytype is `ECP_HYB_KEM` or `ECX_HYB_KEM`. However,
`key->oqsx_provider_ctx.oqsx_evp_ctx->ctx` is also used for signatures.
In that case, a call to `EVP_PKEY_CTX_free` was missing.
ASan trace that helped me finding this one:

```
Indirect leak of 7 byte(s) in 2 object(s) allocated from:
    #0 0x55b6fc8d427e in malloc (/home/pawn/work/oqs-provider/build/test/oqs_test_signatures+0xb927e) (BuildId: 3a8bff15d3315ba7ac1746fdc7b67eb464c607a1)
    open-quantum-safe#1 0x7ff34dfa946b in CRYPTO_malloc /home/pawn/work/openssl/crypto/mem.c:190:12
    open-quantum-safe#2 0x7ff34dfac50a in CRYPTO_strndup /home/pawn/work/openssl/crypto/o_str.c:43:11
    open-quantum-safe#3 0x7ff34df9d152 in ossl_algorithm_get1_first_name /home/pawn/work/openssl/crypto/core_algorithm.c:195:11
    open-quantum-safe#4 0x7ff34df353a1 in keymgmt_from_algorithm /home/pawn/work/openssl/crypto/evp/keymgmt_meth.c:50:31
    open-quantum-safe#5 0x7ff34df15e44 in construct_evp_method /home/pawn/work/openssl/crypto/evp/evp_fetch.c:219:14
    open-quantum-safe#6 0x7ff34df9e3ce in ossl_method_construct_this /home/pawn/work/openssl/crypto/core_fetch.c:109:19
    open-quantum-safe#7 0x7ff34df9d740 in algorithm_do_map /home/pawn/work/openssl/crypto/core_algorithm.c:77:13
    open-quantum-safe#8 0x7ff34df9ce88 in algorithm_do_this /home/pawn/work/openssl/crypto/core_algorithm.c:122:15
    open-quantum-safe#9 0x7ff34dfcce3e in ossl_provider_doall_activated /home/pawn/work/openssl/crypto/provider_core.c:1423:14
    open-quantum-safe#10 0x7ff34df9ca64 in ossl_algorithm_do_all /home/pawn/work/openssl/crypto/core_algorithm.c:162:9
    open-quantum-safe#11 0x7ff34df9dd64 in ossl_method_construct /home/pawn/work/openssl/crypto/core_fetch.c:153:5
    open-quantum-safe#12 0x7ff34df13b6f in inner_evp_generic_fetch /home/pawn/work/openssl/crypto/evp/evp_fetch.c:312:23
    open-quantum-safe#13 0x7ff34df13226 in evp_generic_fetch /home/pawn/work/openssl/crypto/evp/evp_fetch.c:364:14
    open-quantum-safe#14 0x7ff34df36a42 in EVP_KEYMGMT_fetch /home/pawn/work/openssl/crypto/evp/keymgmt_meth.c:220:12
    open-quantum-safe#15 0x7ff34df5d6e4 in int_ctx_new /home/pawn/work/openssl/crypto/evp/pmeth_lib.c:280:23
    open-quantum-safe#16 0x7ff34df5f0a9 in EVP_PKEY_CTX_new_id /home/pawn/work/openssl/crypto/evp/pmeth_lib.c:469:12
    open-quantum-safe#17 0x7ff349865bcb in oqsx_hybsig_init /home/pawn/work/oqs-provider/oqsprov/oqsprov_keys.c:578:20
    open-quantum-safe#18 0x7ff349865157 in oqsx_key_new /home/pawn/work/oqs-provider/oqsprov/oqsprov_keys.c:791:16
    open-quantum-safe#19 0x7ff3498715a3 in oqsx_genkey /home/pawn/work/oqs-provider/oqsprov/oqs_kmgmt.c:487:16
    open-quantum-safe#20 0x7ff34986da08 in oqsx_gen /home/pawn/work/oqs-provider/oqsprov/oqs_kmgmt.c:509:12
    open-quantum-safe#21 0x7ff34df373c7 in evp_keymgmt_gen /home/pawn/work/openssl/crypto/evp/keymgmt_meth.c:366:12
    open-quantum-safe#22 0x7ff34df34b58 in evp_keymgmt_util_gen /home/pawn/work/openssl/crypto/evp/keymgmt_lib.c:517:20
    open-quantum-safe#23 0x7ff34df5a775 in EVP_PKEY_generate /home/pawn/work/openssl/crypto/evp/pmeth_gn.c:189:13
    open-quantum-safe#24 0x55b6fc90f2ad in test_oqs_signatures /home/pawn/work/oqs-provider/test/oqs_test_signatures.c:39:48
    open-quantum-safe#25 0x55b6fc90ee2d in main /home/pawn/work/oqs-provider/test/oqs_test_signatures.c:114:17
    open-quantum-safe#26 0x7ff34d8461c9 in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
```

## 4. Missing `EVP_PKEY_CTX_free` in case of error.

In `oqsx_hybsig_init`, a `EVP_PKEY_CTX` is initialized using `EVP_PKEY_CTX_new_id`.
However, later in the code, some extra work is done, and in case of error, the
`EVP_PKEY_CTX` wasn't freed.

(I couldn't recover the ASan message for this one…)


## 5. `OPENSSL_free` instead of `EVP_PKEY_free`.

In `oqsx_key_gen`, when a classical private key is generated, the field
`classical_pkey` of a `OQSX_KEY` is set with a value of type `EVP_PKEY`.

However, in `oqsx_key_free`, `OPENSSL_free` was used on `classical_pkey`
instead of `EVP_PKEY_free`.

ASan trace that helped me finding this one:

```
   #0 0x555a6b1b728e in malloc (/home/pawn/work/oqs-provider/build/test/oqs_test_signatures+0xb928e) (BuildId: 6cc7ebd4bc73a0c78b2ab8d7b9dcb8b165d88e74)
    open-quantum-safe#1 0x7f59a93a946b in CRYPTO_malloc /home/pawn/work/openssl/crypto/mem.c:190:12
    open-quantum-safe#2 0x7f59a93a94a2 in CRYPTO_zalloc /home/pawn/work/openssl/crypto/mem.c:197:11
    open-quantum-safe#3 0x7f59a909cdcd in BN_new /home/pawn/work/openssl/crypto/bn/bn_lib.c:247:16
    open-quantum-safe#4 0x7f59a920b955 in ossl_ec_group_new_ex /home/pawn/work/openssl/crypto/ec/ec_lib.c:61:25
    open-quantum-safe#5 0x7f59a920dd55 in EC_GROUP_dup /home/pawn/work/openssl/crypto/ec/ec_lib.c:273:14
    open-quantum-safe#6 0x7f59a9207774 in EC_KEY_set_group /home/pawn/work/openssl/crypto/ec/ec_key.c:733:18
    open-quantum-safe#7 0x7f59a975b3b5 in ec_gen_assign_group /home/pawn/work/openssl/providers/implementations/keymgmt/ec_kmgmt.c:1236:12
    open-quantum-safe#8 0x7f59a9758bdb in ec_gen /home/pawn/work/openssl/providers/implementations/keymgmt/ec_kmgmt.c:1274:11
    open-quantum-safe#9 0x7f59a93373c7 in evp_keymgmt_gen /home/pawn/work/openssl/crypto/evp/keymgmt_meth.c:366:12
    open-quantum-safe#10 0x7f59a9334b58 in evp_keymgmt_util_gen /home/pawn/work/openssl/crypto/evp/keymgmt_lib.c:517:20
    open-quantum-safe#11 0x7f59a935a775 in EVP_PKEY_generate /home/pawn/work/openssl/crypto/evp/pmeth_gn.c:189:13
    open-quantum-safe#12 0x7f59a935b404 in EVP_PKEY_keygen /home/pawn/work/openssl/crypto/evp/pmeth_gn.c:274:12
    open-quantum-safe#13 0x7f59a4c69a51 in oqsx_key_gen_evp_key /home/pawn/work/oqs-provider/oqsprov/oqsprov_keys.c:1012:12
    open-quantum-safe#14 0x7f59a4c68faf in oqsx_key_gen /home/pawn/work/oqs-provider/oqsprov/oqsprov_keys.c:1084:16
    open-quantum-safe#15 0x7f59a4c716ca in oqsx_genkey /home/pawn/work/oqs-provider/oqsprov/oqs_kmgmt.c:496:9
    open-quantum-safe#16 0x7f59a4c6da78 in oqsx_gen /home/pawn/work/oqs-provider/oqsprov/oqs_kmgmt.c:509:12
    open-quantum-safe#17 0x7f59a93373c7 in evp_keymgmt_gen /home/pawn/work/openssl/crypto/evp/keymgmt_meth.c:366:12
    open-quantum-safe#18 0x7f59a9334b58 in evp_keymgmt_util_gen /home/pawn/work/openssl/crypto/evp/keymgmt_lib.c:517:20
    open-quantum-safe#19 0x7f59a935a775 in EVP_PKEY_generate /home/pawn/work/openssl/crypto/evp/pmeth_gn.c:189:13
    open-quantum-safe#20 0x555a6b1f2890 in test_oqs_signatures /home/pawn/work/oqs-provider/test/oqs_test_signatures.c:68:44
    open-quantum-safe#21 0x555a6b1f1e3d in main /home/pawn/work/oqs-provider/test/oqs_test_signatures.c:114:17
    open-quantum-safe#22 0x7f59a8c461c9 in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
```

## 6. Some `OPENSSL_free` were missing in `oqs_test_kems.c`

```
==569479==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 346035 byte(s) in 42 object(s) allocated from:
    #0 0x559c76ef524e in malloc (/home/pawn/work/oqs-provider/build/test/oqs_test_kems+0xb924e) (BuildId: 76a98f5859ba3c90334cef4286e89aa76436b1ae)
    open-quantum-safe#1 0x7f04407a946b in CRYPTO_malloc /home/pawn/work/openssl/crypto/mem.c:190:12
    open-quantum-safe#2 0x559c76f30389 in test_oqs_kems /home/pawn/work/oqs-provider/test/oqs_test_kems.c:44:26
    open-quantum-safe#3 0x559c76f2fdfd in main /home/pawn/work/oqs-provider/test/oqs_test_kems.c:93:17
    open-quantum-safe#4 0x7f04400461c9 in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16

Direct leak of 2690 byte(s) in 42 object(s) allocated from:
    #0 0x559c76ef524e in malloc (/home/pawn/work/oqs-provider/build/test/oqs_test_kems+0xb924e) (BuildId: 76a98f5859ba3c90334cef4286e89aa76436b1ae)
    open-quantum-safe#1 0x7f04407a946b in CRYPTO_malloc /home/pawn/work/openssl/crypto/mem.c:190:12
    open-quantum-safe#2 0x559c76f304a5 in test_oqs_kems /home/pawn/work/oqs-provider/test/oqs_test_kems.c:47:29
    open-quantum-safe#3 0x559c76f2fdfd in main /home/pawn/work/oqs-provider/test/oqs_test_kems.c:93:17
    open-quantum-safe#4 0x7f04400461c9 in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16

Direct leak of 2690 byte(s) in 42 object(s) allocated from:
    #0 0x559c76ef524e in malloc (/home/pawn/work/oqs-provider/build/test/oqs_test_kems+0xb924e) (BuildId: 76a98f5859ba3c90334cef4286e89aa76436b1ae)
    open-quantum-safe#1 0x7f04407a946b in CRYPTO_malloc /home/pawn/work/openssl/crypto/mem.c:190:12
    open-quantum-safe#2 0x559c76f303e4 in test_oqs_kems /home/pawn/work/oqs-provider/test/oqs_test_kems.c:45:29
    open-quantum-safe#3 0x559c76f2fdfd in main /home/pawn/work/oqs-provider/test/oqs_test_kems.c:93:17
    open-quantum-safe#4 0x7f04400461c9 in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
```

## 7. Missing `X509_PUBKEY_free` in `oqsx_d2i_PUBKEY`.

This was probably the hardest one for me to track down.

In `oqsx_d2i_PUBKEY`, a call is done to `oqsx_d2i_X509_PUBKEY_INTERNAL`.
However, the returned value (of type `X509_PUBKEY`) wasn't freed.

ASan:

```
Direct leak of 48 byte(s) in 1 object(s) allocated from:
    #0 0x55610d68132e in malloc (/home/pawn/work/oqs-provider/build/test/oqs_test_endecode+0xb932e) (BuildId: 53bfaa0a3b850edc7328f30be8b047ab6675ba3c)
    open-quantum-safe#1 0x7f68999a946b in CRYPTO_malloc /home/pawn/work/openssl/crypto/mem.c:190:12
    open-quantum-safe#2 0x7f68999a94a2 in CRYPTO_zalloc /home/pawn/work/openssl/crypto/mem.c:197:11
    open-quantum-safe#3 0x7f6895295897  (<unknown module>)
    open-quantum-safe#4 0x7f6895295ce4  (<unknown module>)
    open-quantum-safe#5 0x7f6895296a6c  (<unknown module>)
    open-quantum-safe#6 0x7f689986eae8 in decoder_process /home/pawn/work/openssl/crypto/encode_decode/decoder_lib.c:971:14
    open-quantum-safe#7 0x7f6899d13383 in spki2typespki_decode /home/pawn/work/openssl/providers/implementations/encode_decode/decode_spki2typespki.c:111:10
    open-quantum-safe#8 0x7f689986eae8 in decoder_process /home/pawn/work/openssl/crypto/encode_decode/decoder_lib.c:971:14
    open-quantum-safe#9 0x7f6899d114d3 in pem2der_decode /home/pawn/work/openssl/providers/implementations/encode_decode/decode_pem2der.c:204:14
    open-quantum-safe#10 0x7f689986eae8 in decoder_process /home/pawn/work/openssl/crypto/encode_decode/decoder_lib.c:971:14
    open-quantum-safe#11 0x7f689986d106 in OSSL_DECODER_from_bio /home/pawn/work/openssl/crypto/encode_decode/decoder_lib.c:82:10
    open-quantum-safe#12 0x55610d6bd407 in decode_EVP_PKEY_prov /home/pawn/work/oqs-provider/test/oqs_test_endecode.c:151:10
    open-quantum-safe#13 0x55610d6bc75b in test_oqs_encdec /home/pawn/work/oqs-provider/test/oqs_test_endecode.c:188:14
    open-quantum-safe#14 0x55610d6bbf37 in main /home/pawn/work/oqs-provider/test/oqs_test_endecode.c:240:17
    open-quantum-safe#15 0x7f68991671c9 in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16

Indirect leak of 1312 byte(s) in 1 object(s) allocated from:
    #0 0x55610d68132e in malloc (/home/pawn/work/oqs-provider/build/test/oqs_test_endecode+0xb932e) (BuildId: 53bfaa0a3b850edc7328f30be8b047ab6675ba3c)
    open-quantum-safe#1 0x7f68999a946b in CRYPTO_malloc /home/pawn/work/openssl/crypto/mem.c:190:12
    open-quantum-safe#2 0x7f68995d6b99 in ossl_c2i_ASN1_BIT_STRING /home/pawn/work/openssl/crypto/asn1/a_bitstr.c:117:13
    open-quantum-safe#3 0x7f689961f659 in asn1_ex_c2i /home/pawn/work/openssl/crypto/asn1/tasn_dec.c:885:14
    open-quantum-safe#4 0x7f689961bbc9 in asn1_d2i_ex_primitive /home/pawn/work/openssl/crypto/asn1/tasn_dec.c:818:10
    open-quantum-safe#5 0x7f6899618054 in asn1_item_embed_d2i /home/pawn/work/openssl/crypto/asn1/tasn_dec.c:217:16
    open-quantum-safe#6 0x7f689961e02f in asn1_template_noexp_d2i /home/pawn/work/openssl/crypto/asn1/tasn_dec.c:682:15
    open-quantum-safe#7 0x7f689961a7b4 in asn1_template_ex_d2i /home/pawn/work/openssl/crypto/asn1/tasn_dec.c:558:16
    open-quantum-safe#8 0x7f689961986a in asn1_item_embed_d2i /home/pawn/work/openssl/crypto/asn1/tasn_dec.c:422:19
    open-quantum-safe#9 0x7f68996175c8 in asn1_item_ex_d2i_intern /home/pawn/work/openssl/crypto/asn1/tasn_dec.c:118:10
    open-quantum-safe#10 0x7f68996177da in ASN1_item_d2i_ex /home/pawn/work/openssl/crypto/asn1/tasn_dec.c:144:9
    open-quantum-safe#11 0x7f6895295926  (<unknown module>)
    open-quantum-safe#12 0x7f6895295ce4  (<unknown module>)
    open-quantum-safe#13 0x7f6895296a6c  (<unknown module>)
    open-quantum-safe#14 0x7f689986eae8 in decoder_process /home/pawn/work/openssl/crypto/encode_decode/decoder_lib.c:971:14
    open-quantum-safe#15 0x7f6899d13383 in spki2typespki_decode /home/pawn/work/openssl/providers/implementations/encode_decode/decode_spki2typespki.c:111:10
    open-quantum-safe#16 0x7f689986eae8 in decoder_process /home/pawn/work/openssl/crypto/encode_decode/decoder_lib.c:971:14
    open-quantum-safe#17 0x7f6899d114d3 in pem2der_decode /home/pawn/work/openssl/providers/implementations/encode_decode/decode_pem2der.c:204:14
    open-quantum-safe#18 0x7f689986eae8 in decoder_process /home/pawn/work/openssl/crypto/encode_decode/decoder_lib.c:971:14
    open-quantum-safe#19 0x7f689986d106 in OSSL_DECODER_from_bio /home/pawn/work/openssl/crypto/encode_decode/decoder_lib.c:82:10
    open-quantum-safe#20 0x55610d6bd407 in decode_EVP_PKEY_prov /home/pawn/work/oqs-provider/test/oqs_test_endecode.c:151:10
    open-quantum-safe#21 0x55610d6bc75b in test_oqs_encdec /home/pawn/work/oqs-provider/test/oqs_test_endecode.c:188:14
    open-quantum-safe#22 0x55610d6bbf37 in main /home/pawn/work/oqs-provider/test/oqs_test_endecode.c:240:17
    open-quantum-safe#23 0x7f68991671c9 in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16

Indirect leak of 24 byte(s) in 1 object(s) allocated from:
    #0 0x55610d68132e in malloc (/home/pawn/work/oqs-provider/build/test/oqs_test_endecode+0xb932e) (BuildId: 53bfaa0a3b850edc7328f30be8b047ab6675ba3c)
    open-quantum-safe#1 0x7f68999a946b in CRYPTO_malloc /home/pawn/work/openssl/crypto/mem.c:190:12
    open-quantum-safe#2 0x7f68999a94a2 in CRYPTO_zalloc /home/pawn/work/openssl/crypto/mem.c:197:11
    open-quantum-safe#3 0x7f68995fde90 in ASN1_STRING_type_new /home/pawn/work/openssl/crypto/asn1/asn1_lib.c:350:11
    open-quantum-safe#4 0x7f689962a7dd in ASN1_BIT_STRING_new /home/pawn/work/openssl/crypto/asn1/tasn_typ.c:31:1
    open-quantum-safe#5 0x7f68995d6a0f in ossl_c2i_ASN1_BIT_STRING /home/pawn/work/openssl/crypto/asn1/a_bitstr.c:98:20
    open-quantum-safe#6 0x7f689961f659 in asn1_ex_c2i /home/pawn/work/openssl/crypto/asn1/tasn_dec.c:885:14
    open-quantum-safe#7 0x7f689961bbc9 in asn1_d2i_ex_primitive /home/pawn/work/openssl/crypto/asn1/tasn_dec.c:818:10
    open-quantum-safe#8 0x7f6899618054 in asn1_item_embed_d2i /home/pawn/work/openssl/crypto/asn1/tasn_dec.c:217:16
    open-quantum-safe#9 0x7f689961e02f in asn1_template_noexp_d2i /home/pawn/work/openssl/crypto/asn1/tasn_dec.c:682:15
    open-quantum-safe#10 0x7f689961a7b4 in asn1_template_ex_d2i /home/pawn/work/openssl/crypto/asn1/tasn_dec.c:558:16
    open-quantum-safe#11 0x7f689961986a in asn1_item_embed_d2i /home/pawn/work/openssl/crypto/asn1/tasn_dec.c:422:19
    open-quantum-safe#12 0x7f68996175c8 in asn1_item_ex_d2i_intern /home/pawn/work/openssl/crypto/asn1/tasn_dec.c:118:10
    open-quantum-safe#13 0x7f68996177da in ASN1_item_d2i_ex /home/pawn/work/openssl/crypto/asn1/tasn_dec.c:144:9
    open-quantum-safe#14 0x7f6895295926  (<unknown module>)
    open-quantum-safe#15 0x7f6895295ce4  (<unknown module>)
    open-quantum-safe#16 0x7f6895296a6c  (<unknown module>)
    open-quantum-safe#17 0x7f689986eae8 in decoder_process /home/pawn/work/openssl/crypto/encode_decode/decoder_lib.c:971:14
    open-quantum-safe#18 0x7f6899d13383 in spki2typespki_decode /home/pawn/work/openssl/providers/implementations/encode_decode/decode_spki2typespki.c:111:10
    open-quantum-safe#19 0x7f689986eae8 in decoder_process /home/pawn/work/openssl/crypto/encode_decode/decoder_lib.c:971:14
    open-quantum-safe#20 0x7f6899d114d3 in pem2der_decode /home/pawn/work/openssl/providers/implementations/encode_decode/decode_pem2der.c:204:14
    open-quantum-safe#21 0x7f689986eae8 in decoder_process /home/pawn/work/openssl/crypto/encode_decode/decoder_lib.c:971:14
    open-quantum-safe#22 0x7f689986d106 in OSSL_DECODER_from_bio /home/pawn/work/openssl/crypto/encode_decode/decoder_lib.c:82:10
    open-quantum-safe#23 0x55610d6bd407 in decode_EVP_PKEY_prov /home/pawn/work/oqs-provider/test/oqs_test_endecode.c:151:10
    open-quantum-safe#24 0x55610d6bc75b in test_oqs_encdec /home/pawn/work/oqs-provider/test/oqs_test_endecode.c:188:14
    open-quantum-safe#25 0x55610d6bbf37 in main /home/pawn/work/oqs-provider/test/oqs_test_endecode.c:240:17
    open-quantum-safe#26 0x7f68991671c9 in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16

Indirect leak of 16 byte(s) in 1 object(s) allocated from:
    #0 0x55610d68132e in malloc (/home/pawn/work/oqs-provider/build/test/oqs_test_endecode+0xb932e) (BuildId: 53bfaa0a3b850edc7328f30be8b047ab6675ba3c)
    open-quantum-safe#1 0x7f68999a946b in CRYPTO_malloc /home/pawn/work/openssl/crypto/mem.c:190:12
    open-quantum-safe#2 0x7f68999a94a2 in CRYPTO_zalloc /home/pawn/work/openssl/crypto/mem.c:197:11
    open-quantum-safe#3 0x7f6899625ed4 in asn1_item_embed_new /home/pawn/work/openssl/crypto/asn1/tasn_new.c:136:21
    open-quantum-safe#4 0x7f689962620e in ossl_asn1_item_ex_new_intern /home/pawn/work/openssl/crypto/asn1/tasn_new.c:52:12
    open-quantum-safe#5 0x7f68996191a5 in asn1_item_embed_d2i /home/pawn/work/openssl/crypto/asn1/tasn_dec.c:366:21
    open-quantum-safe#6 0x7f689961e02f in asn1_template_noexp_d2i /home/pawn/work/openssl/crypto/asn1/tasn_dec.c:682:15
    open-quantum-safe#7 0x7f689961a7b4 in asn1_template_ex_d2i /home/pawn/work/openssl/crypto/asn1/tasn_dec.c:558:16
    open-quantum-safe#8 0x7f689961986a in asn1_item_embed_d2i /home/pawn/work/openssl/crypto/asn1/tasn_dec.c:422:19
    open-quantum-safe#9 0x7f68996175c8 in asn1_item_ex_d2i_intern /home/pawn/work/openssl/crypto/asn1/tasn_dec.c:118:10
    open-quantum-safe#10 0x7f68996177da in ASN1_item_d2i_ex /home/pawn/work/openssl/crypto/asn1/tasn_dec.c:144:9
    open-quantum-safe#11 0x7f6895295926  (<unknown module>)
    open-quantum-safe#12 0x7f6895295ce4  (<unknown module>)
    open-quantum-safe#13 0x7f6895296a6c  (<unknown module>)
    open-quantum-safe#14 0x7f689986eae8 in decoder_process /home/pawn/work/openssl/crypto/encode_decode/decoder_lib.c:971:14
    open-quantum-safe#15 0x7f6899d13383 in spki2typespki_decode /home/pawn/work/openssl/providers/implementations/encode_decode/decode_spki2typespki.c:111:10
    open-quantum-safe#16 0x7f689986eae8 in decoder_process /home/pawn/work/openssl/crypto/encode_decode/decoder_lib.c:971:14
    open-quantum-safe#17 0x7f6899d114d3 in pem2der_decode /home/pawn/work/openssl/providers/implementations/encode_decode/decode_pem2der.c:204:14
    open-quantum-safe#18 0x7f689986eae8 in decoder_process /home/pawn/work/openssl/crypto/encode_decode/decoder_lib.c:971:14
    open-quantum-safe#19 0x7f689986d106 in OSSL_DECODER_from_bio /home/pawn/work/openssl/crypto/encode_decode/decoder_lib.c:82:10
    open-quantum-safe#20 0x55610d6bd407 in decode_EVP_PKEY_prov /home/pawn/work/oqs-provider/test/oqs_test_endecode.c:151:10
    open-quantum-safe#21 0x55610d6bc75b in test_oqs_encdec /home/pawn/work/oqs-provider/test/oqs_test_endecode.c:188:14
    open-quantum-safe#22 0x55610d6bbf37 in main /home/pawn/work/oqs-provider/test/oqs_test_endecode.c:240:17
    open-quantum-safe#23 0x7f68991671c9 in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16

SUMMARY: AddressSanitizer: 1400 byte(s) leaked in 4 allocation(s).
```


## 8. Bonus point: fix a potential double-free.

In `oqsx_key_gen`, when the keytype isn't `KEY_TYPE_HYB_SIG`, the
generated `pkey` is freed, and the flow goes down to the `err` label:

https://github.com/open-quantum-safe/oqs-provider/blob/cc3895f208730abfe1efc15f7c3ff4358ac6e4c0/oqsprov/oqsprov_keys.c#L1089-L1104

There is a potential double-free bug here: if `oqsx_key_gen_oqs` fails, then
`EVP_PKEY_free` will be called twice on the same `pkey` pointer.

Note that it is very unlikely that `oqsx_key_gen_oqs` fails, since it only
calls `OQS_KEM_keypair`/`OQS_SIG_keypair`.
thb-sb added a commit to thb-sb/oqs-provider that referenced this pull request Sep 5, 2023
@thb-sb
Fix various memory leaks.
fe9d50d 
This commit fixes various memory leaks that have been found using ASan.

Note: I will soon add a CI job for testing oqs-provider against memory leaks
with ASan.


# 1. Memory leak in `oqs_test_signatures`.

In `oqs_test_signatures`, [`OPENSSL_free`] was used with `EVP_PKEY_CTX`. However,
the right destructor function for `EVP_PKEY_CTX` is [`EVP_PKEY_CTX_free`].

This commmit fixes this minor bug.

[`OPENSSL_free`](https://www.openssl.org/docs/man3.0/man3/OPENSSL_free.html)
[`EVP_PKEY_CTX_free`](https://www.openssl.org/docs/man3.0/man3/EVP_PKEY_CTX_free.html)


## 2. a selftest key that wasn't freed.

In `oqsprov/oqsprov_keys.c`, `d2i_PrivateKey` is used to verify that we the
key we've just generated is sound.
However, the returned object wasn't freed.
ASan trace that helped me finding this one:

```
Indirect leak of 96 byte(s) in 4 object(s) allocated from:
    #0 0x5651eef1428e in malloc (/home/pawn/work/oqs-provider/build/test/oqs_test_signatures+0xb928e) (BuildId: 6cc7ebd4bc73a0c78b2ab8d7b9dcb8b165d88e74)
    open-quantum-safe#1 0x7fb7a23a946b in CRYPTO_malloc /home/pawn/work/openssl/crypto/mem.c:190:12
    open-quantum-safe#2 0x7fb7a23a94a2 in CRYPTO_zalloc /home/pawn/work/openssl/crypto/mem.c:197:11
    open-quantum-safe#3 0x7fb7a209cdcd in BN_new /home/pawn/work/openssl/crypto/bn/bn_lib.c:247:16
    open-quantum-safe#4 0x7fb7a2224a00 in ossl_ec_GFp_mont_group_set_curve /home/pawn/work/openssl/crypto/ec/ecp_mont.c:169:11
    open-quantum-safe#5 0x7fb7a220f5fa in EC_GROUP_set_curve /home/pawn/work/openssl/crypto/ec/ec_lib.c:562:12
    open-quantum-safe#6 0x7fb7a2202d75 in EC_GROUP_new_curve_GFp /home/pawn/work/openssl/crypto/ec/ec_cvt.c:61:10
    open-quantum-safe#7 0x7fb7a220184d in ec_group_new_from_data /home/pawn/work/openssl/crypto/ec/ec_curve.c:3179:22
    open-quantum-safe#8 0x7fb7a2200ff9 in EC_GROUP_new_by_curve_name_ex /home/pawn/work/openssl/crypto/ec/ec_curve.c:3287:19
    open-quantum-safe#9 0x7fb7a2201ed9 in EC_GROUP_new_by_curve_name /home/pawn/work/openssl/crypto/ec/ec_curve.c:3303:12
    open-quantum-safe#10 0x7fb7a21f84e2 in EC_GROUP_new_from_ecpkparameters /home/pawn/work/openssl/crypto/ec/ec_asn1.c:863:20
    open-quantum-safe#11 0x7fb7a21f8f28 in d2i_ECPrivateKey /home/pawn/work/openssl/crypto/ec/ec_asn1.c:956:22
    open-quantum-safe#12 0x7fb7a270b4b5 in der2key_decode /home/pawn/work/openssl/providers/implementations/encode_decode/decode_der2key.c:220:19
    open-quantum-safe#13 0x7fb7a226eae8 in decoder_process /home/pawn/work/openssl/crypto/encode_decode/decoder_lib.c:971:14
    open-quantum-safe#14 0x7fb7a226d106 in OSSL_DECODER_from_bio /home/pawn/work/openssl/crypto/encode_decode/decoder_lib.c:82:10
    open-quantum-safe#15 0x7fb7a226f007 in OSSL_DECODER_from_data /home/pawn/work/openssl/crypto/encode_decode/decoder_lib.c:157:9
    open-quantum-safe#16 0x7fb7a200e123 in d2i_PrivateKey_decoder /home/pawn/work/openssl/crypto/asn1/d2i_pr.c:74:11
    open-quantum-safe#17 0x7fb7a200dbba in d2i_PrivateKey_ex /home/pawn/work/openssl/crypto/asn1/d2i_pr.c:162:11
    open-quantum-safe#18 0x7fb7a200e3d2 in d2i_PrivateKey /home/pawn/work/openssl/crypto/asn1/d2i_pr.c:172:12
    open-quantum-safe#19 0x7fb79dc6a22e in oqsx_key_gen_evp_key /home/pawn/work/oqs-provider/oqsprov/oqsprov_keys.c:1036:25
    open-quantum-safe#20 0x7fb79dc68f3f in oqsx_key_gen /home/pawn/work/oqs-provider/oqsprov/oqsprov_keys.c:1077:16
    open-quantum-safe#21 0x7fb79dc7164a in oqsx_genkey /home/pawn/work/oqs-provider/oqsprov/oqs_kmgmt.c:496:9
    open-quantum-safe#22 0x7fb79dc6d9f8 in oqsx_gen /home/pawn/work/oqs-provider/oqsprov/oqs_kmgmt.c:509:12
    open-quantum-safe#23 0x7fb7a23373c7 in evp_keymgmt_gen /home/pawn/work/openssl/crypto/evp/keymgmt_meth.c:366:12
    open-quantum-safe#24 0x7fb7a2334b58 in evp_keymgmt_util_gen /home/pawn/work/openssl/crypto/evp/keymgmt_lib.c:517:20
    open-quantum-safe#25 0x7fb7a235a775 in EVP_PKEY_generate /home/pawn/work/openssl/crypto/evp/pmeth_gn.c:189:13
    open-quantum-safe#26 0x5651eef4f2bd in test_oqs_signatures /home/pawn/work/oqs-provider/test/oqs_test_signatures.c:39:48
    open-quantum-safe#27 0x5651eef4ee3d in main /home/pawn/work/oqs-provider/test/oqs_test_signatures.c:114:17
    open-quantum-safe#28 0x7fb7a1b671c9 in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
```

## 3. a `EVP_PKEY_CTX` that wasn't freed.

In `oqsx_key_free`, `key->oqsx_provider_ctx.oqsx_evp_ctx->ctx` is freed if
the keytype is `ECP_HYB_KEM` or `ECX_HYB_KEM`. However,
`key->oqsx_provider_ctx.oqsx_evp_ctx->ctx` is also used for signatures.
In that case, a call to `EVP_PKEY_CTX_free` was missing.
ASan trace that helped me finding this one:

```
Indirect leak of 7 byte(s) in 2 object(s) allocated from:
    #0 0x55b6fc8d427e in malloc (/home/pawn/work/oqs-provider/build/test/oqs_test_signatures+0xb927e) (BuildId: 3a8bff15d3315ba7ac1746fdc7b67eb464c607a1)
    open-quantum-safe#1 0x7ff34dfa946b in CRYPTO_malloc /home/pawn/work/openssl/crypto/mem.c:190:12
    open-quantum-safe#2 0x7ff34dfac50a in CRYPTO_strndup /home/pawn/work/openssl/crypto/o_str.c:43:11
    open-quantum-safe#3 0x7ff34df9d152 in ossl_algorithm_get1_first_name /home/pawn/work/openssl/crypto/core_algorithm.c:195:11
    open-quantum-safe#4 0x7ff34df353a1 in keymgmt_from_algorithm /home/pawn/work/openssl/crypto/evp/keymgmt_meth.c:50:31
    open-quantum-safe#5 0x7ff34df15e44 in construct_evp_method /home/pawn/work/openssl/crypto/evp/evp_fetch.c:219:14
    open-quantum-safe#6 0x7ff34df9e3ce in ossl_method_construct_this /home/pawn/work/openssl/crypto/core_fetch.c:109:19
    open-quantum-safe#7 0x7ff34df9d740 in algorithm_do_map /home/pawn/work/openssl/crypto/core_algorithm.c:77:13
    open-quantum-safe#8 0x7ff34df9ce88 in algorithm_do_this /home/pawn/work/openssl/crypto/core_algorithm.c:122:15
    open-quantum-safe#9 0x7ff34dfcce3e in ossl_provider_doall_activated /home/pawn/work/openssl/crypto/provider_core.c:1423:14
    open-quantum-safe#10 0x7ff34df9ca64 in ossl_algorithm_do_all /home/pawn/work/openssl/crypto/core_algorithm.c:162:9
    open-quantum-safe#11 0x7ff34df9dd64 in ossl_method_construct /home/pawn/work/openssl/crypto/core_fetch.c:153:5
    open-quantum-safe#12 0x7ff34df13b6f in inner_evp_generic_fetch /home/pawn/work/openssl/crypto/evp/evp_fetch.c:312:23
    open-quantum-safe#13 0x7ff34df13226 in evp_generic_fetch /home/pawn/work/openssl/crypto/evp/evp_fetch.c:364:14
    open-quantum-safe#14 0x7ff34df36a42 in EVP_KEYMGMT_fetch /home/pawn/work/openssl/crypto/evp/keymgmt_meth.c:220:12
    open-quantum-safe#15 0x7ff34df5d6e4 in int_ctx_new /home/pawn/work/openssl/crypto/evp/pmeth_lib.c:280:23
    open-quantum-safe#16 0x7ff34df5f0a9 in EVP_PKEY_CTX_new_id /home/pawn/work/openssl/crypto/evp/pmeth_lib.c:469:12
    open-quantum-safe#17 0x7ff349865bcb in oqsx_hybsig_init /home/pawn/work/oqs-provider/oqsprov/oqsprov_keys.c:578:20
    open-quantum-safe#18 0x7ff349865157 in oqsx_key_new /home/pawn/work/oqs-provider/oqsprov/oqsprov_keys.c:791:16
    open-quantum-safe#19 0x7ff3498715a3 in oqsx_genkey /home/pawn/work/oqs-provider/oqsprov/oqs_kmgmt.c:487:16
    open-quantum-safe#20 0x7ff34986da08 in oqsx_gen /home/pawn/work/oqs-provider/oqsprov/oqs_kmgmt.c:509:12
    open-quantum-safe#21 0x7ff34df373c7 in evp_keymgmt_gen /home/pawn/work/openssl/crypto/evp/keymgmt_meth.c:366:12
    open-quantum-safe#22 0x7ff34df34b58 in evp_keymgmt_util_gen /home/pawn/work/openssl/crypto/evp/keymgmt_lib.c:517:20
    open-quantum-safe#23 0x7ff34df5a775 in EVP_PKEY_generate /home/pawn/work/openssl/crypto/evp/pmeth_gn.c:189:13
    open-quantum-safe#24 0x55b6fc90f2ad in test_oqs_signatures /home/pawn/work/oqs-provider/test/oqs_test_signatures.c:39:48
    open-quantum-safe#25 0x55b6fc90ee2d in main /home/pawn/work/oqs-provider/test/oqs_test_signatures.c:114:17
    open-quantum-safe#26 0x7ff34d8461c9 in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
```

## 4. Missing `EVP_PKEY_CTX_free` in case of error.

In `oqsx_hybsig_init`, a `EVP_PKEY_CTX` is initialized using `EVP_PKEY_CTX_new_id`.
However, later in the code, some extra work is done, and in case of error, the
`EVP_PKEY_CTX` wasn't freed.

(I couldn't recover the ASan message for this one…)


## 5. `OPENSSL_free` instead of `EVP_PKEY_free`.

In `oqsx_key_gen`, when a classical private key is generated, the field
`classical_pkey` of a `OQSX_KEY` is set with a value of type `EVP_PKEY`.

However, in `oqsx_key_free`, `OPENSSL_free` was used on `classical_pkey`
instead of `EVP_PKEY_free`.

ASan trace that helped me finding this one:

```
   #0 0x555a6b1b728e in malloc (/home/pawn/work/oqs-provider/build/test/oqs_test_signatures+0xb928e) (BuildId: 6cc7ebd4bc73a0c78b2ab8d7b9dcb8b165d88e74)
    open-quantum-safe#1 0x7f59a93a946b in CRYPTO_malloc /home/pawn/work/openssl/crypto/mem.c:190:12
    open-quantum-safe#2 0x7f59a93a94a2 in CRYPTO_zalloc /home/pawn/work/openssl/crypto/mem.c:197:11
    open-quantum-safe#3 0x7f59a909cdcd in BN_new /home/pawn/work/openssl/crypto/bn/bn_lib.c:247:16
    open-quantum-safe#4 0x7f59a920b955 in ossl_ec_group_new_ex /home/pawn/work/openssl/crypto/ec/ec_lib.c:61:25
    open-quantum-safe#5 0x7f59a920dd55 in EC_GROUP_dup /home/pawn/work/openssl/crypto/ec/ec_lib.c:273:14
    open-quantum-safe#6 0x7f59a9207774 in EC_KEY_set_group /home/pawn/work/openssl/crypto/ec/ec_key.c:733:18
    open-quantum-safe#7 0x7f59a975b3b5 in ec_gen_assign_group /home/pawn/work/openssl/providers/implementations/keymgmt/ec_kmgmt.c:1236:12
    open-quantum-safe#8 0x7f59a9758bdb in ec_gen /home/pawn/work/openssl/providers/implementations/keymgmt/ec_kmgmt.c:1274:11
    open-quantum-safe#9 0x7f59a93373c7 in evp_keymgmt_gen /home/pawn/work/openssl/crypto/evp/keymgmt_meth.c:366:12
    open-quantum-safe#10 0x7f59a9334b58 in evp_keymgmt_util_gen /home/pawn/work/openssl/crypto/evp/keymgmt_lib.c:517:20
    open-quantum-safe#11 0x7f59a935a775 in EVP_PKEY_generate /home/pawn/work/openssl/crypto/evp/pmeth_gn.c:189:13
    open-quantum-safe#12 0x7f59a935b404 in EVP_PKEY_keygen /home/pawn/work/openssl/crypto/evp/pmeth_gn.c:274:12
    open-quantum-safe#13 0x7f59a4c69a51 in oqsx_key_gen_evp_key /home/pawn/work/oqs-provider/oqsprov/oqsprov_keys.c:1012:12
    open-quantum-safe#14 0x7f59a4c68faf in oqsx_key_gen /home/pawn/work/oqs-provider/oqsprov/oqsprov_keys.c:1084:16
    open-quantum-safe#15 0x7f59a4c716ca in oqsx_genkey /home/pawn/work/oqs-provider/oqsprov/oqs_kmgmt.c:496:9
    open-quantum-safe#16 0x7f59a4c6da78 in oqsx_gen /home/pawn/work/oqs-provider/oqsprov/oqs_kmgmt.c:509:12
    open-quantum-safe#17 0x7f59a93373c7 in evp_keymgmt_gen /home/pawn/work/openssl/crypto/evp/keymgmt_meth.c:366:12
    open-quantum-safe#18 0x7f59a9334b58 in evp_keymgmt_util_gen /home/pawn/work/openssl/crypto/evp/keymgmt_lib.c:517:20
    open-quantum-safe#19 0x7f59a935a775 in EVP_PKEY_generate /home/pawn/work/openssl/crypto/evp/pmeth_gn.c:189:13
    open-quantum-safe#20 0x555a6b1f2890 in test_oqs_signatures /home/pawn/work/oqs-provider/test/oqs_test_signatures.c:68:44
    open-quantum-safe#21 0x555a6b1f1e3d in main /home/pawn/work/oqs-provider/test/oqs_test_signatures.c:114:17
    open-quantum-safe#22 0x7f59a8c461c9 in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
```

## 6. Some `OPENSSL_free` were missing in `oqs_test_kems.c`

```
==569479==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 346035 byte(s) in 42 object(s) allocated from:
    #0 0x559c76ef524e in malloc (/home/pawn/work/oqs-provider/build/test/oqs_test_kems+0xb924e) (BuildId: 76a98f5859ba3c90334cef4286e89aa76436b1ae)
    open-quantum-safe#1 0x7f04407a946b in CRYPTO_malloc /home/pawn/work/openssl/crypto/mem.c:190:12
    open-quantum-safe#2 0x559c76f30389 in test_oqs_kems /home/pawn/work/oqs-provider/test/oqs_test_kems.c:44:26
    open-quantum-safe#3 0x559c76f2fdfd in main /home/pawn/work/oqs-provider/test/oqs_test_kems.c:93:17
    open-quantum-safe#4 0x7f04400461c9 in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16

Direct leak of 2690 byte(s) in 42 object(s) allocated from:
    #0 0x559c76ef524e in malloc (/home/pawn/work/oqs-provider/build/test/oqs_test_kems+0xb924e) (BuildId: 76a98f5859ba3c90334cef4286e89aa76436b1ae)
    open-quantum-safe#1 0x7f04407a946b in CRYPTO_malloc /home/pawn/work/openssl/crypto/mem.c:190:12
    open-quantum-safe#2 0x559c76f304a5 in test_oqs_kems /home/pawn/work/oqs-provider/test/oqs_test_kems.c:47:29
    open-quantum-safe#3 0x559c76f2fdfd in main /home/pawn/work/oqs-provider/test/oqs_test_kems.c:93:17
    open-quantum-safe#4 0x7f04400461c9 in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16

Direct leak of 2690 byte(s) in 42 object(s) allocated from:
    #0 0x559c76ef524e in malloc (/home/pawn/work/oqs-provider/build/test/oqs_test_kems+0xb924e) (BuildId: 76a98f5859ba3c90334cef4286e89aa76436b1ae)
    open-quantum-safe#1 0x7f04407a946b in CRYPTO_malloc /home/pawn/work/openssl/crypto/mem.c:190:12
    open-quantum-safe#2 0x559c76f303e4 in test_oqs_kems /home/pawn/work/oqs-provider/test/oqs_test_kems.c:45:29
    open-quantum-safe#3 0x559c76f2fdfd in main /home/pawn/work/oqs-provider/test/oqs_test_kems.c:93:17
    open-quantum-safe#4 0x7f04400461c9 in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
```

## 7. Missing `X509_PUBKEY_free` in `oqsx_d2i_PUBKEY`.

This was probably the hardest one for me to track down.

In `oqsx_d2i_PUBKEY`, a call is done to `oqsx_d2i_X509_PUBKEY_INTERNAL`.
However, the returned value (of type `X509_PUBKEY`) wasn't freed.

ASan:

```
Direct leak of 48 byte(s) in 1 object(s) allocated from:
    #0 0x55610d68132e in malloc (/home/pawn/work/oqs-provider/build/test/oqs_test_endecode+0xb932e) (BuildId: 53bfaa0a3b850edc7328f30be8b047ab6675ba3c)
    open-quantum-safe#1 0x7f68999a946b in CRYPTO_malloc /home/pawn/work/openssl/crypto/mem.c:190:12
    open-quantum-safe#2 0x7f68999a94a2 in CRYPTO_zalloc /home/pawn/work/openssl/crypto/mem.c:197:11
    open-quantum-safe#3 0x7f6895295897  (<unknown module>)
    open-quantum-safe#4 0x7f6895295ce4  (<unknown module>)
    open-quantum-safe#5 0x7f6895296a6c  (<unknown module>)
    open-quantum-safe#6 0x7f689986eae8 in decoder_process /home/pawn/work/openssl/crypto/encode_decode/decoder_lib.c:971:14
    open-quantum-safe#7 0x7f6899d13383 in spki2typespki_decode /home/pawn/work/openssl/providers/implementations/encode_decode/decode_spki2typespki.c:111:10
    open-quantum-safe#8 0x7f689986eae8 in decoder_process /home/pawn/work/openssl/crypto/encode_decode/decoder_lib.c:971:14
    open-quantum-safe#9 0x7f6899d114d3 in pem2der_decode /home/pawn/work/openssl/providers/implementations/encode_decode/decode_pem2der.c:204:14
    open-quantum-safe#10 0x7f689986eae8 in decoder_process /home/pawn/work/openssl/crypto/encode_decode/decoder_lib.c:971:14
    open-quantum-safe#11 0x7f689986d106 in OSSL_DECODER_from_bio /home/pawn/work/openssl/crypto/encode_decode/decoder_lib.c:82:10
    open-quantum-safe#12 0x55610d6bd407 in decode_EVP_PKEY_prov /home/pawn/work/oqs-provider/test/oqs_test_endecode.c:151:10
    open-quantum-safe#13 0x55610d6bc75b in test_oqs_encdec /home/pawn/work/oqs-provider/test/oqs_test_endecode.c:188:14
    open-quantum-safe#14 0x55610d6bbf37 in main /home/pawn/work/oqs-provider/test/oqs_test_endecode.c:240:17
    open-quantum-safe#15 0x7f68991671c9 in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16

Indirect leak of 1312 byte(s) in 1 object(s) allocated from:
    #0 0x55610d68132e in malloc (/home/pawn/work/oqs-provider/build/test/oqs_test_endecode+0xb932e) (BuildId: 53bfaa0a3b850edc7328f30be8b047ab6675ba3c)
    open-quantum-safe#1 0x7f68999a946b in CRYPTO_malloc /home/pawn/work/openssl/crypto/mem.c:190:12
    open-quantum-safe#2 0x7f68995d6b99 in ossl_c2i_ASN1_BIT_STRING /home/pawn/work/openssl/crypto/asn1/a_bitstr.c:117:13
    open-quantum-safe#3 0x7f689961f659 in asn1_ex_c2i /home/pawn/work/openssl/crypto/asn1/tasn_dec.c:885:14
    open-quantum-safe#4 0x7f689961bbc9 in asn1_d2i_ex_primitive /home/pawn/work/openssl/crypto/asn1/tasn_dec.c:818:10
    open-quantum-safe#5 0x7f6899618054 in asn1_item_embed_d2i /home/pawn/work/openssl/crypto/asn1/tasn_dec.c:217:16
    open-quantum-safe#6 0x7f689961e02f in asn1_template_noexp_d2i /home/pawn/work/openssl/crypto/asn1/tasn_dec.c:682:15
    open-quantum-safe#7 0x7f689961a7b4 in asn1_template_ex_d2i /home/pawn/work/openssl/crypto/asn1/tasn_dec.c:558:16
    open-quantum-safe#8 0x7f689961986a in asn1_item_embed_d2i /home/pawn/work/openssl/crypto/asn1/tasn_dec.c:422:19
    open-quantum-safe#9 0x7f68996175c8 in asn1_item_ex_d2i_intern /home/pawn/work/openssl/crypto/asn1/tasn_dec.c:118:10
    open-quantum-safe#10 0x7f68996177da in ASN1_item_d2i_ex /home/pawn/work/openssl/crypto/asn1/tasn_dec.c:144:9
    open-quantum-safe#11 0x7f6895295926  (<unknown module>)
    open-quantum-safe#12 0x7f6895295ce4  (<unknown module>)
    open-quantum-safe#13 0x7f6895296a6c  (<unknown module>)
    open-quantum-safe#14 0x7f689986eae8 in decoder_process /home/pawn/work/openssl/crypto/encode_decode/decoder_lib.c:971:14
    open-quantum-safe#15 0x7f6899d13383 in spki2typespki_decode /home/pawn/work/openssl/providers/implementations/encode_decode/decode_spki2typespki.c:111:10
    open-quantum-safe#16 0x7f689986eae8 in decoder_process /home/pawn/work/openssl/crypto/encode_decode/decoder_lib.c:971:14
    open-quantum-safe#17 0x7f6899d114d3 in pem2der_decode /home/pawn/work/openssl/providers/implementations/encode_decode/decode_pem2der.c:204:14
    open-quantum-safe#18 0x7f689986eae8 in decoder_process /home/pawn/work/openssl/crypto/encode_decode/decoder_lib.c:971:14
    open-quantum-safe#19 0x7f689986d106 in OSSL_DECODER_from_bio /home/pawn/work/openssl/crypto/encode_decode/decoder_lib.c:82:10
    open-quantum-safe#20 0x55610d6bd407 in decode_EVP_PKEY_prov /home/pawn/work/oqs-provider/test/oqs_test_endecode.c:151:10
    open-quantum-safe#21 0x55610d6bc75b in test_oqs_encdec /home/pawn/work/oqs-provider/test/oqs_test_endecode.c:188:14
    open-quantum-safe#22 0x55610d6bbf37 in main /home/pawn/work/oqs-provider/test/oqs_test_endecode.c:240:17
    open-quantum-safe#23 0x7f68991671c9 in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16

Indirect leak of 24 byte(s) in 1 object(s) allocated from:
    #0 0x55610d68132e in malloc (/home/pawn/work/oqs-provider/build/test/oqs_test_endecode+0xb932e) (BuildId: 53bfaa0a3b850edc7328f30be8b047ab6675ba3c)
    open-quantum-safe#1 0x7f68999a946b in CRYPTO_malloc /home/pawn/work/openssl/crypto/mem.c:190:12
    open-quantum-safe#2 0x7f68999a94a2 in CRYPTO_zalloc /home/pawn/work/openssl/crypto/mem.c:197:11
    open-quantum-safe#3 0x7f68995fde90 in ASN1_STRING_type_new /home/pawn/work/openssl/crypto/asn1/asn1_lib.c:350:11
    open-quantum-safe#4 0x7f689962a7dd in ASN1_BIT_STRING_new /home/pawn/work/openssl/crypto/asn1/tasn_typ.c:31:1
    open-quantum-safe#5 0x7f68995d6a0f in ossl_c2i_ASN1_BIT_STRING /home/pawn/work/openssl/crypto/asn1/a_bitstr.c:98:20
    open-quantum-safe#6 0x7f689961f659 in asn1_ex_c2i /home/pawn/work/openssl/crypto/asn1/tasn_dec.c:885:14
    open-quantum-safe#7 0x7f689961bbc9 in asn1_d2i_ex_primitive /home/pawn/work/openssl/crypto/asn1/tasn_dec.c:818:10
    open-quantum-safe#8 0x7f6899618054 in asn1_item_embed_d2i /home/pawn/work/openssl/crypto/asn1/tasn_dec.c:217:16
    open-quantum-safe#9 0x7f689961e02f in asn1_template_noexp_d2i /home/pawn/work/openssl/crypto/asn1/tasn_dec.c:682:15
    open-quantum-safe#10 0x7f689961a7b4 in asn1_template_ex_d2i /home/pawn/work/openssl/crypto/asn1/tasn_dec.c:558:16
    open-quantum-safe#11 0x7f689961986a in asn1_item_embed_d2i /home/pawn/work/openssl/crypto/asn1/tasn_dec.c:422:19
    open-quantum-safe#12 0x7f68996175c8 in asn1_item_ex_d2i_intern /home/pawn/work/openssl/crypto/asn1/tasn_dec.c:118:10
    open-quantum-safe#13 0x7f68996177da in ASN1_item_d2i_ex /home/pawn/work/openssl/crypto/asn1/tasn_dec.c:144:9
    open-quantum-safe#14 0x7f6895295926  (<unknown module>)
    open-quantum-safe#15 0x7f6895295ce4  (<unknown module>)
    open-quantum-safe#16 0x7f6895296a6c  (<unknown module>)
    open-quantum-safe#17 0x7f689986eae8 in decoder_process /home/pawn/work/openssl/crypto/encode_decode/decoder_lib.c:971:14
    open-quantum-safe#18 0x7f6899d13383 in spki2typespki_decode /home/pawn/work/openssl/providers/implementations/encode_decode/decode_spki2typespki.c:111:10
    open-quantum-safe#19 0x7f689986eae8 in decoder_process /home/pawn/work/openssl/crypto/encode_decode/decoder_lib.c:971:14
    open-quantum-safe#20 0x7f6899d114d3 in pem2der_decode /home/pawn/work/openssl/providers/implementations/encode_decode/decode_pem2der.c:204:14
    open-quantum-safe#21 0x7f689986eae8 in decoder_process /home/pawn/work/openssl/crypto/encode_decode/decoder_lib.c:971:14
    open-quantum-safe#22 0x7f689986d106 in OSSL_DECODER_from_bio /home/pawn/work/openssl/crypto/encode_decode/decoder_lib.c:82:10
    open-quantum-safe#23 0x55610d6bd407 in decode_EVP_PKEY_prov /home/pawn/work/oqs-provider/test/oqs_test_endecode.c:151:10
    open-quantum-safe#24 0x55610d6bc75b in test_oqs_encdec /home/pawn/work/oqs-provider/test/oqs_test_endecode.c:188:14
    open-quantum-safe#25 0x55610d6bbf37 in main /home/pawn/work/oqs-provider/test/oqs_test_endecode.c:240:17
    open-quantum-safe#26 0x7f68991671c9 in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16

Indirect leak of 16 byte(s) in 1 object(s) allocated from:
    #0 0x55610d68132e in malloc (/home/pawn/work/oqs-provider/build/test/oqs_test_endecode+0xb932e) (BuildId: 53bfaa0a3b850edc7328f30be8b047ab6675ba3c)
    open-quantum-safe#1 0x7f68999a946b in CRYPTO_malloc /home/pawn/work/openssl/crypto/mem.c:190:12
    open-quantum-safe#2 0x7f68999a94a2 in CRYPTO_zalloc /home/pawn/work/openssl/crypto/mem.c:197:11
    open-quantum-safe#3 0x7f6899625ed4 in asn1_item_embed_new /home/pawn/work/openssl/crypto/asn1/tasn_new.c:136:21
    open-quantum-safe#4 0x7f689962620e in ossl_asn1_item_ex_new_intern /home/pawn/work/openssl/crypto/asn1/tasn_new.c:52:12
    open-quantum-safe#5 0x7f68996191a5 in asn1_item_embed_d2i /home/pawn/work/openssl/crypto/asn1/tasn_dec.c:366:21
    open-quantum-safe#6 0x7f689961e02f in asn1_template_noexp_d2i /home/pawn/work/openssl/crypto/asn1/tasn_dec.c:682:15
    open-quantum-safe#7 0x7f689961a7b4 in asn1_template_ex_d2i /home/pawn/work/openssl/crypto/asn1/tasn_dec.c:558:16
    open-quantum-safe#8 0x7f689961986a in asn1_item_embed_d2i /home/pawn/work/openssl/crypto/asn1/tasn_dec.c:422:19
    open-quantum-safe#9 0x7f68996175c8 in asn1_item_ex_d2i_intern /home/pawn/work/openssl/crypto/asn1/tasn_dec.c:118:10
    open-quantum-safe#10 0x7f68996177da in ASN1_item_d2i_ex /home/pawn/work/openssl/crypto/asn1/tasn_dec.c:144:9
    open-quantum-safe#11 0x7f6895295926  (<unknown module>)
    open-quantum-safe#12 0x7f6895295ce4  (<unknown module>)
    open-quantum-safe#13 0x7f6895296a6c  (<unknown module>)
    open-quantum-safe#14 0x7f689986eae8 in decoder_process /home/pawn/work/openssl/crypto/encode_decode/decoder_lib.c:971:14
    open-quantum-safe#15 0x7f6899d13383 in spki2typespki_decode /home/pawn/work/openssl/providers/implementations/encode_decode/decode_spki2typespki.c:111:10
    open-quantum-safe#16 0x7f689986eae8 in decoder_process /home/pawn/work/openssl/crypto/encode_decode/decoder_lib.c:971:14
    open-quantum-safe#17 0x7f6899d114d3 in pem2der_decode /home/pawn/work/openssl/providers/implementations/encode_decode/decode_pem2der.c:204:14
    open-quantum-safe#18 0x7f689986eae8 in decoder_process /home/pawn/work/openssl/crypto/encode_decode/decoder_lib.c:971:14
    open-quantum-safe#19 0x7f689986d106 in OSSL_DECODER_from_bio /home/pawn/work/openssl/crypto/encode_decode/decoder_lib.c:82:10
    open-quantum-safe#20 0x55610d6bd407 in decode_EVP_PKEY_prov /home/pawn/work/oqs-provider/test/oqs_test_endecode.c:151:10
    open-quantum-safe#21 0x55610d6bc75b in test_oqs_encdec /home/pawn/work/oqs-provider/test/oqs_test_endecode.c:188:14
    open-quantum-safe#22 0x55610d6bbf37 in main /home/pawn/work/oqs-provider/test/oqs_test_endecode.c:240:17
    open-quantum-safe#23 0x7f68991671c9 in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16

SUMMARY: AddressSanitizer: 1400 byte(s) leaked in 4 allocation(s).
```


## 8. Bonus point: fix a potential double-free.

In `oqsx_key_gen`, when the keytype isn't `KEY_TYPE_HYB_SIG`, the
generated `pkey` is freed, and the flow goes down to the `err` label:

https://github.com/open-quantum-safe/oqs-provider/blob/cc3895f208730abfe1efc15f7c3ff4358ac6e4c0/oqsprov/oqsprov_keys.c#L1089-L1104

There is a potential double-free bug here: if `oqsx_key_gen_oqs` fails, then
`EVP_PKEY_free` will be called twice on the same `pkey` pointer.

Note that it is very unlikely that `oqsx_key_gen_oqs` fails, since it only
calls `OQS_KEM_keypair`/`OQS_SIG_keypair`.
thb-sb added a commit to thb-sb/oqs-provider that referenced this pull request Sep 6, 2023
@thb-sb
Fix various memory leaks.
0ee6a8d 
This commit fixes various memory leaks that have been found using ASan.

Note: I will soon add a CI job for testing oqs-provider against memory leaks
with ASan.


# 1. Memory leak in `oqs_test_signatures`.

In `oqs_test_signatures`, [`OPENSSL_free`] was used with `EVP_PKEY_CTX`. However,
the right destructor function for `EVP_PKEY_CTX` is [`EVP_PKEY_CTX_free`].

This commmit fixes this minor bug.

[`OPENSSL_free`](https://www.openssl.org/docs/man3.0/man3/OPENSSL_free.html)
[`EVP_PKEY_CTX_free`](https://www.openssl.org/docs/man3.0/man3/EVP_PKEY_CTX_free.html)


## 2. a selftest key that wasn't freed.

In `oqsprov/oqsprov_keys.c`, `d2i_PrivateKey` is used to verify that we the
key we've just generated is sound.
However, the returned object wasn't freed.
ASan trace that helped me finding this one:

```
Indirect leak of 96 byte(s) in 4 object(s) allocated from:
    #0 0x5651eef1428e in malloc (/home/pawn/work/oqs-provider/build/test/oqs_test_signatures+0xb928e) (BuildId: 6cc7ebd4bc73a0c78b2ab8d7b9dcb8b165d88e74)
    open-quantum-safe#1 0x7fb7a23a946b in CRYPTO_malloc /home/pawn/work/openssl/crypto/mem.c:190:12
    open-quantum-safe#2 0x7fb7a23a94a2 in CRYPTO_zalloc /home/pawn/work/openssl/crypto/mem.c:197:11
    open-quantum-safe#3 0x7fb7a209cdcd in BN_new /home/pawn/work/openssl/crypto/bn/bn_lib.c:247:16
    open-quantum-safe#4 0x7fb7a2224a00 in ossl_ec_GFp_mont_group_set_curve /home/pawn/work/openssl/crypto/ec/ecp_mont.c:169:11
    open-quantum-safe#5 0x7fb7a220f5fa in EC_GROUP_set_curve /home/pawn/work/openssl/crypto/ec/ec_lib.c:562:12
    open-quantum-safe#6 0x7fb7a2202d75 in EC_GROUP_new_curve_GFp /home/pawn/work/openssl/crypto/ec/ec_cvt.c:61:10
    open-quantum-safe#7 0x7fb7a220184d in ec_group_new_from_data /home/pawn/work/openssl/crypto/ec/ec_curve.c:3179:22
    open-quantum-safe#8 0x7fb7a2200ff9 in EC_GROUP_new_by_curve_name_ex /home/pawn/work/openssl/crypto/ec/ec_curve.c:3287:19
    open-quantum-safe#9 0x7fb7a2201ed9 in EC_GROUP_new_by_curve_name /home/pawn/work/openssl/crypto/ec/ec_curve.c:3303:12
    open-quantum-safe#10 0x7fb7a21f84e2 in EC_GROUP_new_from_ecpkparameters /home/pawn/work/openssl/crypto/ec/ec_asn1.c:863:20
    open-quantum-safe#11 0x7fb7a21f8f28 in d2i_ECPrivateKey /home/pawn/work/openssl/crypto/ec/ec_asn1.c:956:22
    open-quantum-safe#12 0x7fb7a270b4b5 in der2key_decode /home/pawn/work/openssl/providers/implementations/encode_decode/decode_der2key.c:220:19
    open-quantum-safe#13 0x7fb7a226eae8 in decoder_process /home/pawn/work/openssl/crypto/encode_decode/decoder_lib.c:971:14
    open-quantum-safe#14 0x7fb7a226d106 in OSSL_DECODER_from_bio /home/pawn/work/openssl/crypto/encode_decode/decoder_lib.c:82:10
    open-quantum-safe#15 0x7fb7a226f007 in OSSL_DECODER_from_data /home/pawn/work/openssl/crypto/encode_decode/decoder_lib.c:157:9
    open-quantum-safe#16 0x7fb7a200e123 in d2i_PrivateKey_decoder /home/pawn/work/openssl/crypto/asn1/d2i_pr.c:74:11
    open-quantum-safe#17 0x7fb7a200dbba in d2i_PrivateKey_ex /home/pawn/work/openssl/crypto/asn1/d2i_pr.c:162:11
    open-quantum-safe#18 0x7fb7a200e3d2 in d2i_PrivateKey /home/pawn/work/openssl/crypto/asn1/d2i_pr.c:172:12
    open-quantum-safe#19 0x7fb79dc6a22e in oqsx_key_gen_evp_key /home/pawn/work/oqs-provider/oqsprov/oqsprov_keys.c:1036:25
    open-quantum-safe#20 0x7fb79dc68f3f in oqsx_key_gen /home/pawn/work/oqs-provider/oqsprov/oqsprov_keys.c:1077:16
    open-quantum-safe#21 0x7fb79dc7164a in oqsx_genkey /home/pawn/work/oqs-provider/oqsprov/oqs_kmgmt.c:496:9
    open-quantum-safe#22 0x7fb79dc6d9f8 in oqsx_gen /home/pawn/work/oqs-provider/oqsprov/oqs_kmgmt.c:509:12
    open-quantum-safe#23 0x7fb7a23373c7 in evp_keymgmt_gen /home/pawn/work/openssl/crypto/evp/keymgmt_meth.c:366:12
    open-quantum-safe#24 0x7fb7a2334b58 in evp_keymgmt_util_gen /home/pawn/work/openssl/crypto/evp/keymgmt_lib.c:517:20
    open-quantum-safe#25 0x7fb7a235a775 in EVP_PKEY_generate /home/pawn/work/openssl/crypto/evp/pmeth_gn.c:189:13
    open-quantum-safe#26 0x5651eef4f2bd in test_oqs_signatures /home/pawn/work/oqs-provider/test/oqs_test_signatures.c:39:48
    open-quantum-safe#27 0x5651eef4ee3d in main /home/pawn/work/oqs-provider/test/oqs_test_signatures.c:114:17
    open-quantum-safe#28 0x7fb7a1b671c9 in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
```

## 3. a `EVP_PKEY_CTX` that wasn't freed.

In `oqsx_key_free`, `key->oqsx_provider_ctx.oqsx_evp_ctx->ctx` is freed if
the keytype is `ECP_HYB_KEM` or `ECX_HYB_KEM`. However,
`key->oqsx_provider_ctx.oqsx_evp_ctx->ctx` is also used for signatures.
In that case, a call to `EVP_PKEY_CTX_free` was missing.
ASan trace that helped me finding this one:

```
Indirect leak of 7 byte(s) in 2 object(s) allocated from:
    #0 0x55b6fc8d427e in malloc (/home/pawn/work/oqs-provider/build/test/oqs_test_signatures+0xb927e) (BuildId: 3a8bff15d3315ba7ac1746fdc7b67eb464c607a1)
    open-quantum-safe#1 0x7ff34dfa946b in CRYPTO_malloc /home/pawn/work/openssl/crypto/mem.c:190:12
    open-quantum-safe#2 0x7ff34dfac50a in CRYPTO_strndup /home/pawn/work/openssl/crypto/o_str.c:43:11
    open-quantum-safe#3 0x7ff34df9d152 in ossl_algorithm_get1_first_name /home/pawn/work/openssl/crypto/core_algorithm.c:195:11
    open-quantum-safe#4 0x7ff34df353a1 in keymgmt_from_algorithm /home/pawn/work/openssl/crypto/evp/keymgmt_meth.c:50:31
    open-quantum-safe#5 0x7ff34df15e44 in construct_evp_method /home/pawn/work/openssl/crypto/evp/evp_fetch.c:219:14
    open-quantum-safe#6 0x7ff34df9e3ce in ossl_method_construct_this /home/pawn/work/openssl/crypto/core_fetch.c:109:19
    open-quantum-safe#7 0x7ff34df9d740 in algorithm_do_map /home/pawn/work/openssl/crypto/core_algorithm.c:77:13
    open-quantum-safe#8 0x7ff34df9ce88 in algorithm_do_this /home/pawn/work/openssl/crypto/core_algorithm.c:122:15
    open-quantum-safe#9 0x7ff34dfcce3e in ossl_provider_doall_activated /home/pawn/work/openssl/crypto/provider_core.c:1423:14
    open-quantum-safe#10 0x7ff34df9ca64 in ossl_algorithm_do_all /home/pawn/work/openssl/crypto/core_algorithm.c:162:9
    open-quantum-safe#11 0x7ff34df9dd64 in ossl_method_construct /home/pawn/work/openssl/crypto/core_fetch.c:153:5
    open-quantum-safe#12 0x7ff34df13b6f in inner_evp_generic_fetch /home/pawn/work/openssl/crypto/evp/evp_fetch.c:312:23
    open-quantum-safe#13 0x7ff34df13226 in evp_generic_fetch /home/pawn/work/openssl/crypto/evp/evp_fetch.c:364:14
    open-quantum-safe#14 0x7ff34df36a42 in EVP_KEYMGMT_fetch /home/pawn/work/openssl/crypto/evp/keymgmt_meth.c:220:12
    open-quantum-safe#15 0x7ff34df5d6e4 in int_ctx_new /home/pawn/work/openssl/crypto/evp/pmeth_lib.c:280:23
    open-quantum-safe#16 0x7ff34df5f0a9 in EVP_PKEY_CTX_new_id /home/pawn/work/openssl/crypto/evp/pmeth_lib.c:469:12
    open-quantum-safe#17 0x7ff349865bcb in oqsx_hybsig_init /home/pawn/work/oqs-provider/oqsprov/oqsprov_keys.c:578:20
    open-quantum-safe#18 0x7ff349865157 in oqsx_key_new /home/pawn/work/oqs-provider/oqsprov/oqsprov_keys.c:791:16
    open-quantum-safe#19 0x7ff3498715a3 in oqsx_genkey /home/pawn/work/oqs-provider/oqsprov/oqs_kmgmt.c:487:16
    open-quantum-safe#20 0x7ff34986da08 in oqsx_gen /home/pawn/work/oqs-provider/oqsprov/oqs_kmgmt.c:509:12
    open-quantum-safe#21 0x7ff34df373c7 in evp_keymgmt_gen /home/pawn/work/openssl/crypto/evp/keymgmt_meth.c:366:12
    open-quantum-safe#22 0x7ff34df34b58 in evp_keymgmt_util_gen /home/pawn/work/openssl/crypto/evp/keymgmt_lib.c:517:20
    open-quantum-safe#23 0x7ff34df5a775 in EVP_PKEY_generate /home/pawn/work/openssl/crypto/evp/pmeth_gn.c:189:13
    open-quantum-safe#24 0x55b6fc90f2ad in test_oqs_signatures /home/pawn/work/oqs-provider/test/oqs_test_signatures.c:39:48
    open-quantum-safe#25 0x55b6fc90ee2d in main /home/pawn/work/oqs-provider/test/oqs_test_signatures.c:114:17
    open-quantum-safe#26 0x7ff34d8461c9 in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
```

## 4. Missing `EVP_PKEY_CTX_free` in case of error.

In `oqsx_hybsig_init`, a `EVP_PKEY_CTX` is initialized using `EVP_PKEY_CTX_new_id`.
However, later in the code, some extra work is done, and in case of error, the
`EVP_PKEY_CTX` wasn't freed.

(I couldn't recover the ASan message for this one…)


## 5. `OPENSSL_free` instead of `EVP_PKEY_free`.

In `oqsx_key_gen`, when a classical private key is generated, the field
`classical_pkey` of a `OQSX_KEY` is set with a value of type `EVP_PKEY`.

However, in `oqsx_key_free`, `OPENSSL_free` was used on `classical_pkey`
instead of `EVP_PKEY_free`.

ASan trace that helped me finding this one:

```
   #0 0x555a6b1b728e in malloc (/home/pawn/work/oqs-provider/build/test/oqs_test_signatures+0xb928e) (BuildId: 6cc7ebd4bc73a0c78b2ab8d7b9dcb8b165d88e74)
    open-quantum-safe#1 0x7f59a93a946b in CRYPTO_malloc /home/pawn/work/openssl/crypto/mem.c:190:12
    open-quantum-safe#2 0x7f59a93a94a2 in CRYPTO_zalloc /home/pawn/work/openssl/crypto/mem.c:197:11
    open-quantum-safe#3 0x7f59a909cdcd in BN_new /home/pawn/work/openssl/crypto/bn/bn_lib.c:247:16
    open-quantum-safe#4 0x7f59a920b955 in ossl_ec_group_new_ex /home/pawn/work/openssl/crypto/ec/ec_lib.c:61:25
    open-quantum-safe#5 0x7f59a920dd55 in EC_GROUP_dup /home/pawn/work/openssl/crypto/ec/ec_lib.c:273:14
    open-quantum-safe#6 0x7f59a9207774 in EC_KEY_set_group /home/pawn/work/openssl/crypto/ec/ec_key.c:733:18
    open-quantum-safe#7 0x7f59a975b3b5 in ec_gen_assign_group /home/pawn/work/openssl/providers/implementations/keymgmt/ec_kmgmt.c:1236:12
    open-quantum-safe#8 0x7f59a9758bdb in ec_gen /home/pawn/work/openssl/providers/implementations/keymgmt/ec_kmgmt.c:1274:11
    open-quantum-safe#9 0x7f59a93373c7 in evp_keymgmt_gen /home/pawn/work/openssl/crypto/evp/keymgmt_meth.c:366:12
    open-quantum-safe#10 0x7f59a9334b58 in evp_keymgmt_util_gen /home/pawn/work/openssl/crypto/evp/keymgmt_lib.c:517:20
    open-quantum-safe#11 0x7f59a935a775 in EVP_PKEY_generate /home/pawn/work/openssl/crypto/evp/pmeth_gn.c:189:13
    open-quantum-safe#12 0x7f59a935b404 in EVP_PKEY_keygen /home/pawn/work/openssl/crypto/evp/pmeth_gn.c:274:12
    open-quantum-safe#13 0x7f59a4c69a51 in oqsx_key_gen_evp_key /home/pawn/work/oqs-provider/oqsprov/oqsprov_keys.c:1012:12
    open-quantum-safe#14 0x7f59a4c68faf in oqsx_key_gen /home/pawn/work/oqs-provider/oqsprov/oqsprov_keys.c:1084:16
    open-quantum-safe#15 0x7f59a4c716ca in oqsx_genkey /home/pawn/work/oqs-provider/oqsprov/oqs_kmgmt.c:496:9
    open-quantum-safe#16 0x7f59a4c6da78 in oqsx_gen /home/pawn/work/oqs-provider/oqsprov/oqs_kmgmt.c:509:12
    open-quantum-safe#17 0x7f59a93373c7 in evp_keymgmt_gen /home/pawn/work/openssl/crypto/evp/keymgmt_meth.c:366:12
    open-quantum-safe#18 0x7f59a9334b58 in evp_keymgmt_util_gen /home/pawn/work/openssl/crypto/evp/keymgmt_lib.c:517:20
    open-quantum-safe#19 0x7f59a935a775 in EVP_PKEY_generate /home/pawn/work/openssl/crypto/evp/pmeth_gn.c:189:13
    open-quantum-safe#20 0x555a6b1f2890 in test_oqs_signatures /home/pawn/work/oqs-provider/test/oqs_test_signatures.c:68:44
    open-quantum-safe#21 0x555a6b1f1e3d in main /home/pawn/work/oqs-provider/test/oqs_test_signatures.c:114:17
    open-quantum-safe#22 0x7f59a8c461c9 in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
```

## 6. Some `OPENSSL_free` were missing in `oqs_test_kems.c`

```
==569479==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 346035 byte(s) in 42 object(s) allocated from:
    #0 0x559c76ef524e in malloc (/home/pawn/work/oqs-provider/build/test/oqs_test_kems+0xb924e) (BuildId: 76a98f5859ba3c90334cef4286e89aa76436b1ae)
    open-quantum-safe#1 0x7f04407a946b in CRYPTO_malloc /home/pawn/work/openssl/crypto/mem.c:190:12
    open-quantum-safe#2 0x559c76f30389 in test_oqs_kems /home/pawn/work/oqs-provider/test/oqs_test_kems.c:44:26
    open-quantum-safe#3 0x559c76f2fdfd in main /home/pawn/work/oqs-provider/test/oqs_test_kems.c:93:17
    open-quantum-safe#4 0x7f04400461c9 in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16

Direct leak of 2690 byte(s) in 42 object(s) allocated from:
    #0 0x559c76ef524e in malloc (/home/pawn/work/oqs-provider/build/test/oqs_test_kems+0xb924e) (BuildId: 76a98f5859ba3c90334cef4286e89aa76436b1ae)
    open-quantum-safe#1 0x7f04407a946b in CRYPTO_malloc /home/pawn/work/openssl/crypto/mem.c:190:12
    open-quantum-safe#2 0x559c76f304a5 in test_oqs_kems /home/pawn/work/oqs-provider/test/oqs_test_kems.c:47:29
    open-quantum-safe#3 0x559c76f2fdfd in main /home/pawn/work/oqs-provider/test/oqs_test_kems.c:93:17
    open-quantum-safe#4 0x7f04400461c9 in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16

Direct leak of 2690 byte(s) in 42 object(s) allocated from:
    #0 0x559c76ef524e in malloc (/home/pawn/work/oqs-provider/build/test/oqs_test_kems+0xb924e) (BuildId: 76a98f5859ba3c90334cef4286e89aa76436b1ae)
    open-quantum-safe#1 0x7f04407a946b in CRYPTO_malloc /home/pawn/work/openssl/crypto/mem.c:190:12
    open-quantum-safe#2 0x559c76f303e4 in test_oqs_kems /home/pawn/work/oqs-provider/test/oqs_test_kems.c:45:29
    open-quantum-safe#3 0x559c76f2fdfd in main /home/pawn/work/oqs-provider/test/oqs_test_kems.c:93:17
    open-quantum-safe#4 0x7f04400461c9 in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
```

## 7. Missing `X509_PUBKEY_free` in `oqsx_d2i_PUBKEY`.

This was probably the hardest one for me to track down.

In `oqsx_d2i_PUBKEY`, a call is done to `oqsx_d2i_X509_PUBKEY_INTERNAL`.
However, the returned value (of type `X509_PUBKEY`) wasn't freed.

ASan:

```
Direct leak of 48 byte(s) in 1 object(s) allocated from:
    #0 0x55610d68132e in malloc (/home/pawn/work/oqs-provider/build/test/oqs_test_endecode+0xb932e) (BuildId: 53bfaa0a3b850edc7328f30be8b047ab6675ba3c)
    open-quantum-safe#1 0x7f68999a946b in CRYPTO_malloc /home/pawn/work/openssl/crypto/mem.c:190:12
    open-quantum-safe#2 0x7f68999a94a2 in CRYPTO_zalloc /home/pawn/work/openssl/crypto/mem.c:197:11
    open-quantum-safe#3 0x7f6895295897  (<unknown module>)
    open-quantum-safe#4 0x7f6895295ce4  (<unknown module>)
    open-quantum-safe#5 0x7f6895296a6c  (<unknown module>)
    open-quantum-safe#6 0x7f689986eae8 in decoder_process /home/pawn/work/openssl/crypto/encode_decode/decoder_lib.c:971:14
    open-quantum-safe#7 0x7f6899d13383 in spki2typespki_decode /home/pawn/work/openssl/providers/implementations/encode_decode/decode_spki2typespki.c:111:10
    open-quantum-safe#8 0x7f689986eae8 in decoder_process /home/pawn/work/openssl/crypto/encode_decode/decoder_lib.c:971:14
    open-quantum-safe#9 0x7f6899d114d3 in pem2der_decode /home/pawn/work/openssl/providers/implementations/encode_decode/decode_pem2der.c:204:14
    open-quantum-safe#10 0x7f689986eae8 in decoder_process /home/pawn/work/openssl/crypto/encode_decode/decoder_lib.c:971:14
    open-quantum-safe#11 0x7f689986d106 in OSSL_DECODER_from_bio /home/pawn/work/openssl/crypto/encode_decode/decoder_lib.c:82:10
    open-quantum-safe#12 0x55610d6bd407 in decode_EVP_PKEY_prov /home/pawn/work/oqs-provider/test/oqs_test_endecode.c:151:10
    open-quantum-safe#13 0x55610d6bc75b in test_oqs_encdec /home/pawn/work/oqs-provider/test/oqs_test_endecode.c:188:14
    open-quantum-safe#14 0x55610d6bbf37 in main /home/pawn/work/oqs-provider/test/oqs_test_endecode.c:240:17
    open-quantum-safe#15 0x7f68991671c9 in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16

Indirect leak of 1312 byte(s) in 1 object(s) allocated from:
    #0 0x55610d68132e in malloc (/home/pawn/work/oqs-provider/build/test/oqs_test_endecode+0xb932e) (BuildId: 53bfaa0a3b850edc7328f30be8b047ab6675ba3c)
    open-quantum-safe#1 0x7f68999a946b in CRYPTO_malloc /home/pawn/work/openssl/crypto/mem.c:190:12
    open-quantum-safe#2 0x7f68995d6b99 in ossl_c2i_ASN1_BIT_STRING /home/pawn/work/openssl/crypto/asn1/a_bitstr.c:117:13
    open-quantum-safe#3 0x7f689961f659 in asn1_ex_c2i /home/pawn/work/openssl/crypto/asn1/tasn_dec.c:885:14
    open-quantum-safe#4 0x7f689961bbc9 in asn1_d2i_ex_primitive /home/pawn/work/openssl/crypto/asn1/tasn_dec.c:818:10
    open-quantum-safe#5 0x7f6899618054 in asn1_item_embed_d2i /home/pawn/work/openssl/crypto/asn1/tasn_dec.c:217:16
    open-quantum-safe#6 0x7f689961e02f in asn1_template_noexp_d2i /home/pawn/work/openssl/crypto/asn1/tasn_dec.c:682:15
    open-quantum-safe#7 0x7f689961a7b4 in asn1_template_ex_d2i /home/pawn/work/openssl/crypto/asn1/tasn_dec.c:558:16
    open-quantum-safe#8 0x7f689961986a in asn1_item_embed_d2i /home/pawn/work/openssl/crypto/asn1/tasn_dec.c:422:19
    open-quantum-safe#9 0x7f68996175c8 in asn1_item_ex_d2i_intern /home/pawn/work/openssl/crypto/asn1/tasn_dec.c:118:10
    open-quantum-safe#10 0x7f68996177da in ASN1_item_d2i_ex /home/pawn/work/openssl/crypto/asn1/tasn_dec.c:144:9
    open-quantum-safe#11 0x7f6895295926  (<unknown module>)
    open-quantum-safe#12 0x7f6895295ce4  (<unknown module>)
    open-quantum-safe#13 0x7f6895296a6c  (<unknown module>)
    open-quantum-safe#14 0x7f689986eae8 in decoder_process /home/pawn/work/openssl/crypto/encode_decode/decoder_lib.c:971:14
    open-quantum-safe#15 0x7f6899d13383 in spki2typespki_decode /home/pawn/work/openssl/providers/implementations/encode_decode/decode_spki2typespki.c:111:10
    open-quantum-safe#16 0x7f689986eae8 in decoder_process /home/pawn/work/openssl/crypto/encode_decode/decoder_lib.c:971:14
    open-quantum-safe#17 0x7f6899d114d3 in pem2der_decode /home/pawn/work/openssl/providers/implementations/encode_decode/decode_pem2der.c:204:14
    open-quantum-safe#18 0x7f689986eae8 in decoder_process /home/pawn/work/openssl/crypto/encode_decode/decoder_lib.c:971:14
    open-quantum-safe#19 0x7f689986d106 in OSSL_DECODER_from_bio /home/pawn/work/openssl/crypto/encode_decode/decoder_lib.c:82:10
    open-quantum-safe#20 0x55610d6bd407 in decode_EVP_PKEY_prov /home/pawn/work/oqs-provider/test/oqs_test_endecode.c:151:10
    open-quantum-safe#21 0x55610d6bc75b in test_oqs_encdec /home/pawn/work/oqs-provider/test/oqs_test_endecode.c:188:14
    open-quantum-safe#22 0x55610d6bbf37 in main /home/pawn/work/oqs-provider/test/oqs_test_endecode.c:240:17
    open-quantum-safe#23 0x7f68991671c9 in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16

Indirect leak of 24 byte(s) in 1 object(s) allocated from:
    #0 0x55610d68132e in malloc (/home/pawn/work/oqs-provider/build/test/oqs_test_endecode+0xb932e) (BuildId: 53bfaa0a3b850edc7328f30be8b047ab6675ba3c)
    open-quantum-safe#1 0x7f68999a946b in CRYPTO_malloc /home/pawn/work/openssl/crypto/mem.c:190:12
    open-quantum-safe#2 0x7f68999a94a2 in CRYPTO_zalloc /home/pawn/work/openssl/crypto/mem.c:197:11
    open-quantum-safe#3 0x7f68995fde90 in ASN1_STRING_type_new /home/pawn/work/openssl/crypto/asn1/asn1_lib.c:350:11
    open-quantum-safe#4 0x7f689962a7dd in ASN1_BIT_STRING_new /home/pawn/work/openssl/crypto/asn1/tasn_typ.c:31:1
    open-quantum-safe#5 0x7f68995d6a0f in ossl_c2i_ASN1_BIT_STRING /home/pawn/work/openssl/crypto/asn1/a_bitstr.c:98:20
    open-quantum-safe#6 0x7f689961f659 in asn1_ex_c2i /home/pawn/work/openssl/crypto/asn1/tasn_dec.c:885:14
    open-quantum-safe#7 0x7f689961bbc9 in asn1_d2i_ex_primitive /home/pawn/work/openssl/crypto/asn1/tasn_dec.c:818:10
    open-quantum-safe#8 0x7f6899618054 in asn1_item_embed_d2i /home/pawn/work/openssl/crypto/asn1/tasn_dec.c:217:16
    open-quantum-safe#9 0x7f689961e02f in asn1_template_noexp_d2i /home/pawn/work/openssl/crypto/asn1/tasn_dec.c:682:15
    open-quantum-safe#10 0x7f689961a7b4 in asn1_template_ex_d2i /home/pawn/work/openssl/crypto/asn1/tasn_dec.c:558:16
    open-quantum-safe#11 0x7f689961986a in asn1_item_embed_d2i /home/pawn/work/openssl/crypto/asn1/tasn_dec.c:422:19
    open-quantum-safe#12 0x7f68996175c8 in asn1_item_ex_d2i_intern /home/pawn/work/openssl/crypto/asn1/tasn_dec.c:118:10
    open-quantum-safe#13 0x7f68996177da in ASN1_item_d2i_ex /home/pawn/work/openssl/crypto/asn1/tasn_dec.c:144:9
    open-quantum-safe#14 0x7f6895295926  (<unknown module>)
    open-quantum-safe#15 0x7f6895295ce4  (<unknown module>)
    open-quantum-safe#16 0x7f6895296a6c  (<unknown module>)
    open-quantum-safe#17 0x7f689986eae8 in decoder_process /home/pawn/work/openssl/crypto/encode_decode/decoder_lib.c:971:14
    open-quantum-safe#18 0x7f6899d13383 in spki2typespki_decode /home/pawn/work/openssl/providers/implementations/encode_decode/decode_spki2typespki.c:111:10
    open-quantum-safe#19 0x7f689986eae8 in decoder_process /home/pawn/work/openssl/crypto/encode_decode/decoder_lib.c:971:14
    open-quantum-safe#20 0x7f6899d114d3 in pem2der_decode /home/pawn/work/openssl/providers/implementations/encode_decode/decode_pem2der.c:204:14
    open-quantum-safe#21 0x7f689986eae8 in decoder_process /home/pawn/work/openssl/crypto/encode_decode/decoder_lib.c:971:14
    open-quantum-safe#22 0x7f689986d106 in OSSL_DECODER_from_bio /home/pawn/work/openssl/crypto/encode_decode/decoder_lib.c:82:10
    open-quantum-safe#23 0x55610d6bd407 in decode_EVP_PKEY_prov /home/pawn/work/oqs-provider/test/oqs_test_endecode.c:151:10
    open-quantum-safe#24 0x55610d6bc75b in test_oqs_encdec /home/pawn/work/oqs-provider/test/oqs_test_endecode.c:188:14
    open-quantum-safe#25 0x55610d6bbf37 in main /home/pawn/work/oqs-provider/test/oqs_test_endecode.c:240:17
    open-quantum-safe#26 0x7f68991671c9 in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16

Indirect leak of 16 byte(s) in 1 object(s) allocated from:
    #0 0x55610d68132e in malloc (/home/pawn/work/oqs-provider/build/test/oqs_test_endecode+0xb932e) (BuildId: 53bfaa0a3b850edc7328f30be8b047ab6675ba3c)
    open-quantum-safe#1 0x7f68999a946b in CRYPTO_malloc /home/pawn/work/openssl/crypto/mem.c:190:12
    open-quantum-safe#2 0x7f68999a94a2 in CRYPTO_zalloc /home/pawn/work/openssl/crypto/mem.c:197:11
    open-quantum-safe#3 0x7f6899625ed4 in asn1_item_embed_new /home/pawn/work/openssl/crypto/asn1/tasn_new.c:136:21
    open-quantum-safe#4 0x7f689962620e in ossl_asn1_item_ex_new_intern /home/pawn/work/openssl/crypto/asn1/tasn_new.c:52:12
    open-quantum-safe#5 0x7f68996191a5 in asn1_item_embed_d2i /home/pawn/work/openssl/crypto/asn1/tasn_dec.c:366:21
    open-quantum-safe#6 0x7f689961e02f in asn1_template_noexp_d2i /home/pawn/work/openssl/crypto/asn1/tasn_dec.c:682:15
    open-quantum-safe#7 0x7f689961a7b4 in asn1_template_ex_d2i /home/pawn/work/openssl/crypto/asn1/tasn_dec.c:558:16
    open-quantum-safe#8 0x7f689961986a in asn1_item_embed_d2i /home/pawn/work/openssl/crypto/asn1/tasn_dec.c:422:19
    open-quantum-safe#9 0x7f68996175c8 in asn1_item_ex_d2i_intern /home/pawn/work/openssl/crypto/asn1/tasn_dec.c:118:10
    open-quantum-safe#10 0x7f68996177da in ASN1_item_d2i_ex /home/pawn/work/openssl/crypto/asn1/tasn_dec.c:144:9
    open-quantum-safe#11 0x7f6895295926  (<unknown module>)
    open-quantum-safe#12 0x7f6895295ce4  (<unknown module>)
    open-quantum-safe#13 0x7f6895296a6c  (<unknown module>)
    open-quantum-safe#14 0x7f689986eae8 in decoder_process /home/pawn/work/openssl/crypto/encode_decode/decoder_lib.c:971:14
    open-quantum-safe#15 0x7f6899d13383 in spki2typespki_decode /home/pawn/work/openssl/providers/implementations/encode_decode/decode_spki2typespki.c:111:10
    open-quantum-safe#16 0x7f689986eae8 in decoder_process /home/pawn/work/openssl/crypto/encode_decode/decoder_lib.c:971:14
    open-quantum-safe#17 0x7f6899d114d3 in pem2der_decode /home/pawn/work/openssl/providers/implementations/encode_decode/decode_pem2der.c:204:14
    open-quantum-safe#18 0x7f689986eae8 in decoder_process /home/pawn/work/openssl/crypto/encode_decode/decoder_lib.c:971:14
    open-quantum-safe#19 0x7f689986d106 in OSSL_DECODER_from_bio /home/pawn/work/openssl/crypto/encode_decode/decoder_lib.c:82:10
    open-quantum-safe#20 0x55610d6bd407 in decode_EVP_PKEY_prov /home/pawn/work/oqs-provider/test/oqs_test_endecode.c:151:10
    open-quantum-safe#21 0x55610d6bc75b in test_oqs_encdec /home/pawn/work/oqs-provider/test/oqs_test_endecode.c:188:14
    open-quantum-safe#22 0x55610d6bbf37 in main /home/pawn/work/oqs-provider/test/oqs_test_endecode.c:240:17
    open-quantum-safe#23 0x7f68991671c9 in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16

SUMMARY: AddressSanitizer: 1400 byte(s) leaked in 4 allocation(s).
```


## 8. Bonus point: fix a potential double-free.

In `oqsx_key_gen`, when the keytype isn't `KEY_TYPE_HYB_SIG`, the
generated `pkey` is freed, and the flow goes down to the `err` label:

https://github.com/open-quantum-safe/oqs-provider/blob/cc3895f208730abfe1efc15f7c3ff4358ac6e4c0/oqsprov/oqsprov_keys.c#L1089-L1104

There is a potential double-free bug here: if `oqsx_key_gen_oqs` fails, then
`EVP_PKEY_free` will be called twice on the same `pkey` pointer.

Note that it is very unlikely that `oqsx_key_gen_oqs` fails, since it only
calls `OQS_KEM_keypair`/`OQS_SIG_keypair`.
thb-sb added a commit to thb-sb/oqs-provider that referenced this pull request Sep 7, 2023
@thb-sb
Fix various memory leaks.
41a8c1d 
This commit fixes various memory leaks that have been found using ASan.

Note: I will soon add a CI job for testing oqs-provider against memory leaks
with ASan.


# 1. Memory leak in `oqs_test_signatures`.

In `oqs_test_signatures`, [`OPENSSL_free`] was used with `EVP_PKEY_CTX`. However,
the right destructor function for `EVP_PKEY_CTX` is [`EVP_PKEY_CTX_free`].

This commmit fixes this minor bug.

[`OPENSSL_free`](https://www.openssl.org/docs/man3.0/man3/OPENSSL_free.html)
[`EVP_PKEY_CTX_free`](https://www.openssl.org/docs/man3.0/man3/EVP_PKEY_CTX_free.html)


## 2. a selftest key that wasn't freed.

In `oqsprov/oqsprov_keys.c`, `d2i_PrivateKey` is used to verify that we the
key we've just generated is sound.
However, the returned object wasn't freed.
ASan trace that helped me finding this one:

```
Indirect leak of 96 byte(s) in 4 object(s) allocated from:
    #0 0x5651eef1428e in malloc (/home/pawn/work/oqs-provider/build/test/oqs_test_signatures+0xb928e) (BuildId: 6cc7ebd4bc73a0c78b2ab8d7b9dcb8b165d88e74)
    open-quantum-safe#1 0x7fb7a23a946b in CRYPTO_malloc /home/pawn/work/openssl/crypto/mem.c:190:12
    open-quantum-safe#2 0x7fb7a23a94a2 in CRYPTO_zalloc /home/pawn/work/openssl/crypto/mem.c:197:11
    open-quantum-safe#3 0x7fb7a209cdcd in BN_new /home/pawn/work/openssl/crypto/bn/bn_lib.c:247:16
    open-quantum-safe#4 0x7fb7a2224a00 in ossl_ec_GFp_mont_group_set_curve /home/pawn/work/openssl/crypto/ec/ecp_mont.c:169:11
    open-quantum-safe#5 0x7fb7a220f5fa in EC_GROUP_set_curve /home/pawn/work/openssl/crypto/ec/ec_lib.c:562:12
    open-quantum-safe#6 0x7fb7a2202d75 in EC_GROUP_new_curve_GFp /home/pawn/work/openssl/crypto/ec/ec_cvt.c:61:10
    open-quantum-safe#7 0x7fb7a220184d in ec_group_new_from_data /home/pawn/work/openssl/crypto/ec/ec_curve.c:3179:22
    open-quantum-safe#8 0x7fb7a2200ff9 in EC_GROUP_new_by_curve_name_ex /home/pawn/work/openssl/crypto/ec/ec_curve.c:3287:19
    open-quantum-safe#9 0x7fb7a2201ed9 in EC_GROUP_new_by_curve_name /home/pawn/work/openssl/crypto/ec/ec_curve.c:3303:12
    open-quantum-safe#10 0x7fb7a21f84e2 in EC_GROUP_new_from_ecpkparameters /home/pawn/work/openssl/crypto/ec/ec_asn1.c:863:20
    open-quantum-safe#11 0x7fb7a21f8f28 in d2i_ECPrivateKey /home/pawn/work/openssl/crypto/ec/ec_asn1.c:956:22
    open-quantum-safe#12 0x7fb7a270b4b5 in der2key_decode /home/pawn/work/openssl/providers/implementations/encode_decode/decode_der2key.c:220:19
    open-quantum-safe#13 0x7fb7a226eae8 in decoder_process /home/pawn/work/openssl/crypto/encode_decode/decoder_lib.c:971:14
    open-quantum-safe#14 0x7fb7a226d106 in OSSL_DECODER_from_bio /home/pawn/work/openssl/crypto/encode_decode/decoder_lib.c:82:10
    open-quantum-safe#15 0x7fb7a226f007 in OSSL_DECODER_from_data /home/pawn/work/openssl/crypto/encode_decode/decoder_lib.c:157:9
    open-quantum-safe#16 0x7fb7a200e123 in d2i_PrivateKey_decoder /home/pawn/work/openssl/crypto/asn1/d2i_pr.c:74:11
    open-quantum-safe#17 0x7fb7a200dbba in d2i_PrivateKey_ex /home/pawn/work/openssl/crypto/asn1/d2i_pr.c:162:11
    open-quantum-safe#18 0x7fb7a200e3d2 in d2i_PrivateKey /home/pawn/work/openssl/crypto/asn1/d2i_pr.c:172:12
    open-quantum-safe#19 0x7fb79dc6a22e in oqsx_key_gen_evp_key /home/pawn/work/oqs-provider/oqsprov/oqsprov_keys.c:1036:25
    open-quantum-safe#20 0x7fb79dc68f3f in oqsx_key_gen /home/pawn/work/oqs-provider/oqsprov/oqsprov_keys.c:1077:16
    open-quantum-safe#21 0x7fb79dc7164a in oqsx_genkey /home/pawn/work/oqs-provider/oqsprov/oqs_kmgmt.c:496:9
    open-quantum-safe#22 0x7fb79dc6d9f8 in oqsx_gen /home/pawn/work/oqs-provider/oqsprov/oqs_kmgmt.c:509:12
    open-quantum-safe#23 0x7fb7a23373c7 in evp_keymgmt_gen /home/pawn/work/openssl/crypto/evp/keymgmt_meth.c:366:12
    open-quantum-safe#24 0x7fb7a2334b58 in evp_keymgmt_util_gen /home/pawn/work/openssl/crypto/evp/keymgmt_lib.c:517:20
    open-quantum-safe#25 0x7fb7a235a775 in EVP_PKEY_generate /home/pawn/work/openssl/crypto/evp/pmeth_gn.c:189:13
    open-quantum-safe#26 0x5651eef4f2bd in test_oqs_signatures /home/pawn/work/oqs-provider/test/oqs_test_signatures.c:39:48
    open-quantum-safe#27 0x5651eef4ee3d in main /home/pawn/work/oqs-provider/test/oqs_test_signatures.c:114:17
    open-quantum-safe#28 0x7fb7a1b671c9 in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
```

## 3. a `EVP_PKEY_CTX` that wasn't freed.

In `oqsx_key_free`, `key->oqsx_provider_ctx.oqsx_evp_ctx->ctx` is freed if
the keytype is `ECP_HYB_KEM` or `ECX_HYB_KEM`. However,
`key->oqsx_provider_ctx.oqsx_evp_ctx->ctx` is also used for signatures.
In that case, a call to `EVP_PKEY_CTX_free` was missing.
ASan trace that helped me finding this one:

```
Indirect leak of 7 byte(s) in 2 object(s) allocated from:
    #0 0x55b6fc8d427e in malloc (/home/pawn/work/oqs-provider/build/test/oqs_test_signatures+0xb927e) (BuildId: 3a8bff15d3315ba7ac1746fdc7b67eb464c607a1)
    open-quantum-safe#1 0x7ff34dfa946b in CRYPTO_malloc /home/pawn/work/openssl/crypto/mem.c:190:12
    open-quantum-safe#2 0x7ff34dfac50a in CRYPTO_strndup /home/pawn/work/openssl/crypto/o_str.c:43:11
    open-quantum-safe#3 0x7ff34df9d152 in ossl_algorithm_get1_first_name /home/pawn/work/openssl/crypto/core_algorithm.c:195:11
    open-quantum-safe#4 0x7ff34df353a1 in keymgmt_from_algorithm /home/pawn/work/openssl/crypto/evp/keymgmt_meth.c:50:31
    open-quantum-safe#5 0x7ff34df15e44 in construct_evp_method /home/pawn/work/openssl/crypto/evp/evp_fetch.c:219:14
    open-quantum-safe#6 0x7ff34df9e3ce in ossl_method_construct_this /home/pawn/work/openssl/crypto/core_fetch.c:109:19
    open-quantum-safe#7 0x7ff34df9d740 in algorithm_do_map /home/pawn/work/openssl/crypto/core_algorithm.c:77:13
    open-quantum-safe#8 0x7ff34df9ce88 in algorithm_do_this /home/pawn/work/openssl/crypto/core_algorithm.c:122:15
    open-quantum-safe#9 0x7ff34dfcce3e in ossl_provider_doall_activated /home/pawn/work/openssl/crypto/provider_core.c:1423:14
    open-quantum-safe#10 0x7ff34df9ca64 in ossl_algorithm_do_all /home/pawn/work/openssl/crypto/core_algorithm.c:162:9
    open-quantum-safe#11 0x7ff34df9dd64 in ossl_method_construct /home/pawn/work/openssl/crypto/core_fetch.c:153:5
    open-quantum-safe#12 0x7ff34df13b6f in inner_evp_generic_fetch /home/pawn/work/openssl/crypto/evp/evp_fetch.c:312:23
    open-quantum-safe#13 0x7ff34df13226 in evp_generic_fetch /home/pawn/work/openssl/crypto/evp/evp_fetch.c:364:14
    open-quantum-safe#14 0x7ff34df36a42 in EVP_KEYMGMT_fetch /home/pawn/work/openssl/crypto/evp/keymgmt_meth.c:220:12
    open-quantum-safe#15 0x7ff34df5d6e4 in int_ctx_new /home/pawn/work/openssl/crypto/evp/pmeth_lib.c:280:23
    open-quantum-safe#16 0x7ff34df5f0a9 in EVP_PKEY_CTX_new_id /home/pawn/work/openssl/crypto/evp/pmeth_lib.c:469:12
    open-quantum-safe#17 0x7ff349865bcb in oqsx_hybsig_init /home/pawn/work/oqs-provider/oqsprov/oqsprov_keys.c:578:20
    open-quantum-safe#18 0x7ff349865157 in oqsx_key_new /home/pawn/work/oqs-provider/oqsprov/oqsprov_keys.c:791:16
    open-quantum-safe#19 0x7ff3498715a3 in oqsx_genkey /home/pawn/work/oqs-provider/oqsprov/oqs_kmgmt.c:487:16
    open-quantum-safe#20 0x7ff34986da08 in oqsx_gen /home/pawn/work/oqs-provider/oqsprov/oqs_kmgmt.c:509:12
    open-quantum-safe#21 0x7ff34df373c7 in evp_keymgmt_gen /home/pawn/work/openssl/crypto/evp/keymgmt_meth.c:366:12
    open-quantum-safe#22 0x7ff34df34b58 in evp_keymgmt_util_gen /home/pawn/work/openssl/crypto/evp/keymgmt_lib.c:517:20
    open-quantum-safe#23 0x7ff34df5a775 in EVP_PKEY_generate /home/pawn/work/openssl/crypto/evp/pmeth_gn.c:189:13
    open-quantum-safe#24 0x55b6fc90f2ad in test_oqs_signatures /home/pawn/work/oqs-provider/test/oqs_test_signatures.c:39:48
    open-quantum-safe#25 0x55b6fc90ee2d in main /home/pawn/work/oqs-provider/test/oqs_test_signatures.c:114:17
    open-quantum-safe#26 0x7ff34d8461c9 in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
```

## 4. Missing `EVP_PKEY_CTX_free` in case of error.

In `oqsx_hybsig_init`, a `EVP_PKEY_CTX` is initialized using `EVP_PKEY_CTX_new_id`.
However, later in the code, some extra work is done, and in case of error, the
`EVP_PKEY_CTX` wasn't freed.

(I couldn't recover the ASan message for this one…)


## 5. `OPENSSL_free` instead of `EVP_PKEY_free`.

In `oqsx_key_gen`, when a classical private key is generated, the field
`classical_pkey` of a `OQSX_KEY` is set with a value of type `EVP_PKEY`.

However, in `oqsx_key_free`, `OPENSSL_free` was used on `classical_pkey`
instead of `EVP_PKEY_free`.

ASan trace that helped me finding this one:

```
   #0 0x555a6b1b728e in malloc (/home/pawn/work/oqs-provider/build/test/oqs_test_signatures+0xb928e) (BuildId: 6cc7ebd4bc73a0c78b2ab8d7b9dcb8b165d88e74)
    open-quantum-safe#1 0x7f59a93a946b in CRYPTO_malloc /home/pawn/work/openssl/crypto/mem.c:190:12
    open-quantum-safe#2 0x7f59a93a94a2 in CRYPTO_zalloc /home/pawn/work/openssl/crypto/mem.c:197:11
    open-quantum-safe#3 0x7f59a909cdcd in BN_new /home/pawn/work/openssl/crypto/bn/bn_lib.c:247:16
    open-quantum-safe#4 0x7f59a920b955 in ossl_ec_group_new_ex /home/pawn/work/openssl/crypto/ec/ec_lib.c:61:25
    open-quantum-safe#5 0x7f59a920dd55 in EC_GROUP_dup /home/pawn/work/openssl/crypto/ec/ec_lib.c:273:14
    open-quantum-safe#6 0x7f59a9207774 in EC_KEY_set_group /home/pawn/work/openssl/crypto/ec/ec_key.c:733:18
    open-quantum-safe#7 0x7f59a975b3b5 in ec_gen_assign_group /home/pawn/work/openssl/providers/implementations/keymgmt/ec_kmgmt.c:1236:12
    open-quantum-safe#8 0x7f59a9758bdb in ec_gen /home/pawn/work/openssl/providers/implementations/keymgmt/ec_kmgmt.c:1274:11
    open-quantum-safe#9 0x7f59a93373c7 in evp_keymgmt_gen /home/pawn/work/openssl/crypto/evp/keymgmt_meth.c:366:12
    open-quantum-safe#10 0x7f59a9334b58 in evp_keymgmt_util_gen /home/pawn/work/openssl/crypto/evp/keymgmt_lib.c:517:20
    open-quantum-safe#11 0x7f59a935a775 in EVP_PKEY_generate /home/pawn/work/openssl/crypto/evp/pmeth_gn.c:189:13
    open-quantum-safe#12 0x7f59a935b404 in EVP_PKEY_keygen /home/pawn/work/openssl/crypto/evp/pmeth_gn.c:274:12
    open-quantum-safe#13 0x7f59a4c69a51 in oqsx_key_gen_evp_key /home/pawn/work/oqs-provider/oqsprov/oqsprov_keys.c:1012:12
    open-quantum-safe#14 0x7f59a4c68faf in oqsx_key_gen /home/pawn/work/oqs-provider/oqsprov/oqsprov_keys.c:1084:16
    open-quantum-safe#15 0x7f59a4c716ca in oqsx_genkey /home/pawn/work/oqs-provider/oqsprov/oqs_kmgmt.c:496:9
    open-quantum-safe#16 0x7f59a4c6da78 in oqsx_gen /home/pawn/work/oqs-provider/oqsprov/oqs_kmgmt.c:509:12
    open-quantum-safe#17 0x7f59a93373c7 in evp_keymgmt_gen /home/pawn/work/openssl/crypto/evp/keymgmt_meth.c:366:12
    open-quantum-safe#18 0x7f59a9334b58 in evp_keymgmt_util_gen /home/pawn/work/openssl/crypto/evp/keymgmt_lib.c:517:20
    open-quantum-safe#19 0x7f59a935a775 in EVP_PKEY_generate /home/pawn/work/openssl/crypto/evp/pmeth_gn.c:189:13
    open-quantum-safe#20 0x555a6b1f2890 in test_oqs_signatures /home/pawn/work/oqs-provider/test/oqs_test_signatures.c:68:44
    open-quantum-safe#21 0x555a6b1f1e3d in main /home/pawn/work/oqs-provider/test/oqs_test_signatures.c:114:17
    open-quantum-safe#22 0x7f59a8c461c9 in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
```

## 6. Some `OPENSSL_free` were missing in `oqs_test_kems.c`

```
==569479==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 346035 byte(s) in 42 object(s) allocated from:
    #0 0x559c76ef524e in malloc (/home/pawn/work/oqs-provider/build/test/oqs_test_kems+0xb924e) (BuildId: 76a98f5859ba3c90334cef4286e89aa76436b1ae)
    open-quantum-safe#1 0x7f04407a946b in CRYPTO_malloc /home/pawn/work/openssl/crypto/mem.c:190:12
    open-quantum-safe#2 0x559c76f30389 in test_oqs_kems /home/pawn/work/oqs-provider/test/oqs_test_kems.c:44:26
    open-quantum-safe#3 0x559c76f2fdfd in main /home/pawn/work/oqs-provider/test/oqs_test_kems.c:93:17
    open-quantum-safe#4 0x7f04400461c9 in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16

Direct leak of 2690 byte(s) in 42 object(s) allocated from:
    #0 0x559c76ef524e in malloc (/home/pawn/work/oqs-provider/build/test/oqs_test_kems+0xb924e) (BuildId: 76a98f5859ba3c90334cef4286e89aa76436b1ae)
    open-quantum-safe#1 0x7f04407a946b in CRYPTO_malloc /home/pawn/work/openssl/crypto/mem.c:190:12
    open-quantum-safe#2 0x559c76f304a5 in test_oqs_kems /home/pawn/work/oqs-provider/test/oqs_test_kems.c:47:29
    open-quantum-safe#3 0x559c76f2fdfd in main /home/pawn/work/oqs-provider/test/oqs_test_kems.c:93:17
    open-quantum-safe#4 0x7f04400461c9 in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16

Direct leak of 2690 byte(s) in 42 object(s) allocated from:
    #0 0x559c76ef524e in malloc (/home/pawn/work/oqs-provider/build/test/oqs_test_kems+0xb924e) (BuildId: 76a98f5859ba3c90334cef4286e89aa76436b1ae)
    open-quantum-safe#1 0x7f04407a946b in CRYPTO_malloc /home/pawn/work/openssl/crypto/mem.c:190:12
    open-quantum-safe#2 0x559c76f303e4 in test_oqs_kems /home/pawn/work/oqs-provider/test/oqs_test_kems.c:45:29
    open-quantum-safe#3 0x559c76f2fdfd in main /home/pawn/work/oqs-provider/test/oqs_test_kems.c:93:17
    open-quantum-safe#4 0x7f04400461c9 in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
```

## 7. Missing `X509_PUBKEY_free` in `oqsx_d2i_PUBKEY`.

This was probably the hardest one for me to track down.

In `oqsx_d2i_PUBKEY`, a call is done to `oqsx_d2i_X509_PUBKEY_INTERNAL`.
However, the returned value (of type `X509_PUBKEY`) wasn't freed.

ASan:

```
Direct leak of 48 byte(s) in 1 object(s) allocated from:
    #0 0x55610d68132e in malloc (/home/pawn/work/oqs-provider/build/test/oqs_test_endecode+0xb932e) (BuildId: 53bfaa0a3b850edc7328f30be8b047ab6675ba3c)
    open-quantum-safe#1 0x7f68999a946b in CRYPTO_malloc /home/pawn/work/openssl/crypto/mem.c:190:12
    open-quantum-safe#2 0x7f68999a94a2 in CRYPTO_zalloc /home/pawn/work/openssl/crypto/mem.c:197:11
    open-quantum-safe#3 0x7f6895295897  (<unknown module>)
    open-quantum-safe#4 0x7f6895295ce4  (<unknown module>)
    open-quantum-safe#5 0x7f6895296a6c  (<unknown module>)
    open-quantum-safe#6 0x7f689986eae8 in decoder_process /home/pawn/work/openssl/crypto/encode_decode/decoder_lib.c:971:14
    open-quantum-safe#7 0x7f6899d13383 in spki2typespki_decode /home/pawn/work/openssl/providers/implementations/encode_decode/decode_spki2typespki.c:111:10
    open-quantum-safe#8 0x7f689986eae8 in decoder_process /home/pawn/work/openssl/crypto/encode_decode/decoder_lib.c:971:14
    open-quantum-safe#9 0x7f6899d114d3 in pem2der_decode /home/pawn/work/openssl/providers/implementations/encode_decode/decode_pem2der.c:204:14
    open-quantum-safe#10 0x7f689986eae8 in decoder_process /home/pawn/work/openssl/crypto/encode_decode/decoder_lib.c:971:14
    open-quantum-safe#11 0x7f689986d106 in OSSL_DECODER_from_bio /home/pawn/work/openssl/crypto/encode_decode/decoder_lib.c:82:10
    open-quantum-safe#12 0x55610d6bd407 in decode_EVP_PKEY_prov /home/pawn/work/oqs-provider/test/oqs_test_endecode.c:151:10
    open-quantum-safe#13 0x55610d6bc75b in test_oqs_encdec /home/pawn/work/oqs-provider/test/oqs_test_endecode.c:188:14
    open-quantum-safe#14 0x55610d6bbf37 in main /home/pawn/work/oqs-provider/test/oqs_test_endecode.c:240:17
    open-quantum-safe#15 0x7f68991671c9 in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16

Indirect leak of 1312 byte(s) in 1 object(s) allocated from:
    #0 0x55610d68132e in malloc (/home/pawn/work/oqs-provider/build/test/oqs_test_endecode+0xb932e) (BuildId: 53bfaa0a3b850edc7328f30be8b047ab6675ba3c)
    open-quantum-safe#1 0x7f68999a946b in CRYPTO_malloc /home/pawn/work/openssl/crypto/mem.c:190:12
    open-quantum-safe#2 0x7f68995d6b99 in ossl_c2i_ASN1_BIT_STRING /home/pawn/work/openssl/crypto/asn1/a_bitstr.c:117:13
    open-quantum-safe#3 0x7f689961f659 in asn1_ex_c2i /home/pawn/work/openssl/crypto/asn1/tasn_dec.c:885:14
    open-quantum-safe#4 0x7f689961bbc9 in asn1_d2i_ex_primitive /home/pawn/work/openssl/crypto/asn1/tasn_dec.c:818:10
    open-quantum-safe#5 0x7f6899618054 in asn1_item_embed_d2i /home/pawn/work/openssl/crypto/asn1/tasn_dec.c:217:16
    open-quantum-safe#6 0x7f689961e02f in asn1_template_noexp_d2i /home/pawn/work/openssl/crypto/asn1/tasn_dec.c:682:15
    open-quantum-safe#7 0x7f689961a7b4 in asn1_template_ex_d2i /home/pawn/work/openssl/crypto/asn1/tasn_dec.c:558:16
    open-quantum-safe#8 0x7f689961986a in asn1_item_embed_d2i /home/pawn/work/openssl/crypto/asn1/tasn_dec.c:422:19
    open-quantum-safe#9 0x7f68996175c8 in asn1_item_ex_d2i_intern /home/pawn/work/openssl/crypto/asn1/tasn_dec.c:118:10
    open-quantum-safe#10 0x7f68996177da in ASN1_item_d2i_ex /home/pawn/work/openssl/crypto/asn1/tasn_dec.c:144:9
    open-quantum-safe#11 0x7f6895295926  (<unknown module>)
    open-quantum-safe#12 0x7f6895295ce4  (<unknown module>)
    open-quantum-safe#13 0x7f6895296a6c  (<unknown module>)
    open-quantum-safe#14 0x7f689986eae8 in decoder_process /home/pawn/work/openssl/crypto/encode_decode/decoder_lib.c:971:14
    open-quantum-safe#15 0x7f6899d13383 in spki2typespki_decode /home/pawn/work/openssl/providers/implementations/encode_decode/decode_spki2typespki.c:111:10
    open-quantum-safe#16 0x7f689986eae8 in decoder_process /home/pawn/work/openssl/crypto/encode_decode/decoder_lib.c:971:14
    open-quantum-safe#17 0x7f6899d114d3 in pem2der_decode /home/pawn/work/openssl/providers/implementations/encode_decode/decode_pem2der.c:204:14
    open-quantum-safe#18 0x7f689986eae8 in decoder_process /home/pawn/work/openssl/crypto/encode_decode/decoder_lib.c:971:14
    open-quantum-safe#19 0x7f689986d106 in OSSL_DECODER_from_bio /home/pawn/work/openssl/crypto/encode_decode/decoder_lib.c:82:10
    open-quantum-safe#20 0x55610d6bd407 in decode_EVP_PKEY_prov /home/pawn/work/oqs-provider/test/oqs_test_endecode.c:151:10
    open-quantum-safe#21 0x55610d6bc75b in test_oqs_encdec /home/pawn/work/oqs-provider/test/oqs_test_endecode.c:188:14
    open-quantum-safe#22 0x55610d6bbf37 in main /home/pawn/work/oqs-provider/test/oqs_test_endecode.c:240:17
    open-quantum-safe#23 0x7f68991671c9 in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16

Indirect leak of 24 byte(s) in 1 object(s) allocated from:
    #0 0x55610d68132e in malloc (/home/pawn/work/oqs-provider/build/test/oqs_test_endecode+0xb932e) (BuildId: 53bfaa0a3b850edc7328f30be8b047ab6675ba3c)
    open-quantum-safe#1 0x7f68999a946b in CRYPTO_malloc /home/pawn/work/openssl/crypto/mem.c:190:12
    open-quantum-safe#2 0x7f68999a94a2 in CRYPTO_zalloc /home/pawn/work/openssl/crypto/mem.c:197:11
    open-quantum-safe#3 0x7f68995fde90 in ASN1_STRING_type_new /home/pawn/work/openssl/crypto/asn1/asn1_lib.c:350:11
    open-quantum-safe#4 0x7f689962a7dd in ASN1_BIT_STRING_new /home/pawn/work/openssl/crypto/asn1/tasn_typ.c:31:1
    open-quantum-safe#5 0x7f68995d6a0f in ossl_c2i_ASN1_BIT_STRING /home/pawn/work/openssl/crypto/asn1/a_bitstr.c:98:20
    open-quantum-safe#6 0x7f689961f659 in asn1_ex_c2i /home/pawn/work/openssl/crypto/asn1/tasn_dec.c:885:14
    open-quantum-safe#7 0x7f689961bbc9 in asn1_d2i_ex_primitive /home/pawn/work/openssl/crypto/asn1/tasn_dec.c:818:10
    open-quantum-safe#8 0x7f6899618054 in asn1_item_embed_d2i /home/pawn/work/openssl/crypto/asn1/tasn_dec.c:217:16
    open-quantum-safe#9 0x7f689961e02f in asn1_template_noexp_d2i /home/pawn/work/openssl/crypto/asn1/tasn_dec.c:682:15
    open-quantum-safe#10 0x7f689961a7b4 in asn1_template_ex_d2i /home/pawn/work/openssl/crypto/asn1/tasn_dec.c:558:16
    open-quantum-safe#11 0x7f689961986a in asn1_item_embed_d2i /home/pawn/work/openssl/crypto/asn1/tasn_dec.c:422:19
    open-quantum-safe#12 0x7f68996175c8 in asn1_item_ex_d2i_intern /home/pawn/work/openssl/crypto/asn1/tasn_dec.c:118:10
    open-quantum-safe#13 0x7f68996177da in ASN1_item_d2i_ex /home/pawn/work/openssl/crypto/asn1/tasn_dec.c:144:9
    open-quantum-safe#14 0x7f6895295926  (<unknown module>)
    open-quantum-safe#15 0x7f6895295ce4  (<unknown module>)
    open-quantum-safe#16 0x7f6895296a6c  (<unknown module>)
    open-quantum-safe#17 0x7f689986eae8 in decoder_process /home/pawn/work/openssl/crypto/encode_decode/decoder_lib.c:971:14
    open-quantum-safe#18 0x7f6899d13383 in spki2typespki_decode /home/pawn/work/openssl/providers/implementations/encode_decode/decode_spki2typespki.c:111:10
    open-quantum-safe#19 0x7f689986eae8 in decoder_process /home/pawn/work/openssl/crypto/encode_decode/decoder_lib.c:971:14
    open-quantum-safe#20 0x7f6899d114d3 in pem2der_decode /home/pawn/work/openssl/providers/implementations/encode_decode/decode_pem2der.c:204:14
    open-quantum-safe#21 0x7f689986eae8 in decoder_process /home/pawn/work/openssl/crypto/encode_decode/decoder_lib.c:971:14
    open-quantum-safe#22 0x7f689986d106 in OSSL_DECODER_from_bio /home/pawn/work/openssl/crypto/encode_decode/decoder_lib.c:82:10
    open-quantum-safe#23 0x55610d6bd407 in decode_EVP_PKEY_prov /home/pawn/work/oqs-provider/test/oqs_test_endecode.c:151:10
    open-quantum-safe#24 0x55610d6bc75b in test_oqs_encdec /home/pawn/work/oqs-provider/test/oqs_test_endecode.c:188:14
    open-quantum-safe#25 0x55610d6bbf37 in main /home/pawn/work/oqs-provider/test/oqs_test_endecode.c:240:17
    open-quantum-safe#26 0x7f68991671c9 in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16

Indirect leak of 16 byte(s) in 1 object(s) allocated from:
    #0 0x55610d68132e in malloc (/home/pawn/work/oqs-provider/build/test/oqs_test_endecode+0xb932e) (BuildId: 53bfaa0a3b850edc7328f30be8b047ab6675ba3c)
    open-quantum-safe#1 0x7f68999a946b in CRYPTO_malloc /home/pawn/work/openssl/crypto/mem.c:190:12
    open-quantum-safe#2 0x7f68999a94a2 in CRYPTO_zalloc /home/pawn/work/openssl/crypto/mem.c:197:11
    open-quantum-safe#3 0x7f6899625ed4 in asn1_item_embed_new /home/pawn/work/openssl/crypto/asn1/tasn_new.c:136:21
    open-quantum-safe#4 0x7f689962620e in ossl_asn1_item_ex_new_intern /home/pawn/work/openssl/crypto/asn1/tasn_new.c:52:12
    open-quantum-safe#5 0x7f68996191a5 in asn1_item_embed_d2i /home/pawn/work/openssl/crypto/asn1/tasn_dec.c:366:21
    open-quantum-safe#6 0x7f689961e02f in asn1_template_noexp_d2i /home/pawn/work/openssl/crypto/asn1/tasn_dec.c:682:15
    open-quantum-safe#7 0x7f689961a7b4 in asn1_template_ex_d2i /home/pawn/work/openssl/crypto/asn1/tasn_dec.c:558:16
    open-quantum-safe#8 0x7f689961986a in asn1_item_embed_d2i /home/pawn/work/openssl/crypto/asn1/tasn_dec.c:422:19
    open-quantum-safe#9 0x7f68996175c8 in asn1_item_ex_d2i_intern /home/pawn/work/openssl/crypto/asn1/tasn_dec.c:118:10
    open-quantum-safe#10 0x7f68996177da in ASN1_item_d2i_ex /home/pawn/work/openssl/crypto/asn1/tasn_dec.c:144:9
    open-quantum-safe#11 0x7f6895295926  (<unknown module>)
    open-quantum-safe#12 0x7f6895295ce4  (<unknown module>)
    open-quantum-safe#13 0x7f6895296a6c  (<unknown module>)
    open-quantum-safe#14 0x7f689986eae8 in decoder_process /home/pawn/work/openssl/crypto/encode_decode/decoder_lib.c:971:14
    open-quantum-safe#15 0x7f6899d13383 in spki2typespki_decode /home/pawn/work/openssl/providers/implementations/encode_decode/decode_spki2typespki.c:111:10
    open-quantum-safe#16 0x7f689986eae8 in decoder_process /home/pawn/work/openssl/crypto/encode_decode/decoder_lib.c:971:14
    open-quantum-safe#17 0x7f6899d114d3 in pem2der_decode /home/pawn/work/openssl/providers/implementations/encode_decode/decode_pem2der.c:204:14
    open-quantum-safe#18 0x7f689986eae8 in decoder_process /home/pawn/work/openssl/crypto/encode_decode/decoder_lib.c:971:14
    open-quantum-safe#19 0x7f689986d106 in OSSL_DECODER_from_bio /home/pawn/work/openssl/crypto/encode_decode/decoder_lib.c:82:10
    open-quantum-safe#20 0x55610d6bd407 in decode_EVP_PKEY_prov /home/pawn/work/oqs-provider/test/oqs_test_endecode.c:151:10
    open-quantum-safe#21 0x55610d6bc75b in test_oqs_encdec /home/pawn/work/oqs-provider/test/oqs_test_endecode.c:188:14
    open-quantum-safe#22 0x55610d6bbf37 in main /home/pawn/work/oqs-provider/test/oqs_test_endecode.c:240:17
    open-quantum-safe#23 0x7f68991671c9 in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16

SUMMARY: AddressSanitizer: 1400 byte(s) leaked in 4 allocation(s).
```


## 8. Bonus point: fix a potential double-free.

In `oqsx_key_gen`, when the keytype isn't `KEY_TYPE_HYB_SIG`, the
generated `pkey` is freed, and the flow goes down to the `err` label:

https://github.com/open-quantum-safe/oqs-provider/blob/cc3895f208730abfe1efc15f7c3ff4358ac6e4c0/oqsprov/oqsprov_keys.c#L1089-L1104

There is a potential double-free bug here: if `oqsx_key_gen_oqs` fails, then
`EVP_PKEY_free` will be called twice on the same `pkey` pointer.

Note that it is very unlikely that `oqsx_key_gen_oqs` fails, since it only
calls `OQS_KEM_keypair`/`OQS_SIG_keypair`.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@bhess bhess bhess approved these changes

@dstebila dstebila Awaiting requested review from dstebila

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Remove default algs, BIKEr2; add BIKEr3
3 participants
@baentsch @dstebila @bhess