Fix SE050 RSA-PSS signing, key cleanup, and mutex leaks

[LinuxJedi]

RSA-PSS fix:
Skip SE050 hardware path for RSA-PSS sign and verify operations in RsaPublicEncryptEx() and RsaPrivateDecryptEx(). The SE050's PSS sign API (Se05x_API_RSASign) is a hash-then-sign operation, which double-hashes when wolfSSL passes a pre-computed digest (as done during TLS CertificateVerify). PSS operations now fall through to the software RSA path. PKCS#1 v1.5 signing continues to use SE050 hardware.

Key object leak fix:
Add se050_rsa_free_key() called from wc_FreeRsaKey() to erase wolfSSL-allocated RSA key objects from SE050 persistent storage on free. Without this, persistent key slots on the SE050 are never reclaimed and eventually exhaust secure storage. Add matching sss_key_store_erase_key() calls to se050_ecc_free_key(), se050_ed25519_free_key(), and se050_curve25519_free_key(). Only keys with keyId >= SE050_KEYID_START are erased (pre-provisioned keys are left intact).

Mutex leak fix:
Add missing wolfSSL_CryptHwMutexUnLock() calls before early returns in se050_rsa_sign(), se050_rsa_verify(), se050_rsa_public_encrypt(), and se050_rsa_private_decrypt() when the algorithm lookup fails after the mutex has already been acquired.

ZD 21212

[Copilot]

Pull request overview

Fixes SE050 RSA-PSS behavior by routing PSS sign/verify through the software RSA implementation, and addresses SE050 resource leaks (persistent key objects + mutex unlock paths).

Changes:

• Skip SE050 hardware path for RSA-PSS sign/verify to avoid double-hashing.
• Erase wolfSSL-allocated SE050 persistent key objects on key free (RSA/ECC/Ed25519/Curve25519).
• Add missing mutex unlocks on early-return error paths in SE050 RSA operations.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 7 comments.

File	Description
wolfcrypt/src/rsa.c	Routes PSS sign/verify away from SE050 and triggers SE050 RSA key erasure during rsa key free.
wolfcrypt/src/port/nxp/se050_port.c	Adds RSA key erase helper, erases other key types, and fixes missing mutex unlocks; adds SE050 debug logging.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

You can also share your feedback on Copilot code review. Take the survey.

[LinuxJedi]

Copilot comments 4-7 are in debugging code and if we address those we should address across existing code too,

[LinuxJedi]

Jenkins, retest this please - previous run missing

[LinuxJedi]

Jenkins, retest this please - previous run missing

[LinuxJedi]

Rebase push done

[wolfSSL-Fenrir-bot]

Fenrir Automated Review — PR #9912

Scan targets checked: wolfcrypt-bugs, wolfcrypt-port, wolfcrypt-port-bugs, wolfcrypt-src
Findings: 5

Medium (3)

Ignored sss_key_store_erase_key return value in se050_rsa_free_key loses key material tracking

File: wolfcrypt/src/port/nxp/se050_port.c:1060-1065
Function: se050_rsa_free_key
Category: Key material exposure during hardware offload

In se050_rsa_free_key, the return value of sss_key_store_erase_key() is not checked. If the erase operation fails (e.g., hardware communication error, busy SE050), execution continues to clear key->keyId and key->keyIdSet to 0. This means the software loses all reference to the key, but the key material remains in SE050 persistent storage with no way to clean it up later. The same pattern is introduced in se050_ecc_free_key, se050_ed25519_free_key, and se050_curve25519_free_key.

if (key->keyId >= SE050_KEYID_START) {
            sss_key_store_erase_key(&host_keystore, &keyObject);
        }
        sss_key_object_free(&keyObject);
        key->keyId = 0;
        key->keyIdSet = 0;

Recommendation: Check the return value of sss_key_store_erase_key(). If it fails, log a warning and consider either retrying the erase or preserving the keyId so the caller can attempt cleanup later. At minimum, log the failure so operators can detect key slot leakage on the SE050.

---

Ignored sss_key_store_erase_key return value in ECC/ed25519/curve25519 free functions

File: wolfcrypt/src/port/nxp/se050_port.c:2440-2445,2906-2909,3382-3385
Function: se050_ecc_free_key, se050_ed25519_free_key, se050_curve25519_free_key
Category: Key material exposure during hardware offload

The same pattern from se050_rsa_free_key is replicated in three other free functions. In each case, sss_key_store_erase_key() is called without checking its return value, then keyId and keyIdSet are unconditionally cleared. A failed erase leaves key material orphaned in SE050 persistent storage. Over time, repeated failures could exhaust the SE050's limited secure storage slots, which is the very problem this PR aims to fix.

if (key->keyId >= SE050_KEYID_START) {
            sss_key_store_erase_key(&host_keystore, &newKey);
        }
        sss_key_object_free(&newKey);
        key->keyId = 0;
        key->keyIdSet = 0;

Recommendation: Check the return value of sss_key_store_erase_key() in all three functions. On failure, at minimum log a warning. Consider preserving the key ID reference so the leak can be detected or retried.

---

Key erasure failure silently clears key ID, causing permanent SE050 storage leak

File: wolfcrypt/src/port/nxp/se050_port.c:1058-1066
Function: se050_rsa_free_key
Category: Resource leaks on error paths

In se050_rsa_free_key, when sss_key_store_erase_key fails, the function still clears key->keyId and key->keyIdSet to 0 on the next lines. This means the in-memory reference to the SE050 persistent key is lost, making it impossible to retry the erase or ever clean up that key. The comment on this very code block states the purpose is to prevent persistent key objects from leaking and exhausting secure storage, yet the implementation can silently cause exactly that leak. The same pattern is repeated in the additions to se050_ecc_free_key (~line 2443), se050_ed25519_free_key (~line 2906), and se050_curve25519_free_key (~line 3382).

if (key->keyId >= SE050_KEYID_START) {
    sss_key_store_erase_key(&host_keystore, &keyObject);
}
sss_key_object_free(&keyObject);
key->keyId = 0;
key->keyIdSet = 0;

Recommendation: Check the return value of sss_key_store_erase_key. If it fails, either skip clearing keyId/keyIdSet (allowing a future free call to retry), or at minimum emit a WOLFSSL_MSG warning so the failure is observable. This applies to all four free functions modified in this PR. For example:

if (key->keyId >= SE050_KEYID_START) {
    sss_status_t eraseSt = sss_key_store_erase_key(&host_keystore, &keyObject);
    if (eraseSt != kStatus_SSS_Success) {
        WOLFSSL_MSG("se050_rsa_free_key: failed to erase key from SE050");
    }
}

---

Low (2)

Missing NULL check on key parameter in se050_rsa_free_key

File: wolfcrypt/src/port/nxp/se050_port.c:1029-1039
Function: se050_rsa_free_key
Category: NULL pointer dereference

The new se050_rsa_free_key function dereferences key (accessing key->keyIdSet and in debug builds key->keyId) without first checking that key is not NULL. While the caller wc_FreeRsaKey does check for NULL before calling, this function is externally visible and other callers may not. The existing se050_ecc_free_key follows the same unguarded pattern, but this is new code that could add a defensive check. In debug builds, the printf at the top dereferences key->keyId before any validation.

void se050_rsa_free_key(struct RsaKey* key)
{
    sss_status_t    status = kStatus_SSS_Success;
    sss_object_t    keyObject;
    sss_key_store_t host_keystore;

#ifdef SE050_DEBUG
    printf("se050_rsa_free_key: key %p, keyId %d\n", key, key->keyId);
#endif

    if (cfg_se050_i2c_pi == NULL) {
        return;
    }
    if (key->keyIdSet == 0) {
        return;
    }

Recommendation: Add a NULL check on key at the top of se050_rsa_free_key, before the debug printf and before accessing key->keyIdSet. For example: if (key == NULL) { return; }

---

Debug printf dereferences key pointer before any NULL guard

File: wolfcrypt/src/port/nxp/se050_port.c:1030
Function: se050_rsa_free_key
Category: Logic errors

The SE050_DEBUG printf at the top of se050_rsa_free_key dereferences key->keyId before any validation of the key pointer. While the current caller (wc_FreeRsaKey) checks key == NULL before calling this function, se050_rsa_free_key itself has no NULL guard, and the debug printf is the very first statement that touches key. If this function is ever called from another context with a NULL pointer, it will crash. The non-debug path also dereferences key->keyIdSet without a NULL check, but the debug path is more concerning as it's reached unconditionally when SE050_DEBUG is defined.

#ifdef SE050_DEBUG
    printf("se050_rsa_free_key: key %p, keyId %d\n", key, key->keyId);
#endif

    if (cfg_se050_i2c_pi == NULL) {
        return;
    }
    if (key->keyIdSet == 0) {
        return;
    }

Recommendation: Add a NULL check on key at the top of the function, before the debug printf, consistent with defensive practices in public API functions:

if (key == NULL) {
    return;
}

---

This review was generated automatically by Fenrir. Findings are non-blocking.