Skip to content

rsa: keep RSA-PSS keys without parameters unrestricted - #483

Open
yosuke-wolfssl wants to merge 1 commit into
wolfSSL:masterfrom
yosuke-wolfssl:fix/f_12483
Open

rsa: keep RSA-PSS keys without parameters unrestricted#483
yosuke-wolfssl wants to merge 1 commit into
wolfSSL:masterfrom
yosuke-wolfssl:fix/f_12483

Conversation

@yosuke-wolfssl

Copy link
Copy Markdown
Contributor

Problem

An RSA-PSS SubjectPublicKeyInfo or PKCS#8 PrivateKeyInfo whose
AlgorithmIdentifier carries no RSASSA-PSS-params is unrestricted: any
supported digest may be used with it. wp_rsa_pss_get_params() had no way to
tell "parameters absent" from "parameters present" — it never learned where the
AlgorithmIdentifier ended — so it installed the RFC 8017 defaults and set
pssDefSet unconditionally. An unrestricted key decoded as restricted to
SHA-1/MGF1-SHA-1/salt 20.

Path Before After
d2i_PUBKEY_ex on absent-params SPKI loads, pinned to SHA-1 loads, unrestricted
d2i_AutoPrivateKey_ex on absent-params PKCS#8 decode fails outright loads, unrestricted
i2d_PUBKEY of such a key injects RSASSA-PSS-params bare OID, byte-identical

Valid signatures over another supported digest were rejected after decoding.

Fix (src/wp_rsa_kmgmt.c)

  • algIdEnd takes the AlgorithmIdentifier SEQUENCE's own length octet as
    the end of its content, making presence decidable. This replaces a walk that
    stepped over the SPKI BIT STRING and landed inside RSAPublicKey, matched no
    parameter tag, and still reported success.
  • wp_rsa_pss_encode_key_alg_id() encodes the OID alone for an unrestricted
    key. Key encoders use it; the signature AlgorithmIdentifier keeps carrying the
    digest, MGF and salt the signature was actually made with.
  • NULL parameters are rejected, matching ossl_rsa_param_decode().

In src/wp_rsa_sig.c, minSaltLen now uses -1 for "unrestricted" — the
sentinel wp_rsa_setup_md() already tested for but which was never assigned, so
the restriction check fired for every PSS key.

Empty parameters (30 00) still decode as restricted to SHA-1, per the ASN.1
defaults. Closes f-12483.

Tests

  • test_rsa_pss_no_params — decodes both formats, checks the default digest
    is SHA256 and no salt length is reported, signs and verifies under SHA-256/384/512
    in both directions against OpenSSL, asserts byte-exact round-trips, and confirms a
    restricted key is still pinned.
  • test_rsa_sig_alg_id — now A/Bs the PSS signature AlgorithmIdentifier
    against OpenSSL across three digests.

Verification

  • 217/217 unit tests; RSA CLI suite green; ASan + UBSan clean.
  • Interops with the upstream evppkey_rsa.txt unrestricted PSS vector.
  • Negative controls: on base, test_rsa_pss_no_params fails; reverting the
    encoder split makes the alg-id test report 13 bytes against OpenSSL's 67.

@yosuke-wolfssl yosuke-wolfssl self-assigned this Sep 2, 2026
Copilot AI lite review requested due to automatic review settings September 2, 2026 23:57

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

It changes security- and interoperability-critical DER parsing/encoding paths for RSA-PSS keys, so a final human review is warranted even with strong tests.

Pull request overview

This PR fixes RSA-PSS key decode/encode behavior so that keys whose AlgorithmIdentifier omits RSASSA-PSS-params remain unrestricted (rather than being implicitly pinned to RFC 8017 defaults like SHA-1), and ensures the encoded AlgorithmIdentifier round-trips byte-identically for those keys.

Changes:

  • Update RSA-PSS parameter detection/decoding to distinguish “params absent” vs “params present” and only apply defaults when params are actually present (src/wp_rsa_kmgmt.c).
  • Split key AlgorithmIdentifier encoding so unrestricted RSA-PSS keys encode as a bare OID (no injected params), while signature alg-ids still carry digest/MGF/salt (src/wp_rsa_kmgmt.c).
  • Fix PSS restriction handling by initializing/copying minSaltLen with the intended “unrestricted” sentinel and add unit tests covering unrestricted PSS keys and signature alg-id parity with OpenSSL (src/wp_rsa_sig.c, test/test_rsa.c, test/unit.*).
File summaries
File Description
test/unit.h Adds the new unit test prototype to the test registry header.
test/unit.c Registers the new RSA-PSS “no params” unit test in the test list.
test/test_rsa.c Adds DER fixtures and new test logic for unrestricted RSA-PSS keys and PSS signature AlgorithmIdentifier A/B vs OpenSSL.
src/wp_rsa_sig.c Initializes and propagates minSaltLen so unrestricted PSS keys don’t incorrectly trigger restriction checks.
src/wp_rsa_kmgmt.c Corrects RSA-PSS param presence detection and adjusts key AlgorithmIdentifier encoding for unrestricted keys.
Review details
  • Files reviewed: 5/5 changed files
  • Comments generated: 1
  • Review effort level: Lite

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread src/wp_rsa_kmgmt.c Outdated

@wolfSSL-Fenrir-bot wolfSSL-Fenrir-bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fenrir Automated Review — PR #483

Scan targets checked: wolfprovider-bugs, wolfprovider-src

Findings: 1
1 finding(s) posted as inline comments (see file-level comments below)

This review was generated automatically by Fenrir. Reported findings require changes before merge.

Comment thread src/wp_rsa_kmgmt.c
- wp_rsa_pss_get_params() takes the AlgorithmIdentifier's length
  octet as the end of its content, reads a SEQUENCE after the OID as
  parameters present and nothing at all as absent, and rejects any
  other tag. The defaults, the hash, MGF and salt blocks and
  rsa->pssDefSet apply only when parameters are present. The BIT
  STRING step-over is removed.
- wp_rsa_pss_encode_key_alg_id() encodes the OID alone for a key with
  no PSS restrictions, else defers to wp_rsa_pss_encode_alg_id(); the
  six call sites in the SPKI and PKCS#8 encoders use it. rsa_pss_oid
  moves to file scope and both encoders compute their length delta
  with signed operands.
- wp_rsa_ctx_new() sets minSaltLen to -1 and wp_rsa_ctx_dup() copies
  it. For a key with no PSS restrictions wp_rsa_signverify_init()
  uses minSaltLen -1, and saltLen RSA_PSS_SALTLEN_AUTO where
  RSA_PSS_SALTLEN_AUTO_DIGEST_MAX is unavailable.
- test_rsa.c gains rsa_pss_key_der_2048_pkcs8_noparams and
  test_rsa_pss_no_params(), covering an empty parameters SEQUENCE
  restricting the key and a duplicated context keeping the salt
  length restriction; test_rsa_sig_alg_id() compares the PSS
  signature AlgorithmIdentifier against OpenSSL.

Issue: F-12483
Comment thread src/wp_rsa_kmgmt.c

@wolfSSL-Fenrir-bot wolfSSL-Fenrir-bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fenrir Automated Review — PR #483

Scan targets checked: wolfprovider-bugs, wolfprovider-src

Fenrir result: Approved ✅

No new issues found in the changed files.

Advisory only — this automated result does not count as a GitHub approval.

@wolfSSL-Fenrir-bot
wolfSSL-Fenrir-bot dismissed their stale review September 3, 2026 03:08

Fenrir's latest completed scan found no issues; clearing the prior automated change request.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants