Skip to content

fix: PSS mgf1/saltlen setup for auto-promoted OAEP padding#133

Merged
timlegge merged 2 commits intocpan-authors:mainfrom
atoomic:koan.atoomic/fix-pss-auto-promote-mgf1
Mar 22, 2026
Merged

fix: PSS mgf1/saltlen setup for auto-promoted OAEP padding#133
timlegge merged 2 commits intocpan-authors:mainfrom
atoomic:koan.atoomic/fix-pss-auto-promote-mgf1

Conversation

@Koan-Bot
Copy link
Copy Markdown
Contributor

@Koan-Bot Koan-Bot commented Mar 21, 2026

What

Fix sign() and verify() to correctly configure PSS mgf1_md and saltlen when OAEP padding auto-promotes to PSS on OpenSSL 3.x.

Why

When use_pkcs1_oaep_padding is set and sign()/verify() are called, the code auto-promotes OAEP to RSA_PKCS1_PSS_PADDING (lines 1124-1126 and 1202-1204). However, the PSS parameter setup (mgf1 digest and salt length) checked p_rsa->padding — which is still RSA_PKCS1_OAEP_PADDING — instead of the local sign_pad/verify_pad variable that holds the promoted value. This caused the EVP_PKEY_CTX_set_rsa_mgf1_md and EVP_PKEY_CTX_set_rsa_pss_saltlen calls to be silently skipped, falling back to OpenSSL defaults (MGF1=SHA-1, saltlen=max) instead of the intended values (MGF1=hash mode digest, saltlen=digest length).

How

Two-character fix: replace p_rsa->padding with sign_pad (line 1133) and verify_pad (line 1210) in the PSS parameter setup conditionals.

Testing

  • New t/pss_auto_promote.t with 6 tests validating cross-verification between OAEP-auto-promoted and explicit PSS paths
  • Full suite passes: 468 tests, 0 failures

🤖 Generated with Claude Code


Quality Report

Changes: 2 files changed, 65 insertions(+), 2 deletions(-)

Code scan: clean

Tests: passed (OK)

Branch hygiene: clean

Generated by Kōan post-mission quality pipeline

Koan-Bot and others added 2 commits March 21, 2026 00:07
Verify that signatures created with OAEP padding (auto-promoted to PSS
internally) can be verified with explicit PSS padding and vice versa.
This validates that the mgf1 digest and salt length parameters are
consistent between auto-promoted and explicit PSS paths.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
… setup

When OAEP padding auto-promotes to PSS in sign() and verify(), the
mgf1_md and saltlen configuration was skipped because the conditional
checked p_rsa->padding (still OAEP) instead of the local sign_pad /
verify_pad variable (already promoted to PSS). This meant auto-promoted
PSS used OpenSSL defaults (MGF1=SHA-1, saltlen=max) instead of the
intended values (MGF1=hash_mode digest, saltlen=digest_length).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@atoomic atoomic marked this pull request as ready for review March 21, 2026 06:14
Copy link
Copy Markdown
Member

@timlegge timlegge left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

look good

@timlegge timlegge merged commit 9540f1e into cpan-authors:main Mar 22, 2026
28 checks passed
toddr-bot added a commit to toddr-bot/Crypt-OpenSSL-RSA that referenced this pull request Mar 23, 2026
On OpenSSL 3.x, the early CHECK_OPEN_SSL() calls for pctx, params_build,
and push_BN operations would croak immediately on failure, bypassing the
err: cleanup label. This leaked pctx, params_build, and the input
BIGNUMs (n, e, d, p, q).

Convert to THROW()/goto err which routes through the existing cleanup
code. Also initialize `error` to 0 at declaration (was uninitialized
before the if(p||q) block).

Additionally adds t/pss_auto_promote.t to MANIFEST (missing since cpan-authors#133).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants