Moved the CertificateVerify buffers into the TLS session metadata

[fdesbiens]

Fixes #430.

The problem

_nx_secure_tls_send_certificate_verify and _nx_secure_tls_process_certificate_verify built and checked every CertificateVerify signature in file-scope static buffers:

Buffer	Size	Where
handshake_hash	64 + 34 + 64	both files
_nx_secure_padded_signature	600	send
_nx_secure_decrypted_signature	600	process
_nx_secure_pss_scratch	600	process

_nx_secure_tls_protection is released before the handshake runs, so two sessions can be inside those routines at the same time and overwrite each other's working data. A client authenticating on two sessions at once could emit a CertificateVerify covering the other session's transcript; a server verifying two TLS 1.3 clients at once could reject a valid signature. The signature is invalid rather than forgeable, so the visible symptom is an intermittent authentication failure under concurrency that never reproduces in single-session testing.

The fix

Option 2 from the issue: the buffers are carved out of the application-supplied crypto metadata area at session create time, which is already per-session, so no new fixed-size field is added to NX_SECURE_TLS_SESSION.

Sending and processing a CertificateVerify never overlap within one session, so the two paths share one region:

• 764 bytes per session, or 1364 with TLS 1.3 enabled
• nothing at all when NX_SECURE_DISABLE_X509 is defined
• both totals are multiples of four and the region is placed first in the metadata area, so every existing region keeps the alignment it has today

The sizes come from NX_SECURE_TLS_CERTIFICATE_VERIFY_SIGNATURE_SIZE and NX_SECURE_TLS_CERTIFICATE_VERIFY_PSS_SCRATCH_SIZE, both of which an application can lower when its largest certificate key is known to be smaller than the 4096-bit default of NX_CRYPTO_MAX_RSA_MODULUS_SIZE.

The issue was filed against the signing side only. The verify side has the identical bug with the opposite symptom — rejecting a valid peer signature — so it is fixed here too rather than left for a follow-up.

Please note: metadata requirement grows

An application that passes a hard-coded metadata buffer instead of the value from nx_secure_tls_metadata_size_calculate will now get NX_INVALID_PARAMETERS from nx_secure_tls_session_create until it resizes the buffer. This is the main review question on the change.

One thing beyond the issue

Neither routine bounded data_size (the certificate's RSA modulus length) against the signature buffer. It could not bite at the old fixed 600 bytes with NX_CRYPTO_MAX_RSA_MODULUS_SIZE at its 4096-bit default, but it becomes reachable now that the size is configurable, so both routines now reject an oversized modulus with NX_SECURE_TLS_INVALID_CERTIFICATE instead of writing past the end of the buffer.

Tests

nx_secure_tls_certificate_verify_concurrency_test is new. It hooks the SHA-256 handshake hash method to preempt one session at the exact point where the handshake hash has been written but not yet consumed, resumes a higher-priority thread that runs a second session to completion, and then checks that the first session still produced or accepted its own transcript. Verified to fail on the code before this change, by restoring each source file from dev in turn:

• signing side: the captured PKCS#1 block no longer matches the block the session produces alone
• verifying side: 0x131 NX_SECURE_TLS_CERTIFICATE_VERIFY_FAILURE on a valid signature

A third phase covers the two new modulus checks.

nx_secure_tls_client_handshake_coverage_test assembles an NX_SECURE_TLS_SESSION by hand rather than through nx_secure_tls_session_create, so it now supplies the scratch area the same way it already hand-assigns nx_secure_tls_crypto_table.

Suite	Result
nx_secure, all 19 profiles	100% (177 + 18 x 154)
web, 3 profiles	100% (3 x 55)
mqtt, 9 profiles	100% (9 x 35)
crypto, 4 profiles	100% (4 x 33)
netxduo (build_nxd_fast)	builds clean

Two coverage-report steps abort in my environment, both pre-existing and unrelated to this change: addons/web hits a gcovr 7.0 parser limitation on gcc 13's %%%%%:NNNNN-block output, and test/cmake/crypto/coverage.sh (unchanged from dev) passes a --gcov-ignore-parse-errors value gcovr 7.0 does not accept. All tests in both suites pass.

Documentation

Matching documentation PR against rtos-docs-asciidoc: the two new configuration options in the TLS and DTLS option tables, and a paragraph in the TLS "Cryptographic Metadata" section covering the per-session reservation and the hard-coded-buffer consequence.

Note for #399

@EdouardMALOT — this touches the ground #399 builds on. _nx_secure_pss_scratch in the send path does not exist yet on dev, so #399 will need its PSS scratch to come from the same per-session region rather than a new static. The area already reserves NX_SECURE_TLS_CERTIFICATE_VERIFY_PSS_SCRATCH_SIZE under TLS 1.3 at NX_SECURE_TLS_CERTIFICATE_VERIFY_PSS_OFFSET, and the send path can use it exactly as the verify path does, with no further sizing change. The comment on _nx_secure_pss_scratch in #399 that points at this constraint can point at this PR.

[fdesbiens]

@EdouardMALOT would you be willing to review this one? A formal review request is not possible because review requests are limited to repository collaborators, hence the ping.

You found this bug while working on #399, and that PR is the part of the tree most affected: the send path's PSS scratch will want to come from the per-session region this adds rather than a new static. The two review questions I would most value your eyes on are whether sharing one region between the send and process paths is safe in every handshake ordering you have exercised, and whether the growth in the required metadata size is acceptable for the targets you care about.