mbedtls: add support for 4.0.0 by vszakats · Pull Request #19077 · curl/curl

vszakats · 2025-10-15T20:12:37Z

After this patch libcurl requires (as already documented)
the curl_global_init() call when using the curl_formadd() API with
mbedTLS.

Note: NTLM is not supported with mbedTLS 4+, because it lacks
the necessary crypto primitive: DES.

Also:

lib: de-dupe mbedTLS minimum version checks into curl_setup.h.
lib: initialize PSA Crypto as part of curl_global_init().
For MD5, SHA-256, curl_formadd(), and MultiSSL builds with mbedTLS
but where mbedTLS isn't the default backend.
lib1308: fix to call curl_global_init() (for the Form API).
curl_ntlm_core: disable with mbedTLS 4+.
md4: disable mbedTLS implementation when building against 4.x.
md5: use mbedTLS PSA Crypto API when available, otherwise use
the default local implementation.
sha256: use mbedTLS PSA Crypto API when available, otherwise use
the default local implementation.
vtls/mbedtls: drop PSA Crypto initialization in favor of
curl_global_init().
vtls/mbedtls: use PSA Crypto random API with all mbedTLS versions.
vtls/mbedtls: do the same for the SHA-256 callback.
autotools: detect mbedTLS 4+, and disable NTLM for 3.x.
cmake: disable NTLM for mbedTLS 3.x.
GHA/linux: keep building mbedTLS 3.x manually and use it in
an existing job, while also enabling pytest in it.
GHA/linux: bump to mbedTLS 4.0.0.
Closes CI: Update dependency Mbed-TLS/mbedtls to v4 #19075
Closes CI: Update dependency Mbed-TLS/mbedtls to v3.6.5 (try again?) #19074

Refs:
https://github.com/Mbed-TLS/mbedtls/releases/tag/mbedtls-4.0.0
https://github.com/Mbed-TLS/mbedtls/blob/mbedtls-4.0.0/docs/4.0-migration-guide.md
https://github.com/Mbed-TLS/mbedtls/blob/mbedtls-4.0.0/tf-psa-crypto/docs/1.0-migration-guide.md [404]
https://github.com/Mbed-TLS/TF-PSA-Crypto/blob/tf-psa-crypto-1.0.0/docs/1.0-migration-guide.md
https://github.com/Mbed-TLS/TF-PSA-Crypto/blob/tf-psa-crypto-1.0.0/docs/psa-transition.md
https://github.com/Mbed-TLS/TF-PSA-Crypto/tree/627f727bbed3d9319ed548f1c0839a29c223414e/docs/4.0-migration-guide

maybe a better way to fix MD5 and SHA-256 would be to switch to the PSA
API already available in 3.x. It requires calling psa_crypto_init() before use.
add/adjust version guards for PSA init in psa_crypto_init(). Just in case.
Or remove feature guards around PSA functions.
They seem to be available when explicitly including psa/crypto.h.
Which is available in all 3.2.0+ versions.

bagder · 2025-10-15T20:27:30Z

Crazy how mbedTLS keeps breaking compatibility release after release,

Careless is a word that comes to mind! We can also just document not compatible with v4 and leave that work for someone to do later. I mean it is an option.

vszakats · 2025-10-15T21:09:43Z

There is a separate 'migration guide' mentioning some of the breakages: https://github.com/Mbed-TLS/mbedtls/blob/mbedtls-4.0.0/docs/4.0-migration-guide.md

The above doc links to another migration guide for the crypto subsystem, which is a 404:
https://github.com/Mbed-TLS/mbedtls/blob/mbedtls-4.0.0/tf-psa-crypto/docs/1.0-migration-guide.md

What I noticed so far: MD4 and 3DES are gone, MD5 and SHA256 may now be offered via different/moved headers and a new API (possibly introduced in 3.x? This may not be accurate, and will need more digging. edit: Oh, md.h is also "legacy" and has "reduced functionality" in 4.x. edit 2: It looks like the "PSA" API is the recommended way.

There is a per-header collection of further migration guides for the crypto API here:
https://github.com/Mbed-TLS/TF-PSA-Crypto/tree/development/docs/4.0-migration-guide

After major changes in 3.x, the random generator API has seen another overhaul in 4.x.

Agreed with leaving the rest as-is for now. This PR is probably okay to merge, but it's just
the first, light, step to rewrite things for 4.x.

edit: this may be the non-404 link: https://github.com/Mbed-TLS/TF-PSA-Crypto/blob/tf-psa-crypto-1.0.0/docs/1.0-migration-guide.md (which is different from the master branch with the many separate migration guides).

vszakats · 2025-10-16T11:36:57Z

Managed to make curl build with 4.0.0.

Some tests fail with it:

FAIL 1308: 'formpost tests' curl_formadd, curl_formget, FORM
FAIL 2088: 'HTTPS GET with client authentication (mtls)' HTTPS, HTTP GET, Client Auth
FAIL 2089: 'HTTPS GET with client authentication (mtls) and --insecure' HTTPS, HTTP GET, Client Auth

Field notes:

it'd likely help if 3.x made the PSA API non-optional at build time (I mean: non-disablable)...
...and mbedTLS Release Notes made it clear that PSA is the preferred, next mandatory, crypto API.
edit: after looking into this again: it seems the PSA API (MBEDTLS_USE_PSA_CRYPTO) was there but practically always disabled in 3.x, with no clear way to enable it. The headers had a lot of mentions what happens if it's enabled though. edit: It always seems to be available by including psa/crypto.h. I found these confusing.
if incompatible changes would be listed in Releases Notes, without exceptions. (or at least in Git history)
e.g. mbedtls_ssl_conf_tls13_enable_signal_new_session_tickets() was deleted
without a word. This happened in the past with other APIs.
It'd also help if the replacement API (or changed defaults, etc) would be explained.
if mbedTLS's git log -p would show all diffs, instead of blank merge commits. Without it, it's non-trivial to figure out when something was added/removed and why. (Still trying to figure out how to show these diffs with Git.) E.g. git log -S mbedtls_ssl_conf_tls13_enable_signal_new_session_tickets comes up empty, even though this function was definitely added (by 3.6.1) then removed (by 4.0.0).
One can dream but the practice of constantly deleting / adding APIs without any regard for compatibility hurts downstream extremely. Any amount of transition period would help, and compatibility APIs may also help to understand how to migrate to the new APIs by looking at their source.
Testing with 3.2.0 and 3.3.0 is not possible because the (cmake?) builds are broken.

Pending issues:

the mandatory one-time PSA initialization (psa_crypto_init()) isn't called by libcurl for some reason.
→ fixed by explicitly initializing libcurl in lib1308.
I'm getting the transtition from the deleted mbedtls_pk_can_do() to mbedtls_pk_can_do_psa() (or mbedtls_pk_can_do_ext()) wrong. It doesn't help the transition has no documentation, the static documentation has almost zero detail, and there is near zero use of the new APIs on GitHub (or the internet).
Haven't dug through these yet: Permissions 1: create mbedtls_pk_can_do_ext() Mbed-TLS/mbedtls#5826 Permissions 1: create mbedtls_pk_can_do_ext() Mbed-TLS/mbedtls#5833 Implement mbedtls_pk_can_do_psa (improved mbedtls_pk_can_do_ext) Mbed-TLS/TF-PSA-Crypto#351
Mbed-TLS/mbedtls@69e348d
Migrate from mbedtls_pk_can_do_ext to mbedtls_pk_can_do_psa Mbed-TLS/mbedtls#10266
[development] Migrate from mbedtls_pk_can_do_ext to mbedtls_pk_can_do_psa (2/2) Mbed-TLS/mbedtls#10333
→ fixed by replacing PSA_ALG_ECDSA() with MBEDTLS_PK_ALG_ECDSA(). Ref: https://github.com/Mbed-TLS/mbedtls/pull/10333/files

vszakats · 2025-10-16T14:00:35Z

Ready for review!

mbedTLS crypto might be used from other parts of the code, not only SSL. In MultiSSL builds it's possible that a different TLS backend is selected, yet, the code uses mbedTLS crypto in md4/md5/sha256/curl_ntlm_core. This can happen e.g. in an mbedTLS + Schannel MultiSSL build with Schannel selected as default.

icc: remark curl#10441: The Intel(R) C++ Compiler Classic (ICC) is deprecated and will be removed from product release in the second half of 2023. The Intel(R) oneAPI DPC++/C++ Compiler (ICX) is the recommended compiler moving forward. Please transition to use this compiler. Use '-diag-disable=10441' to disable this message. In file included from libtests.c(125): ../../../tests/libtest/lib1308.c(52): error curl#188: enumerated type mixed with another type global_init(CURL_GLOBAL_ALL); ^ https://github.com/curl/curl/actions/runs/18561102955/job/52910049327?pr=19077#step:35:187

vszakats · 2025-10-16T23:47:07Z

  WOLFSSL_VERSION: 5.8.2
  # renovate: datasource=github-tags depName=Mbed-TLS/mbedtls versioning=semver registryUrl=https://github.com
  MBEDTLS_VERSION: 4.0.0
  # renovate: datasource=github-tags depName=Mbed-TLS/mbedtls versioning=semver:^3.0.0 registryUrl=https://github.com


(We'll see if versioning= is correct, if Renovate picks it up to bump to 3.6.5.)

Make autotools and cmake detect DES support in OpenSSL and mbedTLS. Forward feature macros to C and omit NTLM from the feature preview list. Use the feature macros in source. This ensure that `-V` output matches the preview. OpenSSL doesn't support DES when built with `no-des` or `no-deprecated`. mbedTLS 4.x no longer supports it, and it's possible to disable it in <4 with `scripts/config.py unset MBEDTLS_DES_C`. Before this patch this worked for mbedTLS 4 only, and with a regression for pending PR #16973. Also: - drop NTLM feature check from `curl_setup.h` in favour of autotools/ cmake feature macros. This makes `curl_setup.h` no longer need to include an mbedTLS header, which in turn makes tests/server build without depending on mbedTLS. Fixing, in #16973: ``` In file included from tests/server/first.h:40, from bld/tests/server/servers.c:3: lib/curl_setup.h:741:10: fatal error: mbedtls/version.h: No such file or directory 741 | #include <mbedtls/version.h> | ^~~~~~~~~~~~~~~~~~~ ``` Ref: https://github.com/curl/curl/actions/runs/18689537893/job/53291322012?pr=16973 Ref: #19181 (initial fix idea) Follow-up to 3a30583 #19077 - move back mbedTLS header include and version check from `curl_setup.h` to each source which consumes mbedTLS. - GHA/http3-linux: drop workaround that disabled NTLM for `no-deprecated` OpenSSL builds. Follow-up to 0069778 #12384 - curl_ntlm_core: drop pointless macro `CURL_NTLM_NOT_SUPPORTED`. Follow-up to 0069778 #12384 Closes #19206

Follow-up to 3a30583 curl#19077

Fixing: ``` lib/vtls/mbedtls.c:1505:10: error: call to undeclared function 'Curl_sha256it'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration] 1505 | return Curl_sha256it(sha256sum, input, inputlen); | ^ 1 error generated. ``` with mbedTLS configuration: ``` tf-psa-crypto/scripts/config.py unset PSA_WANT_ALG_SHA_256 tf-psa-crypto/scripts/config.py unset PSA_WANT_ALG_TLS12_ECJPAKE_TO_PMS tf-psa-crypto/scripts/config.py unset MBEDTLS_LMS_C ``` Follow-up to 3a30583 #19077 Closes #19223

`mbedtls-arm` and `mbedtls-prev` were matching `contains()` expressions looking for `mbedtls`. Follow-up to 8806035 curl#20240 Follow-up to 3a30583 curl#19077

`mbedtls-arm` and `mbedtls-prev` were matching `contains()` expressions looking for `mbedtls`. It caused an unnecessary cache restore and an redundant mbedtls build on bumps, and made the build flavors require different local directory names to avoid this accidental collision. Also drop `-threadsafe` from internal names. All local builds are. Follow-up to 8806035 #20240 Follow-up to 3a30583 #19077 Closes #20248

Also: - fix incorrect version in cache id. Follow-up to 3a30583 #19077 - latest version 3.6.6 fails pytests. Seems similar to the v4.1.0 regression. #21178 Mbed-TLS/mbedtls#10668 Closes #21262

Also: - fix incorrect version in cache id. Follow-up to 3a30583 curl#19077 - latest version 3.6.6 fails pytests. Seems similar to the v4.1.0 regression. curl#21178 Mbed-TLS/mbedtls#10668 Closes curl#21262

To initialize hash contexts. Ref: https://arm-software.github.io/psa-api/crypto/1.1/api/ops/hashes.html#c.psa_hash_operation_init Follow-up to 3a30583 #19077 Closes #22220

vszakats added the TLS label Oct 15, 2025

This comment was marked as resolved.

Sign in to view

github-actions Bot added the CI Continuous Integration label Oct 16, 2025

This comment was marked as resolved.

Sign in to view

github-actions Bot added the tests label Oct 16, 2025

vszakats changed the title ~~mbedtls: improve compatibility with 4.0.0~~ mbedtls: add support for 4.0.0 Oct 16, 2025

vszakats force-pushed the mbedtls41 branch from e645bc5 to 688ce55 Compare October 16, 2025 14:50

bagder reviewed Oct 16, 2025

View reviewed changes

Comment thread .github/workflows/linux.yml

vszakats mentioned this pull request Oct 16, 2025

mbedtls 4.0.0 mbedtls@3 3.6.5 Homebrew/homebrew-core#249749

Merged

vszakats added 17 commits October 17, 2025 01:24

curl_ntlm_core.c drop unused slot in comment

28f0b1c

curl_ntlm_core: try adding mbedTLS 4 support

0af4983

mbedtls 4 md5

f547147

mbedtls 4 md4

5dc9dc0

mbedtls 4 sha256

d7e88ec

mbedtls 4 vtls fix 1

d2ade98

mbedtls 4 vtls fix 2

6fed84d

move min version check to curl_setup.h

206cdae

fixups

7f06dd6

sp

e4f1f01

cleanup

1535673

init/cleanup cleanup

8c083a0

mbedtls replace md5 API with PSA

39c6ac0

mbedtls replace sha256 API with PSA

dc865fa

mbedtls.c use PSA API for sha256

d1f42b1

disable entropy and drbg for <4

380da7c

vszakats added 8 commits October 17, 2025 01:24

lib1308.c uninit frenzy

a77e659

fixup pk detection

11e19b4

mbedtls: drop PSA Crypto init in favor of curl_global_init()

6c906f2

cleanup a debug message

6aa85e7

GHA/linux: tidy up name (unrelated)

fb1a918

GHA/linux: tidy up cache name for cached mbedtls (unrelated)

051e609

GHA/linux: add mbedtls 3 job with pytest

1de0796

vszakats force-pushed the mbedtls41 branch from 688ce55 to 1de0796 Compare October 16, 2025 23:24

GHA/linux: add mbedtls 3 job with pytest fixup

02d8bab

vszakats commented Oct 16, 2025

View reviewed changes

cleanup

8d8b102

bagder approved these changes Oct 17, 2025

View reviewed changes

vszakats closed this in 3a30583 Oct 17, 2025

vszakats deleted the mbedtls41 branch October 17, 2025 09:51

This was referenced Oct 21, 2025

build: move NTLM guard definitions to curl_ntlm_core.h #19181

Closed

NTLM: disable if DES support missing from OpenSSL or mbedTLS #19206

Closed

vszakats added a commit to vszakats/curl that referenced this pull request Oct 24, 2025

mbedtls: fix building with sha-256 missing from PSA

9137cb8

Follow-up to 3a30583 curl#19077

vszakats mentioned this pull request Oct 24, 2025

mbedtls: fix building with sha-256 missing from PSA #19223

Closed

vszakats mentioned this pull request Jan 10, 2026

GHA/linux: fix 'mbedtls' internal name confusion #20248

Closed

2 tasks

vszakats mentioned this pull request Apr 8, 2026

GHA/linux: bump mbedTLS 3 to 3.6.5 (from 3.6.4), also verify hash #21262

Closed

vszakats mentioned this pull request Jun 29, 2026

mbedtls: replace memset() with psa_hash_operation_init() #22220

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

mbedtls: add support for 4.0.0#19077

mbedtls: add support for 4.0.0#19077
vszakats wants to merge 58 commits into
curl:masterfrom
vszakats:mbedtls41

vszakats commented Oct 15, 2025 •

edited

Loading

Uh oh!

This comment was marked as resolved.

bagder commented Oct 15, 2025

Uh oh!

vszakats commented Oct 15, 2025 •

edited

Loading

Uh oh!

This comment was marked as resolved.

vszakats commented Oct 16, 2025 •

edited

Loading

Uh oh!

vszakats commented Oct 16, 2025

Uh oh!

Uh oh!

vszakats Oct 16, 2025

Uh oh!

Reviewers

Assignees

Labels

Milestone

Development

Uh oh!

3 participants

Uh oh!

Uh oh!

Conversation

vszakats commented Oct 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

This comment was marked as resolved.

bagder commented Oct 15, 2025

Uh oh!

vszakats commented Oct 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

This comment was marked as resolved.

vszakats commented Oct 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

vszakats commented Oct 16, 2025

Uh oh!

Uh oh!

vszakats Oct 16, 2025

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Milestone

Development

Uh oh!

3 participants

vszakats commented Oct 15, 2025 •

edited

Loading

vszakats commented Oct 15, 2025 •

edited

Loading

vszakats commented Oct 16, 2025 •

edited

Loading