hardening: add API guards

[icing]

hardening: add API guards

Add call stacks to easy and multi instances. Record ongoing API calls and callback invocations there to detect recursion and not allowed invocations.

Define enums for easy, multi and callbacks in api.h. In api.c define properties for these functions:

• can they recurse
• is the easy/multi handle destroyed during the call or should it be good afterwards
• is the call allowed when a multi event callback is ongoing
• is the call allowed when a notification callback is ongoing

Entering a guard

• checks that passed CURL*/CURLM* are GOOD on entering
• checks that easy handle's mid is correct and it is known for it in the multi.
• checks that call properties are obeyed (recursion, callback checks)
• checks that passed CURL*/CURLM* are GOOD on leaving, unless call is known to kill it

Checks for ongoing callbacks inspect the whole call stack and catches nested invocations (which our current flags can not).

Call stacks in easy/multi handle are fixed size and will deny recursion when the limit is reached. The current limits are 7 for easy and 15 for multi now.

Removes:

• multi->in_callback, check is done via call stack
• multi->in_ntfy_cb, check is done via call stack

The overhead in my tests seems minimal, if noticeable at all.

[Copilot]

Pull request overview

This PR introduces centralized API/callback “guard” tracking for easy and multi handles by adding fixed-size call stacks and using them to detect invalid handle usage, disallowed recursion, and disallowed API calls during certain callbacks. It updates many public API entrypoints and user-callback invocations to enter/leave these guards, replacing the previous multi->in_callback / multi->in_ntfy_callback style state.

Changes:

• Add new lib/api.[ch] implementing easy/multi/callback call-stack tracking and guard enter/leave helpers.
• Wrap many public easy/multi API functions and internal user-callback invocations with the new guard macros.
• Update internal structs and unit size limits to account for the added call stack storage.

Reviewed changes

Copilot reviewed 30 out of 30 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
tests/unit/unit3214.c	Adjusts max allowed CURLM struct size for added guard fields.
lib/ws.c	Switches callback detection to Curl_api_is_in_callback() and adds easy-API guards around WS APIs.
lib/vtls/openssl.c	Wraps SSL_CTX user callback with CBAPI guard.
lib/vssh/libssh2.c	Wraps SSH-related user callbacks with CBAPI guard.
lib/vssh/libssh.c	Wraps SSH-related user callbacks with CBAPI guard.
lib/urldata.h	Includes api.h and adds easy call stack storage to struct Curl_easy.
lib/url.c	Uses internal Curl_multi_remove_handle() variant.
lib/transfer.h	Adds internal Curl_easy_recv() declaration.
lib/setopt.c	Wraps curl_easy_setopt() with easy-API guard.
lib/sendf.c	Wraps read/seek/ioctl callbacks with CBAPI guard.
lib/rtsp.c	Wraps RTP write callback with CBAPI guard.
lib/progress.c	Wraps progress callbacks with CBAPI guard.
lib/multiif.h	Removes old in-callback APIs; exposes internal multi add/remove and “knows easy” helpers.
lib/multihandle.h	Includes api.h, adds multi call stack storage, and removes old callback-state fields.
lib/multi.c	Adds multi-API guards around public multi entrypoints; introduces internal add/remove variants and call stack usage.
lib/multi_ntfy.c	Wraps notification dispatch in CBAPI multi guard.
lib/multi_ev.c	Wraps socket callback invocations in CBAPI multi guard.
lib/Makefile.inc	Adds api.c/api.h to the build.
lib/http2.c	Wraps multi push callback with CBAPI guard.
lib/http_chunks.c	Wraps trailer callback with CBAPI guard.
lib/hostip.c	Wraps resolver-start callback with CBAPI guard.
lib/ftplistparser.c	Wraps fnmatch callback usage with CBAPI guard.
lib/ftp.c	Wraps chunk/seek callbacks with CBAPI guard.
lib/easy.c	Wraps multiple easy APIs with easy-API guard; introduces internal Curl_easy_recv() used by internal callers.
lib/doh.c	Uses internal Curl_multi_add_handle() / Curl_multi_remove_handle() variants.
lib/cw-out.c	Wraps write callback with CBAPI guard.
lib/curl_trc.c	Wraps debug callback with CBAPI guard.
lib/cf-socket.c	Wraps socket open/close/sockopt callbacks with CBAPI guard.
lib/api.h	Defines guard types/enums/macros and handle “GOOD_*” checks for the guard system.
lib/api.c	Implements guard enter/leave logic and callback stack inspection helpers.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.