feat!: C ABI callback streaming, std::sync::mpsc removed from ffi (refs #482)#490
Merged
Conversation
#482) Rewrites the C ABI streaming layer in ffi/src/streaming.rs to deliver events through callback registration instead of polling. The user- supplied extern "C" fn flows through the SSOT StreamingDispatcher landed in #489 (queued mode) or directly on the FPSS reader thread (inline mode). Removed C ABI symbols: - tdx_unified_start_streaming - tdx_unified_next_event - tdx_fpss_next_event - tdx_fpss_event_free New C ABI symbols (with TdxFpssCallback typedef): - tdx_unified_set_callback / tdx_unified_set_inline_callback - tdx_fpss_set_callback / tdx_fpss_set_inline_callback tdx_fpss_connect now defers FPSS TLS connection until the first set_callback call so callback registration and connect are atomic. tdx_*_dropped_events now reports StreamingDispatcher overflow drops (returns 0 in inline mode -- no queue). tdx_*_reconnect re-uses the previously-registered callback so callers do not re-supply it. The C++ wrapper's next_event / FpssEventPtr / FpssEventDeleter are removed; thetadx.hpp now exposes only configuration / subscription / lifecycle methods. The C++ wrapper migration to the callback API ships in a follow-up PR. Python and TypeScript SDKs do not depend on the FFI poll path -- they use thetadatadx::ThetaDataDx::start_streaming directly with their own internal mpsc shims, so this change does not break their compile. No version bump.
This was referenced May 6, 2026
userFRM
added a commit
that referenced
this pull request
May 6, 2026
Migrate the C++ wrapper to the callback C ABI shipped in PR #490. `tdx::FpssClient` gains two header-only methods that wrap `tdx_fpss_set_callback` / `tdx_fpss_set_inline_callback`: void set_callback(std::function<void(const FpssEvent&)> fn); void set_inline_callback(std::function<void(const FpssEvent&)> fn); The Client owns a `unique_ptr<std::function<...>>` -- a stable address survives moves of the owning client and is handed to the C ABI as the `void* ctx`. A free `extern "C"` shim recovers the function from `ctx` and invokes it with `const FpssEvent&`, swallowing any propagating exception so unwinding cannot cross the Rust/C boundary. The destructor's call to `tdx_fpss_shutdown` runs before the function storage is freed, so the dispatcher / reader threads cannot dereference stale state. `fpss_smoke.cpp` is restored on the callback path -- `#error` directive deleted, example rewritten to subscribe, register a queued callback, print events for five seconds, and exit cleanly. The CMake target builds clean. `sdks/cpp/README.md` streaming section now documents callback registration as the only entry point with a dedicated note on the inline opt-in's microsecond-budget contract. Version bump 8.0.29 -> 8.0.30 via `scripts/bump_version.py`. The [Unreleased] CHANGELOG block becomes `## [8.0.30] - 2026-05-06` -- the single coherent #482 release that bundles the dispatcher core (#489), C ABI callback (#490), and C++ wrapper migration. Python (#492) and TypeScript (#493) entries land when those PRs merge. Closes #482.
userFRM
added a commit
that referenced
this pull request
May 6, 2026
* feat!: C++ callback wrapper + v8.0.30 release (closes #482) Migrate the C++ wrapper to the callback C ABI shipped in PR #490. `tdx::FpssClient` gains two header-only methods that wrap `tdx_fpss_set_callback` / `tdx_fpss_set_inline_callback`: void set_callback(std::function<void(const FpssEvent&)> fn); void set_inline_callback(std::function<void(const FpssEvent&)> fn); The Client owns a `unique_ptr<std::function<...>>` -- a stable address survives moves of the owning client and is handed to the C ABI as the `void* ctx`. A free `extern "C"` shim recovers the function from `ctx` and invokes it with `const FpssEvent&`, swallowing any propagating exception so unwinding cannot cross the Rust/C boundary. The destructor's call to `tdx_fpss_shutdown` runs before the function storage is freed, so the dispatcher / reader threads cannot dereference stale state. `fpss_smoke.cpp` is restored on the callback path -- `#error` directive deleted, example rewritten to subscribe, register a queued callback, print events for five seconds, and exit cleanly. The CMake target builds clean. `sdks/cpp/README.md` streaming section now documents callback registration as the only entry point with a dedicated note on the inline opt-in's microsecond-budget contract. Version bump 8.0.29 -> 8.0.30 via `scripts/bump_version.py`. The [Unreleased] CHANGELOG block becomes `## [8.0.30] - 2026-05-06` -- the single coherent #482 release that bundles the dispatcher core (#489), C ABI callback (#490), and C++ wrapper migration. Python (#492) and TypeScript (#493) entries land when those PRs merge. Closes #482. * fix(cpp): address review on PR #494 -- memory safety on set_callback / move-assign set_callback / set_inline_callback now stage the new std::function into a local unique_ptr and only adopt it into callback_ after tdx_fpss_set_callback returns 0. The C ABI rejects subsequent registrations with -1 while keeping the previously installed (callback, ctx) live, so overwriting callback_ before checking the return code dangled the Rust-side ctx into freed storage. operator=(FpssClient&&) now drains the existing handle via tdx_fpss_shutdown before dropping the old callback_. tdx_fpss_shutdown stops the FPSS reader and joins the dispatcher drain thread before returning, so once it completes no thread can still observe the old ctx pointer.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Stage B of issue #482: rewrite the C ABI streaming layer in
ffi/src/streaming.rsto deliver events through callback registrationinstead of polling. The user-supplied
extern "C" fnflows through theSSOT
StreamingDispatcher(added in #489) for the queued path, ordirectly on the FPSS reader thread for the inline path.
This is the C-ABI-only stage. Python and TypeScript bindings do not
depend on the FFI poll path -- they call
thetadatadx::ThetaDataDx::start_streamingdirectly with their own internal
mpscshims, so this change does notbreak their compile. Their bindings migrate to the callback API in
follow-up PRs.
Removed C ABI symbols
tdx_unified_start_streamingtdx_unified_next_eventtdx_fpss_next_eventtdx_fpss_event_freeAll
std::sync::mpsc::channel,Sender,Receiver,recv_timeoutreferences and the buffered-event closure path are gone. Verified:
rg "std::sync::mpsc" ffi/ --type rustreturns zero matches.New C ABI symbols
set_callback): events flowFPSS reader -> bounded(8192) crossbeam queue -> dispatcher drain thread -> user fn. Reader neverblocks on user code; overflow events are dropped and counted via
tdx_*_dropped_events.set_inline_callback): user fn fires directly on theFPSS reader thread, bypassing the queue. Microsecond-budget contract
-- any allocation, I/O, or lock acquisition will stall the reader and
cause the vendor session to drop. Identical semantics to
start_streaming_inline.Semantic migration
tdx_fpss_next_event(handle, timeout_ms)and freed each returnedevent. New: caller registers an
extern "C" fnonce viatdx_fpss_set_callback; the SDK invokes it for every event with theregistered
ctxopaque pointer.*mut TdxFpssEventwas caller-owned andrequired
tdx_fpss_event_free. New: the*const TdxFpssEventpointer handed to the callback is valid only for the duration of that
call; callers must copy any fields they want to outlive the
callback.
tdx_fpss_connectdefers the FPSS TLS connection until the firstset_callback/set_inline_callbackcall.FpssClient::connectregisters its event handler at construction time, so deferring the
connect lets us avoid an internal queue and atomically bind the C
callback to the FPSS reader.
tdx_*_reconnectre-uses the previously-registered callback socallers do not re-supply it. Returns -1 with a clear error if no
callback was ever installed (the new ABI has no out-of-band buffer
to fall back on).
tdx_*_dropped_eventsnow reportsStreamingDispatcheroverflowdrops (queue full when the reader tried to enqueue), not channel-
disconnect drops. Returns 0 in inline mode (no queue exists).
C++ wrapper change
The C++ wrapper at
sdks/cpp/include/thetadx.hppwas minimallyadjusted, not stubbed with
#error/static_assert:next_eventmethod (MethodKind::NextEvent) wasremoved from the
cpp_fpsstarget insdk_surface.toml. Regeneratedsdks/cpp/include/fpss.hpp.incandsdks/cpp/src/fpss.cpp.incno longer declare or define
FpssClient::next_event.FpssEventDeleterandFpssEventPtrare deleted fromthetadx.hpp.FpssClient::dropped_events()now reports dispatcher overflow drops(semantics changed; method preserved).
sdks/cpp/examples/fpss_smoke.cppexample calls into theremoved
next_eventAPI, so it now opens with#error "fpss_smoke.cpp depends on the removed next_event poll API. Re-enable when the C++ wrapper migrates to the callback C ABI in PR E (refs #482)."-- a static breakage signal so downstream consumersdo not silently miss the API change.
set_callback/set_inline_callbackships in a follow-up PR.
Tests
5 new unit tests in
ffi/src/streaming.rs::testscover the callbackwiring without needing a real FPSS server:
ffi_callback_inline_invokes_user_fn_on_caller_thread-- inlinemode runs the user fn synchronously on the caller thread with the
registered
ctx.ffi_callback_queued_runs_on_dispatcher_thread-- queued mode runsthe user fn on the dispatcher's drain thread (different OS thread id
than the producer).
ffi_callback_is_send_and_sync-- compile-time assertion thatFfiCallbackisSend + Sync.fpss_dropped_events_zero_before_callback-- handle returns 0 whenno callback has been installed.
unified_dropped_events_handles_null-- null-handle defensivereturn + dispatcher steady-state drop count.
No version bump
Cargo.tomlstays at 8.0.29. The version bump rides the coherentrelease that lands when all bindings (Python/TS/C++) are migrated.
Test plan
cargo fmt --all -- --checkcleancargo clippy --workspace --all-targets -- -D warningscleancargo test --workspacepassing (475+ tests, including the 5 newcallback unit tests)
cargo deny checkcleancargo run -p thetadatadx --bin generate_sdk_surfaces --features config-file -- --checkcleancargo check --manifest-path tools/mcp/Cargo.toml --lockedcargo clippy --manifest-path tools/mcp/Cargo.toml --all-targets -- -D warningscargo test --manifest-path tools/mcp/Cargo.toml --no-runcargo check --manifest-path tools/server/Cargo.toml --lockedcargo check --manifest-path sdks/python/Cargo.toml --lockedcargo check --manifest-path sdks/typescript/Cargo.toml --lockedrg "std::sync::mpsc" ffi/ --type rustreturns zeroset_callback/set_inline_callbackfrom a C consumer once the C++ wrapper migration lands