perf(profiling): remove indirect allocator forwarding - #4070
Conversation
Read executor_globals.current_execute_data directly on NTS builds instead of calling through the C FFI wrapper for every allocation and reallocation. Keep the existing wrapper on ZTS builds. A 60-second macOS sample reduced get_current_execute_data self time from 0.412% (207/50,210 main-thread samples) to 0.002% (1/49,737). Known allocation-hook self time fell from 4.184% to 3.601%, a 13.9% relative reduction. Repeated end-to-end throughput remained within system scheduling noise. Validation: cargo test (22 passed); allocation sampling-distance, memory-peak, and GC PHPTs passed.
|
Benchmarks [ profiler ]Benchmark execution time: 2026-07-28 18:27:32 Comparing candidate commit 010c2f7 in PR branch Found 0 performance improvements and 0 performance regressions! Performance is the same for 29 metrics, 7 unstable metrics.
|
Forward PHP 8.4+ allocations and frees from a single ZendMMState read instead of loading a function pointer, making an indirect Rust call, and reloading the state. Preserve neighboring custom allocator support with a predictable previous-handler check. Use unchecked heap extraction in the callbacks: rinit stores the heap before they can be invoked, and rshutdown removes them before clearing it. This avoids an Option discriminant check on every allocation and free. Across six 60-second runs per binary, mean allocation throughput increased from 9,366,444/s to 10,018,262/s (+6.96%) and median throughput increased by 7.56%. A 60-second native sample reduced known forwarding-path self time from 3.599% to 2.438% (-32.3% relative). Validation: cargo test (22 passed); allocation sampling-distance, memory-peak, and GC PHPTs passed. https://datadoghq.atlassian.net/browse/PROF-15506
efd1c5e to
6e87883
Compare
Benchmarks [ tracer ]Benchmark execution time: 2026-07-28 19:29:43 Comparing candidate commit 010c2f7 in PR branch Found 2 performance improvements and 2 performance regressions! Performance is the same for 190 metrics, 0 unstable metrics.
|
Apply the single-state allocation and free forwarding path to PHP 8.3 and older. Preserve the required ZendMM prepare/restore calls and neighboring custom allocator support while removing the intermediate function pointers and wrappers. On ZTS this also avoids repeated TSRM lookups. Across six balanced 60-second runs per binary on PHP 8.3 ZTS, mean allocation throughput increased from 28,079,883/s to 32,661,234/s (+16.32%) and median throughput increased by 16.65%. A full 60-second native sample reduced known forwarding-path self time from 16.31% to 12.30% (-24.6% relative). Validation: PHP 8.3 ZTS cargo test (22 passed); allocation sampling-distance, memory-peak, and GC PHPTs passed; PHP 8.5 release build and cargo test (22 passed). https://datadoghq.atlassian.net/browse/PROF-15506
The free-handler selection test retains allocation callbacks that now call _zend_mm_free directly, but Rust unit-test binaries are not loaded by PHP. Provide a test-only stub with release and debug PHP signatures so both ZendMM API implementations link. Verified with cargo test on PHP 8.5 NTS and PHP 8.3 ZTS; both pass 22 tests.
Forward reallocations from one ZendMMState read instead of loading a function pointer, making an indirect Rust call, and reloading state. Preserve neighboring custom allocator support and the PHP 8.3-and-older heap prepare/restore calls. Remove the realloc forwarding field, intermediate wrappers, and initialization panic callback from both ZendMM implementations. Validation: PHP 8.5 NTS and PHP 8.3 ZTS cargo test (22 passed each); five allocation PHPTs passed on both builds. https://datadoghq.atlassian.net/browse/PROF-15506
realFlowControl
added
Overhead
On PHP 8.3 and older, select the direct ZendMM or neighboring custom allocator callback once during rinit. The normal allocation callback no longer loads or branches on prev_custom_mm_alloc for every allocation; the custom allocator callback remains as a documented cold compatibility path. Across six balanced 60-second PHP 8.3 ZTS runs per binary, mean throughput increased from 31,167,496/s to 32,646,366/s (+4.74%) and median throughput increased by 4.56%. Native samples reduced alloc_prof_malloc self time from 6.13% to 3.54% (-42.3% relative). The equivalent PHP 8.5 NTS experiment measured within system noise (+0.79% mean / +0.39% median) and was not retained. Validation: PHP 8.5 NTS cargo test (22 passed), PHP 8.3 ZTS and PHP 7.3 ZTS cargo check, and five PHP 8.3 allocation PHPTs. https://datadoghq.atlassian.net/browse/PROF-15506
Apply the same RINIT-selected allocation callback used for PHP 8.3 and older to PHP 8.4 and newer. The normal callback has no previous-allocator load or branch, while a separate cold callback preserves neighboring custom allocator support. The PHP 8.5 NTS benchmark was inconclusive but non-negative: a clean six-run retry measured +0.40% mean and +0.69% median with about 1.8% run variance. Keep the implementation for symmetry across ZendMM APIs rather than as a claimed performance win. Validation: PHP 8.5 NTS cargo test (22 passed) and profiler-release build. https://datadoghq.atlassian.net/browse/PROF-15506
…faster-allocation-forwarding
morrisonlevi
left a comment
morrisonlevi
left a comment
There was a problem hiding this comment.
Approved. I have reviewed the final stack and will refactor after it's all landed.
2 participants
Description
Forward allocations, reallocations, and frees from one
ZendMMStateread instead of indirect Rust calls and repeated profiler-global lookups. During RINIT, select compile-time-specialized normal and neighboring-allocator callbacks for malloc, free, and realloc so the common paths have no previous-handler load or branch; neighboring callbacks remain cold compatibility paths. Free and realloc callbacks also specialize heap-live tracking behavior.PHP 8.4+ calls ZendMM directly. PHP 8.3 and older retains the required
prepare_zend_heap()/restore_zend_heap()calls. Test-only_zend_mm_freestubs keep the direct callbacks linkable outside PHP.Benchmark
Fresh release builds from
master(a732d1203) and this branch (010c2f717), using six balanced 60-second runs per binary on macOS ARM64:The PHP 8.3 NTS control remains neutral while PHP 8.3 ZTS improves by 14.42%, indicating that the legacy-path gain primarily comes from eliminating repeated TSRM-backed profiler-global lookups. PHP 8.5 NTS also improves because RINIT-selected callbacks remove previous-handler dispatch from free and realloc. Fresh 70-second native samples reduced selected Rust malloc/free forwarding top-of-stack samples by 37% on PHP 8.3 ZTS and 38% on PHP 8.5 NTS.
Validation: release builds, 22 Rust tests on PHP 8.3/8.5, five allocation PHPTs on PHP 8.3/8.5, PHP 7.3 ZTS Docker validation, and
cargo test --all-featureson PHP 8.5 NTS Linux/macOS (24 passed).Reviewer checklist
https://datadoghq.atlassian.net/browse/PROF-15506