perf(profiling): reuse TSRM cache in allocation hook - #4072
perf(profiling): reuse TSRM cache in allocation hook#4072realFlowControl wants to merge 38 commits into
Conversation
Read executor_globals.current_execute_data directly on NTS builds instead of calling through the C FFI wrapper for every allocation and reallocation. Keep the existing wrapper on ZTS builds. A 60-second macOS sample reduced get_current_execute_data self time from 0.412% (207/50,210 main-thread samples) to 0.002% (1/49,737). Known allocation-hook self time fell from 4.184% to 3.601%, a 13.9% relative reduction. Repeated end-to-end throughput remained within system scheduling noise. Validation: cargo test (22 passed); allocation sampling-distance, memory-peak, and GC PHPTs passed.
Forward PHP 8.4+ allocations and frees from a single ZendMMState read instead of loading a function pointer, making an indirect Rust call, and reloading the state. Preserve neighboring custom allocator support with a predictable previous-handler check. Use unchecked heap extraction in the callbacks: rinit stores the heap before they can be invoked, and rshutdown removes them before clearing it. This avoids an Option discriminant check on every allocation and free. Across six 60-second runs per binary, mean allocation throughput increased from 9,366,444/s to 10,018,262/s (+6.96%) and median throughput increased by 7.56%. A 60-second native sample reduced known forwarding-path self time from 3.599% to 2.438% (-32.3% relative). Validation: cargo test (22 passed); allocation sampling-distance, memory-peak, and GC PHPTs passed. https://datadoghq.atlassian.net/browse/PROF-15506
Apply the single-state allocation and free forwarding path to PHP 8.3 and older. Preserve the required ZendMM prepare/restore calls and neighboring custom allocator support while removing the intermediate function pointers and wrappers. On ZTS this also avoids repeated TSRM lookups. Across six balanced 60-second runs per binary on PHP 8.3 ZTS, mean allocation throughput increased from 28,079,883/s to 32,661,234/s (+16.32%) and median throughput increased by 16.65%. A full 60-second native sample reduced known forwarding-path self time from 16.31% to 12.30% (-24.6% relative). Validation: PHP 8.3 ZTS cargo test (22 passed); allocation sampling-distance, memory-peak, and GC PHPTs passed; PHP 8.5 release build and cargo test (22 passed). https://datadoghq.atlassian.net/browse/PROF-15506
Reuse one ZTS local-storage cache pointer to access both profiler globals and executor globals in the PHP 8.3-and-older allocation callback. This removes one TSRM lookup and the C current_execute_data wrapper per allocation while preserving the post-allocation executor-global read. Across six balanced 60-second runs per binary on PHP 8.3 ZTS, mean allocation throughput increased from 32,677,620/s to 34,432,538/s (+5.37%) and median throughput increased by 5.54%. Full native samples reduced selected TLS and wrapper self time from 17.74% to 12.95% (-27.0% relative). Validation: PHP 8.3 ZTS cargo test (22 passed); allocation sampling-distance, memory-peak, and GC PHPTs passed; PHP 8.5 release build and cargo test (22 passed). https://datadoghq.atlassian.net/browse/PROF-15506
The free-handler selection test retains allocation callbacks that now call _zend_mm_free directly, but Rust unit-test binaries are not loaded by PHP. Provide a test-only stub with release and debug PHP signatures so both ZendMM API implementations link. Verified with cargo test on PHP 8.5 NTS and PHP 8.3 ZTS; both pass 22 tests.
The free-handler selection test retains allocation callbacks that now call _zend_mm_free directly, but Rust unit-test binaries are not loaded by PHP. Provide a test-only stub with release and debug PHP signatures so both ZendMM API implementations link. Verified with cargo test on PHP 8.5 NTS and PHP 8.3 ZTS; both pass 22 tests.
|
Benchmarks [ profiler ]Benchmark execution time: 2026-07-31 17:52:16 Comparing candidate commit 616427c in PR branch Found 0 performance improvements and 2 performance regressions! Performance is the same for 25 metrics, 9 unstable metrics.
|
Benchmarks [ tracer ]Benchmark execution time: 2026-07-28 19:26:33 Comparing candidate commit 85850cc in PR branch Found 2 performance improvements and 4 performance regressions! Performance is the same for 187 metrics, 1 unstable metrics.
|
Forward reallocations from one ZendMMState read instead of loading a function pointer, making an indirect Rust call, and reloading state. Preserve neighboring custom allocator support and the PHP 8.3-and-older heap prepare/restore calls. Remove the realloc forwarding field, intermediate wrappers, and initialization panic callback from both ZendMM implementations. Validation: PHP 8.5 NTS and PHP 8.3 ZTS cargo test (22 passed each); five allocation PHPTs passed on both builds. https://datadoghq.atlassian.net/browse/PROF-15506
Forward reallocations from one ZendMMState read instead of loading a function pointer, making an indirect Rust call, and reloading state. Preserve neighboring custom allocator support and the PHP 8.3-and-older heap prepare/restore calls. Remove the realloc forwarding field, intermediate wrappers, and initialization panic callback from both ZendMM implementations. Validation: PHP 8.5 NTS and PHP 8.3 ZTS cargo test (22 passed each); five allocation PHPTs passed on both builds. https://datadoghq.atlassian.net/browse/PROF-15506
PHP 7.1 through 7.3 expose executor globals through executor_globals_id instead of the fast offset introduced in PHP 7.4. Reuse the cached TSRM storage through the resource-ID path on those versions and keep the offset path for PHP 7.4 and newer. Provide a test-only tsrm_get_ls_cache stub because Rust unit-test binaries are not loaded by PHP. Validation: PHP 7.3 ZTS cargo test (22 passed), release build, and allocation smoke test; PHP 8.3 ZTS and PHP 8.5 NTS cargo test (22 passed each). https://datadoghq.atlassian.net/browse/PROF-15506
realFlowControl
added
AI Generated
…lorian/prof-15506-reuse-tsrm-cache
On PHP 8.3 and older, the execute_internal hook called the full profiler interrupt handler after every internal function. Check EG(vm_interrupt) first so the TLS, request-state, and atomic work only runs when PHP has a pending VM interrupt. Across six balanced 60-second runs per binary on PHP 8.3 ZTS, mean allocation-loop throughput increased from 34,160,664/s to 36,061,991/s (+5.57%) and median throughput increased by 5.92%. Native samples reduced the manual interrupt path from 14.04% to 8.62% of main-thread samples. Wall-time correctness remained unchanged: a four-second CPU/sleep workload attributed 50.23% to sleep before and 50.31% after. Validation: PHP 7.3 ZTS cargo check; PHP 8.3 ZTS and PHP 8.5 NTS cargo test (22 passed each). https://datadoghq.atlassian.net/browse/PROF-15506
On PHP 8.3 and older, select the direct ZendMM or neighboring custom allocator callback once during rinit. The normal allocation callback no longer loads or branches on prev_custom_mm_alloc for every allocation; the custom allocator callback remains as a documented cold compatibility path. Across six balanced 60-second PHP 8.3 ZTS runs per binary, mean throughput increased from 31,167,496/s to 32,646,366/s (+4.74%) and median throughput increased by 4.56%. Native samples reduced alloc_prof_malloc self time from 6.13% to 3.54% (-42.3% relative). The equivalent PHP 8.5 NTS experiment measured within system noise (+0.79% mean / +0.39% median) and was not retained. Validation: PHP 8.5 NTS cargo test (22 passed), PHP 8.3 ZTS and PHP 7.3 ZTS cargo check, and five PHP 8.3 allocation PHPTs. https://datadoghq.atlassian.net/browse/PROF-15506
…lorian/prof-15506-reuse-tsrm-cache # Conflicts: # profiling/src/allocation/allocation_le83.rs
Apply the same RINIT-selected allocation callback used for PHP 8.3 and older to PHP 8.4 and newer. The normal callback has no previous-allocator load or branch, while a separate cold callback preserves neighboring custom allocator support. The PHP 8.5 NTS benchmark was inconclusive but non-negative: a clean six-run retry measured +0.40% mean and +0.69% median with about 1.8% run variance. Keep the implementation for symmetry across ZendMM APIs rather than as a claimed performance win. Validation: PHP 8.5 NTS cargo test (22 passed) and profiler-release build. https://datadoghq.atlassian.net/browse/PROF-15506
…lorian/prof-15506-reuse-tsrm-cache
…faster-allocation-forwarding
…lorian/prof-15506-reuse-tsrm-cache
…lorian/prof-15506-reuse-tsrm-cache
…reuse-tsrm-cache # Conflicts: # profiling/src/allocation/allocation_le83.rs
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: ce7adb22ad
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
This reverts commit be25644.
This reverts commit 02acd65.
…guard-internal-interrupt
This is very similar to the approach taken by Florian, which did a
check for EG(vm_interrupt). Although I was able to reproduce a small
speedup there, I realized that it's more nuanced than that:
1. This doesn't apply to actual VM interrupts, only to things which
need to check for a pending interrupt.
2. EG(vm_interrupt) technically isn't related to the thing we care
about, which is the interrupt count.
So this adds ddog_php_prof_interrupt_function_unlikely which is the
same as ddog_php_prof_interrupt_function at a high level, but it is
optimized to assume that there isn't a pending interrupt (opposite of
ddog_php_prof_interrupt_function).
And use system settings from the global, rather than through the REQUEST_LOCALS.
This reverts commit 32abe92.
This reverts commit 8cb3cde.
These changes specifically speed up the hot paths of the interrupt
function. Notably, this can be a bit hot on 8.3 and below because of
execute_internal, and it's used on frameless functions on 8.4+. The
biggest change comes from avoiding REQUEST_LOCALS. In this case, this
is totally safe because interrupt_count is atomic.
This also removes the profiling_enabled check of the interrupt
function. This is unnecessary because:
1. We do not set interrupts when disabled to begin with, generally.
trigger_time_sample can still trigger but this is test only.
2. For edge cases having a pending interrupt going into a fork, the
child will call `Profiler::kill()` which will cause
`Profiler::get()` to return None.
…guard-internal-interrupt
…che' into florian/prof-15506-reuse-tsrm-cache
morrisonlevi
left a comment
morrisonlevi
left a comment
There was a problem hiding this comment.
I ported the changes to 8.4+ plus as well. I did a tiny bit of refactoring but not too much, will leave the rest of the refactoring another PR.
2 participants
Description
Reuse one ZTS local-storage cache pointer to access both profiler globals and executor globals in the PHP 8.3-and-older allocation callback. This removes one
tsrm_get_ls_cache()call and the Ccurrent_execute_datawrapper per allocation while preserving the post-allocation executor-global read.PHP 7.1 through 7.3 access executor globals through
executor_globals_id; PHP 7.4 and newer use the fastexecutor_globals_offset. Both paths reuse the same cached TSRM storage pointer in either RINIT-selected normal or neighboring custom malloc callback.This PR is stacked on #4070 and contains only the third allocation-profiler optimization.
Across six balanced 60-second runs per binary on PHP 8.3 ZTS:
Validation after syncing #4070's malloc/free/realloc callback selection:
cargo test: 22 passedcargo test: 22 passedcargo test: 22 passed;cargo test --all-featureson Linux/macOS: 24 passedReviewer checklist
https://datadoghq.atlassian.net/browse/PROF-15506