Introduction

Reli has a memory profiling mode, and it can be used with the command like below

reli inspector:memory -p <pid_of_target_process>

You can use this mode to analyze the memory usage of the target process, for finding out memory bottlenecks or memory leaks.

For example, you can see statistics such as whether strings, arrays or objects are particularly dominant in a script's memory usage, or contextual information such as where certain local variables in a given call frame are referenced from elsewhere, and the actual values held by certain memory areas.

The functionality of this mode is similar to php-meminfo, but works in the phpspy-ish way.

It captures the memory contents of the target from outside the process, and analyzes it with the knowledge of the internal structures of the PHP VM, then dumps them all. So target programs don't need any modifications, don't need to load a specific extension for this.

Requirements

FFI and PCNTL
PHP 8.1 or above for execution
PHP 7.0+ for targets
Only NTS targets are tested currently
The capability to stab the target process with ptrace (CAP_SYS_PTRACE)
- Usually running as root is enough

Installation

From Composer

composer create-project reliforp/reli-prof
cd reli-prof
./reli

From Git

git clone git@github.com:reliforp/reli-prof.git
cd reli-prof
composer install
./reli

From Docker

docker pull reliforp/reli-prof
docker run -it --security-opt="apparmor=unconfined" --cap-add=SYS_PTRACE --pid=host reliforp/reli-prof

Options

./reli inspector:memory --help
Description:
  [experimental] get memory usage from an outer process

Usage:
  inspector:memory [options] [--] [<cmd> [<args>...]]

Arguments:
  cmd                                                                command to execute as a target: either pid (via -p/--pid) or cmd must be specified
  args                                                               command line arguments for cmd

Options:
      --stop-process|--no-stop-process                               stop the process while inspecting (default: on)
      --pretty-print|--no-pretty-print                               pretty print the result (default: off)
      --memory-limit-error-file=MEMORY-LIMIT-ERROR-FILE              file path where memory_limit is exceeded
      --memory-limit-error-line=MEMORY-LIMIT-ERROR-LINE              line number where memory_limit is exceeded
      --memory-limit-error-max-depth[=MEMORY-LIMIT-ERROR-MAX-DEPTH]  max attempts to trace back the VM stack on memory_limit error [default: 512]
  -p, --pid=PID                                                      process id
      --php-regex[=PHP-REGEX]                                        regex to find the php binary loaded in the target process
      --libpthread-regex[=LIBPTHREAD-REGEX]                          regex to find the libpthread.so loaded in the target process
      --php-version[=PHP-VERSION]                                    php version (auto|v7[0-4]|v8[01]) of the target (default: auto)
      --php-path[=PHP-PATH]                                          path to the php binary (only needed in tracing chrooted ZTS target)
      --libpthread-path[=LIBPTHREAD-PATH]                            path to the libpthread.so (only needed in tracing chrooted ZTS target)
  -h, --help                                                         Display help for the given command. When no command is given display help for the list command
  -q, --quiet                                                        Do not output any message
  -V, --version                                                      Display this application version
      --ansi|--no-ansi                                               Force (or disable --no-ansi) ANSI output
  -n, --no-interaction                                               Do not ask any interactive question
  -v|vv|vvv, --verbose                                               Increase the verbosity of messages: 1 for normal output, 2 for more verbose output and 3 for debug

Usage

Caution

Don't upload the output of this command to the internet, because it can contain sensitive information of the target script!!!

Warning

This feature is in an experimental stage and may be less stable than others. The contents of the output may change in the near future.

This tool can be used like this.

sudo ./reli i:m --pretty-print -p <pid_of_target_process> >memory_analyzed.json

The analysis takes seconds in many case. During the analysis, the target process is stopped by default.

And the output is like below. The target process is psalm in this example.

{
    "summary": [
        {
            "zend_mm_heap_total": 153092096,
            "zend_mm_heap_usage": 126947784,
            "zend_mm_chunk_total": 153092096,
            "zend_mm_chunk_usage": 126947784,
            "zend_mm_huge_total": 0,
            "zend_mm_huge_usage": 0,
            "vm_stack_total": 262144,
            "vm_stack_usage": 8720,
            "compiler_arena_total": 1900544,
            "compiler_arena_usage": 1791992,
            "possible_allocation_overhead_total": 10196000,
            "possible_array_overhead_total": 21284000,
            "memory_get_usage": 129053816,
            "memory_get_real_usage": 153092096,
            "cached_chunks_size": 0,
            "heap_memory_analyzed_percentage": 98.36809784842008,
            "php_version": "v82",
            "analyzer": "reli 0.9.0"
        }
    ],
    "location_types_summary": {
        "ZendObjectMemoryLocation": {
            "location_count": 124341,
            "memory_usage": 45245784
        },
        "ZendArrayTableOverheadMemoryLocation": {
            "location_count": 108548,
            "memory_usage": 21204736
        },
        "ZendArrayTableMemoryLocation": {
            "location_count": 109462,
            "memory_usage": 17964240
        },
        "ZendStringMemoryLocation": {
            "location_count": 242532,
            "memory_usage": 16149456
        },
<--- snip --->
    "class_objects_summary": {
        "Psalm\\CodeLocation": {
            "count": 32875,
            "total_size": 13413000
        },
        "Psalm\\Type\\Union": {
            "count": 24924,
            "total_size": 12960480
        },
        "Psalm\\Internal\\MethodIdentifier": {
            "count": 8838,
            "total_size": 636336
        },
<--- snip --->
    "context": {
        "call_frames": {
            "#node_id": 0,
            "#type": "CallFramesContext",
            "#count": 4,
            "0": {
                "#node_id": 1,
                "#type": "CallFrameContext",
                "function_name": "Psalm\\IssueBuffer::finish",
                "local_variables": {
                    "#node_id": 2,
                    "#type": "CallFrameVariableTableContext",
                    "project_analyzer": {
                        "#node_id": 3,
                        "#type": "ObjectContext",
                        "#locations": [
                            {
                                "address": 139957204265152,
                                "size": 424,
                                "refcount": 3,
                                "type_info": 3221319688,
                                "class_name": "Psalm\\Internal\\Analyzer\\ProjectAnalyzer"
                            }
                        ],
                        "object_handlers": {
                            "#node_id": 4,
                            "#type": "ObjectHandlersContext",
                            "#locations": [
                                {
                                    "address": 93902816133472,
                                    "size": 200
                                }
                            ]
                        },
                        "object_properties": {
                            "#node_id": 5,
                            "#type": "ObjectPropertiesContext",
                            "#count": 24
                            "config": {
                                "#node_id": 6,
                                "#type": "ObjectContext",
                                "#locations": [
                                    {
                                        "address": 139957201996800,
                                        "size": 1544,
                                        "refcount": 7,
                                        "type_info": 3221239816,
                                        "class_name": "Psalm\\Config"
                                    }
                                ],
                                "object_handlers": {
                                    "#reference_node_id": 4
                                },
<--- snip --->
...

A quick tour of each part of the output

ZendMM allocates 153,092,096 byte of chunks in total
126,947,784 bytes of them are located by this tool
If the target process calls memory_get_usage(false) at this moment, the return value would be 129,053,816
So 98.36809784842008% of areas reported by memory_get_usage() is analyzed
The kind of the top most memory area is the locations for zend_object, which corresponding to the PHP objects in the target process
- The number of objects is 124,341, and the total analyzed size of them is 45,245,784 bytes
The class with the most number of instances is Psalm\CodeLocation
- The number of instances is 32,875, and the total analyzed size of them is 13,413,000 bytes
The "context" field can be used to find out which areas in the script are used in what context. For example, the executing function at this moment is Psalm\IssueBuffer::finish(). Its local variables contain $project_analyzer, which is an instance of Psalm\Internal\Analyzer\ProjectAnalyzer. And you can extract all references to the same instance with jq.

$ cat memory_analized.json | jq 'path(..|objects|select(."#reference_node_id"==3 or ."#node_id"==3))|join(".")'
"context.call_frames.0.local_variables.project_analyzer"
"context.call_frames.1.local_variables.project_analyzer"
"context.class_table.psalm\\internal\\analyzer\\projectanalyzer.static_properties.instance"
"context.objects_store.17"

Useful snippets for analyzing the output