Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
101 changes: 98 additions & 3 deletions Doc/library/profiling.sampling.rst
Original file line number Diff line number Diff line change
Expand Up @@ -200,6 +200,36 @@ On most systems, attaching to another process requires appropriate permissions.
See :ref:`profiling-permissions` for platform-specific requirements.


.. _replay-command:

The ``replay`` command
----------------------

The ``replay`` command converts binary profile files to other output formats::

python -m profiling.sampling replay profile.bin
python -m profiling.sampling replay --flamegraph -o profile.html profile.bin

This command is useful when you have captured profiling data in binary format
and want to analyze it later or convert it to a visualization format. Binary
profiles can be replayed multiple times to different formats without
re-profiling.

::

# Convert binary to pstats (default, prints to stdout)
python -m profiling.sampling replay profile.bin

# Convert binary to flame graph
python -m profiling.sampling replay --flamegraph -o output.html profile.bin

# Convert binary to gecko format for Firefox Profiler
python -m profiling.sampling replay --gecko -o profile.json profile.bin

# Convert binary to heatmap
python -m profiling.sampling replay --heatmap -o my_heatmap profile.bin


Profiling in production
-----------------------

Expand Down Expand Up @@ -1041,6 +1071,57 @@ intuitive view that shows exactly where time is spent without requiring
interpretation of hierarchical visualizations.


Binary format
-------------

Binary format (:option:`--binary`) produces a compact binary file for efficient
storage of profiling data::

python -m profiling.sampling run --binary -o profile.bin script.py
python -m profiling.sampling attach --binary -o profile.bin 12345

The :option:`--compression` option controls data compression:

- ``auto`` (default): Use zstd compression if available, otherwise no
compression
- ``zstd``: Force zstd compression (requires zstd support)
- ``none``: Disable compression

::

python -m profiling.sampling run --binary --compression=zstd -o profile.bin script.py

To analyze binary profiles, use the :ref:`replay-command` to convert them to
other formats like flame graphs or pstats output.


Record and replay workflow
==========================

The binary format combined with the replay command enables a record-and-replay
workflow that separates data capture from analysis. Rather than generating
visualizations during profiling, you capture raw data to a compact binary file
and convert it to different formats later.

This approach has three main benefits. First, sampling runs faster because the
work of building data structures for visualization is deferred until replay.
Second, a single binary capture can be converted to multiple output formats
without re-profiling---pstats for a quick overview, flame graph for visual
exploration, heatmap for line-level detail. Third, binary files are compact
and easy to share with colleagues who can convert them to their preferred
format.

A typical workflow::

# Capture profile in production or during tests
python -m profiling.sampling attach --binary -o profile.bin 12345

# Later, analyze with different formats
python -m profiling.sampling replay profile.bin
python -m profiling.sampling replay --flamegraph -o profile.html profile.bin
python -m profiling.sampling replay --heatmap -o heatmap profile.bin


Live mode
=========

Expand Down Expand Up @@ -1252,6 +1333,10 @@ Global options

Attach to and profile a running process by PID.

.. option:: replay

Convert a binary profile file to another output format.


Sampling options
----------------
Expand Down Expand Up @@ -1335,12 +1420,22 @@ Output options

Generate HTML heatmap with line-level sample counts.

.. option:: --binary

Generate high-performance binary format for later conversion with the
``replay`` command.

.. option:: --compression <type>

Compression for binary format: ``auto`` (use zstd if available, default),
``zstd``, or ``none``.

.. option:: -o <path>, --output <path>

Output file or directory path. Default behavior varies by format:
``--pstats`` writes to stdout, ``--flamegraph`` and ``--gecko`` generate
files like ``flamegraph.PID.html``, and ``--heatmap`` creates a directory
named ``heatmap_PID``.
``--pstats`` writes to stdout, ``--flamegraph``, ``--gecko``, and
``--binary`` generate files like ``flamegraph.PID.html``, and ``--heatmap``
creates a directory named ``heatmap_PID``.


pstats display options
Expand Down
9 changes: 9 additions & 0 deletions Include/internal/pycore_global_objects_fini_generated.h

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

9 changes: 9 additions & 0 deletions Include/internal/pycore_global_strings.h
Original file line number Diff line number Diff line change
Expand Up @@ -376,9 +376,11 @@ struct _Py_global_strings {
STRUCT_FOR_ID(co_varnames)
STRUCT_FOR_ID(code)
STRUCT_FOR_ID(col_offset)
STRUCT_FOR_ID(collector)
STRUCT_FOR_ID(command)
STRUCT_FOR_ID(comment_factory)
STRUCT_FOR_ID(compile_mode)
STRUCT_FOR_ID(compression)
STRUCT_FOR_ID(config)
STRUCT_FOR_ID(consts)
STRUCT_FOR_ID(context)
Expand Down Expand Up @@ -441,7 +443,9 @@ struct _Py_global_strings {
STRUCT_FOR_ID(event)
STRUCT_FOR_ID(eventmask)
STRUCT_FOR_ID(exc)
STRUCT_FOR_ID(exc_tb)
STRUCT_FOR_ID(exc_type)
STRUCT_FOR_ID(exc_val)
STRUCT_FOR_ID(exc_value)
STRUCT_FOR_ID(excepthook)
STRUCT_FOR_ID(exception)
Expand Down Expand Up @@ -697,6 +701,7 @@ struct _Py_global_strings {
STRUCT_FOR_ID(print_file_and_line)
STRUCT_FOR_ID(priority)
STRUCT_FOR_ID(progress)
STRUCT_FOR_ID(progress_callback)
STRUCT_FOR_ID(progress_routine)
STRUCT_FOR_ID(proto)
STRUCT_FOR_ID(protocol)
Expand Down Expand Up @@ -737,6 +742,7 @@ struct _Py_global_strings {
STRUCT_FOR_ID(reversed)
STRUCT_FOR_ID(rounding)
STRUCT_FOR_ID(salt)
STRUCT_FOR_ID(sample_interval_us)
STRUCT_FOR_ID(sched_priority)
STRUCT_FOR_ID(scheduler)
STRUCT_FOR_ID(script)
Expand Down Expand Up @@ -776,8 +782,10 @@ struct _Py_global_strings {
STRUCT_FOR_ID(spam)
STRUCT_FOR_ID(src)
STRUCT_FOR_ID(src_dir_fd)
STRUCT_FOR_ID(stack_frames)
STRUCT_FOR_ID(stacklevel)
STRUCT_FOR_ID(start)
STRUCT_FOR_ID(start_time_us)
STRUCT_FOR_ID(statement)
STRUCT_FOR_ID(stats)
STRUCT_FOR_ID(status)
Expand Down Expand Up @@ -818,6 +826,7 @@ struct _Py_global_strings {
STRUCT_FOR_ID(times)
STRUCT_FOR_ID(timespec)
STRUCT_FOR_ID(timestamp)
STRUCT_FOR_ID(timestamp_us)
STRUCT_FOR_ID(timetuple)
STRUCT_FOR_ID(timeunit)
STRUCT_FOR_ID(top)
Expand Down
9 changes: 9 additions & 0 deletions Include/internal/pycore_runtime_init_generated.h

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

36 changes: 36 additions & 0 deletions Include/internal/pycore_unicodeobject_generated.h

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading
Loading