Skip to content

Docs: Polish profile documentation #110494

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 7 commits into
base: main
Choose a base branch
from
Open
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
40 changes: 22 additions & 18 deletions Doc/library/profile.rst
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,7 @@ This section is provided for users that "don't want to read the manual." It
provides a very brief overview, and allows a user to rapidly perform profiling
on an existing application.

To profile a function that takes a single argument, you can do::
To profile a single statement, you can do::

import cProfile
import re
Expand Down Expand Up @@ -121,6 +121,14 @@ results to a file by specifying a filename to the :func:`run` function::
The :class:`pstats.Stats` class reads profile results from a file and formats
them in various ways.

You can also do inline profiling with :class:`cProfile.Profile <profile.Profile>`::

import cProfile
import re
with cProfile.Profile() as pr:
re.compile("foo|bar")
pr.print_stats()

.. _profile-cli:

The files :mod:`cProfile` and :mod:`profile` can also be invoked as a script to
Expand All @@ -141,6 +149,10 @@ the output by. This only applies when ``-o`` is not supplied.
.. versionadded:: 3.8
Added the ``-m`` option to :mod:`profile`.

Note that if the process is terminated abruptly (for example, by
:data:`~signal.SIGTERM` or :data:`~signal.SIGKILL`) during profiling,
no results will be generated/printed.

The :mod:`pstats` module's :class:`~pstats.Stats` class has a variety of methods
for manipulating and printing the data saved into a profile results file::

Expand Down Expand Up @@ -316,11 +328,6 @@ functions:

Profile ``func(*args, **kwargs)``

Note that profiling will only work if the called command/function actually
returns. If the interpreter is terminated (e.g. via a :func:`sys.exit` call
during the called command/function execution) no profiling results will be
printed.

.. _profile-stats:

The :class:`Stats` Class
Expand Down Expand Up @@ -577,23 +584,17 @@ implementations.
Limitations
===========

One limitation has to do with accuracy of timing information. There is a
fundamental problem with deterministic profilers involving accuracy. The most
obvious restriction is that the underlying "clock" is only ticking at a rate
(typically) of about .001 seconds. Hence no measurements will be more accurate
than the underlying clock. If enough measurements are taken, then the "error"
will tend to average out. Unfortunately, removing this first error induces a
second source of error.
Comment on lines -580 to -586
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Personally, I like this paragraph, and it still contains meaningful information. I'd just update it to leave out specifics about resolution.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure. But in reality, the clock resolution should not be an issue (or significant issue compared to other source) anymore. If we list this as limitation, there are a LOT of factors that we could potentially write here.


The second problem is that it "takes a while" from when an event is dispatched
It "takes a while" from when an event is dispatched
until the profiler's call to get the time actually *gets* the state of the
clock. Similarly, there is a certain lag when exiting the profiler event
handler from the time that the clock's value was obtained (and then squirreled
away), until the user's code is once again executing. As a result, functions
that are called many times, or call many functions, will typically accumulate
this error. The error that accumulates in this fashion is typically less than
the accuracy of the clock (less than one clock tick), but it *can* accumulate
and become very significant.
Comment on lines -594 to -596
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd leave this?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The sentence the error that accumulates in this fashion is typically less than the accuracy of the clock (less than one clock tick) is incorrect now, as the accuracy of the clock has improved a lot for the last .. I don't know how many years. This is definitely more significant than the clock accuracy issue. Do you want me to keep the "become very significant"?

this error.

Also, the logic in the profiler event handler takes time to execute, which
will introduce skew into the results, making the functions slower than they
would be if the profiler was not running.

The problem is more important with :mod:`profile` than with the lower-overhead
:mod:`cProfile`. For this reason, :mod:`profile` provides a means of
Expand All @@ -605,6 +606,9 @@ probability work against you :-). ) Do *not* be alarmed by negative numbers in
the profile. They should *only* appear if you have calibrated your profiler,
and the results are actually better than without calibration.

To avoid infinite recursion, :mod:`profile` and :mod:`cProfile` do not work
inside trace functions set using :func:`sys.settrace` or :func:`sys.setprofile`.


.. _profile-calibration:

Expand Down