This is a Python 3.6 module to profile function performance. It records the number of calls made to select functions, and outputs statistics about their execution times.
There are two ways to install this package from the terminal:
Directly from PyPI (recommended):
pip3 install function-profiler
git clone https://github.com/Datamine/Function-Profiler
sudo python3 setup.py install
In either of these ways, it may be acceptable to use
python instead of
python3 respectively, depending on your local configuration.
Explanation, Basic Use
The profiler has three components:
function_profiler, a decorator for functions to have their calls logged
FunctionLogger, a class that acts as a context manager for the decorated functions, and which stores (as class variables) the number of calls to each decorated function, as well as their execution time lengths. It also has a class method,
FunctionLogger.log_data, which outputs the logged call/time data (to
with_logger, a decorator that calls
profiler.FunctionLogger.log_dataafter the function it wraps has exited (whether by normal return or by exception). This is useful if, for example, you want to output logs just once, after your
mainfunction has exited. (Or if you want to output logs every time some particular function exits, etc.)
import profiler @profiler.function_profiler() def foo(): return foo() foo() foo() profiler.FunctionLogger.log_data('stdout')
foo: 3 calls. Time stats (seconds): Min: 0.000001, Mean: 0.000002, Median: 0.000002, Max: 0.000003, Stddev: 0.000001
Suppose we have a file,
import profiler @profiler.function_profiler() def foo(): return @profiler.with_logger() def main(): foo() if __name__=='__main__': main()
and we run this from the command line, i.e.
python3 example.py. We get this output at the command-line:
foo: 1 call. Time stats (seconds): Min: 0.000001, Mean: 0.000002, Median: 0.000002, Max: 0.000003, Stddev: 0.000001
profiler.FunctionLogger.log_data takes one of three options:
'stdout'if you wish the output to be logged to stdout
'stderr'if you wish the output to be logged to stderr
'suppress'if you wish the output not to be logged
- any other string, e.g.
'myfile.txt'will cause the output to be logged to a file of that name.
with_logger also accepts
the same set of three options.
If you don't want the summary statistics but rather more granular ones, you can access
profiler.FunctionLogger's class variables directly. There are two:
profiler.FunctionLogger.call_frequenciesis a dict mapping each function to the number of times it has been called.
profiler.FunctionLogger.call_timesis a dict mapping each function to a list of how long it took to complete each function call.
profiler.function_profiler accepts one argument: a naming convention, either
'qualname' (default) or
'name'. This is because
data on functions by using their names as keys. By default,
it uses the functions' fully qualified names in order to prevent name collisions, but by
'name' instead, it'll just use the function's local name.
Tests and Examples
It is possible to run tests for
profiler before installing the module.
From the top-level directory, simply run
python3 run-tests.py. (In order to run
the tests without installing the package, the tests use
then try to locate the
profiler module using its relative path, which assumes that
the tests are being run from the top-level directory.)
Examples are located in the
examples/ directory, and should similarly be run from
the top-level directory, e.g.
python3 examples/basic-profiler.py. If you're uncertain
about using this library, then hopefully the examples will be useful references.
profiler.FunctionLogger acts as a context manager on a function call. Consequently,
a function that does not exit will not have its call time logged. Common cases in which you
might face this error include interrupting the program before a function call finishes, or
writing a function that exceeds the maximum recursion depth (so functions are repeatedly entered, but never exited before erring).
This package is intended for Python 3. It doesn't currently work for Python 2, though it should be easy to achieve that compatibility if you need it. I am firmly moving on to Python 3.6+, consequently I will not be writing any code to ensure backwards compatibility. However, I am willing to bless Python 2.7-compatible forks, should they appear.
This package lives here on PyPI.
There's also an entry about this project on my blog.