Pyinstrument is a Python profiler. A profiler is a tool to help you optimize your code - make it faster. To get the biggest speed increase you should focus on the slowest part of your program. Pyinstrument helps you find it!
☕️Not sure where to start? Check out this video tutorial from calmcode.io!
pip install pyinstrument
Pyinstrument supports Python 3.7+.
To run Pyinstrument from a git checkout, there's a build step. Take a look at Contributing for more info.
To learn how to use pyinstrument, or to check the reference, head to the documentation.
- Profiling code inside a Docker container can cause some strange results, because the gettimeofday syscall that pyinstrument uses is slow in that environment. See #83
- When using
script.pycontains a class serialized with
pickle, you might encounter errors because the serialisation machinery doesn't know where
__main__is. See this issue for workarounds
5 November 2022
- Adds the class name to methods in the console & HTML outputs (#203)
- Fix a bug that caused pyinstrument machinery to appear at the start of a profile (#215)
- Frames that set a
__traceback_hide__local variable will now be removed from the output (#217)
- Jupyter/IPython magic now supports async/await, if you run with a
- Fix a crash when more than one root frame is captured in a thread - this can happen with gevent.
- A big refactor to the backend, allowing more than just static information to be captured. This currently is just powering the class name feature, but more is to come!
21 August 2022
- Adds buttons in the HTML output to switch between absolute and proportional (percentage) time.
- Adds a command line flag
--interval(seconds, default 0.001) to change the interval that pyinstrument samples a program. This is useful for long-running programs, where increasing the interval reduces the memory overhead.
- Includes wheels for CPython 3.11.
Adds a command-line option
--render-optionthat allows arbitrary setting of render options. This lets you set options like
filter_thresholdfrom the command line, by doing something like
pyinstrument -p processor_options.filter_threshold=0.
Here's the help output for the option:
-p RENDER_OPTION, --render-option=RENDER_OPTION options to pass to the renderer, in the format 'flag_name' or 'option_name=option_value'. For example, to set the option 'time', pass '-p time=percent_of_total'. To pass multiple options, use the -p option multiple times. You can set processor options using dot-syntax, like '-p processor_options.filter_threshold=0'. option_value is parsed as a JSON value or a string.
Adds the ability to view times in the console output as percentages, rather than absolute times. Use the ConsoleRenderer option
time='percent_of_total', or on the command line, use
pyinstrument -p time=percent_of_total.
Adds command line options for loading and saving pyinstrument sessions. You can save the raw data for a pyinstrument session with
-r session, like
pyinstrument -r session -o session.pyisession myscript.py. Loading is via
pyinstrument --load session.pyisession.
Command line output format is inferred from the
-ooutput file extension. So if you do
pyinstrument -o profile.html myscript.py, you don't need to supply
-r html, pyinstrument will automatically use the HTML renderer. Or if you do
pyinstrument -o profile.pyisession myscript.py, it will save a raw session object.
Adds usage examples for FastAPI and pytest to the documentation.
Fixes a bug causing NotImplementedError when using
Adds support for Python 3.11
- Fixed an issue causing PYINSTRUMENT_PROFILE_DIR_RENDERER to output the wrong file extension when used with the speedscope renderer.
- You can now use pyinstrument natively in an IPython notebook! Just use
%load_ext pyinstrumentat the top of your notebook, and then
%%pyinstrumentin the cell you want to profile.
- Added support for the speedscope format.
This provides a way to view interactive flamecharts using pyinstrument. To
use, profile with
pyinstrument -r speedscope, and upload to the speedscope web app.
- You can now configure renderers for the Django middleware file output,
- Added wheels for Linux aarch64 (64-bit ARM).
- Fix a packaging issue where a package called 'test' was installed alongside pyinstrument
- Use more modern C APIs to resolve deprecation warnings on Python 3.10.
- Minor docs fixes
- CPython 3.10 support
- Improve error messages when trying to use Profiler from multiple threads
- Fix crash when rendering sessions that contain a module in a FrameGroup
- Fix some packaging issues
Async support! Pyinstrument now detects when an async task hits an await, and tracks time spent outside of the async context under this await.
So, for example, here's a simple script with an async task that does a sleep:
import asyncio from pyinstrument import Profiler async def main(): p = Profiler(async_mode='disabled') with p: print('Hello ...') await asyncio.sleep(1) print('... World!') p.print() asyncio.run(main())
Before Pyinstrument 4.0.0, we'd see only time spent in the run loop, like this:
_ ._ __/__ _ _ _ _ _/_ Recorded: 18:33:03 Samples: 2 /_//_/// /_\ / //_// / //_'/ // Duration: 1.006 CPU time: 0.001 / _/ v3.4.2 Program: examples/async_example_simple.py 1.006 _run_once asyncio/base_events.py:1784 └─ 1.005 select selectors.py:553 [3 frames hidden] selectors, <built-in> 1.005 kqueue.control <built-in>:0
Now, with pyinstrument 4.0.0, we get:
_ ._ __/__ _ _ _ _ _/_ Recorded: 18:30:43 Samples: 2 /_//_/// /_\ / //_// / //_'/ // Duration: 1.007 CPU time: 0.001 / _/ v4.0.0 Program: examples/async_example_simple.py 1.006 main async_example_simple.py:4 └─ 1.005 sleep asyncio/tasks.py:641 [2 frames hidden] asyncio 1.005 [await]
Pyinstrument has a documentation site, including full Python API docs!
- Fix a bug that caused
--show-allto be ignored on the command line.
- Under-the-hood modernisation
timelineoption (boolean) to Profiler methods
- Fixed issue with
pyinstrument -m module, where pyinstrument wouldn't find modules in the current directory.
- Dropped support for Python 2.7 and 3.5. Old versions will remain available on PyPI, and pip should choose the correct one automatically.
- Added the ability to track time in C functions. Minor note - Pyinstrument
will record time spent C functions as 'leaf' functions, due to a limitation
in how Python records frames.
Python -> C -> Pythonis recorded as
Python -> Python, but
Python -> Python -> Cwill be attributed correctly. (#103)
<__array_function__ internals>frames appearing as app code in reports
- Added support for timeline mode on HTML and JSON renderers
- Released as a tarball as well as a universal wheel
- Added PYINSTRUMENT_SHOW_CALLBACK option on the Django middleware to add a condition to showing the profile (could be used to run pyinstrument on a live server!)
- Fixed bug in the Django middleware where file would not be written because of a unicode error
- Fixed bug with the Django middleware on Windows where profiling would fail because we were trying to put an illegal character '?' in the profile path. (#66)
--show-regexoptions, to mark certain files to be displayed. This helps to profile inside specific modules, while hiding others. For example,
pyinstrument --show '*/sympy/*' script.py.
- Fix #60: pass all arguments after -m module_name to the called module
- Fix crash during HTML/JSON output when no frames were captured.
Pyinstrument will now hide traces through libraries that you're using by default. So instead of showing you loads of frames going through the internals of something external e.g. urllib, it lets you focus on your code.
To go back to the old behaviour, use
--show-allon the command line.
'Entry' frames of hidden groups are shown, so you know which call is the problem
Really slow frames in the groups are shown too, e.g. the 'read' call on the socket
Application code is highlighted in the console
Additional metrics are shown at the top of the trace - timestamp, number of samples, duration, CPU time
Hidden code is controlled by the
--hide-regexoptions - matching on the path of the code files.
--hide=EXPR glob-style pattern matching the file paths whose frames to hide. Defaults to '*/lib/*'. --hide-regex=REGEX regex matching the file paths whose frames to hide. Useful if --hide doesn't give enough control.
Outputting a timeline is supported from the command line.
-t, --timeline render as a timeline - preserve ordering and don't condense repeated calls
Because there are a few rendering options now, you can load a previous profiling session using
--load-prev- pyinstrument keeps the last 10 sessions.
Hidden groups can also call back into application code, that looks like this:
(internal) When recording timelines, frame trees are completely linear now, allowing for the creation of super-accurate frame charts.
(internal) The HTML renderer has been rewritten as a Vue.js app. All the console improvements apply to the HTML output too, plus it's interactive.
(internal) A lot of unit and integration tests added!
Yikes! See #49 for the gory details. I hope you like it.
- Big refactor!
Recordershave been removed. The frame recording is now internal to the
Profilerobject. This means the 'frame' objects are more general-purpose, which paves the way for...
- Processors! These are functions that mutate the tree to sculpt the output. They are used by the renderers to filter the output to the correct form. Now, instead of a time-aggregating recorder, the profiler just uses timeline-style recording (this is lower-overhead anyway) and the aggregation is done as a processing step.
- The upshot of this is that it's now way easier to alter the tree to filter stuff out, and do more advanced things like combining frames that we don't care about. More features to come that use this in v3.0!
- Importlib frames are removed - you won't see them at all. Their children are retained, so imports are just transparent.
- Django profile file name is now limited to a hundred of characters (#50)
- Fix bug with --html option (#53)
--versioncommand line option
- Fix crash when using on the command line.
Added support for JSON output. Use
pyinstrument --renderer=json scriptfile.py. PR
pyinstrument --htmland you don't pipe the output to a file, pyinstrument will write the console output to a temp file and open that in a browser.
- Added support for running modules with pyinstrument via the command line. The new syntax
pyinstrument -m module_name! PR
- Fix crashes due to multi-threaded use of pyinstrument. The fix is in the C extension, over at joerick/pyinstrument_cext#3
Pyinstrument can now be used in a
profiler = pyinstrument.Profiler() with profiler: # do some work here... print(profiler.output_text())
Middleware fix for older versions of Django
- Fix for max recursion error when used to profile programs with a lot of frames on the stack.
- Ensure license is included in the sdist.
Pyinstrument uses a new profiling mode. Rather than using signals, pyintrument uses a new statistical profiler built on PyEval_SetProfile. This means no more main thread restriction, no more IO errors when using Pyinstrument, and no need for a separate more 'setprofile' mode!
Renderers. Users can customize Pyinstrument to use alternative renderers with the
Profiler.output(), or using the
--rendererargument on the command line.
Recorders. To support other use cases of Pyinstrument (e.g. flame charts), pyinstrument now has a 'timeline' recorder mode. This mode records captured frames in a linear way, so the program execution can be viewed on a timeline.
pyinstrumentcommand. You can now profile python scripts from the shell by running
$ pyinstrument script.py. This is now equivalent to
python -m pyinstrument. Thanks @asmeurer!
Application code is highlighted in HTML traces to make it easier to spot
PYINSTRUMENT_PROFILE_DIRoption to the Django interface, which will log profiles of all requests to a file the specified folder. Useful for profiling API calls.
PYINSTRUMENT_USE_SIGNALoption to the Django interface, for use when signal mode presents problems.
To setup a dev environment:
virtualenv --python=python3 env . env/bin/activate pip install --upgrade pip pip install -r requirements-dev.txt pre-commit install --install-hooks
To get some sample output:
To run the tests:
To run linting checks locally:
pre-commit run --all-files
Some of the pre-commit checks, like
black, will auto-fix
the problems they find. So if the above command returns an error, try
running it again, it might succeed the second time :)
Running all the checks can be slow, so you can also run checks
individually, e.g., to format source code that fails
pre-commit run --all-files isort pre-commit run --all-files black
To diagnose why
pyright checks are failing:
pre-commit run --all-files pyright
The HTML renderer Vue.js app
To edit the html renderer style, do:
cd html_renderer npm ci npm run serve
When launched without a top-level
window.profileSession object, it will
fetch a sample profile so you can work with it.
To compile the JS app and bundle it back into the pyinstrument python tool: