Skip to content

Latest commit

 

History

History
147 lines (103 loc) · 5.56 KB

usage.rst

File metadata and controls

147 lines (103 loc) · 5.56 KB
Raw

Usage

Installation

This plugin can be installed using pip:

pip install pytest-memray

pytest-memray is a pytest plugin. It is enabled when you pass --memray to pytest:

pytest tests/ --memray

Allocation tracking

By default, the plugin will track allocations at the high watermark in all tests. This information is reported after tests run ends:

.. command-output:: env COLUMNS=92 pytest --memray demo
   :returncode: 1

Markers

This plugin provides markers that can be used to enforce additional checks and validations on tests when this plugin is enabled.

Important

These markers do nothing when the plugin is not enabled.

limit_memory

.. py:function:: limit_memory(memory_limit: str)

   Fail the execution of the test if the test allocates more memory than allowed

When this marker is applied to a test, it will cause the test to fail if the execution of the test allocates more memory than allowed. It takes a single argument with a string indicating the maximum memory that the test can allocate.

The format for the string is <NUMBER> ([KMGTP]B|B). The marker will raise ValueError if the string format cannot be parsed correctly.

Warning

As the Python interpreter has its own object allocator is possible that memory is not immediately released to the system when objects are deleted, so tests using this marker may need to give some room to account for this.

Example of usage:

@pytest.mark.limit_memory("24 MB")
def test_foobar():
    pass  # do some stuff that allocates memory

limit_leaks

.. py:function:: limit_leaks(max_leaked_memory: str , *, warmups: int=0)

      Fail the execution of the test if the test leaks more memory than allowed.

Important

To detect leaks, Memray needs to intercept calls to the Python allocators. This is adds significant overhead, and will slow your test down. Additionally, if the warmups argument is provided, the test will be run multiple times before the final execution, which will also increase the total time it takes to complete the test run.

When this marker is applied to a test, it will cause the test to fail if the execution of the test leaks more memory than allowed. It takes a single positional argument with a string indicating the maximum memory that the test is allowed to leak.

Leaks are defined as memory that is allocated in the marked test that is not freed before leaving the test body.

The format for the string is <NUMBER> ([KMGTP]B|B). The marker will raise ValueError if the string format cannot be parsed correctly.

The marker also takes an optional keyword-only argument warmups. This argument indicates the number of times the test body (and not any of the fixtures) will be executed before the memory leak check is performed. This is useful to avoid false positives where memory allocated below the test is cached and survives the test run, as well as to account for memory allocated (and not released) by the interpreter under normal execution.

Tip

You can pass the --memray-bin-path argument to pytest to specify a directory where Memray will store the binary files with the results. You can then use the memray CLI to further investigate the allocations and the leaks using any Memray reporters you'd like. Check the memray docs for more information.

Example of usage:

@pytest.mark.limit_leaks("1 MB", warmups=3)
def test_foobar():
    pass # do some stuff that cannot leak memory

Warning

Is very challenging to write tests that do not "leak" memory in some way. This marker is intended to be used to detect leaks in very simple unit tests and can be challenging to use reliably when applied to integration tests or more complex scenarios. Also, pytest itself does things that will be interpreted as leaking memory, for example:

  • pytest captures stdout/stderr by default and will not release the memory associated to these strings until the test ends. This will be interpreted as a leak.
  • pytest captures logging records by default and will not release the memory associated to these strings until the test ends. This will be interpreted as a leak.
  • Memory allocated in the test body from fixtures that is only released at fixture teardown time will be interpreted as a leak because the plugin cannot see the fixtures being deallocated.
  • The Python interpreter has some internal caches that will not be cleared when the test ends. The plugin does is best to warmup all available interpreter caches but there are some that cannot be correctly detected so you may need to allow some small amount of leaked memory or use the warmups argument to avoid false positive leak reports caused by objects that the interpreter plans to reuse later. These caches are implementation details of the interpreter, so the amount of memory allocated, the location of the allocation, and the allocator that was used can all change from one Python version to another.

For this reason, using this marker with "0B" as the maximum allowed leaked memory in tests that are not very simple unit tests will almost always require a nonzero value for the warmups argument and some times that will not be enough.