Documentation error in Getting Started, re. grouping tests in classes

[bobobox]

In the "Getting Started" document, in section "Group multiple tests in a class" (https://github.com/pytest-dev/pytest/blob/master/doc/en/getting-started.rst#group-multiple-tests-in-a-class), the document states:

Something to be aware of when grouping tests inside classes is that each test has a unique instance of the class.
Having each test share the same class instance would be very detrimental to test isolation and would promote poor test practices.
This is outlined below:

However, in what is shown below, both tests are demonstrated to share the same instance of the class, at 0xdeadbeef.

This seems to demonstrate the opposite of what is stated (that the instances are not the same).

[RonnyPfannschmidt]

@bobobox thanks for bringing this to attention, we have a script that makes those auto generated output sections snippets more diff-friendly replacing all addresses with 0xdeadbeef as of now, we ought to investigate if we can get it changed to something like 0xfake{someindex}

[nicoddemus]

While we don't fix regendoc, we might as well at least add a note below that output explaining the situation (possibly pointing to this issue).

[andrew-kramer]

It looks like the documentation now uses two separate object instances at 0xdeadbeef0001 and 0xdeadbeef0002. Does that satisfy this issue?

[nkr0]

I think this issue is a bit more complicated. In TestAll, I've 2 immutable objects and 2 mutable objects.

import dataclasses
import pytest

@dataclasses.dataclass
class DC:
    a: str

class TestAll:
    x = 1
    s = "1"
    dc = DC("1")
    dct = {"a":"1"}
    
    def test_one(self):
        print(self)
        print(self.x)
        print(self.s)
        print(self.dc)
        print(self.dct)
        self.x = 2
        self.s = "2"
        self.dc.a = "2"
        self.dct["a"] = "2"
        print(self.x)
        print(self.s)
        print(self.dc)
        print(self.dct)
        assert False
        

    def test_two(self):
        print(self)
        print(self.x)
        print(self.s)
        print(self.dc)
        print(self.dct)
        assert False

Output shows that the 2 tests have separate TestAll objects. And the changes made to the immutable members in test_one are not propagated to test_two. But changes made to mutable members persist.

========================= test session starts ==========================
platform linux -- Python 3.13.3, pytest-8.3.5, pluggy-1.5.0
rootdir: /home/nkr/work/temp
plugins: cov-6.1.1
collected 2 items                                                      

test_all.py FF                                                   [100%]

=============================== FAILURES ===============================
___________________________ TestAll.test_one ___________________________

self = <test_all.TestAll object at 0x7feb04d28410>

    def test_one(self):
        print(self)
        print(self.x)
        print(self.s)
        print(self.dc)
        print(self.dct)
        self.x = 2
        self.s = "2"
        self.dc.a = "2"
        self.dct["a"] = "2"
        print(self.x)
        print(self.s)
        print(self.dc)
        print(self.dct)
>       assert False
E       assert False

test_all.py:28: AssertionError
------------------------- Captured stdout call -------------------------
<test_all.TestAll object at 0x7feb04d28410>
1
1
DC(a='1')
{'a': '1'}
2
2
DC(a='2')
{'a': '2'}
___________________________ TestAll.test_two ___________________________

self = <test_all.TestAll object at 0x7feb04d28cd0>

    def test_two(self):
        print(self)
        print(self.x)
        print(self.s)
        print(self.dc)
        print(self.dct)
>       assert False
E       assert False

test_all.py:37: AssertionError
------------------------- Captured stdout call -------------------------
<test_all.TestAll object at 0x7feb04d28cd0>
1
1
DC(a='2')
{'a': '2'}
======================= short test summary info ========================
FAILED test_all.py::TestAll::test_one - assert False
FAILED test_all.py::TestAll::test_two - assert False
========================== 2 failed in 0.07s ===========================

I'm not sure if this is intended behaviour and you are expected to use setup/tear-down. But I think at-least the documentation needs to be updated.

[nkr0]

Furthermore, looking at the id of objects,

import dataclasses
import pytest

@dataclasses.dataclass
class DC:
    a: str

class TestAll:
    x = 1
    s = "1"
    dc = DC("1")
    dct = {"a":"1"}
    
    def test_one(self):
        print(self)
        print(id(self.x))
        print(id(self.s))
        print(id(self.dc))
        print(id(self.dct))
        print(self.x)
        print(self.s)
        print(self.dc)
        print(self.dct)
        self.x = 2
        self.s = "2"
        self.dc.a = "2"
        self.dct["a"] = "2"
        print("After edit")
        print(id(self.x))
        print(id(self.s))
        print(id(self.dc))
        print(id(self.dct))
        print(self.x)
        print(self.s)
        print(self.dc)
        print(self.dct)
        assert False
        

    def test_two(self):
        print(self)
        print(id(self.x))
        print(id(self.s))
        print(id(self.dc))
        print(id(self.dct))
        print(self.x)
        print(self.s)
        print(self.dc)
        print(self.dct)
        assert False

Within test_one, when an immutable object is changed, its id changes. But then in test_two it has the old id again. Mutable objects have the same id throughout.

========================= test session starts ==========================
platform linux -- Python 3.13.3, pytest-8.3.5, pluggy-1.5.0
rootdir: /home/nkr/work/temp
plugins: cov-6.1.1
collected 2 items                                                      

test_all.py FF                                                   [100%]

=============================== FAILURES ===============================
___________________________ TestAll.test_one ___________________________

self = <test_all.TestAll object at 0x7fa83e91c410>

    def test_one(self):
        print(self)
        print(id(self.x))
        print(id(self.s))
        print(id(self.dc))
        print(id(self.dct))
        print(self.x)
        print(self.s)
        print(self.dc)
        print(self.dct)
        self.x = 2
        self.s = "2"
        self.dc.a = "2"
        self.dct["a"] = "2"
        print("After edit")
        print(id(self.x))
        print(id(self.s))
        print(id(self.dc))
        print(id(self.dct))
        print(self.x)
        print(self.s)
        print(self.dc)
        print(self.dct)
>       assert False
E       assert False

test_all.py:37: AssertionError
------------------------- Captured stdout call -------------------------
<test_all.TestAll object at 0x7fa83e91c410>
10734128
10796392
140360582473648
140360581174656
1
1
DC(a='1')
{'a': '1'}
After edit
10734160
10796440
140360582473648
140360581174656
2
2
DC(a='2')
{'a': '2'}
___________________________ TestAll.test_two ___________________________

self = <test_all.TestAll object at 0x7fa83e91ccd0>

    def test_two(self):
        print(self)
        print(id(self.x))
        print(id(self.s))
        print(id(self.dc))
        print(id(self.dct))
        print(self.x)
        print(self.s)
        print(self.dc)
        print(self.dct)
>       assert False
E       assert False

test_all.py:50: AssertionError
------------------------- Captured stdout call -------------------------
<test_all.TestAll object at 0x7fa83e91ccd0>
10734128
10796392
140360582473648
140360581174656
1
1
DC(a='2')
{'a': '2'}
======================= short test summary info ========================
FAILED test_all.py::TestAll::test_one - assert False
FAILED test_all.py::TestAll::test_two - assert False
========================== 2 failed in 0.08s ===========================

[The-Compiler]

As pointed out by @andrew-kramer, this issue (about the example output in the docs) has been long fixed, namely in c21ef7b via pytest-dev/regendoc@a6e143e.

@nkr0 What you describe is unrelated to the documentation, and in fact to pytest entirely, that's just how (class) variables work in Python.