Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Multiple use of fixture in a single test #2703

Open
defrank opened this Issue Aug 18, 2017 · 19 comments

Comments

Projects
None yet
7 participants
@defrank
Copy link

defrank commented Aug 18, 2017

Feature request to add the ability to use a fixture more than once in the test function arguments. Off the top of my head, appending a number to the fixture name could be a decent API. Obviously don't override existing fixtures and perhaps require the appended numbers to be ascending.

@pytest.fixture
def rand_int():
    yield random.randint(0, 100)

def test_rand_ints(rand_int0, rand_int1, rand_int2):
    # do something with 3 random integers

I can get around this by creating a fixture that generates values.

@python.fixture
def randints():
    def gen(count):
        for x in range(count):
            yield random.randint(0, 100)
    yield gen

def test_randint_gen(randints):
    r1, r2, r3 = randints(3)
    # do something with 3 random integers

I realize these examples could just be utility functions, but consider the following examples that require cleanup.

@python.fixture
def user():
    u = models.User()
    u.commit()
    yield u
    u.delete()

@python.fixture
def usergen(request):
    def gen(count):
        for x in range(count):
            u = models.User()
            u.commit()
            request.addfinalizer(lambda u=u: u.delete())
            yield u

    yield gen

def test_user(user0, user1, user2):
    # do something with 3 users

def test_usergen(usergen):
    user1, user2, user3 = usergen(3)
    # do something with 3 users

I think this deserves debate as my suggestion requires less code and I believe to be more intuitive.

It should be noted that adding scope to either solution will be confusing and probably not recommended (e.g., @fixture(scope='module')).

@RonnyPfannschmidt

This comment has been minimized.

Copy link
Member

RonnyPfannschmidt commented Aug 19, 2017

that would require fixtures to match argument names in a new way,
its hard to impossible wit the current internals and requries a rearchitecting, but i consider that worthwhile

@massich

This comment has been minimized.

Copy link
Contributor

massich commented Aug 23, 2017

If the argument names matching needs to be reworked anyway, would this call be easier?

@pytest.fixture
def rand_int():
    yield random.randint(0, 100)

def test_rand_ints(r0=rand_int, r1=rand_int, r3=rand_int):
    # do something with 3 random integers
@nicoddemus

This comment has been minimized.

Copy link
Member

nicoddemus commented Aug 23, 2017

How would we handle fixture inter-dependencies with this feature?

For example, from the original example, what happens if another fixture also depends on the user fixture?

@pytest.fixture
def subscribed_user(user):
    # subscribe user to newsletter
    return Subscription(user)

def test_user(user0, user1, user2, subscribed_user):
    ...

Which of the user0, user1, user2, is subscribed by "subscribed_user" in this test?

Also, one of the criticisms of pytest is being a little implicit with the fixture injection mechanism, so I'm not sure adding more implicitly to it would be a good idea. @massich's suggestion of using keyword arguments might be a solution to that though.

@RonnyPfannschmidt

This comment has been minimized.

Copy link
Member

RonnyPfannschmidt commented Aug 23, 2017

well - from my pov - until we rework the fixture mechanism into a externally usable library its going to be a huge disservice for any user

@defrank

This comment has been minimized.

Copy link
Author

defrank commented Aug 24, 2017

For example, from the original example, what happens if another fixture also depends on the user fixture?

@pytest.fixture
def subscribed_user(user):
   # subscribe user to newsletter
   return Subscription(user)

def test_user(user0, user1, user2, subscribed_user):
   ...

Which of the user0, user1, user2, is subscribed by "subscribed_user" in this test?

This is partly what sparked this proposal. Honestly, I don't like that the same fixture value is used for every instance. I realize it's a function scope, but most likely, I can access subscribed_user.user. If I specify another user in the test case, then I want it to be different. Actually, I assumed user would be different when I first tried it.

@pytest.fixture
def test_user(user, subscribed_user):
    assert user != subscribed_user.user  # I want this to succeed

Perhaps another scope such as @pytest.fixture(scope='unique')?

I very much like @massich's keyword argument suggestion.

@RonnyPfannschmidt

This comment has been minimized.

Copy link
Member

RonnyPfannschmidt commented Aug 25, 2017

we tried to introduce such scopes, but it was demonstrate that this currently breaks the world - internal refactorings are needed

@nicoddemus

This comment has been minimized.

Copy link
Member

nicoddemus commented Aug 25, 2017

Honestly, I don't like that the same fixture value is used for every instance.

I see @defrank, thanks. I've seen people on occasion with the same impression, but I think the current system is more powerful because reusing fixture instances within the same context is what allows you to actually create more complex systems.

For example, consider the tmpdir fixture; if a different instance was created for each fixture involved in a test function invocation, its usefulness would be severely limited.

By the way, a simple way to obtain the functionality you need is to make your fixture a factory instead; that gives you full control when and where to create the instances.

def test_user(user_factory):
    user0, user1 = user_factory.create('calvin', 'hobbes')

For convenience, you might also have a user fixture which creates a default user:

@pytest.fixture
def user(user_factory):
    return user_factory.create('jonh')

Pytest itself follows the same approach with the tmpdir_factory fixture.

@defrank

This comment has been minimized.

Copy link
Author

defrank commented Aug 25, 2017

Yeah, the usergen was my factory solution. What about some sort of @pytest.factory fixture that could be special-cased for these uses?

@nicoddemus

This comment has been minimized.

Copy link
Member

nicoddemus commented Aug 25, 2017

What about some sort of @pytest.factory fixture that could be special-cased for these uses?

Hmm how would that work? Could you provide an usage example?

@nicoddemus

This comment has been minimized.

Copy link
Member

nicoddemus commented Aug 25, 2017

Yeah, the usergen was my factory solution.

Oh I must have forgotten about it, sorry about that.

@defrank

This comment has been minimized.

Copy link
Author

defrank commented Aug 25, 2017

What about some sort of @pytest.factory fixture that could be special-cased for these uses?

Hmm how would that work? Could you provide an usage example?

Factory

Essentially, mark factory fixtures.

@pytest.factory
def user_factory(request):
    def factory(*names):
        for name in names:
            user = models.User(name=name)
            user.commit()
            request.addfinalizer(lambda u=user: u.delete())
            yield user

    yield factory

Idea 1

@pytest.fixture
def user(bob=user_factory):
    yield bob


def test_user_factory(user, amy=user_factory, Joe=user_factory):
    assert user.name == 'bob'
    assert amy.name == 'amy'
    assert Joe.name == 'joe'

I don't like this:

  1. Restricts keys to be alphanumeric and valid variable names
  2. Mixing case for keys like Joe is against PEP8
  3. user_factory as the default argument will break linters if user_factory isn't in scope

Idea 2

Require factory from user_factory be replaced with unique identifier.

@pytest.fixture
def user(user_0='bob'):
    yield user_0


def test_user_factory(user, user_1='joe', user_amy='amy'):
    assert user.name == 'bob'
    assert user_1.name == 'joe'
    assert user_amy.name == 'amy'

Since my factory requires names, I'm not considering the following, although it should be considered:

def test_user_factory(user, user_amy, user_john):
    assert user != user_amy
    assert user != user_john
    assert user_amy != user_john
@RonnyPfannschmidt

This comment has been minimized.

Copy link
Member

RonnyPfannschmidt commented Aug 26, 2017

imho we need a mechanism that handles a mapping of parameter names and fixtures/parameter sets

all proposals i have seen here so far cant even hope to fit the bill

@BrenBarn

This comment has been minimized.

Copy link

BrenBarn commented May 7, 2018

What is the status of this? I find this is pretty common for fixtures that model some kind of database object. For instance, I have a fixture for a user model. That's great if I only need one. But what if I'm writing a test that needs more than user (because it has to test some kind of interaction between them)? If the user fixture includes teardown implemented via a yield statement, as far as I can see there is no way to "manually" call the fixture function from within the test function while ensuring teardown happens appropriately. That is, I can't do this:

@pytest.fixture
def user():
    u = models.User()
    u.commit()
    yield u
    u.delete()

def test_two_users():
    user1 = user()
    user2 = user()
    # test them

. . . because calling user() directly gives me a generator and I'd have to implement the iterate-then-cleanup procedure myself.

It seems like handling the teardown is done by _pytest.run_fixture_function, but this function is not exposed publically, so there's no way for a test to manually call a generator-fixture and have cleanup happen appropriately. What is the recommended solution?

It seems like at the least it should be possible to provide a context manager wrapper around fixture functions that allows tests to preserve the teardown when calling the fixture function manually. So a test could do:

def test_two_users():
    with context(user) as user1, context(user) as user2:
        # test

This would at least let test functions use a fixture more than once by manually calling it multiple times from within the test function body.

@RonnyPfannschmidt

This comment has been minimized.

Copy link
Member

RonnyPfannschmidt commented May 7, 2018

@BrenBarn nothing happened because even the underpinnings for this need quite some work - which currently isn't available in terms of able manpower

@nicoddemus

This comment has been minimized.

Copy link
Member

nicoddemus commented May 7, 2018

It seems like handling the teardown is done by _pytest.run_fixture_function, but this function is not exposed publically, so there's no way for a test to manually call a generator-fixture and have cleanup happen appropriately. What is the recommended solution?

One solution is to make your fixture return a factory instead of the resource directly:

@pytest.fixture(name='make_user')
def make_user_():
    created = []
    def make_user():
        u = models.User()
        u.commit()
        created.append(u)
        return u

    yield make_user

    for u in created:
        u.delete()

def test_two_users(make_user):
    user1 = make_user()
    user2 = make_user()
    # test them

# you can even have the normal fixture when you only need a single user
@pytest.fixture
def user(make_user):
    return make_user()

def test_one_user(user):
    # test him/her

This is what we have with tmpdir_factory for example.

@BrenBarn

This comment has been minimized.

Copy link

BrenBarn commented May 10, 2018

Thanks, that is useful. Are there any examples of that in the docs?

@nicoddemus

This comment has been minimized.

Copy link
Member

nicoddemus commented May 10, 2018

@BrenBarn good point, there isn't. I created #3461 for it (PRs are welcome!).

@asottile

This comment has been minimized.

Copy link
Member

asottile commented Apr 21, 2019

Duplicate of #456 I believe, if not feel free to reopen (let's consolidate discussion on this there)

@RonnyPfannschmidt

This comment has been minimized.

Copy link
Member

RonnyPfannschmidt commented Apr 21, 2019

this one is about something slightly different

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.