Fixture parametrization with a generator

[Kkevsterrr]

I would like to be able to use a fixture parametrized with a generator.

For example:

def get_dbs():
     """
     Depending on how many tests are running concurrently, not every database
     will be available. Yield a database only if it's available.
     """
     # Rough sketch of the code: 
     while DB_LIST:
         for db in DB_LIST:
             if db.available():
                 # remove db from list, etc
                 yield db
@pytest.fixture(params=get_dbs())
def db(request):
    mydb = request.params
    ... # do stuff

However, it seems that when the params are given a generator, the generator is immediately casted to a list, and the generator is not used. The power of using a generator over a list is it gives you dynamic control of the fixture setup, like how you can do with tests already with parametrize.

You could, of course, use params as a placeholder to retrieve a database, and then dynamically retrieve an available database inside the fixture, but then the fixture name/id is lost to just "db0", instead of the actual useful name of "aurora_database5" (for example).

Am I missing something, or another way to accomplish this? Thanks for your help.

[RonnyPfannschmidt]

each test would need a new generator for the parameterizazion, so no, this is definitively not the right way - you should layer the availiable dbs differently and allow test to skip early/be generated differently

[Kkevsterrr]

I'm not sure I understand what you mean that each test would need a new generator for parametrization.

To extend my previous example

# Each of these corresponds to a real, physical resource that each test must use. 
# If multiple test sessions  are running simultaneously, ideally a test will get a database that is available
# instead of having to block to wait for the database to become available. 
DB_LIST = ["aurora_database", "postgres_database", "mysql"]

def get_dbs():
     """
     Depending on how many tests are running concurrently, not every database
     will be available. Yield a database only if it's available.
     """
     # Rough sketch of the code: 
     while DB_LIST:
         for db in DB_LIST:
             if db.available():
                 # remove db from list, etc
                 yield db

@pytest.fixture(params=get_dbs())
def db(request):
    mydb = request.params
    ... # do stuff

def test_db(testdata, db):
    # perform a test on the given db with the given testdata

If two tests are running concurrently on a machine, this will allow each test to have a test_db with correct naming (test_db[data1:aurora_database]) and ensures that no test session is blocking on another test session while waiting for a database to become available.

If you have another suggestion on how to accomplish this goal (support multiple simultaneous test sessions, which must share some limited pool of external resources that preserves test names) I'm game to explore other solutions!

[Kkevsterrr]

@Zac-HD Why was this issue closed? Was there any progress made towards a solution I missed?

Also, @RonnyPfannschmidt, can you clarify your comment? Thanks!

[RonnyPfannschmidt]

@Kkevsterrr its not possible to correctly manage parametrization having a generator as value, always use a list yourself

[nicoddemus]

Adding to the explanation, pytest will iterate over the object given to fixture(params= every time it needs to instantiate the fixture, not only once, so you might as well use fixture(params=list(get_dbs())).