How to apply @parameterized.expand on Class?

[legshort]

I would like to apply @parameterized.expand on Class so that every test method runs with the common parameter.

I was making decorator for wrapping the expand decorator but got stuck where to apply decorator manually.

What I was trying to do is that overring the method with the expand.

setattr(cls, method_name, parameterized.expand(method)

However, parameterized.expand is @classmethod which does not take a function as the argument.
Has anyone any idea?

[charlsagente]

I'm also interested in this feature, but I ended implementing my own workaround for this. @wolever can I submit a pull request for this?

[legshort]

@charlsagente why not? even though if your pr won't be merged, your code may inspire others including me. I'm thrilled to see your code.

[wolever]

I'd definitely consider merging a class-level parameterized!

Please post a couple examples of how it could be used so we can have something specific to discuss.

[charlsagente]

Ok, my current workaround for this is:

def parameterized_class(test_fields, attribute_name='test_field'):
    """
    Use it as @parametrized_class(test_fields) over your test class.
    :param test_fields: Array of dictionaries ex: [{"user_group_id": 1,
               "user_group": "principal"
               },
              {"user_group_id": 5,
               "user_group": "student"
               }]
    :param attribute_name: By default is test_field but you can put your own, one in your method call it as
    self.test_field['user_group']
    :return: Setts in your object an attribute specified by field_name
    """

    def decorator(base_class):
        module = sys.modules[base_class.__module__].__dict__
        for i, test_field in enumerate(test_fields):
            d = dict(base_class.__dict__)
            d[attribute_name] = test_field
            name = "%s_%s" % (base_class.__name__, i + 1)
            module[name] = type(name, (base_class,), d)

    return decorator

And this is the test case

test_data = [{"username": "user1",
              "password": "1233"
              },
             {"username": "user2",
              "password": "123"
              }]


@parameterized_class(test_data)
class TestParameterizedClass(TestCase):
    def test_method_a(self):
        self.assertIn('username', self.test_field)
        self.assertIn('password', self.test_field)

    def test_method_b(self):
        self.assertIn('username', self.test_field)
        self.assertIn('password', self.test_field)

Now I can use in all my methods the properties from the test_data, in my case i'm using dictionaries. What do you guys think?

[wolever]

Ohh, nifty! I like this idea!

A couple of tweaks I'd like to see to make it more closely match the @parameterized style:

• Accept two arguments, parameter names, and then a list of parameters similar to those used by @parameterized
• Use the first parameter as the suffix (following the style of @parameterized.expand)

For example:

@parameterize_class("username password", [
    ("user1", "1234"),
    ("user2", "5678"),
])
class TestUserAccounts(object):
    def test_login(self):
        assert can_login(self.username, self.password)

I'd also like to see some real-world examples of how this would be used… since I've often heard it talked about, but never seen a compelling case for it. But I'm very happy to be proved wrong!

[charlsagente]

For the first parameter in parameterize_class, do you suggest a string and then split it or maybe another kind of object like a tuple?, example:

@parameterize_class(("username","password"), [
    ("user1", "1234"),
    ("user2", "5678"),
])

[wolever]

Oh actually that's a good point; I was using a string to mirror namedtuple, but a tuple does better mirror the arguments. So yea, that!

I would like to see some real-world use cases for this, though. It definitely seems like something useful, but I don't know if it just seems that way, or if there are actually use cases that are significantly improved by it.

[legshort]

Thanks for sharing your code @charlsagente!!
For me, I was going to use this kind of approach for testing through API version.

ex)

@parameterize_class(['/v1.1/', '/v1.2/'])
class ViewTestCase:
  def test(self, version_url):
    self.client.get(version_url + '/users')

I think it would be nice if the param is wrapped instead of a class variable so that it won't conflict with others.
ex)

# parameterized class variable
self.assertEqual('username', self.param.username)
self.assertEqual('password', self.param.password)

# regular class variable
self.assertEqual('gender', self.gender)

[wolever]

Ah, API versions is a good call!

I do still prefer explicitly naming the target attributes, though, because it makes things more explicit, and the "all params in one attribute" case is still supported:

@parameterized_class("param", [
  {"username": "...", "password": "..."},
])

And you could even write your own tiny wrapper to do the same. The converse, however, isn't true.

[charlsagente]

Updated Function:

def parameterized_class(properties, test_values):

    def decorator(base_class):

        module = sys.modules[base_class.__module__].__dict__
        for i, test_field in enumerate(test_values):
            if len(properties) == len(test_field):
                d = dict(base_class.__dict__)
                for j, property_key in enumerate(properties):
                    d[property_key] = test_field[j]
                name = "%s_%s" % (base_class.__name__, i + 1)
                module[name] = type(name, (base_class,), d)

    return decorator

For now I'm avoiding too much validations, I saw in your code you have input_as_callable that could be used to validate the tuples but for now we can use this to call it.

@parameterized_class(("user", "password"), [
    ("user1", "pass1"),
    ("user2", "pass2")
])
class Test(TestCase):
    pass

For a real world case scenario I have the following:

I have a system in django with different user types, each type can perform certain amount of operations and other types have restrictions. I want to test the restrictions:

• As a user type A I want to validate I have access to view the dashboard

• As a user type B I want to validate I have access to view the dashboard

• As a user type A I want to validate that I can create a new user in the dashboard

• As a user type B I want to validate that I cannot create a new user in the dashboard

@parameterized_class(("user_name", "user_type"), [
    ("userA", 1),
    ("userB", 2)
])
class TestDashboard(TestCase):
    def setUp(self):
        self.client.force_login(self.user_name)

    def test_get_dashboard(self):
        response = self.client.get('/en/dashboard')
        self.assertEqual(response.status_code, 200)

    def test_create_new_user(self):
        request = new_user_data_generator()
        response = self.client.post('/en/dashboard', data=request)
        if self.user_type == 1:
            self.assertEqual(json.loads(response.content)['result'], "bad")

        elif self.user_type == 2:
            self.assertNotEqual(json.loads(response.content)['result'], "bad")
            

[legshort]

@charlsagente I really appreciate sharing your code, I have inspired by your code and I have made some changes to support pass only one param name which you can see below.

def parameterized_class(properties, test_values):
    def decorator(base_class):
        test_class_module = sys.modules[base_class.__module__].__dict__
        for test_value_key, test_field in enumerate(test_values):
            test_class_dict = dict(base_class.__dict__)

            if isinstance(properties, str):
                test_class_dict[properties] = test_field
                _create_module(base_class, test_class_module, test_value_key, test_class_dict)
            elif len(properties) == len(test_field):
                for j, property_key in enumerate(properties):
                    test_class_dict[property_key] = test_field[j]
                _create_module(base_class, test_class_module, test_value_key, test_class_dict)

    def _create_module(base_class, test_class_module, test_value_key, test_class_dict):
        name = '{method_name}_{index}'.format(method_name=base_class.__name__, index=test_value_key + 1)
        test_class_module[name] = type(name, (base_class,), test_class_dict)

    return decorator


@parameterized_class('version_url', ('v1.0', 'v1.1'))
class SingleParamNameTestCase(APITestCase):
    def test(self):
        self.assertTrue(self.version_url)


@parameterized_class(('user_name', 'user_type'), [
    ('userA', 1),
    ('userB', 2)
])
class MultipleParamNamesTestCase(APITestCase):
    def test(self):
        self.assertTrue(self.user_name)
        self.assertTrue(self.user_type)

[wolever]

Cool! I like it :)

Please start on a PR for this!

It should include:

• Documentation with some examples of real use-cases you listed above (I like the API one)
• The class names should follow the style of parameterized.expand, where the first parameter is used if it's a string, and otherwise the index
• Tests (… duh)

I look forward to reviewing it!

[charlsagente]

@wolever I submitted a PR. I used the last edit from @legshort for the decorator wrapper. I added some tests cases and are working for me in py.test, green, unittest and nose.

I also integrated the funcion inside your principal class and now I used the variable string_types to ensure python 2 compatibility.