Issue with BaseCase and pytest-steps ?

[smarie]

Hi there, I am the author or pytest-steps and an user recently reported an issue with BaseCase.

smarie/python-pytest-steps#40

It is not clear to me if this use case should work or not, as I do not know seleniumbase. Would you be able to provide a few hints about the origin of the issue ? If there is something I can do on pytest-steps to make it compliant with seleniumbase I'll happily do it. Thanks!

[mdmintz]

Hi @smarie , I'm familiar with this situation. It's explained by pytest-dev/pytest#3095 and pytest-dev/pytest#2535 (The origin of this issue comes from inheriting unittest.TestCase, which SeleniumBase's BaseCase does). There are a few different solutions to this. One is explained by the links above, which point to using autouse in the fixture definition. (See: https://docs.pytest.org/en/latest/unittest.html#using-autouse-fixtures-and-accessing-other-fixtures)
With the correct modifications, you can now use fixtures that pass multiple parameters, as seen with https://github.com/seleniumbase/SeleniumBase/blob/master/examples/parameterized_test.py ->

from seleniumbase import BaseCase
from parameterized import parameterized

class GoogleTestClass(BaseCase):

    @parameterized.expand([
        ["pypi", "pypi.org"],
        ["wikipedia", "wikipedia.org"],
        ["seleniumbase", "seleniumbase/SeleniumBase"],
    ])
    def test_parameterized_google_search(self, search_term, expected_text):
        self.open('https://google.com/ncr')
        self.type('input[title="Search"]', search_term + '\n')
        self.assert_element('#result-stats')
        self.assert_text(expected_text, '#search')

For the same reason that parameterized works with SeleniumBase, pytest-steps can also be modified to work.

The second solution is a workaround that involves using SeleniumBase as a pytest fixture. That already exists: sb. For libraries that haven't implemented the workaround, (such as pytest-parametrize), you'll have to use the SeleniumBase sb fixture for thing to work, as seen from https://github.com/seleniumbase/SeleniumBase/blob/master/examples/test_pytest_parametrize.py

import pytest

@pytest.mark.parametrize('value', ["pytest", "selenium"])
def test_sb_fixture_with_no_class(sb, value):
    sb.open("https://google.com/ncr")
    sb.type('input[title="Search"]', value + '\n')
    sb.assert_text(value, "div#center_col")

class Test_SB_Fixture():
    @pytest.mark.parametrize('value', ["pytest", "selenium"])
    def test_sb_fixture_inside_class(self, sb, value):
        sb.open("https://google.com/ncr")
        sb.type('input[title="Search"]', value + '\n')
        sb.assert_text(value, "div#center_col")

Using this technique, that modifies the original pytest-steps test to be the following: (note the sb fixture passed into the test args):

from pytest_steps import test_steps

class Test_MMMM():

    @test_steps('step_a', 'step_b', 'step_c')
    def test_suite(self, sb):
        # Step A
        print("step a")
        assert not False  # replace with your logic
        intermediate_a = 'hello'
        yield

        # Step B
        print("step b")
        assert not False  # replace with your logic
        yield

        # Step C
        print("step c")
        new_text = intermediate_a + " ... augmented"
        print(new_text)
        assert len(new_text) == 56
        yield

So to summarize, either pytest-steps needs to be slightly modified to work the way parameterized does (see https://github.com/seleniumbase/SeleniumBase/blob/master/examples/parameterized_test.py for a working example), or users need to use the SeleniumBase sb fixture workaround, as it was for pytest-parametrize (https://github.com/seleniumbase/SeleniumBase/blob/master/examples/test_pytest_parametrize.py).

[smarie]

This is great @mdmintz !! Thanks so much for the workaround. I'll ask the user to try it.

Concerning the issue description though, pytest-steps does not create or use a parametrized autouse fixture, it just

1. generates a test function wrapper test_<name>(<all_other_args>, ________step_name_, requests)
2. decorates it with @pytest.mark.parametrize("________step_name_", steps). So this is direct parametrization, not indirect.

So there is indeed a dependency to the requests fixture added in (1). But the requests fixture is not itself parametrized I believe. Maybe this is already too much for unittest.TestCase ?

Anyway if the workaround works, this is great ! Thanks again for answering so quickly