Two scenarios steps use same fixture and second step should get value from example table but does not

[silygose]

Two scenarios steps use same fixture and second step should get value from example table but does not

@cucumber-basket
Feature: Cucumber Basket
As a gardener,
I want to carry cucumbers in a basket,
So that I don't drop them all.

@remove
Scenario Outline: Remove cucumbers from the basket
Given the basket has "" cucumbers
When "1" cucumbers are removed from the basket
When "<some>" cucumbers are removed from the basket
Then the basket contains "" cucumbers

Examples:
  | initial | some | total |
  | 8       | 3    | 4     |
  | 10      | 4    | 5     |
  | 7       | 0    | 6     |

@when(parsers.cfparse('"{some:Number}" cucumbers are removed from the basket', extra_types=EXTRA_TYPES))
@when('"<some>" cucumbers are removed from the basket')
def remove_cucumbers(basket, some):
print(f'remove {some} items')
basket.remove(some)

Test Ouput:
tests/steps/test_cucumber_steps.py::test_remove_cucumbers_from_the_basket[8-3-4] <- ..........\develop\python_venv.my_python_venv_amd64_3.7.5\lib\site-packages\pytest_bdd\scenario.py remove 1 items
remove 1 items
Step failed: Then "the basket contains "" cucumbers"
FAILED

The first When statement passes a 1 to the fixture.
Expected: The second When step should get from the Examples table but does not.
Actual: The second When uses the same value from the first When.

Using pytest-bdd 4.0.2, python v3.7.5 (64 bit)

[elchupanebrej]

Seems like the #293 issue

@when('"<some>" cucumbers are removed from the basket')

Won't work with the Feature file you've provided.

[silygose]

I have escaped the braces in the feature file. They were not being displayed.

[dcendents]

We got the same problem yesterday and it also happens with 2 different methods, each has its own annotation but they happen to use the same parameter name.

e.g.:

@given(parsers.parse("some {attribute}")
def given1(attribute):
    pass

@given("completely unrelated to the other <attribute>")
def given2(attribute):
    pass

It really looks like Example variables are treated like global variables and can be renamed by other unrelated steps.

My 2 cents on what is the problem using the example in the opening statement:

• The scenario starts, the variable some (or is it a pytest fixture?) is initialized with the value from the Example so 3 the first time
• The first when step is executed, the string is parsed and the (global?) variable some is given the value 1
• The second when is executed and is passed the variable some, which at this point contains 1 and not 3 as expected by that step

[dcendents]

@elchupanebrej BTW I think this is unrelated to #293 (or even #409 which I opened) which is more about how pytest-bdd finds steps.

This is in my opinion a bug in the way pytest-bdd initializes and stores example values for the scenario that can be modified externally by other steps and will alter the rest of the scenario.

[elchupanebrej]

@dcendents you are right. Parsed value is injected as a fixture(https://github.com/pytest-dev/pytest-bdd#step-arguments-are-fixtures-as-well) at execution time without respect of fixtures provided by the Examples section which are provided during collection time

When pytest_bdd tries to inject a new fixture, it could check if this fixture already defined. From a user perspective, the Examples section has to be inlined before step execution and must not be affected by step execution. This is a pretty major bug because some not-well defined step with parameter name collision could break pretty big suite of tests

[dcendents]

@elchupanebrej I don't know all the internals, but one solution could be to use a context manager when there is a name collision and restore the fixture value after the step execution.

If this is not possible then possibly "reset" all fixtures values from the Example table after each step execution.

[olegpidsadnyi]

@dcendents The example values are no longer fixtures. Is it still a case with the latest version?

[elchupanebrej]

Parsed parameters are injected as fixtures and overwrite elder ones:

pytest-bdd/pytest_bdd/scenario.py

Line 54 in f4ed62d

inject_fixture(request, arg, value)

[olegpidsadnyi]

Parsed parameters are injected as fixtures and overwrite elder ones:

pytest-bdd/pytest_bdd/scenario.py

Line 54 in f4ed62d

inject_fixture(request, arg, value)

ouch. i thought it was already undone. then we have to address it. There's no reason for those params to be fixtures

[elchupanebrej]

#493 Partially address this.

[dcendents]

Hi,

All I can say is the specific scenario I described is fixed in 5.0.0
Just to be safe I tested with the following simplified scenarios and it fails with 4.1.0 and passes with 5.0.0

params.feature:

Feature: Test parameters

    Scenario: att1 only
        Given some val
        Then att1 value is val

    Scenario Outline: att2 only
        Given completely unrelated to the other <attribute>
        Then att2 value is other value

        Examples:
            | attribute   |
            | other value |

    Scenario Outline: mix
        Given some val
        Given completely unrelated to the other <attribute>
        Then att1 value is val
        Then att2 value is other value

        Examples:
            | attribute   |
            | other value |

    Scenario Outline: reverse mix
        Given completely unrelated to the other <attribute>
        Given some val
        Then att1 value is val
        Then att2 value is other value

        Examples:
            | attribute   |
            | other value |

params_test.py:

from pytest_bdd import parsers
from pytest_bdd.scenario import scenarios
from pytest_bdd.steps import given, then

scenarios("params.feature")

@given(parsers.parse("some {attribute}"), target_fixture="att1")
def given1(attribute):
    return attribute

# Keep the correct given statement depending on pytest-bdd version
# @given("completely unrelated to the other <attribute>", target_fixture="att2")
@given(parsers.parse("completely unrelated to the other {attribute}"), target_fixture="att2")
def given2(attribute):
    return attribute

@then(parsers.parse("att1 value is {value}"))
def validate_att1(att1, value):
    assert att1 == value

@then(parsers.parse("att2 value is {value}"))
def validate_att2(att2, value):
    assert att2 == value

[elchupanebrej]

I propose the next solution for injecting fixtures into pytest scope, please check link elchupanebrej#64