Scenario State Management (v2.x)

Page version: 3.x / 2.x

Scenario state management
Practices and patterns

Scenario state management

The typical scenario is composed of setup (given), action (when) and assertion(s) (then).

In LightBDD, such scenario steps are represented by methods. While in some cases it is possible to write steps not interacting directly with themselves, in most cases they have to share some state, like in this scenario:

Runner.RunScenario(
    Given_an_empty_basket,
    When_I_add_blue_sweater,
    And_I_add_red_sweater,
    Then_the_basket_should_contain_two_sweaters
);

where the steps here shares information about basket and sweaters.

LightBDD allows to share scenario state between steps in following ways:

via feature fixture fields / properties,
via context fields / properties,
via step method parameters.

Note: LightBDD does not support step methods returning state nor sharing it this way (more information on step method signature can be found on Scenario Steps Definition and Composite Steps Definition wiki page).

Sharing state via feature fixture members

Using feature fixture members is the easiest and most typical way of sharing state between scenario steps. It bases on the same concept that is used in writing regular unit tests, where test class methods uses class fields and properties to hold state they are working on. NUnit users are definitely familiar with the [SetUp]/[TearDown] attributes and the methods preparing / cleaning-up the system under test to execute test against. XUnit users would think here about using constructors and IDisposable interfaces. Regardless the test framework, users write also helper methods that modifies the test state and are used by multiple tests within the test class.

In LightBDD, the same approach can be used, where the given steps initializes test object state, when steps performs action against the system and then steps performs asserts the outcome.

A very simplistic implementation of the above scenario could look like this:

using LightBDD.Framework.Scenarios.Basic;

public class Basket_management : FeatureFixture
{
    [Scenario]
    public void Adding_items()
    {
        Runner.RunScenario(
            Given_an_empty_basket,
            When_I_add_red_sweater,
            And_I_add_blue_sweater,
            Then_the_basket_should_contain_two_sweaters);
    }

    /* Steps part */
    private Basket _basket;

    private void Given_an_empty_basket()
    {
        _basket = new Basket();
    }

    private void When_I_add_red_sweater()
    {
        _basket.Add(new Sweater("red"));
    }

    private void And_I_add_blue_sweater()
    {
        _basket.Add(new Sweater("blue"));
    }

    private void Then_the_basket_should_contain_two_sweaters()
    {
        Assert.Equal(2, _basket.Items.OfType<Sweater>().Count());
    }
}

Sharing state via context members

Sometimes using the feature class to share state is not suitable.

Let's take a look at following code:

public class Basket_management : FeatureFixture
{
    [Scenario]
    public void Adding_items()
    {
        Runner.RunScenario(
            Given_an_empty_basket,
            When_I_add_red_sweater,
            And_I_add_blue_sweater,
            Then_the_basket_should_contain_two_sweaters);
    }

    [Scenario]
    public void Removing_items()
    {
        Runner.RunScenario(
            Given_a_basket_with_red_sweater,
            When_I_remove_the_sweater,
            Then_the_basket_should_be_empty);
    }

    /* Steps part */
    private Basket _basket;

    private void Given_an_empty_basket()
    {
        _basket = new Basket();
    }

    private void Given_a_basket_with_red_sweater()
    {
        Given_an_empty_basket();
        When_I_add_red_sweater();
    }

    private void When_I_add_red_sweater()
    {
        _basket.Add(new Sweater("red"));
    }

    private void And_I_add_blue_sweater()
    {
        _basket.Add(new Sweater("blue"));
    }

    private void When_I_remove_the_sweater()
    {
        _basket.Remove(_basket.Items.OfType<Sweater>().First());
    }

    private void Then_the_basket_should_contain_two_sweaters()
    {
        Assert.AreEqual(2, _basket.Items.OfType<Sweater>().Count());
    }

    private void Then_the_basket_should_be_empty()
    {
        Assert.IsEmpty(_basket.Items);
    }
}

This time, there are two scenarios. If we implement basket and run it with nunit 3 or xunit 2 (after porting Assert code to xunit) the tests will pass.

However, if we decide to run all our tests in parallel by adding [assembly: Parallelizable(ParallelScope.All)] in nunit, we may get a following outcome:

SCENARIO: Removing items
  STEP 1/3: GIVEN a basket with red sweater...
  STEP 1/3: GIVEN a basket with red sweater (Passed after 11ms)
  STEP 2/3: WHEN I remove the sweater...
  STEP 2/3: WHEN I remove the sweater (Passed after 1ms)
  STEP 3/3: THEN the basket should be empty...
  STEP 3/3: THEN the basket should be empty (Failed after 63ms)
  SCENARIO RESULT: Failed after 117ms
    Step 3: NUnit.Framework.AssertionException :   Expected: <empty>
    	  But was:  < <BasketTests.Sweater>, <BasketTests.Sweater> >
    	
    	at BasketTests.Basket_management.Then_the_basket_should_be_empty()
    	at LightBDD.Framework.Scenarios.Basic.Implementation.BasicStepCompiler.StepExecutor.Execute(Object context, Object[] args)

We are getting this issue because nunit is using the same instance of the test class, and because we requested to run all the test methods in parallel, we have got the race condition between scenarios accessing and overriding the _basket field of the same instance. By the way, if we use [assembly: ClassCollectionBehavior(AllowTestParallelization = true)] attribute (described on Test-Framework-Integrations wiki page, LightBDD.XUnit2 section) and run the same tests with xunit framework, they will pass despite running in parallel as xunit instantiates the test class object per test.

In such situation, it is possible to define context objects and use them in scenario (or composite step) while still using the favorite test framework.

Conceptually, sharing the state with context classes is not much different than via feature fixture members, except the fact that state and step methods are implemented in the context class. The context class is instantiated in each scenario method and used to call all the steps.

using LightBDD.Framework.Scenarios.Contextual;
using LightBDD.Framework.Scenarios.Extended;

class BasketContext
{
    private Basket _basket;

    public void Given_an_empty_basket()
    {
        _basket = new Basket();
    }

    public void Given_a_basket_with_red_sweater()
    {
        Given_an_empty_basket();
        When_I_add_red_sweater();
    }

    public void When_I_add_red_sweater()
    {
        _basket.Add(new Sweater("red"));
    }

    public void And_I_add_blue_sweater()
    {
        _basket.Add(new Sweater("blue"));
    }

    public void When_I_remove_the_sweater()
    {
        _basket.Remove(_basket.Items.OfType<Sweater>().First());
    }

    public void Then_the_basket_should_contain_two_sweaters()
    {
        Assert.AreEqual(2, _basket.Items.OfType<Sweater>().Count());
    }

    public void Then_the_basket_should_be_empty()
    {
        Assert.IsEmpty(_basket.Items);
    }
}

public class Basket_management : FeatureFixture
{
    [Scenario]
    public void Adding_items()
    {
        Runner
            .WithContext<BasketContext>()
            .RunScenario(
                x => x.Given_an_empty_basket(),
                x => x.When_I_add_red_sweater(),
                x => x.And_I_add_blue_sweater(),
                x => x.Then_the_basket_should_contain_two_sweaters());
    }

    [Scenario]
    public void Removing_items()
    {
        Runner
            .WithContext<BasketContext>()
            .RunScenario(
                x => x.Given_a_basket_with_red_sweater(),
                x => x.When_I_remove_the_sweater(),
                x => x.Then_the_basket_should_be_empty());
    }
}

As it is presented above, steps and the _basket field has been extracted to BasketContext and the steps have been made public.

Each scenario implementation has been changed as well:

.WithContext<BasketContext>() has been added to instruct runner that scenario uses context object,
.RunScenario() has been changed to use extended syntax to allow calling steps on the context instance via x lambda parameter.

After this change, scenarios are executed against different instances of the context classes, making them passing even if run in parallel in nunit.

The usage of context classes, while more complex, brings additional benefits as well:

it allows to encapsulate the logic around specific features and use it in various fixtures, without need of using inheritance hierarchy (so it is cleaner and more flexible),
it works well with steps implemented as extension methods, so the example x => x.Then_response_should_have_status_code(HttpStatusCode.Ok) step could be implemented as extension method and used on any context class, as long as it is related to handling HTTP requests,
it allows to use dependency injections, described on the DI Containers wiki page,
finally, it allows to compose context objects, which is described later in this page.

Sharing state via step method parameters

The extended scenario syntax allows to use parameterized steps and pass state this way.

The parameterized steps allows to reduce the number of methods by making them more generic purpose:

public class Basket_management : FeatureFixture
{
    [Scenario]
    public void Adding_items()
    {
        Runner.RunScenario(
            _ => Given_an_empty_basket(),
            _ => When_I_add_COLOR_sweater("red"),
            _ => When_I_add_COLOR_sweater("blue"),
            _ => Then_the_basket_should_contain_COUNT_sweaters(2));
    }

    /* Steps part */
    private Basket _basket;

    private void Given_an_empty_basket()
    {
        _basket = new Basket();
    }

    private void When_I_add_COLOR_sweater(string color)
    {
        _basket.Add(new Sweater(color));
    }

    private void Then_the_basket_should_contain_COUNT_sweaters(int count)
    {
        Assert.Equal(count, _basket.Items.OfType<Sweater>().Count());
    }
}

SCENARIO: Adding items
  STEP 1/4: GIVEN an empty basket...
  STEP 1/4: GIVEN an empty basket (Passed after 10ms)
  STEP 2/4: WHEN I add "red" sweater...
  STEP 2/4: WHEN I add "red" sweater (Passed after <1ms)
  STEP 3/4: AND I add "blue" sweater...
  STEP 3/4: AND I add "blue" sweater (Passed after <1ms)
  STEP 4/4: THEN the basket should contain "2" sweaters...
  STEP 4/4: THEN the basket should contain "2" sweaters (Passed after 4ms)
  SCENARIO RESULT: Passed after 66ms

The above example represents the most typical usage of parameterized steps, where all parameters are constants specified in the scenario.
LightBDD allows however to make scenarios a bit more dynamic, as it allows to pass any expressions (almost*) as step parameters, including ones using scenario state.

* any expressions that can be represented as expression tree are supported, except ref/out method calls

Let's see the following example to visualize it better:

public class Basket_management : FeatureFixture
{
    [Scenario]
    public void Pricing_basket()
    {
        Runner.RunScenario(
            _ => Given_an_empty_basket(),
            _ => When_I_add_COLOR_sweater_with_price_EUR("red", 20),
            _ => When_I_add_COLOR_sweater_with_price_EUR("blue", 15),
            _ => Then_all_items_in_the_basket_should_cost_EUR(_basket.Items.Sum(x => x.Price)),

            _ => When_I_add_COLOR_sweater_with_price_EUR("yellow", 20),
            _ => Then_all_items_in_the_basket_should_cost_EUR(_basket.Items.Sum(x => x.Price)));
    }

    /* Steps part */

    private Basket _basket;

    private void Given_an_empty_basket()
    {
        _basket = new Basket();
    }

    private void When_I_add_COLOR_sweater_with_price_EUR(string color, int price)
    {
        _basket.Add(new Sweater(color, price));
    }

    private void Then_all_items_in_the_basket_should_cost_EUR(int cost)
    {
        Assert.Equal(cost, _basket.TotalCost);
    }
}

SCENARIO: Pricing basket
  STEP 1/6: GIVEN an empty basket...
  STEP 1/6: GIVEN an empty basket (Passed after 11ms)
  STEP 2/6: WHEN I add "red" sweater with price "20" EUR...
  STEP 2/6: WHEN I add "red" sweater with price "20" EUR (Passed after <1ms)
  STEP 3/6: AND I add "blue" sweater with price "15" EUR...
  STEP 3/6: AND I add "blue" sweater with price "15" EUR (Passed after <1ms)
  STEP 4/6: THEN all items in the basket should cost "35" EUR...
  STEP 4/6: THEN all items in the basket should cost "35" EUR (Passed after 4ms)
  STEP 5/6: WHEN I add "yellow" sweater with price "20" EUR...
  STEP 5/6: WHEN I add "yellow" sweater with price "20" EUR (Passed after <1ms)
  STEP 6/6: THEN all items in the basket should cost "55" EUR...
  STEP 6/6: THEN all items in the basket should cost "55" EUR (Passed after <1ms)
  SCENARIO RESULT: Passed after 74ms

As presented above, the Then_all_items_in_the_basket_should_cost_EUR() step uses the current basket state to calculate the expected cost. This is because all the parameters evaluation is done during step execution.
It is worth to mention that an additional effort has been made to ensure that parameter expression is evaluated once, so even the side effect expressions are safe to use (as long as this is the developer intention).

Practices and patterns

Business vs internal state

As it was described above, LightBDD supports various ways of sharing state within the scenario. Those ways could be grouped into two kinds:

state shared behind the scene via class members,
state shared explicitly via step parameters.

The difference here is that state shared via step parameters affects the readability of the scenario as well as is always visible in scenario execution progress as well as in reports.

Because of the above, there are following recommendations:

Use state shared explicitly if:

displaying state adds value from the scenario perspective (WHEN I add "yellow" sweater with price "20" EUR), and
state value can be represented as readable text, or used advance parameter types (tabular, verifiable etc), and
expression obtaining state to pass as parameter is simple and readable.

Use state shared via class members if:

state represent internal implementation detail (like: Id of item added to the basket), or
state cannot be represented as readable text (WHEN I add "red" sweater with price "Demo.Tests.Price"), or
state does not add any value to the scenario (WHEN I add "red" sweater [basket: "Demo.Tests.Basket"]).

Generally, the rule of thumb is to use step parameters, if those parameter values are part of the scenario from the business perspective ("red" sweater, "5" items in basket etc.), but avoid them if they would make scenario unreadable or introduce unnecessary noise (item with id "4f648637-6292-49a6-9889-41c9d10b087b").

Ensuring state is initialized before use

The Scenario state management section describes that state sharing can be done via class members, where steps like Given_ or When_ usually sets those members values to proper state while Then_ steps verifies it.
One of the downside of this approach is that it's pretty easy to write the scenario where steps will try to access the members that has not been initialized before, causing NullReferenceException to be thrown.

Let's take a look at this code:

public class My_feature : FeatureFixture
{
    private Customer _customer;

    private void Given_customer_management_window_is_open() { /* ... */ }

    private void When_I_load_customer_using_email(string email)
    {
        _customer = LoadCustomerByEmail(email);
    }

    private void When_update_customer_with_phone(string phone)
    {
        _customer.Phone = phone;
        SaveCustomer(_customer);
    }

    private void Then_customer_phone_should_have_value(string value)
    {
        Assert.Equal(value, _customer.Phone);
    }

    /* ... */
}

If we run this scenario:

Runner.RunScenario(
  _ => Given_customer_management_window_is_open(),
  _ => When_update_customer_with_phone("123456"), // <- customer is not loaded
  _ => Then_customer_phone_should_have_value("123456")
);

... we will get:

SCENARIO: Updating customer phone number
  STEP 1/3: GIVEN customer management window is open...
  STEP 1/3: GIVEN customer management window is open (Passed after 14ms)
  STEP 2/3: WHEN update customer with phone "123456"...
  STEP 2/3: WHEN update customer with phone "123456" (Failed after 4ms)
  SCENARIO RESULT: Failed after 104ms
    Step 2: System.NullReferenceException : Object reference not set to an instance of an object.
    	at Demo.Tests.My_feature.When_update_customer_with_phone(String phone)

With LightBDD 2.4.2 it is possible to use State<T> helper struct that will protect members from access without former initialization:

public class My_feature : FeatureFixture
{
    private State<Customer> _customer;
    private Customer Customer => _customer.GetValue(nameof(Customer));

    private void Given_customer_management_window_is_open() { /* ... */ }

    private void When_I_load_customer_using_email(string email)
    {
        _customer = LoadCustomerByEmail(email);
    }

    private void When_update_customer_with_phone(string phone)
    {
        Customer.Phone = phone;
        SaveCustomer(_customer);
    }

    private void Then_customer_phone_should_have_value(string value)
    {
        Assert.Equal(value, Customer.Phone);
    }

    /* ... */
}

If we run the same scenario now, we will get more meaningful exception:

SCENARIO: Updating customer phone number
  STEP 1/3: GIVEN customer management window is open...
  STEP 1/3: GIVEN customer management window is open (Passed after 10ms)
  STEP 2/3: WHEN update customer with phone "123456"...
  STEP 2/3: WHEN update customer with phone "123456" (Failed after 1ms)
  SCENARIO RESULT: Failed after 86ms
    Step 2: System.InvalidOperationException : The Customer state is not initialized.
    	at LightBDD.Framework.State`1.GetValue(String memberName)
    	at Demo.Tests.My_feature.get_Customer()
    	at Demo.Tests.My_feature.When_update_customer_with_phone(String phone)

The State<T> is a struct with following members:

IsInitialized property returning false if state has not been initialized,
GetValue() method retrieving value or throwing meaningful exception for uninitialized state (it's possible to specify the member name that will be included in exception)
GetValueOrDefault(T defaultValue = default(T)) method returning default value if state is not initialized or has null value (in case T is a reference type),

implicit conversions from T and to T, allowing code like:

State<int> _field;
/* ... */
_field = 5; // sets value
/* ... */
int variable = _field; // gets value or throws if not initialized

Usage patterns:

Simple usage when state is only set or get

class My_feature: FeatureFixture
{
    private State<int> _productId;
    private State<Product> _product;

    public void Given_product_id(int id)
    {
        // implicit cast
        _productId = id;
    }

    public void When_product_is_loaded()
    {
        // implicit cast of State<int> _id to int - it will throw meaningful exception if uninitialized
        _product = GetProductById(_id);
    }
}

If additional operations are made on state, it's better to wrap it in property returning T type

public class My_feature: FeatureFixture
{
    private State<Product> _product;

    //will throw meaningful exception if used without former initialization
    private Product Product => _product.GetValue(nameof(Product));

    public void When_I_request_a_product_by_reference(string reference)
    {
        //it's possible to assign Product to State<Product> field
        _product = _client.GetProduct(reference);
    }

    public void Then_the_product_should_have_name(string name)
    {
        Assert.That(Product.Name, Is.EqualTo(name));
    }
}

Maintaining code readability

Maintaining code readability is an important aspect of writing test suite.

Below there are few suggestions of how to keep LightBDD scenarios clean:

The fixture classes should contain scenarios build around similar logic and use similar set of state members
If scenarios belonging to the same class use different set of fields/properties for keeping state, such class becomes more difficult to maintain and understand. In such situation it's worth considering to split the class into smaller parts.
Keep step methods small and simple by extracting common logic out
Each project is simple and clean in the beginning. Over the time, the step methods are getting bigger and often the code gets duplicated in other feature class or step that does similar but not the same job.
In such situation it is worth to identify such code duplicates and extract them into separate, generic purpose methods. If code was duplicated within the same class, that would be enough, however if it was duplicated across different fixtures, it is worth considering following actions:

move the logic out to helper utility classes (common examples are: XXXApiClient for web apis, LoginPageDriver for Selenium UI tests, etc.) and instantiate them when needed,
extract the logic out to extension methods, where the extension method could be a helper method working on HttpClient or even the step method itself (which works pretty well with context classes).

Use context classes for heavier tests
Sometimes extracting a few methods is not enough. If a whole group of the same fields and steps is used in many fixtures, it is worth to extract them all into dedicated context classes and use it instead.
Use composite steps if step implementation refers to other steps
Often it happens that we write a step method that is composed from other steps in order to simplify usage. Sometimes it happen that such step is doing a lot of stuff which also means that it may take a while to execute it as well as it may fail due to many reasons. The composite steps are designed to increase the transparency of such step by including all of the sub-steps in the progress execution logs and reports.
Avoid using inheritance as a way to sharing common code
At the beginning of the project it is tempting to make a base class with a few common step methods and fields and make all features deriving from it. While it looks OK at that stage, over the time it tends to grow causing fixture classes to have deep inheritance tree and leading to code duplication as at some point it would be needed to derive from multiple base classes which is not possible in C#.
All of the previously described solutions are much better than inheritance, as they offer more flexibility.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Scenario State Management (v2.x)