Skip to content

What Is New

Wojtek edited this page Mar 26, 2019 · 42 revisions

Releases

LightBDD 3.0.1

This release is a quick fix of #177 where scenarios built fluently were not executing properly.

LightBDD 3.0

I am happy to announce LightBDD 3.0, that offers simplified experience with working on BDD scenarios, signed assemblies as well as API cleared off deprecated code!

To learn how to migrate LightBDD 2.x to 3.0, please visit Migrating LightBDD Versions wiki page.

Please beware that this release is binary incompatible with LightBDD 2.x, and requires update of all LightBDD packages to version 3.0 to work properly.

List of changes:

  • With #152 a major work has been done in LightBDD to bring it to version 3.0:
    • unified scenario extension namespaces to LightBDD.Framework.Scenarios to avoid confusion and simiplify usage,
    • unified configuration extension namespaces to LightBDD.Core.Configuration and LightBDD.Framework.Configuration,
    • removed Runner.NewScenario() in favor of Runner being fluent by default (see Scenario Steps Definition > Fluent Scenarios wiki page),
    • removed Runner.RunScenarioActionsAsync() method in favor of Runner.AddSteps(/*async void steps*/).RunAsync() (see Scenario Steps Definition > Fluent Scenarios wiki page),
    • removed obsolete code,
    • reworked and simplified a lot of internals;
  • With #157 all packages except LightBDD.Fixie2 are signed, which means they can be used to test signed projects!
  • With #163 the "End of stack trace" lines got removed from the stack traces in the LightBDD reports.
  • With #165 the basic syntax compiler will report error when lambda expressions are used for basic scenarios. Previously it was allowed to use lambdas, causing unreadable step names being rendered in reports.
  • With #170 LightBDD.NUnit2 has been deprecated.
  • With #174 Allowed IgnoreScenarioAttribute on class level to ignore all scenarios in given feature.

Happy coding!

LightBDD 2.5.0

1. Added Project Templates and Code Snippets to VSIX

With changes #60, #61 the LightBDD for Visual Studio VSIX extension has been extended to offer Project Templates and Code Snippets to ease work with LightBDD.

Note: As Code snippets are not treated as item/project templates, a VSIX with new ID had to be registered in the Visual Studio Gallery. Please ensure that the new extension is downloaded from this location.

Please feel free to take a look at Visual Studio Extensions wiki page for more details.

2. Added compact scenarios support

With change #156 the compact steps have been introduced, allowing to define self-contained scenarios. This change will make LightBDD easier to use in unit-test like scenarios.

Please feel free to take a look at Scenario Steps Definition > Compact Scenarios wiki page for more details.

3. Other minor improvements in 2.5.0

A following list of small improvements has also been included in the 2.5.0 release:

  • #148 (Framework)(Fix) Expect.To.BeAnyTrue() with no sub-expectations should fail
  • #153 (LightBDD.XUnit2)(Examples) Updated dotnet test examples for xunit2 to reflect the newest framework changes
  • #160 (LightBDD.Fixie2)(Change) Updated Fixie dependency to v2.0.2 to detect tests in VS 15.8 and higher

LightBDD 2.4.3

1. NULL value not assignable to Nullable parameter

With change #145 an edge case has been fixed where LightBDD was not honoring parameterized scenarios with null values being assigned to Nullable<> parameters.

Let's consider a following scenario:

[Scenario]
[TestCase(1)]
[TestCase(null)]
public void Some_scenario(int? parameter)
{
    Runner.RunScenario(
        _ => Given_xxx(),
        _ => When_yyy(),
        _ => Then_zzz());
}

Before the fix, running this scenario were passing for TestCase(1) but failing for TestCase(null) with following exception:

System.InvalidOperationException : Provided argument <null> is not assignable to parameter index 0 of method Void Some_scenario(System.Nullable`1[System.Int32])
   at LightBDD.Core.Extensibility.ScenarioDescriptor.BuildParameters(MethodBase methodInfo, Object[] arguments)
   at LightBDD.Core.Extensibility.ScenarioDescriptor..ctor(MethodBase methodInfo, Object[] arguments)
   at LightBDD.NUnit3.Implementation.NUnit3MetadataProvider.CaptureCurrentScenario()
   at LightBDD.Core.Extensibility.Implementation.ScenarioRunner.WithCapturedScenarioDetails()
   at LightBDD.Framework.Scenarios.Extended.Implementation.ExtendedScenarioRunnerFactory`1.BuildScenario(Expression`1[] steps)
   at LightBDD.Framework.Scenarios.Extended.ExtendedScenarioExtensions.RunScenario[TContext](IBddRunner`1 runner, Expression`1[] steps)
   at Example.LightBDD.NUnit3.Features.My_feature.Some_scenario(Nullable`1 parameter)

With the fix, such scenarios can be executed properly:

SCENARIO: Some scenario [parameter: "<null>"]
  STEP 1/3: GIVEN xxx...
  STEP 1/3: GIVEN xxx (Passed after 11ms)
  STEP 2/3: WHEN yyy...
  STEP 2/3: WHEN yyy (Passed after <1ms)
  STEP 3/3: THEN zzz...
  STEP 3/3: THEN zzz (Passed after <1ms)
  SCENARIO RESULT: Passed after 54ms

LightBDD 2.4.2

1. Integration with SourceLink

With change #135 LightBDD packages are integrated now with SourceLink which should make debugging seamless for people using Visual Studio 15.3+ or other tools supporting it.

2. Added State type to guard shared state from NullReferenceException

With change #134 the LightBDD.Framework.State<T> struct has been added to help protecting shared state fields from accessing without former initialization.

The State<T> struct offers following members:

  • IsInitialized property returning false if state has not been initialized,
  • GetValue() method retrieving value or throwing meaningful exception for uninitialized state (it's possible to specify the member name that will be included in exception)
  • GetValueOrDefault(T defaultValue = default(T)) method returning default value if state is not initialized or has null value (in case T is a reference type),
  • implicit conversions from T and to T, allowing code like:
    State<int> _field;
    /* ... */
    _field = 5; // sets value
    /* ... */
    int variable = _field; // gets value or throws if not initialized

Usage patterns:

  1. Simple usage when state is only set or get
class My_feature: FeatureFixture
{
    private State<int> _productId;
    private State<Product> _product;

    public void Given_product_id(int id)
    {
        // implicit cast
        _productId = id;
    }

    public void When_product_is_loaded()
    {
        // implicit cast of State<int> _id to int - it will throw meaningful exception if uninitialized
        _product = GetProductById(_id);
    }
}
  1. If additional operations are made on state, it's better to wrap it in property returning T type
public class My_feature: FeatureFixture
{
    private State<Product> _product;

    //will throw meaningful exception if used without former initialization
    private Product Product => _product.GetValue(nameof(Product));

    public void When_I_request_a_product_by_reference(string reference)
    {
        //it's possible to assign Product to State<Product> field
        _product = _client.GetProduct(reference);
    }

    public void Then_the_product_should_have_name(string name)
    {
        Assert.That(Product.Name, Is.EqualTo(name));
    }
}

3. Support for parallel scenario progress notification in LightBDD.MsTest2

With change #128, LightBDD.MsTest2 uses now TestContext.WriteLine() method to notify execution progress, which allows proper output capture in scenarios running in parallel.

In addition to that, the FeatureFixture class has been extended with TestContext property allowing to use it in all scenarios.

4. Corrected report writers to support properly absolute and UNC paths

With fix #137, report file writers can be now properly configured with absolute and UNC paths.

Let's see following cases to understand better the fix:

  • specifying path like \reports\result.html on Windows will make it expanded with the working directory drive letter, but preserve the rest as is, so for current directory being d:\temp it would be: d:\reports\result.html,
  • specifying absolute path like /out/result.html on Linux will make it being used as is, i.e /out/result.html,
  • specifying path starting from two backslashes will make it being treated as UNC and left as is, i.e. \\machine\reports\result.html will produce \\machine\reports\result.html.

To read more about reports configuration, please check the Generating Reports wiki page.

5. Features gets reported multiple times in LightBDD.XUnit2

With fix #127 the race condition around reporting start of feature processing has been fixed. While fix is general, the bug was only observable in LightBDD.XUnit2, due to the fact how LightBDD integrates with xunit.

LightBDD 2.4.1

1. Added Microsoft DI compatible containers integration

With change #122 it is now possible to install LightBDD.Extensions.DependencyInjection package and configure LightBDD to use any DI container that is compatible with Microsoft.Extensions.DependencyInjection.Abstractions and implements IServiceProvider interface.

Below, there is a sample code showing usage of the new integration:

using LightBDD.Extensions.DependencyInjection;

public class ConfiguredLightBddScopeAttribute : LightBddScopeAttribute
{
	protected override void OnConfigure(LightBddConfiguration configuration)
	{
		IServiceProvider container = new ServiceCollection()
			/* .Add... */
			.BuildServiceProvider();

		configuration.DependencyContainerConfiguration().UseContainer(container);
		/* ... */
	}
	/* ... */
}

For more information, please check the DI Containers wiki page.

2. Improved experience with running parameterized scenarios in LightBDD.XUnit2

With change #123 the experience of running parameterized LightBDD.XUnit2 scenarios has been improved significantly:

  • the parameterized scenarios are pre-enumerated now before execution, that improves the experience of running them from Resharper,
  • the preEnumerateTheories flag (described on xunit configration wiki page) is honored now by LightBDD,
  • the skipped cases, such as: [InlineData(123, Skip="reason")] are properly handled now, making scenario case skipped.

LightBDD 2.4.0

Hello, I am happy to announce the LightBDD 2.4.0 that introduces few new concepts described below.

1. Added in-line verifiable parameters

With change #38 it is possible to use in-line verifiable parameters allowing to:

  • perform verification of all step parameters before failing the step,
  • report parameter validation failure in the step name, rather than in the summary block.

Let's consider following scenario:

Runner.RunScenario(
  _ => Given_a_user_with_id_name_surname_and_email(124, "Joe", "Johnson","jj@gmail.com"),
  _ => When_I_request_user_details_for_id(124),
  _ => Then_I_should_receive_user_with_id_name_surname_and_email(124, "Joe", "Johnson","jj@gmail.com")
);

Before LightBDD 2.4.0, the then step had to be implemented as follows:

void Then_I_should_receive_user_with_id_name_surname_and_email(int id, string name, string surname, string email)
{
   Assert.That(_receivedUser.Id, Is.EqualTo(id), "Wrong ID");
   Assert.That(_receivedUser.Name, Is.EqualTo(name), "Wrong name");
   Assert.That(_receivedUser.Surname, Is.EqualTo(surname), "Wrong surname");
   Assert.That(_receivedUser.Email, Is.EqualTo(email), "Wrong email");
}

... where any of the assertion failure were stopping step execution, giving no information about other assertions.

With LightBDD 2.4.0, it is possible to implement that step as follows:

void Then_I_should_receive_user_with_id_name_surname_and_email(Verifiable<int> id, Verifiable<string> name, Verifiable<string> surname, Verifiable<string> email)
{
   id.SetActual(_receivedUser.Id);
   name.SetActual(_receivedUser.Name);
   surname.SetActual(_receivedUser.Surname);
   email.SetActual(_receivedUser.Email);
}

When executed, the SetActual() methods will perform the comparison against the expectations, however they will not throw in case of failure, allowing whole step to execute. The verification outcomes will be captured by LightBDD after step finish where any failures (including situation where SetActual() has not been called at all) will:

  • make step to fail,
  • get included in-line in step name,
  • get included in step error details.

Below, there are sample progress messages and report:

  SCENARIO: Retrieving user details
  STEP 1/3: GIVEN a user with id "124" name "Joe" surname "Johnson" and email "jj@gmail.com"...
  STEP 1/3: GIVEN a user with id "124" name "Joe" surname "Johnson" and email "jj@gmail.com" (Passed after 13ms)
  STEP 2/3: WHEN I request user details for id "124"...
  STEP 2/3: WHEN I request user details for id "124" (Passed after <1ms)
  STEP 3/3: THEN I should receive user with id "expected: equals '124'" name "expected: equals 'Joe'" surname "expected: equals 'Johnson'" and email "expected: equals 'jj@gmail.com'"...
  STEP 3/3: THEN I should receive user with id "124" name "Joe" surname "expected: equals 'Johnson', but got: 'JOHNSON'" and email "expected: equals 'jj@gmail.com', but got: 'JJ@GMAIL.COM'" (Failed after 7ms)
  SCENARIO RESULT: Failed after 120ms
    Step 3: System.InvalidOperationException : Parameter 'surname' verification failed: expected: equals 'Johnson', but got: 'JOHNSON'
    	Parameter 'email' verification failed: expected: equals 'jj@gmail.com', but got: 'JJ@GMAIL.COM'

Verifiable class and Expect.To expressions

The above example shows usage of Verifiable<T> class (from LightBDD.Framework.Parameters namespace).

As presented above, the Verifiable<T> class has an implicit operator, allowing to initialize it with T value, i.e.: Verifiable<int> myVerifiable = 5;

Such assignment will make a verifiable instance expecting actual value to be equal 5.

While it is handy for most of the cases, the LightBDD 2.4.0 got implemented a set of expressions allowing to make more complex expectations with the usage of Expect.To.*** expressions (from LightBDD.Framework.Expectations namespace).

Let's see following example:

Runner.RunScenario(
    _ => Given_a_user_with_id_name_surname_and_email(124, "Joe", "Johnson", "jj@gmail.com"),
    _ => When_I_request_user_details_for_id(124),
    _ => Then_I_should_receive_user_with_id_name_surname_and_email(
        124, "Joe",
        Expect.To.Not.BeEmpty<string>(),
        Expect.To.MatchIgnoreCase("[a-z]+@([a-z]+)(\\.[a-z]+)+")));

and the outcome:

SCENARIO: Retrieving user details
  STEP 1/3: GIVEN a user with id "124" name "Joe" surname "Johnson" and email "jj@gmail.com"...
  STEP 1/3: GIVEN a user with id "124" name "Joe" surname "Johnson" and email "jj@gmail.com" (Passed after 13ms)
  STEP 2/3: WHEN I request user details for id "124"...
  STEP 2/3: WHEN I request user details for id "124" (Passed after <1ms)
  STEP 3/3: THEN I should receive user with id "expected: equals '124'" name "expected: equals 'Joe'" surname "expected: not empty" and email "expected: matches '[a-z]+@([a-z]+)(\.[a-z]+)+' ignore case"...
  STEP 3/3: THEN I should receive user with id "124" name "Joe" surname "JOHNSON" and email "JJ@GMAIL.COM" (Passed after 6ms)
  SCENARIO RESULT: Passed after 99ms

More details can be found on Advanced Step Parameters wiki page.

2. Added tabular parameters

In additions to the verifiable parameters and the expectation expressions described above, the change #70 introduces tabular parameters in three forms:

  • InputTable<TRow> allowing to pass collections to the step methods, that will be rendered nicely in progress and the reports,
  • VerifiableDataTable<TRow> allowing to define expected collection that, will be compared to the actual collection in the step, on the row and column basis,
  • TableValidator<TRow> allowing to define expectation for all columns, that will have to be fulfilled for each provided actual row value.

It also provides a static class Table (LightBDD.Framework.Parameters namespace) allowing to build those tabular parameters in easy way.

Let's take a look at the example:

[Scenario]
public void Retrieving_user_details()
{
    Runner.RunScenario(

        _ => Given_users(Table.For(
            new User(1, "Joe", "Andersen", "jj@foo.com"),
            new User(2, "Henry", "Hansen", "henry123@foo.com"),
            new User(3, "Marry", "Davis", "ma31@bar.com"),
            new User(4, "Monica", "Larsen", "monsmi22@bar.com"))),

        _ => When_I_search_for_users_by_surname_pattern(".*sen"),

        _ => Then_I_should_receive_users(Table.ExpectData(
            new User(1, "Josh", "Andersen", "jj@foo.com"),
            new User(2, "Henry", "Hansen", "henry123@foo.com"))));
}

// Implementation
private InputTable<User> _users;
private User[] _received;

private void When_I_search_for_users_by_surname_pattern(string pattern)
{
    _received = _users.Where(x => Regex.IsMatch(x.Surname, pattern)).ToArray();
}

private void Given_users(InputTable<User> users)
{
    _users = users;
}

void Then_I_should_receive_users(VerifiableDataTable<User> users)
{
    users.SetActual(_received);
}

class User
{
    public User(int id, string name, string surname, string email)
    {
        Id = id;
        Name = name;
        Surname = surname;
        Email = email;
    }
    public int Id { get; }
    public string Name { get; }
    public string Surname { get; }
    public string Email { get; }
}

The given step uses parameter of type InputTable<User> which is instantiated with Table.For() method.

The then step uses parameter of type VerifiableDataTable<User> which is instantiated with Table.ExpectData() method and, similarly to Verifiable<T> type, gets the actual value provided with users.SetActual() method.

When executed, the progress and report will be as follows:

SCENARIO: Retrieving user details
  STEP 1/3: GIVEN users "<table>"...
  STEP 1/3: GIVEN users "<table>" (Passed after 13ms)
    users:
    +----------------+--+------+--------+
    |Email           |Id|Name  |Surname |
    +----------------+--+------+--------+
    |jj@foo.com      |1 |Joe   |Andersen|
    |henry123@foo.com|2 |Henry |Hansen  |
    |ma31@bar.com    |3 |Marry |Davis   |
    |monsmi22@bar.com|4 |Monica|Larsen  |
    +----------------+--+------+--------+
  STEP 2/3: WHEN I search for users by surname pattern ".*sen"...
  STEP 2/3: WHEN I search for users by surname pattern ".*sen" (Passed after 6ms)
  STEP 3/3: THEN I should receive users "<table>"...
  STEP 3/3: THEN I should receive users "<table>" (Failed after 24ms)
    users:
    +-+-----------------------+--------+-------------+-------------+
    |#|Email                  |Id      |Name         |Surname      |
    +-+-----------------------+--------+-------------+-------------+
    |!|jj@foo.com             |1       |Joe/Josh     |Andersen     |
    |=|henry123@foo.com       |2       |Henry        |Hansen       |
    |+|monsmi22@bar.com/<none>|4/<none>|Monica/<none>|Larsen/<none>|
    +-+-----------------------+--------+-------------+-------------+
  SCENARIO RESULT: Failed after 196ms
    Step 3: System.InvalidOperationException : Parameter 'users' verification failed: [0].Name: expected: equals 'Josh', but got: 'Joe'
    		[2].Email: unexpected value
    		[2].Id: unexpected value
    		[2].Name: unexpected value
    		[2].Surname: unexpected value

While VerifiableDataTable<T> allows to compare expected and actual collections, the TableValidator<T> allows to validate all actual records.

[Scenario]
public void Retrieving_user_details()
{
    Runner.RunScenario(

        _ => Given_users(Table.For(
            new User(0, "Joe", "Andersen", "jj@foo.com"),
            new User(2, "Henry", "Hansen", "henry123@foo2.com"),
            new User(3, "Marry", "Davis", "ma31@bar.com"),
            new User(4, "Monica", "Larsen", "monsmi22@bar.com"))),

        _ => When_I_search_for_users_by_surname_pattern(".*sen"),

        _ => Then_I_should_receive_users(Table.Validate<User>(builder => builder
            .WithColumn(x => x.Id, Expect.To.BeGreaterThan(0))
            .WithColumn(x => x.Name, Expect.To.Not.BeEmpty())
            .WithColumn(x => x.Surname, Expect.To.BeLikeIgnoreCase("*sen"))
            .WithColumn(x => x.Email, Expect.To.MatchIgnoreCase("[\\w]+@([a-z]+)(\\.[a-z]+)+"))
        )));
}

// Implementation
void Then_I_should_receive_users(TableValidator<User> users)
{
    users.SetActual(_received);
}
SCENARIO: Retrieving user details
  STEP 1/3: GIVEN users "<table>"...
  STEP 1/3: GIVEN users "<table>" (Passed after 19ms)
    users:
    +-----------------+--+------+--------+
    |Email            |Id|Name  |Surname |
    +-----------------+--+------+--------+
    |jj@foo.com       |0 |Joe   |Andersen|
    |henry123@foo2.com|2 |Henry |Hansen  |
    |ma31@bar.com     |3 |Marry |Davis   |
    |monsmi22@bar.com |4 |Monica|Larsen  |
    +-----------------+--+------+--------+
  STEP 2/3: WHEN I search for users by surname pattern ".*sen"...
  STEP 2/3: WHEN I search for users by surname pattern ".*sen" (Passed after 6ms)
  STEP 3/3: THEN I should receive users "<table>"...
  STEP 3/3: THEN I should receive users "<table>" (Failed after 18ms)
    users:
    +-+------------------+------+--------+-----------------------------------------------------------------+
    |#|Id                |Name  |Surname |Email                                                            |
    +-+------------------+------+--------+-----------------------------------------------------------------+
    |!|0/greater than '0'|Joe   |Andersen|jj@foo.com                                                       |
    |!|2                 |Henry |Hansen  |henry123@foo2.com/matches '[\w]+@([a-z]+)(\.[a-z]+)+' ignore case|
    |=|4                 |Monica|Larsen  |monsmi22@bar.com                                                 |
    +-+------------------+------+--------+-----------------------------------------------------------------+
  SCENARIO RESULT: Failed after 216ms
    Step 3: System.InvalidOperationException : Parameter 'users' verification failed: [0].Id: expected: greater than '0', but got: '0'
    		[1].Email: expected: matches '[\w]+@([a-z]+)(\.[a-z]+)+' ignore case, but got: 'henry123@foo2.com'

Table customization

The Table class allows to customize the created tables by:

  • defining the list of columns to display, including automatic detection as well as manual specification,
  • defining the key column(s) that will be used in expected-actual row comparison (where otherwise, the row order is used).

As mentioned above, the columns can be automatically detected, where following rules are used:

  • if row is simple type, one column Item is inferred,
  • if row type implements IDictionary<string,object> (which includes ExpandoObject type), the dictionary keys will be used as columns,
  • if row type itself is a collection implementing IList, the longest collection will be used to generate columns with names like [N] (also the Length column will be added),
  • finally, if none of above is true, the rows type will be scanned for fields and properties which will be used as columns.

Finally, the tables themselves support async provisioning of the actual values and VerifiableDataTable<T> offers the SetActual*() overloads with lookup function, allowing to find the actual row for given expected row.

More details can be found on Advanced Step Parameters wiki page.

3. Added DI container support

With change #59, LightBDD 2.4.0 got support for DI containers:

  • a basic DI container has been implemented in LightBDD itself,
  • the LightBDD.Autofac package has been provided if more advanced DI capabilities are needed,
  • the Runner.WithContext<T>() method has been changed, so that T no longer has to have parameterless constructor (DI is used to resolve the dependencies),
  • the StepExecution.GetScenarioDependencyResolver() has been introduced to get access to DI container in non-contextual scenarios,
  • the ResourcePool and ResourceHandle helper types has been added, that, when used with DI, allows to better manage access to heavy but limited resources.

Container scopes

The LightBDD DI containers are designed to support multiple scopes. Every time when a new scenario is run, or new composite-step is executed the new container scope is created (with the information about it's parent). Any dependencies created then are added to the actual scope and disposed when composite-step or scenario is finished.

DI container implementations

By default, LightBDD uses the BasicDependencyContainer which offers following functionality:

  • it allows to resolve types (classes and structures) with 1 public constructor,
  • it supports constructor dependency injections,
  • it supports singleton registrations in LightBDD configuration code: configuration.DependencyContainerConfiguration().UseDefaultContainer(cfg => cfg.RegisterInstance(new MySingleton(), new RegistrationOptions()));,
  • it supports disposal of dependencies upon disposal, if dependency implements IDisposable interface,
  • it supports container scopes.

If more complex DI is needed, it is possible to use LightBDD.Autofac package and use Autofac container, which can be registered as follows:

configuration.DependencyContainerConfiguration().UseAutofac(autofacBuilder);

Accessing container

The Runner.WithContext<T>() method (as well as composite-step equivalent) has been changed to use DI to instantiate the context. It means that context can now have parameterized constructor, where dependencies will be injected by the container.

For the steps that does not use context objects, it is possible to use StepExecution.GetScenarioDependencyResolver() in order to get the container and resolve dependencies manually.

ResourcePool and ResourceHandle

There are scenarios where some heavy dependencies have to be used (such as selenium drivers). Their characteristic is that they can be used by 1 test at a time and they are very costly to create.

The ResourcePool<T> class fits well to this scenario. It allows to create a pool of such objects that will be lent for the scenario execution, but given back after the test is done.

The LightBDD.AcceptanceTests project uses it currently to speed-up the test execution.

The sample registration code is as follows (source):

public class ConfiguredLightBddScopeAttribute : LightBddScopeAttribute
{
    protected override void OnConfigure(LightBddConfiguration configuration)
    {
        configuration.DependencyContainerConfiguration()
            .UseDefaultContainer(ConfigureContainer);
    }

    private void ConfigureContainer(ContainerConfigurator config)
    {
        config.RegisterInstance(
            new ResourcePool<ChromeDriver>(CreateDriver),
            new RegistrationOptions());
    }

    private ChromeDriver CreateDriver()
    {
        var driver = new ChromeDriver();
        driver.Manage().Timeouts().ImplicitWait = TimeSpan.FromMilliseconds(0);
        return driver;
    }
}

... and the sample usage is here (source):

// the context is created for each scenario

public class HtmlReportContext
{
    private readonly ResourceHandle<ChromeDriver> _driverHandle;

    public HtmlReportContext(ResourceHandle<ChromeDriver> driverHandle)
    {
        _driverHandle = driverHandle;
        /* ... */
    }

    public async Task When_a_html_report_is_opened()
    {
        // this will obtain the resource
        Driver = await _driverHandle.ObtainAsync();
        /* ... */
    }
}

For more information, please check the DI Containers wiki page.

4. Added Fixie integration

With change #67 it is possible now to use LightBDD with Fixie framework by using new LightBDD.Fixie2 package!

To see sample usage, please take a look at the Example.LightBDD.Fixie2 project.

The wiki page describing integration with Fixie can be found in Test Framework Integrations wiki section.

5. Added experimental inter-class test execution for LightBDD.XUnit2

With change #117 it is possible now to add

[assembly: ClassCollectionBehavior(AllowTestParallelization = true)]

attribute, that will make all the scenario methods (including parameterized scenarios) to run in parallel, as long as:

  • test class does not implement IClassFixture nor ICollectionFixture attribute,
  • test class does not have defined named collection [Collection] attribute,
  • parallelization is enabled on xunit level.

Please note that xunit itself does not support inter-class parallel test execution and this attribute is rather experimental.

6. StepExecution.Current.Comment() does not work after leaving sub-step in composite step

This change is a bug fix #120 that was causing StepExecution.Current.Comment() to throw if used in composite steps. Please check the issue itself for more details.

LightBDD 2.3.6

1. Allowed IStepDecoratorAttribute and IScenarioDecoratorAttribute to be applied on class level

With change #108 it is now possible to apply decorator attributes implementing IStepDecoratorAttribute and/or IScenarioDecoratorAttribute on class level to cover all the scenarios or steps declared in that class.

The wiki pages have been updated with Extending Test Behavior section covering this topic.

LightBDD 2.3.5

1. Updated LightBDD.XUnit2 integration to recognize xunit Category Traits

With change #104, [Trait("Category", "...")] attribute applied on Feature class or Scenario method is recognized now by LightBDD and included in reports.

Additionally, the Tests Structure and Conventions wiki page has been updated to better describe scenario categories behavior.

LightBDD 2.3.4

1. Implemented console immediate progress notification for LightBDD.NUnit3

With change #34, dotnet test or nunit3-console.exe commands prints now the immediate scenario progress as it occurs, allowing to track long running tests on CI.

If Reshaper is used to run the test, it's output window also contains the progress messages printed as they occurs (however please note that there is a bug reported for Resharper as when tests are run in parallel, their output is incorrectly displayed: see RSRP-466415).

The immediate progress is printed in following format, allowing to track scenarios executed in parallel:

Fi=000,Fa=000,Pe=003 #  1> SCENARIO: [Ticket-6] No product in stock
Fi=000,Fa=000,Pe=003 #  2> SCENARIO: [Ticket-3] Anonymous login name should allow to log in
Fi=000,Fa=000,Pe=003 #  3> SCENARIO: [Ticket-12] Ordering products
Fi=000,Fa=000,Pe=003 #  2>   STEP 1/5: GIVEN the user is about to login...
Fi=000,Fa=000,Pe=003 #  3>   STEP 1/4: GIVEN customer is logged in...
Fi=000,Fa=000,Pe=003 #  1>   STEP 1/4: GIVEN product is out of stock...
Fi=000,Fa=000,Pe=003 #  3>   STEP 1.1/1.5: GIVEN the user is about to login...
Fi=000,Fa=000,Pe=003 #  2>   STEP 1/5: GIVEN the user is about to login (Passed after 13ms)
Fi=000,Fa=000,Pe=003 #  1>   STEP 1/4: GIVEN product is out of stock (Passed after 13ms)

while the scenario summary is still printed in simple form:

Skipped  Ordering_products
Error Message:
 Not implemented yet
Standard Output Messages:
 SCENARIO: [Ticket-12] Ordering products
   STEP 1/4: GIVEN customer is logged in...
   STEP 1.1/1.5: GIVEN the user is about to login...
   STEP 1.1/1.5: GIVEN the user is about to login (Passed after <1ms)
   STEP 1.2/1.5: AND the user entered valid login...
   STEP 1.2/1.5: AND the user entered valid login (Passed after <1ms)
   STEP 1.3/1.5: AND the user entered valid password...
   STEP 1.3/1.5: AND the user entered valid password (Passed after <1ms)
...

Please visit Example.LightBDD.NUnit3 sample project to see more details on running scenarios with LightBDD.NUnit3.

2. Enhanced scenario progress notification

As part of the work described above (issue #34), a few more improvements have been implemented:

  • The LightBDD.XUnit2 progress notification has been changed to print scenario exection summary in simplified format:

    Example.LightBDD.XUnit2.Features.Login_feature.Anonymous_login_name_should_allow_to_log_in [FAIL]
      Login should succeeded
      Expected: True
      Actual:   False
      Stack Trace:
        d:\dev\LightBDD_234\examples\Example.LightBDD.XUnit2\Features\Login_feature.Steps.cs(57,0): at Example.LightBDD.XUnit2.Features.Login_feature.Then_the_login_operation_should_be_successful()
        d:\dev\LightBDD_234\src\LightBDD.Framework\Scenarios\Basic\Implementation\BasicStepCompiler.cs(81,0): at LightBDD.Framework.Scenarios.Basic.Implementation.BasicStepCompiler.StepExecutor.Execute(Object context, Object[] args)
        --- End of stack trace from previous location where exception was thrown ---
           at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
        d:\dev\LightBDD_234\src\LightBDD.Framework\Scenarios\Basic\BasicScenarioExtensions.cs(54,0): at LightBDD.Framework.Scenarios.Basic.BasicScenarioExtensions.RunScenario(IBddRunner runner, Action[] steps)
        d:\dev\LightBDD_234\examples\Example.LightBDD.XUnit2\Features\Login_feature.cs(85,0): at Example.LightBDD.XUnit2.Features.Login_feature.Anonymous_login_name_should_allow_to_log_in()
      Output:
        SCENARIO: [Ticket-3] Anonymous login name should allow to log in
          STEP 1/5: GIVEN the user is about to login...
          STEP 1/5: GIVEN the user is about to login (Passed after <1ms)
          STEP 2/5: AND the user entered anonymous login...
          STEP 2/5: /* Presentation of failed scenario */
          STEP 2/5: AND the user entered anonymous login (Passed after <1ms)
          STEP 3/5: WHEN the user clicks login button...
          STEP 3/5: WHEN the user clicks login button (Passed after 1s 516ms)
          STEP 4/5: THEN the login operation should be successful...
          STEP 4/5: THEN the login operation should be successful (Failed after 4ms)
          SCENARIO RESULT: Failed after 1s 537ms
    

    The change affects VisualStudio Test Explorer output / Resharper output window and console when test output is printed. The immediate progress is still printed in extended form, allowing to track scenarios executed in parallel.

    Please visit Example.LightBDD.XUnit2 sample project to see more details on running scenarios with LightBDD.XUnit2.

  • The FeatureProgressNotifierConfiguration and ScenarioProgressNotifierConfiguration has been extended with methods allowing to combine currently configured notifiers with new ones as well as clear them:

    configuration.Get<FeatureProgressNotifierConfiguration>()
        .ClearNotifiers()
        .AppendNotifiers(new CustomFeatureProgressNotifier())
        .AppendNotifiers(new CustomFeatureProgressNotifier2());
    
    configuration.Get<ScenarioProgressNotifierConfiguration>()
        .ClearNotifierProviders()
        .AppendNotifierProviders(() => new CustomScenarioProgressNotifier())
        .AppendNotifierProviders<FeatureFixture>(fixture => new CustomScenarioProgressNotifier2(fixture));
    
  • All integrations (LightBDD.NUnit2, LightBDD.NUnit3, LightBDD.XUnit2, LightBDD.MsTest2) have been updated with configuration methods allowing to apply default progress notifiers after they have been cleaned up:

    configuration.Get<FeatureProgressNotifierConfiguration>()
        .ClearNotifiers()
        .AppendFrameworkDefaultProgressNotifiers();
    
    configuration.Get<ScenarioProgressNotifierConfiguration>()
        .ClearNotifierProviders()
        .AppendFrameworkDefaultProgressNotifiers();
    

3. Lifted requirement for parameterized constructors in LightBDD.XUnit2 FeatureFixture

With change #98, it is no longer required to provide a parameterized constructor for FeatureFixture class in LightBDD.XUnit2.

Before the change a following constructor had to be added to each feature class:

public My_feature1(ITestOutputHelper output) : base(output)
{
}

After the change, this boilerplate code is no longer required.

Finally, the VSIX extension feature class template for LightBDD.XUnit2 has been updated to no longer generate this constructor.

LightBDD 2.3.3

This is the last LightBDD update in year 2017 :wink:, happy New Year 2018!

1. Simplified exception stack trace for failed scenarios

With change #69, the stack trace of any exceptions thrown by steps has been simplified to make it easier to analyze what went wrong in the step.

The improvement covers two areas:

  • the exceptions no longer have LightBDD internal methods frames on their stack trace,
  • the LightBDD reports and progress notifications have now predefined filters for frames that are related to test framework assertion methods as well as system methods responsible to handle exceptions in async methods.

In addition to the improvements, the number of stack trace lines printed in the reports has been increased from 4 to 8.

Finally, it is possible now to configure exception stack trace printing behaviour with code:

configuration.ExceptionHandlingConfiguration()
    .UpdateExceptionDetailsFormatter(new DefaultExceptionFormatter()
        .WithTestFrameworkDefaults() // apply default framework exclusions such as "NUnit\\..*" for LightBDD.NUnit3
        .WithStackTraceLinesLimit(3) // change max number of printed lines
        .WithMembersExcludedFromStackTrace("LightBDD\\..*", "System\\..*") // filter out any lines starting with "LightBDD." or "System."
        .Format);

Please note that above configuration only affects reports/notifications generated by LightBDD but not by underlying test frameworks.


Sample scenario:

[Label("SCENARIO-1")]
[Scenario]
public void Template_extended_scenario()
{
	Runner.RunScenario(
		_ => Given_method(),
		_ => When_method(),
		_ => Then_method());
}
private void Given_method()
{
}

private void When_method()
{
}

private void Then_method()
{
	Assert.Fail("Not implemented yet");
}

After the change (Resharper# output window):

Not implemented yet
at ClassLibrary1.My_feature1.Then_method() in c:\tmp\LightBDD-Compat\ClassLibrary1\My_feature1.Steps.cs:line 18
at lambda_method(Closure , NoContext , Object[] )
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at LightBDD.Framework.Scenarios.Extended.ExtendedScenarioExtensions.RunScenario[TContext](IBddRunner`1 runner, Expression`1[] steps)
at ClassLibrary1.My_feature1.Template_extended_scenario() in c:\tmp\LightBDD-Compat\ClassLibrary1\My_feature1.cs:line 30


SCENARIO: [SCENARIO-1] Template extended scenario
  STEP 1/3: GIVEN method...
  STEP 1/3: GIVEN method (Passed after <1ms)
  STEP 2/3: WHEN method...
  STEP 2/3: WHEN method (Passed after <1ms)
  STEP 3/3: THEN method...
  STEP 3/3: THEN method (Failed after 1ms)
  SCENARIO RESULT: Failed after 6ms
    Step 3: NUnit.Framework.AssertionException : Not implemented yet
    	at ClassLibrary1.My_feature1.Then_method() in c:\tmp\LightBDD-Compat\ClassLibrary1\My_feature1.Steps.cs:line 18
    	at lambda_method(Closure , NoContext , Object[] )

The stack trace contains the information about step where error has been thrown and then in what scenario (with no LightBDD internals).

Before the change:

Not implemented yet
at ClassLibrary1.My_feature1.Then_method() in c:\tmp\LightBDD-Compat\ClassLibrary1\My_feature1.Steps.cs:line 18
at lambda_method(Closure , NoContext , Object[] )
at LightBDD.Core.Extensibility.Implementation.ScenarioRunner.InvocationResultTransformer.<InvokeAsync>d__5.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at LightBDD.Core.Execution.Implementation.RunnableStep.<InvokeStepMethodAsync>d__22.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at LightBDD.Core.Execution.Implementation.RunnableStep.<InvokeStepMethodAsync>d__22.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at LightBDD.Core.Execution.Implementation.RunnableStep.<InvokeStepAsync>d__19.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at LightBDD.Framework.Commenting.Implementation.StepCommentingDecorator.<ExecuteAsync>d__0.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at LightBDD.Core.Execution.Implementation.RunnableStep.<TimeMeasuredInvokeAsync>d__18.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at LightBDD.Core.Execution.Implementation.RunnableStep.<RunAsync>d__16.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at LightBDD.Core.Execution.Implementation.RunnableScenario.<RunAsync>d__17.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at LightBDD.Core.Extensibility.Implementation.ScenarioRunner.RunSynchronously()
at ClassLibrary1.My_feature1.Template_extended_scenario() in c:\tmp\LightBDD-Compat\ClassLibrary1\My_feature1.cs:line 30


SCENARIO: [SCENARIO-1] Template extended scenario
  STEP 1/3: GIVEN method...
  STEP 1/3: GIVEN method (Passed after 12ms)
  STEP 2/3: WHEN method...
  STEP 2/3: WHEN method (Passed after <1ms)
  STEP 3/3: THEN method...
  STEP 3/3: THEN method (Failed after 3ms)
  SCENARIO RESULT: Failed after 137ms
    Step 3: NUnit.Framework.AssertionException : Not implemented yet
    	at NUnit.Framework.Assert.ReportFailure(String message)
    	at ClassLibrary1.My_feature1.Then_method() in c:\tmp\LightBDD-Compat\ClassLibrary1\My_feature1.Steps.cs:line 18
    	at lambda_method(Closure , NoContext , Object[] )
    	at LightBDD.Core.Extensibility.Implementation.ScenarioRunner.InvocationResultTransformer.<InvokeAsync>d__5.MoveNext()

Before the change, a long (mostly meaningless) stack trace was reported and reports themselves contained frames such as at NUnit.Framework.Assert.ReportFailure(String message).

2. Improved integration with NCrunch

With change #90, it is now possible to reuse the same process/AppDomain to execute LightBDD tests multiple times, which improves the experience of using NCrunch to run LightBDD tests.

Before the change, adding LightBDD scenarios to project monitored by NCrunch was often ending up with NCrunch reporting following error and executing tests forever:

System.InvalidOperationException: FeatureCoordinator of LightBDD.XUnit2.Implementation.XUnit2FeatureCoordinator type is already installed
at LightBDD.Core.Execution.Coordination.FeatureCoordinator.Install(FeatureCoordinator coordinator)
at LightBDD.XUnit2.LightBddScopeAttribute.SetUp()
at LightBDD.XUnit2.Implementation.Customization.TestFrameworkExecutor.RunTestCases(IEnumerable`1 testCases, IMessageSink executionMessageSink, ITestFrameworkExecutionOptions executionOptions)
at Xunit.Sdk.TestFrameworkExecutor`1.RunTests(IEnumerable`1 testCases, IMessageSink executionMessageSink, ITestFrameworkExecutionOptions executionOptions)
at nCrunch.Module.XUnit2.Integration.XUnit2FrameworkRuntimeEnvironment.<>c__DisplayClass3_1.<RunTests>b__3()

3. Improved support for extension methods used as steps

With change #92, it is possible now to use extension methods as steps where LightBDD will render them with the same rules as regular steps.

Example code:

public class MyFeature : FeatureFixture
{
    public MyFeature(ITestOutputHelper output) : base(output)
    {
    }

    [Scenario]
    public void My_scenario()
    {
        Runner
            .WithContext<MyContext>()
            .RunScenario(
                x => x.Given_value(5),
                x => x.When_I_multiply_value_by_MULTIPLIER(2),
                x => x.Then_value_should_be_RESULT(10));
    }
}
class MyContext
{
    public int Value { get; set; }
}

static class MyContextExtensions
{
    public static void Given_value(this MyContext ctx, int value)
    {
        ctx.Value = value;
    }

    public static void When_I_multiply_value_by_MULTIPLIER(this MyContext ctx, int multiplier)
    {
        ctx.Value *= multiplier;
    }

    public static void Then_value_should_be_RESULT(this MyContext ctx, int result)
    {
        Assert.Equal(result, ctx.Value);
    }
}

Example output after the change:

SCENARIO: My scenario
  STEP 1/3: GIVEN value "5"...
  STEP 1/3: GIVEN value "5" (Passed after 9ms)
  STEP 2/3: WHEN I multiply value by "2"...
  STEP 2/3: WHEN I multiply value by "2" (Passed after <1ms)
  STEP 3/3: THEN value should be "10"...
  STEP 3/3: THEN value should be "10" (Passed after 3ms)
  SCENARIO RESULT: Passed after 58ms

Before the change, LightBDD was rendering also this method parameter making step less readable.

Also, as a part of this change a small fix has been applied where step method like void Given_value(MyContext ctx, int value) { } was rendered as GIVEN value [ctx: "MyContext"] "5" instead of GIVEN value "5" [ctx: "MyContext"]

4. Added support for disposable scenario and step context instances

With change #94, the contextual scenarios and composite steps supports now disposal of context instances implementing IDisposable interface.

The WithContext methods have been extended with new behavior where:

  • the .WithContext<MyContext>() will always dispose disposable context instances,
  • the .WithContext(new MyContext()) will not dispose context by default as instance is passed explicitly by caller. However, as the method has now an optional parameter bool takeOwnership = false, it is possible to make context automatically disposed by calling .WithContext(new MyContext(), true),
  • the .WithContext(() => new MyContext()) will dispose context by default as factory method is being passed indicating that it will produce a new instance on each call. Similarly to method above, it is possible to change default behavior and make context not disposed by calling .WithContext(() => new MyContext(), false).

LightBDD 2.3.2

1. Updated LightBDD.MsTest2 to use MsTest.TestFramework 1.2.0

It is a fix for #80 issue where it was not possible to use LightBDD.MsTest2 with MsTest.TestFramework 1.2.0 as it is binary incompatible with version 1.1.18.

It is worth to note that the dependency version upgrade brought a fix for improper status of ignored async scenarios (Microsoft/testfx#249).

2. Fixed directory separator for Unix systems

It is a fix for #87 issue where LightBDD was incorrectly defining path for reports on Unix systems. With this change, the default report paths uses System.IO.Path.DirectorySeparatorChar as directory separator.

LightBDD 2.3.1

1. Fixed "Tests accessing Application Settings fails on NET45x with SerializationException"

It is a fix for #73 issue where LightBDD targeting NET45x was not able to execute tests accessing Application Settings and failing with exception:

System.Runtime.Serialization.SerializationException : Type 'LightBDD.NUnit2.Implementation.TestContextProvider' in assembly 'LightBDD.NUnit2, Version=2.3.0.0, Culture=neutral, PublicKeyToken=null' is not marked as serializable.
		at System.AppDomain.GetHostEvidence(Type type)
		at System.Security.Policy.AppDomainEvidenceFactory.GenerateEvidence(Type evidenceType)
		at System.Security.Policy.Evidence.GenerateHostEvidence(Type type, Boolean hostCanGenerate)
		at System.Security.Policy.Evidence.GetHostEvidenceNoLock(Type type)

The issue was affecting:

  • LightBDD targeting NET45x only (the same tests were working correctly for LightBDD targeting >= NET46),
  • LightBDD.NUnit2, LightBDD.XUnit2, LightBDD.MsTest2 integrations.

LightBDD 2.3.0

1. Multi assert support for steps

The default behavior of LightBDD runner is to stop execution on first failed or ignored step. In most cases the fail fast approach is an expected behavior, however there are scenarios where it is useful to see all failures (such as when we are asserting multiple fields of the API response or checking various parameter combinations for given algorithm).

With this version of LightBDD a MultiAssertAttribute has been added and it can be applied on scenario or composite step method. When applied, all direct sub-steps belonging to the parent will be executed irrespectively to the previous step result and their failure status will be reported after all sub-steps are finished.

Example code:

[Label("Ticket-13")]
[Scenario]
[MultiAssert]
public void Adding_numbers()
{
    Runner.RunScenario(
        _ => Given_a_calculator(),
        _ => Then_adding_X_to_Y_should_give_RESULT(2, 3, 5),
        _ => Then_adding_X_to_Y_should_give_RESULT(-3, 2, -1),
        _ => Then_adding_X_to_Y_should_give_RESULT(0, 1, 1),
        _ => Then_adding_X_to_Y_should_give_RESULT(-2, -1, -3));
}

Example output:

Note: The MultiAssertAttribute applies only on direct sub-steps, which means that if current scenario/step has a sub-step that is a composite, it's sub-steps will execute in normal way. The MultiAssertAttribute has to be applied explicitly on that composite step as well in order to run those sub-steps in multi-assert mode.

To see example usage, please check Calculator_feature.cs.

2. Declarative way for ignoring scenarios and steps

The IgnoredScenarioAttribute has been added to all integrations (LightBDD.NUnit3, LightBDD.NUnit2, LightBDD.XUnit2, LightBDD.MsTest2) and can be applied on scenario method or any step method to immediately ignore it when runner start executing it.

This attribute should be now used instead of test framework specific [Ignore] or [Fact(Skip="reason")] attributes, because scenarios ignore with IgnoredScenarioAttribute will be included in execution reports.

Example code for ignored scenario:

[Scenario]
[IgnoreScenario("Not implemented yet")]
public async Task Successful_payment()
{
    await Runner.RunScenarioAsync(
        _ => Given_customer_has_some_products_in_basket(),
        _ => Given_customer_has_enough_money_to_pay_for_products(),
        _ => When_customer_requests_to_pay(),
        _ => Then_payment_should_be_successful());
}

Example output:

Example code with ignored step:

[Scenario]
public async Task Successful_payment()
{
    await Runner.RunScenarioAsync(
        _ => Given_customer_has_some_products_in_basket(),
        _ => Given_customer_has_enough_money_to_pay_for_products(),
        _ => When_customer_requests_to_pay(),
        _ => Then_payment_should_be_successful());
}

[IgnoreScenario("Not implemented yet")]
private async Task Given_customer_has_enough_money_to_pay_for_products()
{
    /* ... */
}

Example output:

3. Improved scenario/step parameter formatting

The scenario/step parameter formatting feature has been significantly reworked:

  • a default formatter for collections (implementing IEnumerable) and dictionaries (implementing IDictionary) has been added, so that any parameters of those types will be formatted by LightBDD without need of specifying [Format*] attributes,
  • the formatting of the collections (both, default and attribute based) has been extended to honor custom formatting of collection items,
  • the null values are formatted now as <null> instead of empty string,
  • it is possible now to specify multiple [Format*] attributes on a parameter, where first attribute that can format given type will be used (based on Order property and CanFormat() method of the attribute),
  • FormatAttribute has been extended with SupportedType property that is used to specify which type (including descendants) it applies to,
  • it is now possible to register custom formatters on a assembly level with configuration.ValueFormattingConfiguration() and RegisterExplicit() or RegisterGeneral() methods, where
    • RegisterExplicit() is used to register formatter for specific, closed type, such as string or DateTime,
    • RegisterGeneral() is used to register formatter for any type that would fulfil the criteria specified in CanFormat() method of the formatter,
  • a ISelfFormattable interface has been added, allowing to implement formatting method on the type.

The rules for picking a valid formatter for given parameter value are following now:

  1. if value is null, it is formatted as <null>, otherwise
  2. all attributes that are applied on the parameter declaration and extend ParameterFormatterAttribute are sorted by Order property and first matching is used (based on CanFormat() method), otherwise
  3. if type implements ISelfFormattable, it is used then, otherwise
  4. matching strict formatter is used if defined in ValueFormattingConfiguration, otherwise,
  5. the first matching general formatter that is defined in ValueFormattingConfiguration is used (they are evaluated in registration order), otherwise
  6. value is formatted with string.Format() method.

Example code:

[assembly: ConfiguredLightBddScope]
/* ... */

internal class ConfiguredLightBddScopeAttribute : LightBddScopeAttribute
{
    protected override void OnConfigure(LightBddConfiguration configuration)
    {
        configuration
            .ValueFormattingConfiguration()
            // default formatting for all dates
            .RegisterExplicit(typeof(DateTime), new DateTimeValueFormatter());
    }
}

internal class DateTimeValueFormatter : IValueFormatter
{
    public string FormatValue(object value, IValueFormattingService formattingService)
    {
        var date = (DateTime)value;
        return date.ToString("yyyy-MM-dd HH:mm");
    }
}

public class My_feature : FeatureFixture
{
    [Scenario]
    public void My_scenario()
    {
        Runner.RunScenario(
            _ => Given_there_are_invoices_for_dates(
                new DateTime(2015, 01, 01), 
                new DateTime(2015, 02, 01), 
                new DateTime(2015, 03, 02)),
            _ => When_I_request_invoices_before_date(new DateTime(2015, 03, 01)),
            _ => Then_I_should_get_invoices_with_dates(
                new DateTime(2015, 01, 01), 
                new DateTime(2015, 02, 01)));
    }

    // default collection formatting and default date formatting
    private void Given_there_are_invoices_for_dates(params DateTime[] dates)
    {
    }

    // default date formatting
    private void When_I_request_invoices_before_date(DateTime date)
    {
    }

    // custom formatting for dates, while preserving default collection formatting
    private void Then_I_should_get_invoices_with_dates([Format("date={0:yyyy/MM/dd}", SupportedType = typeof(DateTime))] params DateTime[] dates)
    {
    }
}

Example outcome:

SCENARIO: My scenario
  STEP 1/3: GIVEN there are invoices for dates "2015-01-01 00:00, 2015-02-01 00:00, 2015-03-02 00:00"...
  STEP 1/3: GIVEN there are invoices for dates "2015-01-01 00:00, 2015-02-01 00:00, 2015-03-02 00:00" (Passed after 9ms)
  STEP 2/3: WHEN I request invoices before date "2015-03-01 00:00"...
  STEP 2/3: WHEN I request invoices before date "2015-03-01 00:00" (Passed after <1ms)
  STEP 3/3: THEN I should get invoices with dates "date=2015/01/01, date=2015/02/01"...
  STEP 3/3: THEN I should get invoices with dates "date=2015/01/01, date=2015/02/01" (Passed after <1ms)
  SCENARIO RESULT: Passed after 54ms

4. Fluent interface for scenario definition

It is possible now to define scenarios in fluent way and mix basic, extended, async and sync steps together (however at the cost of test readability). The use case for fluent scenario definitions is when scenario is constructed in programmatic manner (like when number of steps varies on input data or other dynamic reasons).

Example code:

using LightBDD.Framework;
using LightBDD.Framework.Scenarios.Basic;
using LightBDD.Framework.Scenarios.Extended;
using LightBDD.Framework.Scenarios.Fluent;
using LightBDD.NUnit3;

/* .. */

[Label("Ticket-14")]
[Scenario]
public async Task Browsing_invoices()
{
    await Runner
        .NewScenario()
        .AddAsyncSteps(
            _ => Given_invoice("Invoice-1"),
            _ => Given_invoice("Invoice-2"))
        .AddSteps(
            When_I_request_all_historical_invoices)
        .AddSteps(
            _ => Then_I_should_see_invoices("Invoice-1", "Invoice-2"))
        .RunAsync();
}

The fluent scenario definition is:

  • initiated with .NewScenario() method that can be called directly on Runner or Runner.WithContext(/* ... */),
  • finished with .RunAsync() that executes the scenario.

Note: The basic syntax is available only if .NewScenario() is called without context.

To see example usage, please check Invoice_history_feature.cs.

LightBDD 2.2.0

1. Capturing scenario parameters

The LightBDD.NUnit3, LightBDD.XUnit2 and LightBDD.MsTest2 integrations captures now scenario method parameters, including them in execution progress as well as in reports. The parameters are formatted in exactly same way as in step methods.

Example code:

[Scenario]
[TestCase(2, 3)]
[TestCase(3, -2)]
public void Adding_X_to_Y(int x, int y)
{
    Runner.RunScenario(
        _ => Given_a_calculator(),
        _ => When_I_add_X_to_Y(x, y),
        _ => Then_the_result_should_be_Z(x + y));
}

Example output:

SCENARIO: Adding "2" to "3"
  STEP 1/3: GIVEN a calculator...
  STEP 1/3: GIVEN a calculator (Passed after 11ms)
  STEP 2/3: WHEN I add "2" to "3"...
  STEP 2/3: WHEN I add "2" to "3" (Passed after <1ms)
  STEP 3/3: THEN the result should be "5"...
  STEP 3/3: THEN the result should be "5" (Passed after <1ms)
  SCENARIO RESULT: Passed after 92ms

SCENARIO: Adding "3" to "-2"
  STEP 1/3: GIVEN a calculator...
  STEP 1/3: GIVEN a calculator (Passed after <1ms)
  STEP 2/3: WHEN I add "3" to "-2"...
  STEP 2/3: WHEN I add "3" to "-2" (Passed after <1ms)
  STEP 3/3: THEN the result should be "1"...
  STEP 3/3: THEN the result should be "1" (Passed after <1ms)
  SCENARIO RESULT: Passed after 9ms

2. Composite steps

It is now possible to define composite steps that are made of other steps. When composite step is executed, all sub-steps are executed in declaration order making parent step passing if all are successful.
The sub-steps would be included in progress notification as well as in final execution reports, where step number would be consist of step number in composite step, prefixed by parent step number.
It is possible to make multiple level composite steps i.e. use composite step as a sub-step of higher level composite step.

To define a composite step, the step method has to return either CompositeStep or Task<CompositeStep> type.
The composite step return value should be created in following way: CompositeStep.DefineNew().AddSteps(...).Build().

The composite steps can be then used to describe scenarios in the same way as regular steps:

  • the Extended scenario syntax fully supports composite steps,
  • the Basic scenario syntax supports composite steps only if RunScenarioAsync method is used due to fact that others will not accept non-void steps.

Example usage:

private async Task<CompositeStep> Given_customer_is_logged_in()
{
    return CompositeStep.DefineNew()
        .AddSteps(
            Given_the_user_is_about_to_login,
            Given_the_user_entered_valid_login,
            Given_the_user_entered_valid_password,
            When_the_user_clicks_login_button,
            Then_the_login_operation_should_be_successful)
        .Build();
}

[Scenario]
public async Task Ordering_products()
{
    await Runner.RunScenarioAsync(
        Given_customer_is_logged_in,
        When_customer_adds_products_to_basket,
        When_customer_pays_for_products_in_basket,
        Then_customer_should_receive_order_email);
}

Example output:

SCENARIO: Ordering products
  STEP 1/4: GIVEN customer is logged in...
  STEP 1.1/1.5: GIVEN the user is about to login...
  STEP 1.1/1.5: GIVEN the user is about to login (Passed after 1ms)
  STEP 1.2/1.5: AND the user entered valid login...
  STEP 1.2/1.5: AND the user entered valid login (Passed after <1ms)
  STEP 1.3/1.5: AND the user entered valid password...
  STEP 1.3/1.5: AND the user entered valid password (Passed after <1ms)
  STEP 1.4/1.5: WHEN the user clicks login button...
  STEP 1.4/1.5: WHEN the user clicks login button (Passed after 144ms)
  STEP 1.5/1.5: THEN the login operation should be successful...
  STEP 1.5/1.5: THEN the login operation should be successful (Passed after <1ms)
  STEP 1/4: GIVEN customer is logged in (Passed after 196ms)
  STEP 2/4: WHEN customer adds products to basket...

Finally, it is possible to specify context instance that would be shared between all sub-steps within the composite by using WithContext() method: CompositeStep.DefineNew().WithContext(...).AddSteps(...).Build()

Note: The CompositeStep.DefineNew()...Build() does not execute sub-steps but just creates a definition of the composite. The specified sub-steps are executed by the engine just after parent step method returns. It may sound a bit confusing but allows to predefine composite steps and reuse them in various steps as well as properly build a step hierarchy to render progress and report.

3. Relaxed rules about step lambda parameter names

Since this version, all single character lambda parameter names would be ignored from rendering in step names (while in previous versions only _ was treated as special name to be ignored).

Any other lambda parameter names would be still used to render as step types (as described in Scenario Steps Definition wiki page).

To visualize the change, the following scenario:

await Runner.RunScenarioAsync(
    x => Given_customer_has_some_products_in_basket(),
    x => Given_customer_has_enough_money_to_pay_for_products(),
    x => When_customer_requests_to_pay(),
    x => Then_payment_should_be_successful());

would render as:

STEP 1/4: GIVEN customer has some products in basket...
STEP 1/4: GIVEN customer has some products in basket (Passed after 129ms)
STEP 2/4: AND customer has enough money to pay for products...
STEP 2/4: AND customer has enough money to pay for products (Passed after 127ms)
STEP 3/4: WHEN customer requests to pay...
STEP 3/4: WHEN customer requests to pay (Passed after 145ms)
STEP 4/4: THEN payment should be successful...
STEP 4/4: THEN payment should be successful (Passed after 159ms)

It is possible to revert this behaviour to one from previous version of LightBDD in LightBDD Configuration by using UpdateUseLambdaNameAsStepType() method:

configuration
    .StepTypeConfiguration()
    .UpdateUseLambdaNameAsStepType(name => name != "_");

4. Better async support

The methods such as Runner.RunScenarioAsync() or Runner.RunScenarioActionsAsync() are now explicitly async which makes compiler suggesting that they should be awaited.

5. More detailed failure reasons

The step and scenario failure reasons are more detailed now and includes step number, exception type, inner exception(s) as well as simplified call stack (up to 4 stack frames):

Step 4: NUnit.Framework.AssertionException :   Login should succeeded
      Expected: True
      But was:  False
    
    at NUnit.Framework.Assert.ReportFailure(String message)
    at NUnit.Framework.Assert.That[TActual](TActual actual, IResolveConstraint expression, String message, Object[] args)
    at Example.LightBDD.NUnit3.Features.Login_feature.Then_the_login_operation_should_be_successful() in d:\dev\LightBDD\examples\Example.LightBDD.NUnit3\Features\Login_feature.Steps.cs:line 58
    at LightBDD.Framework.Scenarios.Basic.Implementation.BasicStepCompiler.StepExecutor.Execute(Object context, Object[] args)

It is possible now to control how exception details are captured by using UpdateExceptionDetailsFormatter() in LightBDD Configuration:

configuration
    .ExceptionHandlingConfiguration()
    .UpdateExceptionDetailsFormatter(ex => ex.Message)

6. VSIX templates for Visual Studio 2017

The VSIX item templates have been updated to support Visual Studio 2017 including old-style and new-style csprojs (desktop, .NET Core and .NET Standard projects).

Clone this wiki locally
You can’t perform that action at this time.