Skip to content

What Is New

Wojtek edited this page Jun 18, 2022 · 56 revisions
Clone this wiki locally

Releases

LightBDD 3.4.2

This is a very small release containing only a bug fix:

  • #257 (Core)(Bug) Added reset of step and scenario results upon retry in decorator

LightBDD 3.4.1

The LightBDD 3.4.1 brings small usability improvements.

With #248, MessageListener.EnsureReceived() provides now more friendly message dumps for common types such as DateTimeOffset, DateTime (both formatted with O format) or all types that implement IFormattable interface.

With #252, the LightBdd.XUnit2 LightBddScopeAttribute provides now DiagnosticMessageSink property, allowing to use diagnostic messages during OnSetUp(), OnTearDown() and OnConfigure() methods.

LightBDD 3.4.0

The LightBDD 3.4.0 brings #245 (Framework)(New) Implemented MessageListener to help testing message based services.

The MessageListener class was added to help write test scenarios for the services using messaging-based communication, where Then step need to verify that a specific message has been sent/published by the service under test.

Assuming that service tests use queueing mechanism (like Rebus, NServiceBus, or similar), the MessageListener can be instantiated at the beginning of each test to listen and collect all received messages. Having it done, it is possible to verify that the service has sent/published a given message with code like follows:

private async Task Then_an_OrderPlacedEvent_should_be_published_for_this_order_id()
{
     await _listener.EnsureReceived<OrderPlacedEvent>(x => x.OrderId == _order.Id);
}

In the method above, the listener will return immediately if the matching message was already received or wait for the pre-defined time until it arrives.

Listening for the messages can be initiated with method MessageListener.Start(source) where the source is a service test specific implementation of:

public interface IMessageSource
{
    public event Action<object> OnMessage;
}

Please note that the MessageListener instances have to be disposed when no longer required, otherwise they will collect messages for the entire duration of the service tests run.

LightBDD 3.3.1

The LightBDD 3.3.1 contains the following changes:

  • #238 (Core)(Fix) Included parameter results in failed steps
  • #231 (Framework)(New) Updated ResourcePool with constructor accepting async resource factory method

LightBDD 3.3.0

I am happy to announce LightBDD 3.3.0 containing significant improvements around Dependency Injection and few other improvements:

  • In response to #219 the support for Dependency Injection has been significantly improved (details below)
  • As part of #219 work, the #143 has been implemented, allowing to setup of the scenario and step contexts after instantiation but before steps run
  • With #214 all projects have been updated to support netstandard2.0 and net461. The support for .NET Core 1.0 and .NET Framework versions below 4.6.1 has been dropped
  • With #221 the VSIX Project Templates for .NET Framework and .NET Core has been merged in single template, where Target Framework has been updated to net5
  • Finally the LightBDD.Fixie2 got Fixie upgrade from 2.2.1 to 2.2.2

1. Dependency Injection improvements

The LightBDD Dependency Injection mechanisms have been extended with LifetimeScopes to provide the context for container scopes:

  • LifetimeScope.Global - used for the container root scope, which is shared between all tests (one for the entire test run),
  • LifetimeScope.Scenario - used for scopes created for each scenario (one per scenario),
  • LifetimeScope.Local - used for all composite steps (one per composite step), where it is possible to have multiple nested local scopes if nested composite steps are used.

2. Default DI Container improvements

The Default LightBDD DI Container has been rewritten to support instance scopes covering single, scenario, local and transient scopes.

To allow easy configuration of the container, the new UseDefault() configuration method has been added, while UseDefaultContainer() has been deprecated:

class ConfiguredLightBddScopeAttribute : LightBddScopeAttribute
{
    protected override void OnConfigure(LightBddConfiguration configuration)
    {
        configuration.DependencyContainerConfiguration()
            .UseDefault(ConfigureDI);
    }

    private void ConfigureDI(IDefaultContainerConfigurator cfg)
    {
        cfg.RegisterType<UserRepository>(InstanceScope.Single, //singleton
            opt => opt.As<IUserRepository>()); //register as interface only

        cfg.RegisterType<ApiClient>(InstanceScope.Scenario); //register as self, one per scenario, shared with nested composite steps

        cfg.RegisterType<FeatureContext>(InstanceScope.Local); //register as self, one per any scope

        cfg.RegisterType(InstanceScope.Transient, // register transient
            r => new ModelBuilder<User>("users", r.Resolve<ApiClient>()), // use custom factory method
            opt => opt.As<IModelBuiler<User>>()); // register as interface

        cfg.RegisterInstance(new Host(), //singleton
            opt => opt.ExternallyOwned() //don't dispose
                .As<Host>().As<IHost>()); //register as both types
    }
}

The above example demonstrates:

  • ability to define Single, Scenario, Local and Transient instance scopes,
  • registration of given type as self, implemented interface or multiple types,
  • ability to instantiate a type using its default public constructor, with inferred dependencies,
  • ability to instantiate a type using custom factory method, with access to the resolver allowing dependency resolution,
  • ability to register singleton instances and control their disposal

Additionally, it is possible now to configure how the container should treat not explicitly registered types:

private void ConfigureDI(IDefaultContainerConfigurator cfg)
{
    // all types not explicitly registered will be resolved as transient instances
    cfg.ConfigureFallbackBehavior(FallbackResolveBehavior.ResolveTransient);
    
    // resolution of any not registered type will fail with exception
    cfg.ConfigureFallbackBehavior(FallbackResolveBehavior.ThrowException);
}

The default behavior is FallbackResolveBehavior.ResolveTransient, which is similar to previous versions of LightBDD.
The FallbackResolveBehavior.ThrowException can help however in less obvious configurations involving multiple instance scopes, as it will enforce all the dependent types being registered within the reachable scopes.

3. LightBDD.Autofac improvements

The LightBDD.Autofac integration has been updated to allow registering types that should be shared within scenario and all nested scopes (composite steps), using InstancePerMatchingLifetimeScope(LifetimeScope.Scenario) method.

using Autofac;
using LightBDD.Core.Configuration;
using LightBDD.Core.Dependencies;

//...

internal class ConfiguredLightBddScopeAttribute : LightBddScopeAttribute
{
    protected override void OnConfigure(LightBddConfiguration configuration)
    {
        configuration.DependencyContainerConfiguration()
            .UseAutofac(ConfigureDI());
    }

    private ContainerBuilder ConfigureDI()
    {
        var builder = new ContainerBuilder();

        builder.RegisterType<ApiClient>().InstancePerMatchingLifetimeScope(LifetimeScope.Scenario);

        return builder;
    }
}

Please note that InstancePerLifetimeScope() method makes the instance shared only within a given scope, not nested ones, thus before this update, it was not possible to register instances to be shared within the scenario and its composite steps but different between scenarios.

4. LightBDD.Extensions.DependencyInjection improvements

Similarly to Autofac's InstancePerLifetimeScope(), the ServiceCollection.AddScoped<T>() makes the given instance shareable within the specified scope, but not the nested ones.
Moreover, the IServiceProvider.CreateScope() method does not support named scopes and does not provide an option to make such type shareable with nested scopes.
In this case, a different strategy had to be implemented instead.

The LightBDD.Extensions.DependencyInjection integration has been updated to make AddScoped() types shared within the scenario and its composite steps by default, which makes more sense from the LightBDD scenarios perspective.

Please note that it is a potentially breaking change, however it is possible to control this behavior with new configuration options:

internal class ConfiguredLightBddScopeAttribute : LightBddScopeAttribute
{
    protected override void OnConfigure(LightBddConfiguration configuration)
    {
        configuration.DependencyContainerConfiguration()
            .UseContainer(
                ConfigureDI(),
                options => options.EnableScopeNestingWithinScenarios(false));
                // ^-- configures to share scope between scenario and composite steps (default behavior)
    }

    private ServiceProvider ConfigureDI()
    {
        var services = new ServiceCollection();

        services.AddScoped<ApiClient>(); //registered as scoped instance

        return services.BuildServiceProvider();
    }
}

The EnableScopeNestingWithinScenarios() works as follows:

  • true: composite steps executed within the scenario will run in the nested scope. All instances resolved in that scope will get disposed upon composite step finish, but composite steps will receive different instances of types registered as scoped than the scenario.
  • false: composite steps executed within the scenario will run in the scenario scope. The lifetime of all instances resolved from the composite steps will match scenario lifetime, and composite steps, as well as parent scenario, will receive the same instances of types registered as scoped.

The default value is false.

5. Context configuration improvements

It is often needed to configure the context after its instantiation to provide the scenario or composite step-specific data.
With #143 the IBddRunner.WithContext() and ICompositeStepBuilder.WithContext() received overloads supporting this need.

For contexts requiring customization on the constructor level, it is possible to write:

Runner
  .WithContext(resolver => new ModelBuilder<Car>(_scenarioUserId, resolver.Resolve<ApiClient>())) // ModelBuilder<Car>

or

CompositeStep
  .DefineNew()
  .WithContext(resolver => new ModelBuilder<Car>(_scenarioUserId, resolver.Resolve<ApiClient>())) // ModelBuilder<Car>

For contexts requiring additional setup, it is possible to write:

Runner
  .WithContext<ModelBuilder<Car>>(model => model.UserId = _scenarioUserId)

or

CompositeStep
  .DefineNew()
  .WithContext<ModelBuilder<Car>>(model => model.UserId = _scenarioUserId)

Finally, it is possible to mix both customization methods:

Runner
  .WithContext(
    resolver => new ModelBuilder<Car>(_scenarioUserId, resolver.Resolve<ApiClient>()),
    model => model.ItemsCount = 15)

or

CompositeStep
  .DefineNew()
  .WithContext(
    resolver => new ModelBuilder<Car>(_scenarioUserId, resolver.Resolve<ApiClient>()),
    model => model.ItemsCount = 15)

LightBDD 3.2.0

I am happy to announce LightBDD 3.2.0 containing a few modernizations.

  • With #206 all suitable LightBDD packages has been updated to explicitly support netstandard2.0 in order to simplify the dependency graph for the library, which has been implemented based on Microsoft guidelines

  • With #210 the test library dependencies have been updated to the recent ones:

    • LightBDD.Fixie2 got Fixie upgrade from 2.0.3 to 2.2.1
    • LightBDD.MsTest2 got MSTest.TestFramework upgrade from 1.4.0 to 2.1.1 (please note that while it is a breaking change, I do not believe MSTest.TestFramework is mature enough to introduce distinct LightBDD.MsTest projects differentiating 1.x and 2.x series of this framework)
    • LightBDD.NUnit3 got NUnit upgrade from 3.7.1 to 3.12.0
    • LightBDD.XUnit2 got xunit upgrade from 2.3.1 to 2.4.1

Finally, in addition to those changes, the LightBDD license has been updated to cover the year 2020 and reformatted to be properly recognized by Github: link to license

LightBDD 3.1.1

I'm happy to announce LightBDD 3.1.1 containing a set of small improvements for the existing behaviors:

  • With #181 the compact steps name parsing mechanism has been improved to:

    • replace \t and new line characters with white space,
    • remove any other control characters,
    • trim any leading and following white characters,
    • do not perform any additional formatting such as replacing _ with space.

    Also, the HTML report renderer has been improved to properly render HTML control characters in the step name, such as <, > etc.
    To make it possible, a few changes have been made in core classes, which are described in the ticket and PR #197.

  • With #182 the step parsing issues have been made easier to discover. Before the change, any misconfigured steps were causing scenarios to fail before execution causing reports to contain the scenario name with exception but no details about the steps: With this change, the reports will contain steps, indicating the faulty ones, and allowing easier identification of the issue: More information can be found in the ticket and PR #196

  • With #185 a bug in LightBDD.XUnit2 integration has been fixed, where it was not possible to use [InlineData] with the specified Skip property. In addition to fixing this particular issue, the overall support to Skip property has been added to LightBDD:

    [Scenario]
    [InlineData(2)]
    [InlineData(3)]
    [InlineData(4, Skip = "Not running it!")]
    public void Eating_N_cakes(int n)
    {
        var myCakes = 3;
        Runner.RunScenario(
            _ => Given_I_have_N_cakes(myCakes),
            _ => When_I_eat_N_of_them(n),
            _ => Then_I_should_have_N_left(myCakes - n));
    }
    
    [Scenario(Skip = "It will just not happen!")]
    public void Eating_lemons()
    {
        Runner.RunScenario(
            _ => Given_I_have_a_lemon(),
            _ => When_I_eat_it(),
            _ => Then_I_should_have_none());
    }

    Before the change, the skipped tests were not included in the reports, but with this change they are captured now:

    The old behavior can be restored by adding [assembly: UseXUnitSkipBehavior] to the test project.

  • With #187 the disposal behvaior of pre-configured DI containers has been improved and made required to be explicitly specified. Before the change, the pre-configured containers were never disposed by LightBDD which could make the tests process hanging upon shutdown.
    With the change, the user can choose if LightBDD should take ownership of the container and dispose it upon shutdown, or leave it to the user to manage it. To prevent unexpected behaviors, this decision has been made mandatory.
    Affected integrations:

    More information can be found in the ticket.

  • With #199 the HTML/XML/Text reports will always contain scenarios ordered by name. Before the change, the scenario order varied between test runs, making reports more difficult to compare and work with.

LightBDD 3.1

This release includes only one change, #191 which extends IScenarioInfo and IStepInfo interfaces with Parent property that allows navigating up the stack and obtain details of what scenario runs the current step or what feature contains the current scenario.
In addition to that change, the IFeatureInfo, IScenarioInfo and IStepInfo interfaces are extended also with an unique RuntimeId GUID.

Both changes should make the custom integrations using progress notification, report generation and decorators more flexible.

LightBDD 3.0.1

This release is a quick fix of #177 where scenarios built fluently were not executing properly.

LightBDD 3.0

I am happy to announce LightBDD 3.0, that offers simplified experience with working on BDD scenarios, signed assemblies as well as API cleared off deprecated code!

To learn how to migrate LightBDD 2.x to 3.0, please visit Migrating LightBDD Versions wiki page.

Please beware that this release is binary incompatible with LightBDD 2.x, and requires update of all LightBDD packages to version 3.0 to work properly.

List of changes:

  • With #152 a major work has been done in LightBDD to bring it to version 3.0:
    • unified scenario extension namespaces to LightBDD.Framework.Scenarios to avoid confusion and simiplify usage,
    • unified configuration extension namespaces to LightBDD.Core.Configuration and LightBDD.Framework.Configuration,
    • removed Runner.NewScenario() in favor of Runner being fluent by default (see Scenario Steps Definition > Fluent Scenarios wiki page),
    • removed Runner.RunScenarioActionsAsync() method in favor of Runner.AddSteps(/*async void steps*/).RunAsync() (see Scenario Steps Definition > Fluent Scenarios wiki page),
    • removed obsolete code,
    • reworked and simplified a lot of internals;
  • With #157 all packages except LightBDD.Fixie2 are signed, which means they can be used to test signed projects!
  • With #163 the "End of stack trace" lines got removed from the stack traces in the LightBDD reports.
  • With #165 the basic syntax compiler will report error when lambda expressions are used for basic scenarios. Previously it was allowed to use lambdas, causing unreadable step names being rendered in reports.
  • With #170 LightBDD.NUnit2 has been deprecated.
  • With #174 Allowed IgnoreScenarioAttribute on class level to ignore all scenarios in given feature.

Happy coding!

LightBDD 2.5.0

1. Added Project Templates and Code Snippets to VSIX

With changes #60, #61 the LightBDD for Visual Studio VSIX extension has been extended to offer Project Templates and Code Snippets to ease work with LightBDD.

Note: As Code snippets are not treated as item/project templates, a VSIX with new ID had to be registered in the Visual Studio Gallery. Please ensure that the new extension is downloaded from this location.

Please feel free to take a look at Visual Studio Extensions wiki page for more details.

2. Added compact scenarios support

With change #156 the compact steps have been introduced, allowing to define self-contained scenarios. This change will make LightBDD easier to use in unit-test like scenarios.

Please feel free to take a look at Scenario Steps Definition > Compact Scenarios wiki page for more details.

3. Other minor improvements in 2.5.0

A following list of small improvements has also been included in the 2.5.0 release:

  • #148 (Framework)(Fix) Expect.To.BeAnyTrue() with no sub-expectations should fail
  • #153 (LightBDD.XUnit2)(Examples) Updated dotnet test examples for xunit2 to reflect the newest framework changes
  • #160 (LightBDD.Fixie2)(Change) Updated Fixie dependency to v2.0.2 to detect tests in VS 15.8 and higher

LightBDD 2.4.3

1. NULL value not assignable to Nullable parameter

With change #145 an edge case has been fixed where LightBDD was not honoring parameterized scenarios with null values being assigned to Nullable<> parameters.

Let's consider a following scenario:

[Scenario]
[TestCase(1)]
[TestCase(null)]
public void Some_scenario(int? parameter)
{
    Runner.RunScenario(
        _ => Given_xxx(),
        _ => When_yyy(),
        _ => Then_zzz());
}

Before the fix, running this scenario were passing for TestCase(1) but failing for TestCase(null) with following exception:

System.InvalidOperationException : Provided argument <null> is not assignable to parameter index 0 of method Void Some_scenario(System.Nullable`1[System.Int32])
   at LightBDD.Core.Extensibility.ScenarioDescriptor.BuildParameters(MethodBase methodInfo, Object[] arguments)
   at LightBDD.Core.Extensibility.ScenarioDescriptor..ctor(MethodBase methodInfo, Object[] arguments)
   at LightBDD.NUnit3.Implementation.NUnit3MetadataProvider.CaptureCurrentScenario()
   at LightBDD.Core.Extensibility.Implementation.ScenarioRunner.WithCapturedScenarioDetails()
   at LightBDD.Framework.Scenarios.Extended.Implementation.ExtendedScenarioRunnerFactory`1.BuildScenario(Expression`1[] steps)
   at LightBDD.Framework.Scenarios.Extended.ExtendedScenarioExtensions.RunScenario[TContext](IBddRunner`1 runner, Expression`1[] steps)
   at Example.LightBDD.NUnit3.Features.My_feature.Some_scenario(Nullable`1 parameter)

With the fix, such scenarios can be executed properly:

SCENARIO: Some scenario [parameter: "<null>"]
  STEP 1/3: GIVEN xxx...
  STEP 1/3: GIVEN xxx (Passed after 11ms)
  STEP 2/3: WHEN yyy...
  STEP 2/3: WHEN yyy (Passed after <1ms)
  STEP 3/3: THEN zzz...
  STEP 3/3: THEN zzz (Passed after <1ms)
  SCENARIO RESULT: Passed after 54ms

LightBDD 2.4.2

1. Integration with SourceLink

With change #135 LightBDD packages are integrated now with SourceLink which should make debugging seamless for people using Visual Studio 15.3+ or other tools supporting it.

2. Added State type to guard shared state from NullReferenceException

With change #134 the LightBDD.Framework.State<T> struct has been added to help protecting shared state fields from accessing without former initialization.

The State<T> struct offers following members:

  • IsInitialized property returning false if state has not been initialized,
  • GetValue() method retrieving value or throwing meaningful exception for uninitialized state (it's possible to specify the member name that will be included in exception)
  • GetValueOrDefault(T defaultValue = default(T)) method returning default value if state is not initialized or has null value (in case T is a reference type),
  • implicit conversions from T and to T, allowing code like:
    State<int> _field;
    /* ... */
    _field = 5; // sets value
    /* ... */
    int variable = _field; // gets value or throws if not initialized

Usage patterns:

  1. Simple usage when state is only set or get
class My_feature: FeatureFixture
{
    private State<int> _productId;
    private State<Product> _product;

    public void Given_product_id(int id)
    {
        // implicit cast
        _productId = id;
    }

    public void When_product_is_loaded()
    {
        // implicit cast of State<int> _id to int - it will throw meaningful exception if uninitialized
        _product = GetProductById(_id);
    }
}
  1. If additional operations are made on state, it's better to wrap it in property returning T type
public class My_feature: FeatureFixture
{
    private State<Product> _product;

    //will throw meaningful exception if used without former initialization
    private Product Product => _product.GetValue(nameof(Product));

    public void When_I_request_a_product_by_reference(string reference)
    {
        //it's possible to assign Product to State<Product> field
        _product = _client.GetProduct(reference);
    }

    public void Then_the_product_should_have_name(string name)
    {
        Assert.That(Product.Name, Is.EqualTo(name));
    }
}

3. Support for parallel scenario progress notification in LightBDD.MsTest2

With change #128, LightBDD.MsTest2 uses now TestContext.WriteLine() method to notify execution progress, which allows proper output capture in scenarios running in parallel.

In addition to that, the FeatureFixture class has been extended with TestContext property allowing to use it in all scenarios.

4. Corrected report writers to support properly absolute and UNC paths

With fix #137, report file writers can be now properly configured with absolute and UNC paths.

Let's see following cases to understand better the fix:

  • specifying path like \reports\result.html on Windows will make it expanded with the working directory drive letter, but preserve the rest as is, so for current directory being d:\temp it would be: d:\reports\result.html,
  • specifying absolute path like /out/result.html on Linux will make it being used as is, i.e /out/result.html,
  • specifying path starting from two backslashes will make it being treated as UNC and left as is, i.e. \\machine\reports\result.html will produce \\machine\reports\result.html.

To read more about reports configuration, please check the Generating Reports wiki page.

5. Features gets reported multiple times in LightBDD.XUnit2

With fix #127 the race condition around reporting start of feature processing has been fixed. While fix is general, the bug was only observable in LightBDD.XUnit2, due to the fact how LightBDD integrates with xunit.

LightBDD 2.4.1

1. Added Microsoft DI compatible containers integration

With change #122 it is now possible to install LightBDD.Extensions.DependencyInjection package and configure LightBDD to use any DI container that is compatible with Microsoft.Extensions.DependencyInjection.Abstractions and implements IServiceProvider interface.

Below, there is a sample code showing usage of the new integration:

using LightBDD.Extensions.DependencyInjection;

public class ConfiguredLightBddScopeAttribute : LightBddScopeAttribute
{
	protected override void OnConfigure(LightBddConfiguration configuration)
	{
		IServiceProvider container = new ServiceCollection()
			/* .Add... */
			.BuildServiceProvider();

		configuration.DependencyContainerConfiguration().UseContainer(container);
		/* ... */
	}
	/* ... */
}

For more information, please check the DI Containers wiki page.

2. Improved experience with running parameterized scenarios in LightBDD.XUnit2

With change #123 the experience of running parameterized LightBDD.XUnit2 scenarios has been improved significantly:

  • the parameterized scenarios are pre-enumerated now before execution, that improves the experience of running them from Resharper,
  • the preEnumerateTheories flag (described on xunit configration wiki page) is honored now by LightBDD,
  • the skipped cases, such as: [InlineData(123, Skip="reason")] are properly handled now, making scenario case skipped.

LightBDD 2.4.0

Hello, I am happy to announce the LightBDD 2.4.0 that introduces few new concepts described below.

1. Added in-line verifiable parameters

With change #38 it is possible to use in-line verifiable parameters allowing to:

  • perform verification of all step parameters before failing the step,
  • report parameter validation failure in the step name, rather than in the summary block.

Let's consider following scenario:

Runner.RunScenario(
  _ => Given_a_user_with_id_name_surname_and_email(124, "Joe", "Johnson","jj@gmail.com"),
  _ => When_I_request_user_details_for_id(124),
  _ => Then_I_should_receive_user_with_id_name_surname_and_email(124, "Joe", "Johnson","jj@gmail.com")
);

Before LightBDD 2.4.0, the then step had to be implemented as follows:

void Then_I_should_receive_user_with_id_name_surname_and_email(int id, string name, string surname, string email)
{
   Assert.That(_receivedUser.Id, Is.EqualTo(id), "Wrong ID");
   Assert.That(_receivedUser.Name, Is.EqualTo(name), "Wrong name");
   Assert.That(_receivedUser.Surname, Is.EqualTo(surname), "Wrong surname");
   Assert.That(_receivedUser.Email, Is.EqualTo(email), "Wrong email");
}

... where any of the assertion failure were stopping step execution, giving no information about other assertions.

With LightBDD 2.4.0, it is possible to implement that step as follows:

void Then_I_should_receive_user_with_id_name_surname_and_email(Verifiable<int> id, Verifiable<string> name, Verifiable<string> surname, Verifiable<string> email)
{
   id.SetActual(_receivedUser.Id);
   name.SetActual(_receivedUser.Name);
   surname.SetActual(_receivedUser.Surname);
   email.SetActual(_receivedUser.Email);
}

When executed, the SetActual() methods will perform the comparison against the expectations, however they will not throw in case of failure, allowing whole step to execute. The verification outcomes will be captured by LightBDD after step finish where any failures (including situation where SetActual() has not been called at all) will:

  • make step to fail,
  • get included in-line in step name,
  • get included in step error details.

Below, there are sample progress messages and report:

  SCENARIO: Retrieving user details
  STEP 1/3: GIVEN a user with id "124" name "Joe" surname "Johnson" and email "jj@gmail.com"...
  STEP 1/3: GIVEN a user with id "124" name "Joe" surname "Johnson" and email "jj@gmail.com" (Passed after 13ms)
  STEP 2/3: WHEN I request user details for id "124"...
  STEP 2/3: WHEN I request user details for id "124" (Passed after <1ms)
  STEP 3/3: THEN I should receive user with id "expected: equals '124'" name "expected: equals 'Joe'" surname "expected: equals 'Johnson'" and email "expected: equals 'jj@gmail.com'"...
  STEP 3/3: THEN I should receive user with id "124" name "Joe" surname "expected: equals 'Johnson', but got: 'JOHNSON'" and email "expected: equals 'jj@gmail.com', but got: 'JJ@GMAIL.COM'" (Failed after 7ms)
  SCENARIO RESULT: Failed after 120ms
    Step 3: System.InvalidOperationException : Parameter 'surname' verification failed: expected: equals 'Johnson', but got: 'JOHNSON'
    	Parameter 'email' verification failed: expected: equals 'jj@gmail.com', but got: 'JJ@GMAIL.COM'

Verifiable class and Expect.To expressions

The above example shows usage of Verifiable<T> class (from LightBDD.Framework.Parameters namespace).

As presented above, the Verifiable<T> class has an implicit operator, allowing to initialize it with T value, i.e.: Verifiable<int> myVerifiable = 5;

Such assignment will make a verifiable instance expecting actual value to be equal 5.

While it is handy for most of the cases, the LightBDD 2.4.0 got implemented a set of expressions allowing to make more complex expectations with the usage of Expect.To.*** expressions (from LightBDD.Framework.Expectations namespace).

Let's see following example:

Runner.RunScenario(
    _ => Given_a_user_with_id_name_surname_and_email(124, "Joe", "Johnson", "jj@gmail.com"),
    _ => When_I_request_user_details_for_id(124),
    _ => Then_I_should_receive_user_with_id_name_surname_and_email(
        124, "Joe",
        Expect.To.Not.BeEmpty<string>(),
        Expect.To.MatchIgnoreCase("[a-z]+@([a-z]+)(\\.[a-z]+)+")));

and the outcome:

SCENARIO: Retrieving user details
  STEP 1/3: GIVEN a user with id "124" name "Joe" surname "Johnson" and email "jj@gmail.com"...
  STEP 1/3: GIVEN a user with id "124" name "Joe" surname "Johnson" and email "jj@gmail.com" (Passed after 13ms)
  STEP 2/3: WHEN I request user details for id "124"...
  STEP 2/3: WHEN I request user details for id "124" (Passed after <1ms)
  STEP 3/3: THEN I should receive user with id "expected: equals '124'" name "expected: equals 'Joe'" surname "expected: not empty" and email "expected: matches '[a-z]+@([a-z]+)(\.[a-z]+)+' ignore case"...
  STEP 3/3: THEN I should receive user with id "124" name "Joe" surname "JOHNSON" and email "JJ@GMAIL.COM" (Passed after 6ms)
  SCENARIO RESULT: Passed after 99ms

More details can be found on Advanced Step Parameters wiki page.

2. Added tabular parameters

In additions to the verifiable parameters and the expectation expressions described above, the change #70 introduces tabular parameters in three forms:

  • InputTable<TRow> allowing to pass collections to the step methods, that will be rendered nicely in progress and the reports,
  • VerifiableDataTable<TRow> allowing to define expected collection that, will be compared to the actual collection in the step, on the row and column basis,
  • TableValidator<TRow> allowing to define expectation for all columns, that will have to be fulfilled for each provided actual row value.

It also provides a static class Table (LightBDD.Framework.Parameters namespace) allowing to build those tabular parameters in easy way.

Let's take a look at the example:

[Scenario]
public void Retrieving_user_details()
{
    Runner.RunScenario(

        _ => Given_users(Table.For(
            new User(1, "Joe", "Andersen", "jj@foo.com"),
            new User(2, "Henry", "Hansen", "henry123@foo.com"),
            new User(3, "Marry", "Davis", "ma31@bar.com"),
            new User(4, "Monica", "Larsen", "monsmi22@bar.com"))),

        _ => When_I_search_for_users_by_surname_pattern(".*sen"),

        _ => Then_I_should_receive_users(Table.ExpectData(
            new User(1, "Josh", "Andersen", "jj@foo.com"),
            new User(2, "Henry", "Hansen", "henry123@foo.com"))));
}

// Implementation
private InputTable<User> _users;
private User[] _received;

private void When_I_search_for_users_by_surname_pattern(string pattern)
{
    _received = _users.Where(x => Regex.IsMatch(x.Surname, pattern)).ToArray();
}

private void Given_users(InputTable<User> users)
{
    _users = users;
}

void Then_I_should_receive_users(VerifiableDataTable<User> users)
{
    users.SetActual(_received);
}

class User
{
    public User(int id, string name, string surname, string email)
    {
        Id = id;
        Name = name;
        Surname = surname;
        Email = email;
    }
    public int Id { get; }
    public string Name { get; }
    public string Surname { get; }
    public string Email { get; }
}

The given step uses parameter of type InputTable<User> which is instantiated with Table.For() method.

The then step uses parameter of type VerifiableDataTable<User> which is instantiated with Table.ExpectData() method and, similarly to Verifiable<T> type, gets the actual value provided with users.SetActual() method.

When executed, the progress and report will be as follows:

SCENARIO: Retrieving user details
  STEP 1/3: GIVEN users "<table>"...
  STEP 1/3: GIVEN users "<table>" (Passed after 13ms)
    users:
    +----------------+--+------+--------+
    |Email           |Id|Name  |Surname |
    +----------------+--+------+--------+
    |jj@foo.com      |1 |Joe   |Andersen|
    |henry123@foo.com|2 |Henry |Hansen  |
    |ma31@bar.com    |3 |Marry |Davis   |
    |monsmi22@bar.com|4 |Monica|Larsen  |
    +----------------+--+------+--------+
  STEP 2/3: WHEN I search for users by surname pattern ".*sen"...
  STEP 2/3: WHEN I search for users by surname pattern ".*sen" (Passed after 6ms)
  STEP 3/3: THEN I should receive users "<table>"...
  STEP 3/3: THEN I should receive users "<table>" (Failed after 24ms)
    users:
    +-+-----------------------+--------+-------------+-------------+
    |#|Email                  |Id      |Name         |Surname      |
    +-+-----------------------+--------+-------------+-------------+
    |!|jj@foo.com             |1       |Joe/Josh     |Andersen     |
    |=|henry123@foo.com       |2       |Henry        |Hansen       |
    |+|monsmi22@bar.com/<none>|4/<none>|Monica/<none>|Larsen/<none>|
    +-+-----------------------+--------+-------------+-------------+
  SCENARIO RESULT: Failed after 196ms
    Step 3: System.InvalidOperationException : Parameter 'users' verification failed: [0].Name: expected: equals 'Josh', but got: 'Joe'
    		[2].Email: unexpected value
    		[2].Id: unexpected value
    		[2].Name: unexpected value
    		[2].Surname: unexpected value

While VerifiableDataTable<T> allows to compare expected and actual collections, the TableValidator<T> allows to validate all actual records.

[Scenario]
public void Retrieving_user_details()
{
    Runner.RunScenario(

        _ => Given_users(Table.For(
            new User(0, "Joe", "Andersen", "jj@foo.com"),
            new User(2, "Henry", "Hansen", "henry123@foo2.com"),
            new User(3, "Marry", "Davis", "ma31@bar.com"),
            new User(4, "Monica", "Larsen", "monsmi22@bar.com"))),

        _ => When_I_search_for_users_by_surname_pattern(".*sen"),

        _ => Then_I_should_receive_users(Table.Validate<User>(builder => builder
            .WithColumn(x => x.Id, Expect.To.BeGreaterThan(0))
            .WithColumn(x => x.Name, Expect.To.Not.BeEmpty())
            .WithColumn(x => x.Surname, Expect.To.BeLikeIgnoreCase("*sen"))
            .WithColumn(x => x.Email, Expect.To.MatchIgnoreCase("[\\w]+@([a-z]+)(\\.[a-z]+)+"))
        )));
}

// Implementation
void Then_I_should_receive_users(TableValidator<User> users)
{
    users.SetActual(_received);
}
SCENARIO: Retrieving user details
  STEP 1/3: GIVEN users "<table>"...
  STEP 1/3: GIVEN users "<table>" (Passed after 19ms)
    users:
    +-----------------+--+------+--------+
    |Email            |Id|Name  |Surname |
    +-----------------+--+------+--------+
    |jj@foo.com       |0 |Joe   |Andersen|
    |henry123@foo2.com|2 |Henry |Hansen  |
    |ma31@bar.com     |3 |Marry |Davis   |
    |monsmi22@bar.com |4 |Monica|Larsen  |
    +-----------------+--+------+--------+
  STEP 2/3: WHEN I search for users by surname pattern ".*sen"...
  STEP 2/3: WHEN I search for users by surname pattern ".*sen" (Passed after 6ms)
  STEP 3/3: THEN I should receive users "<table>"...
  STEP 3/3: THEN I should receive users "<table>" (Failed after 18ms)
    users:
    +-+------------------+------+--------+-----------------------------------------------------------------+
    |#|Id                |Name  |Surname |Email                                                            |
    +-+------------------+------+--------+-----------------------------------------------------------------+
    |!|0/greater than '0'|Joe   |Andersen|jj@foo.com                                                       |
    |!|2                 |Henry |Hansen  |henry123@foo2.com/matches '[\w]+@([a-z]+)(\.[a-z]+)+' ignore case|
    |=|4                 |Monica|Larsen  |monsmi22@bar.com                                                 |
    +-+------------------+------+--------+-----------------------------------------------------------------+
  SCENARIO RESULT: Failed after 216ms
    Step 3: System.InvalidOperationException : Parameter 'users' verification failed: [0].Id: expected: greater than '0', but got: '0'
    		[1].Email: expected: matches '[\w]+@([a-z]+)(\.[a-z]+)+' ignore case, but got: 'henry123@foo2.com'

Table customization

The Table class allows to customize the created tables by:

  • defining the list of columns to display, including automatic detection as well as manual specification,
  • defining the key column(s) that will be used in expected-actual row comparison (where otherwise, the row order is used).

As mentioned above, the columns can be automatically detected, where following rules are used:

  • if row is simple type, one column Item is inferred,
  • if row type implements IDictionary<string,object> (which includes ExpandoObject type), the dictionary keys will be used as columns,
  • if row type itself is a collection implementing IList, the longest collection will be used to generate columns with names like [N] (also the Length column will be added),
  • finally, if none of above is true, the rows type will be scanned for fields and properties which will be used as columns.

Finally, the tables themselves support async provisioning of the actual values and VerifiableDataTable<T> offers the SetActual*() overloads with lookup function, allowing to find the actual row for given expected row.

More details can be found on Advanced Step Parameters wiki page.

3. Added DI container support

With change #59, LightBDD 2.4.0 got support for DI containers:

  • a basic DI container has been implemented in LightBDD itself,
  • the LightBDD.Autofac package has been provided if more advanced DI capabilities are needed,
  • the Runner.WithContext<T>() method has been changed, so that T no longer has to have parameterless constructor (DI is used to resolve the dependencies),
  • the StepExecution.GetScenarioDependencyResolver() has been introduced to get access to DI container in non-contextual scenarios,
  • the ResourcePool and ResourceHandle helper types has been added, that, when used with DI, allows to better manage access to heavy but limited resources.

Container scopes

The LightBDD DI containers are designed to support multiple scopes. Every time when a new scenario is run, or new composite-step is executed the new container scope is created (with the information about it's parent). Any dependencies created then are added to the actual scope and disposed when composite-step or scenario is finished.

DI container implementations

By default, LightBDD uses the BasicDependencyContainer which offers following functionality:

  • it allows to resolve types (classes and structures) with 1 public constructor,
  • it supports constructor dependency injections,
  • it supports singleton registrations in LightBDD configuration code: configuration.DependencyContainerConfiguration().UseDefaultContainer(cfg => cfg.RegisterInstance(new MySingleton(), new RegistrationOptions()));,
  • it supports disposal of dependencies upon disposal, if dependency implements IDisposable interface,
  • it supports container scopes.

If more complex DI is needed, it is possible to use LightBDD.Autofac package and use Autofac container, which can be registered as follows:

configuration.DependencyContainerConfiguration().UseAutofac(autofacBuilder);

Accessing container

The Runner.WithContext<T>() method (as well as composite-step equivalent) has been changed to use DI to instantiate the context. It means that context can now have parameterized constructor, where dependencies will be injected by the container.

For the steps that does not use context objects, it is possible to use StepExecution.GetScenarioDependencyResolver() in order to get the container and resolve dependencies manually.

ResourcePool and ResourceHandle

There are scenarios where some heavy dependencies have to be used (such as selenium drivers). Their characteristic is that they can be used by 1 test at a time and they are very costly to create.

The ResourcePool<T> class fits well to this scenario. It allows to create a pool of such objects that will be lent for the scenario execution, but given back after the test is done.

The LightBDD.AcceptanceTests project uses it currently to speed-up the test execution.

The sample registration code is as follows (source):

public class ConfiguredLightBddScopeAttribute : LightBddScopeAttribute
{
    protected override void OnConfigure(LightBddConfiguration configuration)
    {
        configuration.DependencyContainerConfiguration()
            .UseDefaultContainer(ConfigureContainer);
    }

    private void ConfigureContainer(ContainerConfigurator config)
    {
        config.RegisterInstance(
            new ResourcePool<ChromeDriver>(CreateDriver),
            new RegistrationOptions());
    }

    private ChromeDriver CreateDriver()
    {
        var driver = new ChromeDriver();
        driver.Manage().Timeouts().ImplicitWait = TimeSpan.FromMilliseconds(0);
        return driver;
    }
}

... and the sample usage is here (source):

// the context is created for each scenario

public class HtmlReportContext
{
    private readonly ResourceHandle<ChromeDriver> _driverHandle;

    public HtmlReportContext(ResourceHandle<ChromeDriver> driverHandle)
    {
        _driverHandle = driverHandle;
        /* ... */
    }

    public async Task When_a_html_report_is_opened()
    {
        // this will obtain the resource
        Driver = await _driverHandle.ObtainAsync();
        /* ... */
    }
}

For more information, please check the DI Containers wiki page.

4. Added Fixie integration

With change #67 it is possible now to use LightBDD with Fixie framework by using new LightBDD.Fixie2 package!

To see sample usage, please take a look at the Example.LightBDD.Fixie2 project.

The wiki page describing integration with Fixie can be found in Test Framework Integrations wiki section.

5. Added experimental inter-class test execution for LightBDD.XUnit2

With change #117 it is possible now to add

[assembly: ClassCollectionBehavior(AllowTestParallelization = true)]

attribute, that will make all the scenario methods (including parameterized scenarios) to run in parallel, as long as:

  • test class does not implement IClassFixture nor ICollectionFixture attribute,
  • test class does not have defined named collection [Collection] attribute,
  • parallelization is enabled on xunit level.

Please note that xunit itself does not support inter-class parallel test execution and this attribute is rather experimental.

6. StepExecution.Current.Comment() does not work after leaving sub-step in composite step

This change is a bug fix #120 that was causing StepExecution.Current.Comment() to throw if used in composite steps. Please check the issue itself for more details.

LightBDD 2.3.6

1. Allowed IStepDecoratorAttribute and IScenarioDecoratorAttribute to be applied on class level

With change #108 it is now possible to apply decorator attributes implementing IStepDecoratorAttribute and/or IScenarioDecoratorAttribute on class level to cover all the scenarios or steps declared in that class.

The wiki pages have been updated with Extending Test Behavior section covering this topic.

LightBDD 2.3.5

1. Updated LightBDD.XUnit2 integration to recognize xunit Category Traits

With change #104, [Trait("Category", "...")] attribute applied on Feature class or Scenario method is recognized now by LightBDD and included in reports.

Additionally, the Tests Structure and Conventions wiki page has been updated to better describe scenario categories behavior.

LightBDD 2.3.4

1. Implemented console immediate progress notification for LightBDD.NUnit3

With change #34, dotnet test or nunit3-console.exe commands prints now the immediate scenario progress as it occurs, allowing to track long running tests on CI.

If Reshaper is used to run the test, it's output window also contains the progress messages printed as they occurs (however please note that there is a bug reported for Resharper as when tests are run in parallel, their output is incorrectly displayed: see RSRP-466415).

The immediate progress is printed in following format, allowing to track scenarios executed in parallel:

Fi=000,Fa=000,Pe=003 #  1> SCENARIO: [Ticket-6] No product in stock
Fi=000,Fa=000,Pe=003 #  2> SCENARIO: [Ticket-3] Anonymous login name should allow to log in
Fi=000,Fa=000,Pe=003 #  3> SCENARIO: [Ticket-12] Ordering products
Fi=000,Fa=000,Pe=003 #  2>   STEP 1/5: GIVEN the user is about to login...
Fi=000,Fa=000,Pe=003 #  3>   STEP 1/4: GIVEN customer is logged in...
Fi=000,Fa=000,Pe=003 #  1>   STEP 1/4: GIVEN product is out of stock...
Fi=000,Fa=000,Pe=003 #  3>   STEP 1.1/1.5: GIVEN the user is about to login...
Fi=000,Fa=000,Pe=003 #  2>   STEP 1/5: GIVEN the user is about to login (Passed after 13ms)
Fi=000,Fa=000,Pe=003 #  1>   STEP 1/4: GIVEN product is out of stock (Passed after 13ms)

while the scenario summary is still printed in simple form:

Skipped  Ordering_products
Error Message:
 Not implemented yet
Standard Output Messages:
 SCENARIO: [Ticket-12] Ordering products
   STEP 1/4: GIVEN customer is logged in...
   STEP 1.1/1.5: GIVEN the user is about to login...
   STEP 1.1/1.5: GIVEN the user is about to login (Passed after <1ms)
   STEP 1.2/1.5: AND the user entered valid login...
   STEP 1.2/1.5: AND the user entered valid login (Passed after <1ms)
   STEP 1.3/1.5: AND the user entered valid password...
   STEP 1.3/1.5: AND the user entered valid password (Passed after <1ms)
...

Please visit Example.LightBDD.NUnit3 sample project to see more details on running scenarios with LightBDD.NUnit3.

2. Enhanced scenario progress notification

As part of the work described above (issue #34), a few more improvements have been implemented:

  • The LightBDD.XUnit2 progress notification has been changed to print scenario exection summary in simplified format:

    Example.LightBDD.XUnit2.Features.Login_feature.Anonymous_login_name_should_allow_to_log_in [FAIL]
      Login should succeeded
      Expected: True
      Actual:   False
      Stack Trace:
        d:\dev\LightBDD_234\examples\Example.LightBDD.XUnit2\Features\Login_feature.Steps.cs(57,0): at Example.LightBDD.XUnit2.Features.Login_feature.Then_the_login_operation_should_be_successful()
        d:\dev\LightBDD_234\src\LightBDD.Framework\Scenarios\Basic\Implementation\BasicStepCompiler.cs(81,0): at LightBDD.Framework.Scenarios.Basic.Implementation.BasicStepCompiler.StepExecutor.Execute(Object context, Object[] args)
        --- End of stack trace from previous location where exception was thrown ---
           at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
        d:\dev\LightBDD_234\src\LightBDD.Framework\Scenarios\Basic\BasicScenarioExtensions.cs(54,0): at LightBDD.Framework.Scenarios.Basic.BasicScenarioExtensions.RunScenario(IBddRunner runner, Action[] steps)
        d:\dev\LightBDD_234\examples\Example.LightBDD.XUnit2\Features\Login_feature.cs(85,0): at Example.LightBDD.XUnit2.Features.Login_feature.Anonymous_login_name_should_allow_to_log_in()
      Output:
        SCENARIO: [Ticket-3] Anonymous login name should allow to log in
          STEP 1/5: GIVEN the user is about to login...
          STEP 1/5: GIVEN the user is about to login (Passed after <1ms)
          STEP 2/5: AND the user entered anonymous login...
          STEP 2/5: /* Presentation of failed scenario */
          STEP 2/5: AND the user entered anonymous login (Passed after <1ms)
          STEP 3/5: WHEN the user clicks login button...
          STEP 3/5: WHEN the user clicks login button (Passed after 1s 516ms)
          STEP 4/5: THEN the login operation should be successful...
          STEP 4/5: THEN the login operation should be successful (Failed after 4ms)
          SCENARIO RESULT: Failed after 1s 537ms
    

    The change affects VisualStudio Test Explorer output / Resharper output window and console when test output is printed. The immediate progress is still printed in extended form, allowing to track scenarios executed in parallel.

    Please visit Example.LightBDD.XUnit2 sample project to see more details on running scenarios with LightBDD.XUnit2.

  • The FeatureProgressNotifierConfiguration and ScenarioProgressNotifierConfiguration has been extended with methods allowing to combine currently configured notifiers with new ones as well as clear them:

    configuration.Get<FeatureProgressNotifierConfiguration>()
        .ClearNotifiers()
        .AppendNotifiers(new CustomFeatureProgressNotifier())
        .AppendNotifiers(new CustomFeatureProgressNotifier2());
    
    configuration.Get<ScenarioProgressNotifierConfiguration>()
        .ClearNotifierProviders()
        .AppendNotifierProviders(() => new CustomScenarioProgressNotifier())
        .AppendNotifierProviders<FeatureFixture>(fixture => new CustomScenarioProgressNotifier2(fixture));
    
  • All integrations (LightBDD.NUnit2, LightBDD.NUnit3, LightBDD.XUnit2, LightBDD.MsTest2) have been updated with configuration methods allowing to apply default progress notifiers after they have been cleaned up:

    configuration.Get<FeatureProgressNotifierConfiguration>()
        .ClearNotifiers()
        .AppendFrameworkDefaultProgressNotifiers();
    
    configuration.Get<ScenarioProgressNotifierConfiguration>()
        .ClearNotifierProviders()
        .AppendFrameworkDefaultProgressNotifiers();
    

3. Lifted requirement for parameterized constructors in LightBDD.XUnit2 FeatureFixture

With change #98, it is no longer required to provide a parameterized constructor for FeatureFixture class in LightBDD.XUnit2.

Before the change a following constructor had to be added to each feature class:

public My_feature1(ITestOutputHelper output) : base(output)
{
}

After the change, this boilerplate code is no longer required.

Finally, the VSIX extension feature class template for LightBDD.XUnit2 has been updated to no longer generate this constructor.

LightBDD 2.3.3

This is the last LightBDD update in year 2017 :wink:, happy New Year 2018!

1. Simplified exception stack trace for failed scenarios

With change #69, the stack trace of any exceptions thrown by steps has been simplified to make it easier to analyze what went wrong in the step.

The improvement covers two areas:

  • the exceptions no longer have LightBDD internal methods frames on their stack trace,
  • the LightBDD reports and progress notifications have now predefined filters for frames that are related to test framework assertion methods as well as system methods responsible to handle exceptions in async methods.

In addition to the improvements, the number of stack trace lines printed in the reports has been increased from 4 to 8.

Finally, it is possible now to configure exception stack trace printing behaviour with code:

configuration.ExceptionHandlingConfiguration()
    .UpdateExceptionDetailsFormatter(new DefaultExceptionFormatter()
        .WithTestFrameworkDefaults() // apply default framework exclusions such as "NUnit\\..*" for LightBDD.NUnit3
        .WithStackTraceLinesLimit(3) // change max number of printed lines
        .WithMembersExcludedFromStackTrace("LightBDD\\..*", "System\\..*") // filter out any lines starting with "LightBDD." or "System."
        .Format);

Please note that above configuration only affects reports/notifications generated by LightBDD but not by underlying test frameworks.


Sample scenario:

[Label("SCENARIO-1")]
[Scenario]
public void Template_extended_scenario()
{
	Runner.RunScenario(
		_ => Given_method(),
		_ => When_method(),
		_ => Then_method());
}
private void Given_method()
{
}

private void When_method()
{
}

private void Then_method()
{
	Assert.Fail("Not implemented yet");
}

After the change (Resharper# output window):

Not implemented yet
at ClassLibrary1.My_feature1.Then_method() in c:\tmp\LightBDD-Compat\ClassLibrary1\My_feature1.Steps.cs:line 18
at lambda_method(Closure , NoContext , Object[] )
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at LightBDD.Framework.Scenarios.Extended.ExtendedScenarioExtensions.RunScenario[TContext](IBddRunner`1 runner, Expression`1[] steps)
at ClassLibrary1.My_feature1.Template_extended_scenario() in c:\tmp\LightBDD-Compat\ClassLibrary1\My_feature1.cs:line 30


SCENARIO: [SCENARIO-1] Template extended scenario
  STEP 1/3: GIVEN method...
  STEP 1/3: GIVEN method (Passed after <1ms)
  STEP 2/3: WHEN method...
  STEP 2/3: WHEN method (Passed after <1ms)
  STEP 3/3: THEN method...
  STEP 3/3: THEN method (Failed after 1ms)
  SCENARIO RESULT: Failed after 6ms
    Step 3: NUnit.Framework.AssertionException : Not implemented yet
    	at ClassLibrary1.My_feature1.Then_method() in c:\tmp\LightBDD-Compat\ClassLibrary1\My_feature1.Steps.cs:line 18
    	at lambda_method(Closure , NoContext , Object[] )

The stack trace contains the information about step where error has been thrown and then in what scenario (with no LightBDD internals).

Before the change:

Not implemented yet
at ClassLibrary1.My_feature1.Then_method() in c:\tmp\LightBDD-Compat\ClassLibrary1\My_feature1.Steps.cs:line 18
at lambda_method(Closure , NoContext , Object[] )
at LightBDD.Core.Extensibility.Implementation.ScenarioRunner.InvocationResultTransformer.<InvokeAsync>d__5.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at LightBDD.Core.Execution.Implementation.RunnableStep.<InvokeStepMethodAsync>d__22.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at LightBDD.Core.Execution.Implementation.RunnableStep.<InvokeStepMethodAsync>d__22.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at LightBDD.Core.Execution.Implementation.RunnableStep.<InvokeStepAsync>d__19.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at LightBDD.Framework.Commenting.Implementation.StepCommentingDecorator.<ExecuteAsync>d__0.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at LightBDD.Core.Execution.Implementation.RunnableStep.<TimeMeasuredInvokeAsync>d__18.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at LightBDD.Core.Execution.Implementation.RunnableStep.<RunAsync>d__16.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at LightBDD.Core.Execution.Implementation.RunnableScenario.<RunAsync>d__17.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at LightBDD.Core.Extensibility.Implementation.ScenarioRunner.RunSynchronously()
at ClassLibrary1.My_feature1.Template_extended_scenario() in c:\tmp\LightBDD-Compat\ClassLibrary1\My_feature1.cs:line 30


SCENARIO: [SCENARIO-1] Template extended scenario
  STEP 1/3: GIVEN method...
  STEP 1/3: GIVEN method (Passed after 12ms)
  STEP 2/3: WHEN method...
  STEP 2/3: WHEN method (Passed after <1ms)
  STEP 3/3: THEN method...
  STEP 3/3: THEN method (Failed after 3ms)
  SCENARIO RESULT: Failed after 137ms
    Step 3: NUnit.Framework.AssertionException : Not implemented yet
    	at NUnit.Framework.Assert.ReportFailure(String message)
    	at ClassLibrary1.My_feature1.Then_method() in c:\tmp\LightBDD-Compat\ClassLibrary1\My_feature1.Steps.cs:line 18
    	at lambda_method(Closure , NoContext , Object[] )
    	at LightBDD.Core.Extensibility.Implementation.ScenarioRunner.InvocationResultTransformer.<InvokeAsync>d__5.MoveNext()

Before the change, a long (mostly meaningless) stack trace was reported and reports themselves contained frames such as at NUnit.Framework.Assert.ReportFailure(String message).

2. Improved integration with NCrunch

With change #90, it is now possible to reuse the same process/AppDomain to execute LightBDD tests multiple times, which improves the experience of using NCrunch to run LightBDD tests.

Before the change, adding LightBDD scenarios to project monitored by NCrunch was often ending up with NCrunch reporting following error and executing tests forever:

System.InvalidOperationException: FeatureCoordinator of LightBDD.XUnit2.Implementation.XUnit2FeatureCoordinator type is already installed
at LightBDD.Core.Execution.Coordination.FeatureCoordinator.Install(FeatureCoordinator coordinator)
at LightBDD.XUnit2.LightBddScopeAttribute.SetUp()
at LightBDD.XUnit2.Implementation.Customization.TestFrameworkExecutor.RunTestCases(IEnumerable`1 testCases, IMessageSink executionMessageSink, ITestFrameworkExecutionOptions executionOptions)
at Xunit.Sdk.TestFrameworkExecutor`1.RunTests(IEnumerable`1 testCases, IMessageSink executionMessageSink, ITestFrameworkExecutionOptions executionOptions)
at nCrunch.Module.XUnit2.Integration.XUnit2FrameworkRuntimeEnvironment.<>c__DisplayClass3_1.<RunTests>b__3()

3. Improved support for extension methods used as steps

With change #92, it is possible now to use extension methods as steps where LightBDD will render them with the same rules as regular steps.

Example code:

public class MyFeature : FeatureFixture
{
    public MyFeature(ITestOutputHelper output) : base(output)
    {
    }

    [Scenario]
    public void My_scenario()
    {
        Runner
            .WithContext<MyContext>()
            .RunScenario(
                x => x.Given_value(5),
                x => x.When_I_multiply_value_by_MULTIPLIER(2),
                x => x.Then_value_should_be_RESULT(10));
    }
}
class MyContext
{
    public int Value { get; set; }
}

static class MyContextExtensions
{
    public static void Given_value(this MyContext ctx, int value)
    {
        ctx.Value = value;
    }

    public static void When_I_multiply_value_by_MULTIPLIER(this MyContext ctx, int multiplier)
    {
        ctx.Value *= multiplier;
    }

    public static void Then_value_should_be_RESULT(this MyContext ctx, int result)
    {
        Assert.Equal(result, ctx.Value);
    }
}

Example output after the change:

SCENARIO: My scenario
  STEP 1/3: GIVEN value "5"...
  STEP 1/3: GIVEN value "5" (Passed after 9ms)
  STEP 2/3: WHEN I multiply value by "2"...
  STEP 2/3: WHEN I multiply value by "2" (Passed after <1ms)
  STEP 3/3: THEN value should be "10"...
  STEP 3/3: THEN value should be "10" (Passed after 3ms)
  SCENARIO RESULT: Passed after 58ms

Before the change, LightBDD was rendering also this method parameter making step less readable.

Also, as a part of this change a small fix has been applied where step method like void Given_value(MyContext ctx, int value) { } was rendered as GIVEN value [ctx: "MyContext"] "5" instead of GIVEN value "5" [ctx: "MyContext"]

4. Added support for disposable scenario and step context instances

With change #94, the contextual scenarios and composite steps supports now disposal of context instances implementing IDisposable interface.

The WithContext methods have been extended with new behavior where:

  • the .WithContext<MyContext>() will always dispose disposable context instances,
  • the .WithContext(new MyContext()) will not dispose context by default as instance is passed explicitly by caller. However, as the method has now an optional parameter bool takeOwnership = false, it is possible to make context automatically disposed by calling .WithContext(new MyContext(), true),
  • the .WithContext(() => new MyContext()) will dispose context by default as factory method is being passed indicating that it will produce a new instance on each call. Similarly to method above, it is possible to change default behavior and make context not disposed by calling .WithContext(() => new MyContext(), false).

LightBDD 2.3.2

1. Updated LightBDD.MsTest2 to use MsTest.TestFramework 1.2.0

It is a fix for #80 issue where it was not possible to use LightBDD.MsTest2 with MsTest.TestFramework 1.2.0 as it is binary incompatible with version 1.1.18.

It is worth to note that the dependency version upgrade brought a fix for improper status of ignored async scenarios (Microsoft/testfx#249).

2. Fixed directory separator for Unix systems

It is a fix for #87 issue where LightBDD was incorrectly defining path for reports on Unix systems. With this change, the default report paths uses System.IO.Path.DirectorySeparatorChar as directory separator.

LightBDD 2.3.1

1. Fixed "Tests accessing Application Settings fails on NET45x with SerializationException"

It is a fix for #73 issue where LightBDD targeting NET45x was not able to execute tests accessing Application Settings and failing with exception:

System.Runtime.Serialization.SerializationException : Type 'LightBDD.NUnit2.Implementation.TestContextProvider' in assembly 'LightBDD.NUnit2, Version=2.3.0.0, Culture=neutral, PublicKeyToken=null' is not marked as serializable.
		at System.AppDomain.GetHostEvidence(Type type)
		at System.Security.Policy.AppDomainEvidenceFactory.GenerateEvidence(Type evidenceType)
		at System.Security.Policy.Evidence.GenerateHostEvidence(Type type, Boolean hostCanGenerate)
		at System.Security.Policy.Evidence.GetHostEvidenceNoLock(Type type)

The issue was affecting:

  • LightBDD targeting NET45x only (the same tests were working correctly for LightBDD targeting >= NET46),
  • LightBDD.NUnit2, LightBDD.XUnit2, LightBDD.MsTest2 integrations.

LightBDD 2.3.0

1. Multi assert support for steps

The default behavior of LightBDD runner is to stop execution on first failed or ignored step. In most cases the fail fast approach is an expected behavior, however there are scenarios where it is useful to see all failures (such as when we are asserting multiple fields of the API response or checking various parameter combinations for given algorithm).

With this version of LightBDD a MultiAssertAttribute has been added and it can be applied on scenario or composite step method. When applied, all direct sub-steps belonging to the parent will be executed irrespectively to the previous step result and their failure status will be reported after all sub-steps are finished.

Example code:

[Label("Ticket-13")]
[Scenario]
[MultiAssert]
public void Adding_numbers()
{
    Runner.RunScenario(
        _ => Given_a_calculator(),
        _ => Then_adding_X_to_Y_should_give_RESULT(2, 3, 5),
        _ => Then_adding_X_to_Y_should_give_RESULT(-3, 2, -1),
        _ => Then_adding_X_to_Y_should_give_RESULT(0, 1, 1),
        _ => Then_adding_X_to_Y_should_give_RESULT(-2, -1, -3));
}

Example output:

Note: The MultiAssertAttribute applies only on direct sub-steps, which means that if current scenario/step has a sub-step that is a composite, it's sub-steps will execute in normal way. The MultiAssertAttribute has to be applied explicitly on that composite step as well in order to run those sub-steps in multi-assert mode.

To see example usage, please check Calculator_feature.cs.

2. Declarative way for ignoring scenarios and steps

The IgnoredScenarioAttribute has been added to all integrations (LightBDD.NUnit3, LightBDD.NUnit2, LightBDD.XUnit2, LightBDD.MsTest2) and can be applied on scenario method or any step method to immediately ignore it when runner start executing it.

This attribute should be now used instead of test framework specific [Ignore] or [Fact(Skip="reason")] attributes, because scenarios ignore with IgnoredScenarioAttribute will be included in execution reports.

Example code for ignored scenario:

[Scenario]
[IgnoreScenario("Not implemented yet")]
public async Task Successful_payment()
{
    await Runner.RunScenarioAsync(
        _ => Given_customer_has_some_products_in_basket(),
        _ => Given_customer_has_enough_money_to_pay_for_products(),
        _ => When_customer_requests_to_pay(),
        _ => Then_payment_should_be_successful());
}

Example output:

Example code with ignored step:

[Scenario]
public async Task Successful_payment()
{
    await Runner.RunScenarioAsync(
        _ => Given_customer_has_some_products_in_basket(),
        _ => Given_customer_has_enough_money_to_pay_for_products(),
        _ => When_customer_requests_to_pay(),
        _ => Then_payment_should_be_successful());
}

[IgnoreScenario("Not implemented yet")]
private async Task Given_customer_has_enough_money_to_pay_for_products()
{
    /* ... */
}

Example output:

3. Improved scenario/step parameter formatting

The scenario/step parameter formatting feature has been significantly reworked:

  • a default formatter for collections (implementing IEnumerable) and dictionaries (implementing IDictionary) has been added, so that any parameters of those types will be formatted by LightBDD without need of specifying [Format*] attributes,
  • the formatting of the collections (both, default and attribute based) has been extended to honor custom formatting of collection items,
  • the null values are formatted now as <null> instead of empty string,
  • it is possible now to specify multiple [Format*] attributes on a parameter, where first attribute that can format given type will be used (based on Order property and CanFormat() method of the attribute),
  • FormatAttribute has been extended with SupportedType property that is used to specify which type (including descendants) it applies to,
  • it is now possible to register custom formatters on a assembly level with configuration.ValueFormattingConfiguration() and RegisterExplicit() or RegisterGeneral() methods, where
    • RegisterExplicit() is used to register formatter for specific, closed type, such as string or DateTime,
    • RegisterGeneral() is used to register formatter for any type that would fulfil the criteria specified in CanFormat() method of the formatter,
  • a ISelfFormattable interface has been added, allowing to implement formatting method on the type.

The rules for picking a valid formatter for given parameter value are following now:

  1. if value is null, it is formatted as <null>, otherwise
  2. all attributes that are applied on the parameter declaration and extend ParameterFormatterAttribute are sorted by Order property and first matching is used (based on CanFormat() method), otherwise
  3. if type implements ISelfFormattable, it is used then, otherwise
  4. matching strict formatter is used if defined in ValueFormattingConfiguration, otherwise,
  5. the first matching general formatter that is defined in ValueFormattingConfiguration is used (they are evaluated in registration order), otherwise
  6. value is formatted with string.Format() method.

Example code:

[assembly: ConfiguredLightBddScope]
/* ... */

internal class ConfiguredLightBddScopeAttribute : LightBddScopeAttribute
{
    protected override void OnConfigure(LightBddConfiguration configuration)
    {
        configuration
            .ValueFormattingConfiguration()
            // default formatting for all dates
            .RegisterExplicit(typeof(DateTime), new DateTimeValueFormatter());
    }
}

internal class DateTimeValueFormatter : IValueFormatter
{
    public string FormatValue(object value, IValueFormattingService formattingService)
    {
        var date = (DateTime)value;
        return date.ToString("yyyy-MM-dd HH:mm");
    }
}

public class My_feature : FeatureFixture
{
    [Scenario]
    public void My_scenario()
    {
        Runner.RunScenario(
            _ => Given_there_are_invoices_for_dates(
                new DateTime(2015, 01, 01), 
                new DateTime(2015, 02, 01), 
                new DateTime(2015, 03, 02)),
            _ => When_I_request_invoices_before_date(new DateTime(2015, 03, 01)),
            _ => Then_I_should_get_invoices_with_dates(
                new DateTime(2015, 01, 01), 
                new DateTime(2015, 02, 01)));
    }

    // default collection formatting and default date formatting
    private void Given_there_are_invoices_for_dates(params DateTime[] dates)
    {
    }

    // default date formatting
    private void When_I_request_invoices_before_date(DateTime date)
    {
    }

    // custom formatting for dates, while preserving default collection formatting
    private void Then_I_should_get_invoices_with_dates([Format("date={0:yyyy/MM/dd}", SupportedType = typeof(DateTime))] params DateTime[] dates)
    {
    }
}

Example outcome:

SCENARIO: My scenario
  STEP 1/3: GIVEN there are invoices for dates "2015-01-01 00:00, 2015-02-01 00:00, 2015-03-02 00:00"...
  STEP 1/3: GIVEN there are invoices for dates "2015-01-01 00:00, 2015-02-01 00:00, 2015-03-02 00:00" (Passed after 9ms)
  STEP 2/3: WHEN I request invoices before date "2015-03-01 00:00"...
  STEP 2/3: WHEN I request invoices before date "2015-03-01 00:00" (Passed after <1ms)
  STEP 3/3: THEN I should get invoices with dates "date=2015/01/01, date=2015/02/01"...
  STEP 3/3: THEN I should get invoices with dates "date=2015/01/01, date=2015/02/01" (Passed after <1ms)
  SCENARIO RESULT: Passed after 54ms

4. Fluent interface for scenario definition

It is possible now to define scenarios in fluent way and mix basic, extended, async and sync steps together (however at the cost of test readability). The use case for fluent scenario definitions is when scenario is constructed in programmatic manner (like when number of steps varies on input data or other dynamic reasons).

Example code:

using LightBDD.Framework;
using LightBDD.Framework.Scenarios.Basic;
using LightBDD.Framework.Scenarios.Extended;
using LightBDD.Framework.Scenarios.Fluent;
using LightBDD.NUnit3;

/* .. */

[Label("Ticket-14")]
[Scenario]
public async Task Browsing_invoices()
{
    await Runner
        .NewScenario()
        .AddAsyncSteps(
            _ => Given_invoice("Invoice-1"),
            _ => Given_invoice("Invoice-2"))
        .AddSteps(
            When_I_request_all_historical_invoices)
        .AddSteps(
            _ => Then_I_should_see_invoices("Invoice-1", "Invoice-2"))
        .RunAsync();
}

The fluent scenario definition is:

  • initiated with .NewScenario() method that can be called directly on Runner or Runner.WithContext(/* ... */),
  • finished with .RunAsync() that executes the scenario.

Note: The basic syntax is available only if .NewScenario() is called without context.

To see example usage, please check Invoice_history_feature.cs.

LightBDD 2.2.0

1. Capturing scenario parameters

The LightBDD.NUnit3, LightBDD.XUnit2 and LightBDD.MsTest2 integrations captures now scenario method parameters, including them in execution progress as well as in reports. The parameters are formatted in exactly same way as in step methods.

Example code:

[Scenario]
[TestCase(2, 3)]
[TestCase(3, -2)]
public void Adding_X_to_Y(int x, int y)
{
    Runner.RunScenario(
        _ => Given_a_calculator(),
        _ => When_I_add_X_to_Y(x, y),
        _ => Then_the_result_should_be_Z(x + y));
}

Example output:

SCENARIO: Adding "2" to "3"
  STEP 1/3: GIVEN a calculator...
  STEP 1/3: GIVEN a calculator (Passed after 11ms)
  STEP 2/3: WHEN I add "2" to "3"...
  STEP 2/3: WHEN I add "2" to "3" (Passed after <1ms)
  STEP 3/3: THEN the result should be "5"...
  STEP 3/3: THEN the result should be "5" (Passed after <1ms)
  SCENARIO RESULT: Passed after 92ms

SCENARIO: Adding "3" to "-2"
  STEP 1/3: GIVEN a calculator...
  STEP 1/3: GIVEN a calculator (Passed after <1ms)
  STEP 2/3: WHEN I add "3" to "-2"...
  STEP 2/3: WHEN I add "3" to "-2" (Passed after <1ms)
  STEP 3/3: THEN the result should be "1"...
  STEP 3/3: THEN the result should be "1" (Passed after <1ms)
  SCENARIO RESULT: Passed after 9ms

2. Composite steps

It is now possible to define composite steps that are made of other steps. When composite step is executed, all sub-steps are executed in declaration order making parent step passing if all are successful.
The sub-steps would be included in progress notification as well as in final execution reports, where step number would be consist of step number in composite step, prefixed by parent step number.
It is possible to make multiple level composite steps i.e. use composite step as a sub-step of higher level composite step.

To define a composite step, the step method has to return either CompositeStep or Task<CompositeStep> type.
The composite step return value should be created in following way: CompositeStep.DefineNew().AddSteps(...).Build().

The composite steps can be then used to describe scenarios in the same way as regular steps:

  • the Extended scenario syntax fully supports composite steps,
  • the Basic scenario syntax supports composite steps only if RunScenarioAsync method is used due to fact that others will not accept non-void steps.

Example usage:

private async Task<CompositeStep> Given_customer_is_logged_in()
{
    return CompositeStep.DefineNew()
        .AddSteps(
            Given_the_user_is_about_to_login,
            Given_the_user_entered_valid_login,
            Given_the_user_entered_valid_password,
            When_the_user_clicks_login_button,
            Then_the_login_operation_should_be_successful)
        .Build();
}

[Scenario]
public async Task Ordering_products()
{
    await Runner.RunScenarioAsync(
        Given_customer_is_logged_in,
        When_customer_adds_products_to_basket,
        When_customer_pays_for_products_in_basket,
        Then_customer_should_receive_order_email);
}

Example output:

SCENARIO: Ordering products
  STEP 1/4: GIVEN customer is logged in...
  STEP 1.1/1.5: GIVEN the user is about to login...
  STEP 1.1/1.5: GIVEN the user is about to login (Passed after 1ms)
  STEP 1.2/1.5: AND the user entered valid login...
  STEP 1.2/1.5: AND the user entered valid login (Passed after <1ms)
  STEP 1.3/1.5: AND the user entered valid password...
  STEP 1.3/1.5: AND the user entered valid password (Passed after <1ms)
  STEP 1.4/1.5: WHEN the user clicks login button...
  STEP 1.4/1.5: WHEN the user clicks login button (Passed after 144ms)
  STEP 1.5/1.5: THEN the login operation should be successful...
  STEP 1.5/1.5: THEN the login operation should be successful (Passed after <1ms)
  STEP 1/4: GIVEN customer is logged in (Passed after 196ms)
  STEP 2/4: WHEN customer adds products to basket...

Finally, it is possible to specify context instance that would be shared between all sub-steps within the composite by using WithContext() method: CompositeStep.DefineNew().WithContext(...).AddSteps(...).Build()

Note: The CompositeStep.DefineNew()...Build() does not execute sub-steps but just creates a definition of the composite. The specified sub-steps are executed by the engine just after parent step method returns. It may sound a bit confusing but allows to predefine composite steps and reuse them in various steps as well as properly build a step hierarchy to render progress and report.

3. Relaxed rules about step lambda parameter names

Since this version, all single character lambda parameter names would be ignored from rendering in step names (while in previous versions only _ was treated as special name to be ignored).

Any other lambda parameter names would be still used to render as step types (as described in Scenario Steps Definition wiki page).

To visualize the change, the following scenario:

await Runner.RunScenarioAsync(
    x => Given_customer_has_some_products_in_basket(),
    x => Given_customer_has_enough_money_to_pay_for_products(),
    x => When_customer_requests_to_pay(),
    x => Then_payment_should_be_successful());

would render as:

STEP 1/4: GIVEN customer has some products in basket...
STEP 1/4: GIVEN customer has some products in basket (Passed after 129ms)
STEP 2/4: AND customer has enough money to pay for products...
STEP 2/4: AND customer has enough money to pay for products (Passed after 127ms)
STEP 3/4: WHEN customer requests to pay...
STEP 3/4: WHEN customer requests to pay (Passed after 145ms)
STEP 4/4: THEN payment should be successful...
STEP 4/4: THEN payment should be successful (Passed after 159ms)

It is possible to revert this behaviour to one from previous version of LightBDD in LightBDD Configuration by using UpdateUseLambdaNameAsStepType() method:

configuration
    .StepTypeConfiguration()
    .UpdateUseLambdaNameAsStepType(name => name != "_");

4. Better async support

The methods such as Runner.RunScenarioAsync() or Runner.RunScenarioActionsAsync() are now explicitly async which makes compiler suggesting that they should be awaited.

5. More detailed failure reasons

The step and scenario failure reasons are more detailed now and includes step number, exception type, inner exception(s) as well as simplified call stack (up to 4 stack frames):

Step 4: NUnit.Framework.AssertionException :   Login should succeeded
      Expected: True
      But was:  False
    
    at NUnit.Framework.Assert.ReportFailure(String message)
    at NUnit.Framework.Assert.That[TActual](TActual actual, IResolveConstraint expression, String message, Object[] args)
    at Example.LightBDD.NUnit3.Features.Login_feature.Then_the_login_operation_should_be_successful() in d:\dev\LightBDD\examples\Example.LightBDD.NUnit3\Features\Login_feature.Steps.cs:line 58
    at LightBDD.Framework.Scenarios.Basic.Implementation.BasicStepCompiler.StepExecutor.Execute(Object context, Object[] args)

It is possible now to control how exception details are captured by using UpdateExceptionDetailsFormatter() in LightBDD Configuration:

configuration
    .ExceptionHandlingConfiguration()
    .UpdateExceptionDetailsFormatter(ex => ex.Message)

6. VSIX templates for Visual Studio 2017

The VSIX item templates have been updated to support Visual Studio 2017 including old-style and new-style csprojs (desktop, .NET Core and .NET Standard projects).