Skip to content

Test Framework Integrations

Wojtek edited this page Oct 28, 2019 · 20 revisions

Page version: 3.x / 2.x / 1.x

LightBDD.NUnit3

LightBDD supports NUnit series 3.x on:

  • .NET Framework (>= 4.5)
  • .NET Standard (>=1.6)
[assembly: LightBddScope]

[Description(
@"In order to buy products
As a customer
I want to add products to basket")]
[Label("Story-4")]
public partial class Basket_feature
{
    [Scenario]
    [Label("Ticket-6")]
    [Category(Categories.Sales)]
    public void No_product_in_stock()
    {
        Runner.RunScenario(
            Given_product_is_out_of_stock,
            When_customer_adds_it_to_the_basket,
            Then_the_product_addition_should_be_unsuccessful,
            Then_the_basket_should_not_contain_the_product);
    }
}

The NUnit [Test] attribute is replaced by [Scenario] attribute, which is required to run the LightBDD scenarios.

The LightBddScopeAttribute is present and has to be used in test assembly in order to support LightBDD. It is possible to make a subclass of this attribute and use it instead, which would allow to customize LightBDD configuration.

In addition to standard LightBDD behavior a following list of NUnit specific features/types are supported:

  • DescriptionAttribute - this attribute is recognized and have the same meaning like LightBDD FeatureDescriptionAttribute,
  • CategoryAttribute - this attribute is recognized and have the same meaning like LightBDD ScenarioCategoryAttribute (please note that ScenarioCategoryAttribute is not recognized by NUnit),
  • Assert.Ignore() and Assert.Inconclusive() - those assertions effects with scenario with Ignored status.

The DefaultProgressNotifier is used by default to report execution summary for scenarios displayed on VisualStudio Test Explorer output / Resharper output window and console when test output is printed.

The ParallelProgressNotifierProvider is used by default to report immediate execution progress for scenarios and features, allowing to track long running tests on CI and Resharper output window.

Please browse Example.LightBDD.NUnit3 project to see it in action and learn more how to run tests from console.

LightBDD.MsTest2

LightBDD supports MSTest.TestFramework on:

  • .NET Framework (>=4.5)
  • .NET Standard (>=1.3)
[TestClass]
public class LightBddIntegration
{
    [AssemblyInitialize]
    public static void Setup(TestContext testContext) { LightBddScope.Initialize(); }
    [AssemblyCleanup]
    public static void Cleanup() { LightBddScope.Cleanup(); }
}

[FeatureDescription(
@"In order to buy products
As a customer
I want to add products to basket")]
[TestClass]
[Label("Story-4")]
public partial class Basket_feature
{
    [Scenario]
    [Label("Ticket-6")]
    [TestCategory(Categories.Sales)]
    public void No_product_in_stock()
    {
        Runner.RunScenario(
            Given_product_is_out_of_stock,
            When_customer_adds_it_to_the_basket,
            Then_the_product_addition_should_be_unsuccessful,
            Then_the_basket_should_not_contain_the_product);
    }
}

The MsTest [TestMethod] attribute is replaced by [Scenario] attribute, which is required to run the LightBDD scenarios.

The test assembly has to contain a class with:

  • an [AssemblyInitialize] annotated setup method has to be present and call LightBddScope.Initialize(),
  • an [AssemblyCleanup] annotated setup method has to be present and call LightBddScope.Cleanup(). The class initializing LightBDD scope can customize the configuration by passing configuration delegate to LightBddScope.Initialize() method.

In addition to standard LightBDD behavior a following list of MsTest specific features/types are supported:

  • TestCategoryAttribute - this attribute is recognized and have the same meaning like LightBDD ScenarioCategoryAttribute (please note that ScenarioCategoryAttribute is not recognized by MsTest),
  • Assert.Inconclusive() - those assertions effects with scenario with Ignored status,
  • (since 2.4.2) FeatureFixture exposes TestContext property giving access to test context.

The DefaultProgressNotifier is used by default to report execution progress for both features and scenarios.

Note: Please note that MsTest instantiates a new test instance for each test method. Because the same instance of BDDRunner has to be used for all tests in given test class, a dedicated BDDRunnerFactory class is available and it is used by FeatureFixture to retrieve proper BDDRunner instance.

Note: Please note that by default dotnet test command does not print any output during test execution, except situation when test fails. It is possible to force it to print output by enabling console logging with normal verbosity: dotnet test --logger:"console;verbosity=normal", however the scenario output will be displayed only upon test completion not during its execution.

Please browse Example.LightBDD.MsTest2 project to see it in action and learn more how to run tests from console.

LightBDD.XUnit2

LightBDD supports xUnit series 2.x on:

  • .NET Framework (>=4.5.2)
  • .NET Standard (>=1.3)
[assembly: LightBddScope]

[FeatureDescription(
@"In order to buy products
As a customer
I want to add products to basket")]
[Label("Story-4")]
public partial class Basket_feature
{
    [Scenario]
    [Label("Ticket-6")]
    [ScenarioCategory(Categories.Sales)]
    public void No_product_in_stock()
    {
        Runner.RunScenario(
            Given_product_is_out_of_stock,
            When_customer_adds_it_to_the_basket,
            Then_the_product_addition_should_be_unsuccessful,
            Then_the_basket_should_not_contain_the_product);
    }
}

The XUnit [Fact]/[Theory] attributes are replaced by [Scenario] attribute, which is required to run the LightBDD scenarios.

The LightBddScopeAttribute is present and has to be used in test assembly in order to support LightBDD. It is possible to make a subclass of this attribute and use it instead, which would allow customizing LightBDD configuration.

The StepExecution.Current.IgnoreScenario() method is available and it behaves like Assert.Ignore()/Assert.Inconclusive() methods in other frameworks. It stops current test execution, marking it 'Skipped' in xUnit report and 'Ignored' in LightBDD.

In addition to standard LightBDD behavior a following list of XUnit specific features/types are supported:

  • (since version 2.3.5) TraitAttribute - this attribute is recognized if used with Category name, i.e.: [Trait("Category", "...")], and have the same meaning like LightBDD ScenarioCategoryAttribute (please note that ScenarioCategoryAttribute is not recognized by XUnit),

  • (since version 2.4.0) ClassCollectionBehaviorAttribute - this is an experimental, assembly-level attribute enabling parallel execution of all tests and test cases in test class, as long as all following conditions are meet:

    • test class does not implement IClassFixture nor ICollectionFixture attribute,
    • test class does not have defined named collection [Collection] attribute,
    • parallelization is enabled on xunit level.

    Example usage: [assembly: ClassCollectionBehavior(AllowTestParallelization = true)]

  • (since version 3.1.1) The Skip property on [Scenario] and [InlineData] attributes is recognized and such scenarios are included in LightBDD reports.
    It means that any code before LightBDD Runner.Run... call is executed for such tests.
    If such behavior is undesired, the native xunit behavior can be restored by adding [assembly: UseXUnitSkipBehavior] to the test project.

The DefaultProgressNotifier is used by default to report execution summary for scenarios displayed on VisualStudio Test Explorer output / Resharper output window and console when test output is printed.

The ParallelProgressNotifier by default to report immediate execution progress of scenarios and features (see below how to enable it for dotnet test).

Before version 2.3.4, the FeatureFixture constructor required also ITestOutputHelper parameter to be provided in order to properly initialize progress notifiers - since 2.3.4 there is no need to provide any parameterized constructors.

Note: Please note that by default dotnet test command does not print any output during test execution, except situation when test fails. It is possible to force it to print output by enabling console logging with normal verbosity: dotnet test --logger:"console;verbosity=normal", however the scenario output will be displayed only upon test completion not during its execution.

To see the progress of all scenarios during their execution, please use xunit.console.exe from xunit.runner.console package or dotnet xunit command.

Please browse Example.LightBDD.XUnit2 project to see it in action and learn more how to run tests from console.

LightBDD.Fixie2

LightBDD supports Fixie series 2.x on:

  • .NET Framework (>=4.5.2)
  • .NET Core (>=2.0)
internal class WithLightBddConventions : LightBddDiscoveryConvention
{
}

internal class ConfiguredLightBddScope : LightBddScope
{
}

[FeatureDescription(
@"In order to buy products
As a customer
I want to add products to basket")]
[Label("Story-4")]
public partial class Basket_feature: FeatureFixture
{
    [Scenario]
    [Label("Ticket-6")]
    [ScenarioCategory(Categories.Sales)]
    public void No_product_in_stock()
    {
        Runner.RunScenario(
            Given_product_is_out_of_stock,
            When_customer_adds_it_to_the_basket,
            Then_the_product_addition_should_be_unsuccessful,
            Then_the_basket_should_not_contain_the_product);
    }
}

The [Scenario] attribute describes LightBDD scenario methods.

The LightBddDiscoveryConvention type has to be extended in the test assembly in order to support LightBDD conventions. When extended, it is possible to pass category filter that will make only scenarios with matching [ScenarioCategory("...")] attributes run:

internal class WithLightBddConventions : LightBddDiscoveryConvention
{
    /// <summary>
    /// This constructor allows to specify category filters.
    /// </summary>
    public WithLightBddConventions(string[] category)
    {
        IncludeCategories(category);
    }
}

The LightBddScope type has to be extended in the test assembly in order to properly execute LightBDD scenarios. When extended, it offers a set of methods to configure LightBDD as well as run code during startup/shutdown:

internal class ConfiguredLightBddScope : LightBddScope
{
    protected override void OnConfigure(LightBddConfiguration configuration)
    {
        /* ... */
    }

    protected override void OnSetUp()
    {
        /* ... */
    }

    protected override void OnTearDown()
    {
        /* ... */
    }
}

The StepExecution.Current.IgnoreScenario() method is available and stops current test execution, marking it 'Skipped' in Fixie and 'Ignored' in LightBDD reports.

The InlineCaseAttribute allows to specify parameters for the parameterized scenarios:

[Scenario]
[InlineCase(true)]
[InlineCase(false)]
public void Given_flag_is_in_state([FormatBoolean("on","off")bool state)
{
}

It is also possible to make more complex scenario case source attributes by implementing own attributes with IScenarioCaseSourceAttribute interface (see example implementation).

The ParallelProgressNotifier is used by default to report execution summary for scenarios displayed on VisualStudio Test Explorer output / Resharper output window and console when test output is printed.

The immediate execution progress of scenarios and features is not available at this moment.

Please browse Example.LightBDD.Fixie2 project to see it in action and learn more how to run tests from console.

Continue reading: LightBDD Configuration

Clone this wiki locally
You can’t perform that action at this time.