Tests Structure and Conventions (v2.x)

Page version: 3.x / 2.x / 1.x

• Test classes structure
  • Test class
  • Test method
    • ScenarioCategoryAttribute
  • Test method body
• Conventions
  • Acceptance tests definition and implementation separation
    • Partial classes and private step methods
  • Naming convention
  • Steps behavior
    • MultiAssertAttribute

Test classes structure

LightBDD is implemented on top of the well known test frameworks which makes it very easy to adopt.

Test class

Test classes are treated as Features, where:

Code	Meaning	Notes
class name	feature name	preferred method names with underscores
class level [FeatureDescription] attribute, NUnit [Description] attribute	feature description (narration)	optional, [FeatureDescription] precedes implementation specific attributes
class level [Label] attribute	can be used to correlate feature with ticket number	optional, multiple allowed
class level [ScenarioCategory] attribute, NUnit [Category] attribute, MsTest [TestCategory] attribute, XUnit [Trait("Category", "...")] attribute	associates all feature scenarios with a logical group like business epic	optional, multiple allowed, attribute type mixing allowed, equivalent of applying those categories to each test method in class

Note: The LightBDD feature classes should derive from FeatureFixture type.

Test method

Test methods are treated as Scenarios, where:

Code	Meaning	Notes
method level [Scenario] attribute	represents method to be scenario	It is equivalent of NUnit [Test], XUnit [Fact]/[Theory], MSTest [TestMethod] attributes, where test runner specific test method attributes are not supported.
method name	scenario name	preferred method names with underscores
method level [Label] attribute	can be used to correlate scenario with ticket number	optional, multiple allowed
method level [ScenarioCategory] attribute, NUnit [Category] attribute, MsTest [TestCategory] attribute, XUnit [Trait("Category", "...")] attribute	associates scenario with a logical group like business epic	optional, multiple allowed, attribute type mixing allowed, scenario would be associated to all categories defined on test method level and class level
method level [MultiAssert] attribute	can be used to instruct runner to run all its direct sub-steps no matter of their outcome and collect results after all are finished	optional; it overrides the default behavior where scenario execution will stop on first failed/ignored step
method level [IgnoreScenario] attribute	can be used to instruct runner to mark scenario as ignored without executing it and include it in reports	optional

ScenarioCategoryAttribute

As presented above, it is possible to categorize the scenarios by applying [ScenarioCategory] or underlying test framework specific category attributes on feature class or scenario method. Usage of those attributes will effect with including such categories in the feature reports.

Please note however that ScenarioCategoryAttribute is pure LightBDD concept and it is not recognized by the underlying frameworks, which means that it is not possible to filter tests by it when executed from command line - if such behavior is desired, please use instead native attributes:

• [Category] for NUnit,
• [TestCategory] for MsTest,
• [Trait("Category", "...")] for XUnit (since version 2.3.5).

Test method body

The test method body is a single call to Runner.RunScenario() with list of methods to execute in specified order, which are treated as Steps, where:

Code	Meaning	Notes
method name	step name	preferred method names with underscores
method level [MultiAssert] attribute	if step is a composite, it can be used to instruct runner to run all its direct sub-steps no matter of their outcome and collect results after all are finished	optional; it overrides the default behavior where composite step execution will stop on first failed/ignored sub-step
method level [IgnoreScenario]	can be used to instruct runner to mark step as ignored without executing it	optional

See example below:

[TestFixture]
[FeatureDescription(
@"In order to access personal data
As an user
I want to login into system")] //feature description
[Label("Story-1")]
[ScenarioCategory("Security")]
public partial class Login_feature //feature name
{
	[Test]
	[Label("Ticket-1")]
	public void Successful_login() //scenario name
	{
		Runner.RunScenario(

			Given_the_user_is_about_to_login, //scenario steps
			Given_the_user_entered_valid_login,
			Given_the_user_entered_valid_password,
			When_the_user_clicks_login_button,
			Then_the_login_operation_should_be_successful,
			Then_a_welcome_message_containing_user_name_should_be_returned);
	}
}

Please note that LightBDD supports multiple formats of scenario definitions - for more, please see Scenario Steps Definition.

Conventions

Acceptance tests definition and implementation separation

An important convention in LightBDD is that acceptance tests should be divided into two separate files:

• acceptance tests definition part containing easy to read scenarios,
• implementation part containing scenario test implementation. With it, all tests written with LightBDD would be easy to read even by people who are not experts in writing code and to maintain by developers.

Partial classes and private step methods

It is suggested to use partial classes and private step methods over other ways of separating test definition and it's implementation, because it simplifies tests maintenance in future. If after requirements change some of steps become not used any more, VisualStudio (or Resharper) will help developers with keeping test code clean, by indicating unused methods.

Naming convention

The naming convention for all names used by LightBDD is that the _ character is replaced with white space - for more details see DefaultNameFormatter class.

Steps behavior

The default steps behavior is interpreted as follows:

• if all steps exit without exception, the scenario finishes with passed status,
• if a given step has [IgnoreScenario] attribute or call Assert.Ignore() (NUnit) or Assert.Inconclusive() (NUnit/MsTest) or StepExecution.Current.IgnoreScenario() (xUnit), other steps are not executed and the scenario finishes with ignored status,
• if a given step throws any exception, other steps are not executed and scenario finishes with failed status,

Steps can also finish with bypassed status if StepExecution.Current.Bypass() method is called. Such steps however would not stop scenario execution. The scenario result would be as follows:

• if all others steps are passed/bypassed, the test would be reported as passed in testing framework, but LightBDD would report it as bypassed,
• if any consecutive step would be ignored or failed, the entire scenario would finish in a way described above.

A table below shows steps behavior. A first matching row will correspond to overall scenario status.

Trigger	Step result	Scenario behavior	Test result	Scenario result
Assert.Fail() / exception	Failed	Stop	Failed	Failed
[IgnoreScenario] / Assert.Ignore() / Assert.Inconclusive() / StepExecution.Current.IgnoreScenario()	Ignored	Stop	Ignored / Skipped / Inconclusive	Ignored
StepExecution.Current.Bypass()	Bypassed	Continue	Passed	Bypassed
no exceptions	Passed	Continue	Passed	Passed

Note: please note that applying underlying test framework [Ignore] or [Fact(Skip="reason")] attribute on scenario method will affect the test as being ignored in the test framework and it will not appear in the report as Runner.RunScenario() would be never be executed.

MultiAssertAttribute

If [MultiAssert] attribute is applied on a scenario method or composite step method, runner will always execute all direct sub-steps of the scenario/composite irrespectively to the previous sub-step result and their failure status will be reported after all sub-steps are finished. The overall status of parent step/scenario will be set to the most severe status of its sub-steps.

Continue reading: Scenario Steps Definition