Test Progress Notification (v1.x)

Page version: 2.x / 1.x

Standard progress notification

During run, the test progress would be printed on console, like:

FEATURE: [Story-1] Login feature
  In order to access personal data
  As an user
  I want to login into system

SCENARIO: [Ticket-1] Successful login
  STEP 1/6: GIVEN the user is about to login...
  STEP 1/6: GIVEN the user is about to login (Passed after 9ms)
  STEP 2/6: AND the user entered valid login...
  STEP 2/6: AND the user entered valid login (Passed after <1ms)
  STEP 3/6: AND the user entered valid password...
  STEP 3/6: AND the user entered valid password (Passed after <1ms)
  STEP 4/6: WHEN the user clicks login button...
  STEP 4/6: WHEN the user clicks login button (Passed after 182ms)
  STEP 5/6: THEN the login operation should be successful...
  STEP 5/6: THEN the login operation should be successful (Passed after 5ms)
  STEP 6/6: AND a welcome message containing user name should be returned...
  STEP 6/6: AND a welcome message containing user name should be returned (Passed after 14ms)
  SCENARIO RESULT: Passed after 233ms

Before each step execution, a step number and name is printed, which allows developers to track long running scenarios. After step execution is finished, a step status and execution time is printed.

The presented representation of progress text is provided by ConsoleProgressNotifier class which is default for most of the LightBDD integrations.

Other notifiers implementations

Since version 1.7.0 a following notifier classes are also present:

SimplifiedConsoleProgressNotifier

A notifier dedicated for parallel test execution. It captures only events related to scenario start and finish and prints one line of text per each, containing scenario name, it's state and simplified execution progress (number of finished, pending and failed scenarios).

Below, there is an example output generated by this notifier:

Finished=0, Failed=0, Pending=1: starting: Successful addition
Finished=0, Failed=0, Pending=2: starting: Contact book should allow me to add multiple contacts
Finished=0, Failed=0, Pending=3: starting: Wrong password provided causes login to fail
Finished=0, Failed=0, Pending=4: starting: Receiving invoice for products
Finished=0, Failed=0, Pending=5: starting: Should dispatch product after payment is finalized
Finished=1, Failed=0, Pending=4: PASSED: Contact book should allow me to add multiple contacts
Finished=1, Failed=0, Pending=5: starting: Contact book should allow me to remove contacts
Finished=2, Failed=0, Pending=4: PASSED: Contact book should allow me to remove contacts
Finished=2, Failed=0, Pending=5: starting: Contact book should allow me to remove all contacts
Finished=3, Failed=0, Pending=4: BYPASSED: Contact book should allow me to remove all contacts
Finished=4, Failed=0, Pending=3: PASSED: Should dispatch product after payment is finalized
Finished=5, Failed=0, Pending=2: IGNORED: Receiving invoice for products
Finished=6, Failed=0, Pending=1: PASSED: Wrong password provided causes login to fail
Finished=6, Failed=0, Pending=2: starting: Wrong login provided causes login to fail
Finished=7, Failed=0, Pending=1: IGNORED: Successful addition
Finished=7, Failed=0, Pending=2: starting: No product in stock
Finished=8, Failed=0, Pending=1: PASSED: Wrong login provided causes login to fail
Finished=8, Failed=0, Pending=2: starting: Anonymous login name should allow to log in
Finished=9, Failed=0, Pending=1: PASSED: No product in stock
Finished=10, Failed=1, Pending=0: FAILED: Anonymous login name should allow to log in
Finished=10, Failed=1, Pending=1: starting: Successful login
Finished=11, Failed=1, Pending=0: PASSED: Successful login

It does not print any other events, to preserve output text readability. By default, LightBDD.XUnit uses this notifier, together with XUnitOutputProgressNotifier.

---

DelegatingProgressNotifier

A notifier allowing to delegate all notifications to many notifier instances. It does not emit any output by itself.

By default, LightBDD.XUnit uses it to redirect all notifications to SimplifiedConsoleProgressNotifier and XUnitOutputProgressNotifier.

---

XUnitOutputProgressNotifier (LightBDD.XUnit)

This notifier is only available if LightBDD.XUnit package is used. Displays execution progress with usage of xUnit specific ITestOutputHelper class instance, so it can be displayed in Visual Studio test windows. All notifications are in the same format as ones from ConsoleProgressNotifier.

By default, LightBDD.XUnit uses this notifier, together with SimplifiedConsoleProgressNotifier.

---

NUnit3OutputProgressNotifier (LightBDD.NUnit3)

This notifier is only available if LightBDD.NUnit3 package is used. Displays execution progress with usage of NUnit3 specific TestContext class, so it can be displayed in Visual Studio test windows as well as on console.

By default, LightBDD.NUnit3 uses this notifier.

---

TextProgressNotifier (abstract)

This notifier is an abstract class, providing a standard formatting for all notifications. ConsoleProgressNotifier and XUnitOutputProgressNotifier derives from it.

Customizing progress notification

A custom implementation of progress notification can be injected to BDDRunner class by its constructor. If tests are using FeatureFixture class, it is possible to customize progress notification by overriding CreateProgressNotifier() method.

Continue reading: Generating Reports