Implement event-based progress notification to replace IScenarioNotifier / IFeatureNotifier

[Suremaker]

Description

Implement event-based progress notification capability, that:

• provides time-series events related to feature / scenario / step execution,
• provides easy to consume listening mechanism,
• provides the ability to extend the event types without impacting the existing consumers.

The issue with IScenarioNotifier / IFeatureNotifier

• The existing interfaces do not allow adding new types of notifications without breaking existing implementations.
• Even if the consumer is interested to consume specific notification, all methods of the interface needs to be implemented.
• The existing notifications do not provide information about the time when they occurred in a consistent manner.

Progress

• Feature is implemented,
• Ensured backward-compatibility,
• Ensured good debugging experience

[Suremaker]

The IProgressNotifier interface has been introduced and replaced IScenarioProgressNotifier and IFeatureProgressNotifier.

Unlike previous interfaces IProgressNotifier has only one method:

public interface IProgressNotifier
{
    void Notify(ProgressEvent e);
}

It is used now to send notifications about all events happening during test execution, where LightBDD.Core.Notification.Events namespace has all available events defined.

Registration

While all existing notifiers (DefaultProgressNotifier, NoProgressNotifier, ParallelProgressNotifierProvider) have been updated to support the new interface, they need to be registered using new configuration section:

configuration.ProgressNotifierConfiguration()
    .Clear()
    .Append(new DefaultProgressNotifier(Console.WriteLine));

While it was possible to create IScenarioProgressNotifier for the specific scenario, all IProgressNotifier instances are created at once and used for all fixtures and scenarios.

In order to get access to fixture specific outputs, the ScenarioExecutionContext can be used:

configuration.ProgressNotifierConfiguration()
    .Clear()
    .Append(new DefaultProgressNotifier(message => ScenarioExecutionContext
        .GetCurrentScenarioFixtureIfPresent<ITestOutputProvider>()
        ?.TestOutput
        .WriteLine(message)));

Please note that GetCurrentScenarioFixtureIfPresent() will return null when the notification is made out of the scenario context, thus it has to be checked before using TestOutput.

This use case may get simplified before the release, but if anything, it will get implemented in separate ticket

Benefits of the new approach

The main benefit is that LightBDD can now get extended to provide more notifications without breaking the interfaces and requiring all implementations to handle all notifications.