Documentation how to write custom generator/plugin not up to date

[zplan]

SpecFlow Version:

• 3.0
• 2.4
• 2.3
• 2.2
• 2.1
• 2.0
• 1.9

Used Test Runner

• SpecFlow+Runner
• MSTest
• NUnit
• Xunit

Visual Studio Version

• VS 2019
• VS 2017
• VS 2015
• VS 2013

Are the latest Visual Studio updates installed?

• Yes
• No

.NET Framework:

• >= .NET 4.5
• before .NET 4.5
• .NET Core 2.0
• .NET Core 2.1
• .NET Core 2.2
• .NET Core 3.0

Test Execution Method:

• Visual Studio Test Explorer
• TFS/VSTS/Azure DevOps – Task – PLEASE SPECIFY THE NAME OF THE TASK
• Command line – PLEASE SPECIFY THE FULL COMMAND LINE

Issue Description

The online documentation how to write a custom plugin or a generator is not up to date with the implementation of SpecFlow3. Me and a lot of other user are currently struggling when trying to migrate our custom generator from SpecFlow 2.x to 3.0.

For example: The documentation still contains unitTestProvider in app.config or specflow.json which does not exist anymore in SpecFlow3.

It is currently a mysterious how to migrate our custom plugins and generators. Please update the docu as fast as possible and provide some real life examples. I loved SpecFlow so far but this release together with this unusable docu is really disappointing.

[SabotageAndi]

We are working on it. We didn't expect so much plugins out there and that they migrate so fast to SpecFlow 3.0.

[StarWars999123]

I agree with it. But as far as I understood, app.config should still be usable for NetFramework 4.6 projects, but for Plugins it does not work.

[Cylox]

Note that generator plugins now need at least .NET 4.7.1, also had to find that out the hard way.

[Cylox]

We are working on it. We didn't expect so much plugins out there and that they migrate so fast to SpecFlow 3.0.

I have the very strong feeling that the fast updates are due to the per default enabled breaking update of the VS SpecFlow extension. At least this is the reason I'm currently working on the update.

[Shamshiel]

Has someone an idea how to get Specflow to pick up a Runtime plugin in .NET Core only with specflow.json?

{
  "language": {
    "feature": "en-US"
  },
  "plugins": [
    {
      "name": "SpecFlow.MSDependencyInjection",
      "type": " Runtime"
    }
  ]
}

Is the syntax correct? If I reference my Plugin project directly it works but as a Nuget package it doesn't.

[SabotageAndi]

You don't configure it anymore. It has to be in the output directory and end with .SpecFlowPlugin.dll.
See https://github.com/techtalk/SpecFlow/blob/master/TechTalk.SpecFlow/Plugins/RuntimePluginLocator.cs#L33

To get it copied via a NuGet package, you need some MSBuild magic. Something like this: https://github.com/techtalk/SpecFlow/blob/master/Plugins/TechTalk.SpecFlow.MSTest.Generator.SpecFlowPlugin/build/SpecFlow.MsTest.props#L5

[zplan]

You don't configure it anymore. It has to be in the output directory and end with .SpecFlowPlugin.dll.
See https://github.com/techtalk/SpecFlow/blob/master/TechTalk.SpecFlow/Plugins/RuntimePluginLocator.cs#L33

To get it copied via a NuGet package, you need some MSBuild magic. Something like this: https://github.com/techtalk/SpecFlow/blob/master/Plugins/TechTalk.SpecFlow.MSTest.Generator.SpecFlowPlugin/build/SpecFlow.MsTest.props#L5

I am getting more confused now.
We have the scenario, that we have our own MSTest-Generator but we use the MSTest-Runtime from Specflow.

To have the MSTest-Runtime from Specflow, I have to install the Specflow.MSTest nuget package and within this package, the MS-Test Generator from Specflow gets installed automatically via MSBuild. How can I now only use the Runtime with our own Generator? Do I have to package the Specflow Runtime again in our own Nuget package together with our Generator?

[SabotageAndi]

@zplan What are your changes to the MSTest- Generator? Perhaps it makes sense to contribute it back?
If not, you have to create your own NuGet package and package our MSTest Runtime package into it.

[Cylox]

You don't configure it anymore. It has to be in the output directory and end with .SpecFlowPlugin.dll.
See https://github.com/techtalk/SpecFlow/blob/master/TechTalk.SpecFlow/Plugins/RuntimePluginLocator.cs#L33
To get it copied via a NuGet package, you need some MSBuild magic. Something like this: https://github.com/techtalk/SpecFlow/blob/master/Plugins/TechTalk.SpecFlow.MSTest.Generator.SpecFlowPlugin/build/SpecFlow.MsTest.props#L5

I am getting more confused now.
We have the scenario, that we have our own MSTest-Generator but we use the MSTest-Runtime from Specflow.

To have the MSTest-Runtime from Specflow, I have to install the Specflow.MSTest nuget package and within this package, the MS-Test Generator from Specflow gets installed automatically via MSBuild. How can I now only use the Runtime with our own Generator? Do I have to package the Specflow Runtime again in our own Nuget package together with our Generator?

You don't have to reference the SpecFlow.MSTest nuget package in order to use the MSTest generator functionality in your own plugin. The generators are included in the SpecFlow.CustomPlugin nuget package (exactly: in the TechTalk.SpecFlow.Generator assembly), all the generator plugin packages just alter the configuration to tell specflow to use the desired generator. Just have a look at the generator plugins, you'll find out that they are more or less one-liners that set the desired generator plugin name.
When you have subclassed or wrapped the MSTest generator plugin in your own generator plugin this will work just like before. I'm acually doing the exact same thing but with the NUnit3 generator.
You need to remove the SpecFlow.MSTest plugin from your project, this is the reason why your own plugin is not loaded.

[SabotageAndi]

@Cylox Currently the generator plugins only set the unit Test provider. But I want to move the whole generation code for the different frameworks into the plugins.
The code is located in 3-4 locations and I want it to be in one place.

[Cylox]

@SabotageAndi From a structural perspective of view I think this is the right thing to do. The generator code itself should however still be usable without enforcing the corresponding plugin to be active. Otherwise use cases like those of @zplan and mine where the generator code is extended in our own plugin will be made impossible.

[SabotageAndi]

@Cylox & @zplan: what are you customizing in your MSTest generator. Would be interesting to know the concrete use-case.

[Cylox]

@SabotageAndi I have wrapped the NUnit test generator inside of my own generator to be able to add [NUnit.Framework.OrderAttribute] to the test methods. Each test method gets an [OrderAttribute] that reflects its position in the .feature file. The aim is to make sure that the scenarios in the feature file are always executed in the order as they are written in the feature file.
I need this because many of our scenarios are dependent on each other. Before telling me that this is an awful thing to do: We are executing UI tests through these scenarios and of course the state of the target UI changes with each scenario. We have a convention that all tests that depend on each other are contained within one feature file, so that if one scenario fails the next feature can still be executed independently.

[zplan]

@SabotageAndi I do basically the same as @Cylox . We need to attribute the test methods because they are later on analysed via reflection for specific attributes.
To do so we set tags before the Scenario like:

@Testdate:12-12-2019
@Testowner:myName
Scenario abc...

--> The generator just wraps the ms test generator from specflow and and additionally creates MethodAttributes for the TestMethods from the @-Tags. So the generator is not really replaced, just enhanced.

[SabotageAndi]

Writing this documentation is in our next sprint.

[ParagRaut]

@SabotageAndi I'm using custom plugin "MsTestContext.SpecflowPlugin" to add test context in generated code behind files at compile time and then use "Specflow.Autofac.SpecFlowPlugin" plugin to register dependencies at runtime. Currently it seems like Specflow.Autofac plugin is not updated for Specflow v3.
Do you have any idea when we can expect an update ?
I have already put an issue here
https://github.com/gasparnagy/SpecFlow.Autofac/issues/20

[SabotageAndi]

@ParagRaut SpecFlow.Autofac is not yet part of SpecFlow. I talked with @gasparnagy to make it part of it. He hasn't yet send the PR to contribute it.

[ParagRaut]

Hi @gasparnagy, can you please do the needful ? as many of my projects depends on that plugin 😀
Thanks in advance

[gasparnagy]

@ParagRaut I try to do it on Monday

[gasparnagy]

I have released a v3 compatible version of the SpecFlow.Autofac repo. For long term, track #1533.

[ParagRaut]

Thanks for quick feedback and contribution @gasparnagy
I will check if my project works now 😅

[SabotageAndi]

PR with first version of documenation is up: #1539

If you find something missing, please let me know.

[SabotageAndi]

PR is merged. Docs is located at https://github.com/techtalk/SpecFlow/blob/master/docs/plugins/Writing%20Plugins%20for%20SpecFlow%203.md or https://specflow.org/documentation/Plugins/

We also created a Gitter channel special for plugin development: https://gitter.im/techtalk/specflow-plugin-dev
We did this, that the developer of plugins can easily talk to each other and help out if needed.
We will also use this channel to announce changes to the plugin interfaces, so that you aren't surprised when a new version is released.

[lock]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.