New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Discussion] How to handle the VS Integration for SpecFlow 3 #1357

Stephen-Mc opened this Issue Dec 6, 2018 · 4 comments


None yet
3 participants

Stephen-Mc commented Dec 6, 2018

This is the place to discuss the VSIX upgrade process for SpecFlow 3. As announced here, we need to update the SpecFlow Visual Studio extension (VSIX) for SpecFlow 3. The updates mean the new extension will not be compatible with older versions of SpecFlow ( <2.3.2).

Our intention is to update the existing extension with a new version. By default, the extension updates automatically, which means this approach will break older projects using versions of SpecFlow prior to 2.3.2. An alternative approach would be to release a dedicated SpecFlow 3 extension. However, we ended up discounting this approach for a number of reasons that are summarised below. Each approach has disadvantages and could potentially cause projects to stop working.

Approach 1: Release a New SpecFlow 3 Extension

In this case, we would release a new VSIX for SpecFlow 3, and leave the current extension online for users of earlier versions.


  • Users of earlier versions do not have to do anything (i.e. disable automatic updates) to prevent their projects from breaking.


  • Having two separate versions of a SpecFlow 3 extension in the gallery will cause confusion. Some users will invariably install the wrong version, increasing the number of support issues. This will persist for as long as both extensions are available, i.e. way past the transition period where most users will have migrated from 2 to 3.
  • There will be conflicts if both extensions are installed. There is no way to guarantee that only one of the extensions is installed.
  • Users suddenly need another extension for what appears to be "no good reason".
  • Punishes users who consistently update to new versions, as these are the ones who need to jump through hoops to use the new release.

Approach 2: Upgrade the Existing Extension

In this case, we would update the existing VSIX for compatibility with SpecFlow 3.


  • While there may be some short-term confusion and support requests relating to projects suddenly not working (automatic update), these will die down quickly. After ~6 months, the dust will probably have settled.
  • All users will have their extension automatically upgraded to be compatible with the latest version. This is how it has always been.


  • Requires users not interested in Specflow 3 to make changes to their setup to avoid breaking changes. However, this is a one-off and involves clicking a check box.
  • There will invariably be some users who need to continue using SpecFlow <2.3.2 for now, and who will forget that automatic updates need to be enabled when they make the switch to version 3. Again, the number should decrease as time goes on.


With that in mind, we feel that upgrading the existing extension is better approach. We have begun communicating this change to users, so that automatic upgrades can be disabled in good time for those users who will not be migrating to SpecFlow 3 and are using older SpecFlow versions.

It will undoubtedly cause some teething problems for a few months while the switch is made. After a while, the dust will settle. Conversely, we feel that providing two separate extensions runs the risk of causing confusion due to multiple versions of the extension, and the confusion will persist indefinitely. It will not simply die down after a few months if there are two mutually exclusive extensions available for download.


Both approaches will make it difficult to work with older SpecFlow versions and SpecFlow 3 on the same machine; we are not sure to what extent this will be an issue. Let us know you thoughts

Call for Input

If you think we have overlooked some important aspects or have have alternative suggestions on how to improve the upgrade experience, please share them with us.

Furthermore, if you are planning on sticking with an older version of SpecFlow (particular prio to 2.3.2), and do regularly upgrade, let us know your motivation too.


This comment has been minimized.

Tragedian commented Dec 7, 2018

What specific things will the SpecFlow 3 extension break when used with an older version?

Is it worth exploring the possibility of maintaining backwards compatibility?

I'm in the position where I will need to use SpecFlow 2 and SpecFlow 3 depending on what project I'm working on that day, and having to jump between two different VISX packages is far from an ideal world.


This comment has been minimized.


SabotageAndi commented Dec 10, 2018

We will remove the "AppDomain" generation mode and always use the OutOfProcess generation mode.
This works reliabler than the AppDomain mode, but has problems with older SpecFlow versions.

@Tragedian which SpecFlow 2 versions are you using? Everything since 2.3.2 will work. It's only about version before that.


This comment has been minimized.

Tragedian commented Dec 10, 2018

Great to get clarity on version support; 2.3.2 is certainly a more achievable upgrade goal than 3.0. Will the new tooling try to "migrate" the older projects in any way? I'm trying to understand if it will make changes that would typically be visible via source-control which would force other team members to upgrade their VISX packages at the same time.

The oldest project targets 2.2.1. I believe there are plugins in use which stopped functioning when trying to move to 2.3.0.


This comment has been minimized.


SabotageAndi commented Dec 10, 2018

The new version does not make any migration for your project. You have to do it your self.

Which plugins are not working with 2.3.? We changed in every minor version the plugin interface. They aren't big so updating these, should not be a lot of work.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment