This repo contains code that supports the tutorial Testing using SpecFlow and Gherkin Scenarios in C# showcasing the integration between Xray Test Management on Jira and SpecFlow (C#).
The test automation code implements some basic tests (i.e. scenarios/scenario outlines) for a dummy calculator, using a story (i.e. addition) as the feature(s) we aim to deliver.
The code for this tutorial is written in C#. In order to run this tutorial, you need to have Net 5 (i.e. .NET Core 5.0) and mono; mono is required by SpecFlow to generate the Cucumber compatible JSON report.
Dependencies are installed automatically whenever running dotnet restore.
- to generate the Cucumber compatible JSON report, we need to use the "SpecFlow+ Runner" test runner, which supports .NET Core up to .NET 5.0, and take advantage of the possibility of generating custom reports
- in January 2021, SpecFlow team decided to end up with development of the "SpecFlow+ Runner" test runner
- for .NET 6.0 there is no way of generating Cucumber JSON reports, as the supported test runners (e.g., NUnit, xUnit, MSTest) don't support it; if using SpecFlow and integrating it with Xray, or any other tool that uses Cucumber JSON reports, is important to you, then you should reach out SpecFlow team
The straighforward approach to run everything in a single shot is to invoke the auxiliary script go.sh. You need to update the client_id and client_secret to interact with your Xray cloud instance (obtainable from Xray API Keys section); you also need to update the issue keys of the corresponding stories in Jira, that you'll use as basis to generate the .feature file(s).
Remember that to be able to run the scenarios, we need to have the corresponding .feature files. The previous script assumes you are using Xray to manage the specification of the scenarios, so you need to extract them from Jira (more on the possible workflows here).
When you have the .feature files, containing the Feature along with the Scenario properly tagged, you can finally run the tests using the dotnet tool, from within the folder containing your test project (e.g. Calculator.Specs).
dotnet restore
dotnet clean
dotnet testWe need to specify that we want a Cucumber JSON report as output, and also the path where the .feature files are at.
Tests can also run inside a Docker container; local directory/file should be mounted so that Cucumber results are stored locally.
docker build . -t specflow_specrun_net5_tests
docker run --rm -v $(pwd)/TestResults:/source/SpecFlowCalculator.Specs/TestResults -t specflow_specrun_net5_testsResults can be submitted to Jira so that they can be shared with the team and their impacts be easily analysed. This can be achieved using Xray Test Management as shown in further detail in this tutorial. This repo contains an auxiliary script import_results_cloud.sh that does that; it uses a configuration file to have Xray's client_id and client_secret. You can also have a look at the workflows implemented in this repo using GitHub Actions.
This repo also contains some auxiliary shell scripts, provided as an example; feel free to download and customize them to your needs. You may find scripts for interacting both with Xray server/DC or Xray cloud, which have slightly different APIs.
Any questions related with this code, please raise issues in this GitHub project. Feel free to contribute and submit PR's. For Xray specific questions, please contact Xray's support team.