-
Notifications
You must be signed in to change notification settings - Fork 65
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
XUnit steps refactoring #278
Merged
Merged
Changes from all commits
Commits
Show all changes
5 commits
Select commit
Hold shift + click to select a range
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,77 @@ | ||
## Allure.Net.Commons [![](http://img.shields.io/nuget/vpre/Allure.Net.Commons.svg?style=flat)](https://www.nuget.org/packages/Allure.Net.Commons) | ||
.Net implementation of [Allure java-commons](https://github.com/allure-framework/allure-java/tree/master/allure-java-commons). | ||
|
||
Can be targeted either by legacy .net 4.5+ or .net standard 2.* projects. | ||
|
||
Use this library to create custom Allure adapters for .Net test frameworks. | ||
|
||
### Configuration | ||
Allure lifecycle is configured via json file with default name `allureConfig.json`. NuGet package installs `allureConfig.Template.json` which you can use as an example. There are 2 ways to specify config file location: | ||
- set ALLURE_CONFIG environment variable to the full path of json config file. This option is preferable for .net core projects which utilize nuget libraries directly from nuget packages folder. See this example of setting it via code: https://github.com/allure-framework/allure-csharp/blob/bdf11bd3e1f41fd1e4a8fd22fa465b90b68e9d3f/Allure.Commons.NetCore.Tests/AllureConfigTests.cs#L13-L15 | ||
|
||
- place `allureConfig.json` to the location of `Allure.Commons.dll`. This option can be used with .net classic projects which copy all referenced package libraries into binary folder. Do not forget to set 'Copy to Output Directory' property to 'Copy always' or 'Copy if newer' in your test project or set it in .csproj: | ||
``` | ||
<ItemGroup> | ||
<None Update="allureConfig.json"> | ||
<CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory> | ||
</None> | ||
</ItemGroup> | ||
``` | ||
Allure lifecycle will start with default configuration settings if `allureConfig.json` is not found. | ||
|
||
Raw json configuration can be accessed from `AllureLifeCycle.Instance.JsonConfiguration` to extend configuration by adapters. See extension example here: https://github.com/allure-framework/allure-csharp/blob/bdf11bd3e1f41fd1e4a8fd22fa465b90b68e9d3f/Allure.SpecFlowPlugin/PluginHelper.cs#L20-L29 | ||
|
||
|
||
Base configuration params are stored in `AllureLifeCycle.Instance.Configuration` | ||
|
||
Allure configuration section is used to setup output directory and link patterns, e.g.: | ||
``` | ||
{ | ||
"allure": { | ||
"directory": "allure-results", // optional, default value is "allure-results" | ||
"title": "custom run title", // optional | ||
"links": //optional | ||
[ | ||
"https://example.org/{link}", | ||
"https://example.org/{issue}", | ||
"https://example.org/{tms}" | ||
] | ||
} | ||
} | ||
``` | ||
All | ||
Link pattern placeholders will be replaced with URL value of corresponding link type, e.g. | ||
|
||
`link(type: "issue", url: "BUG-01") => https://example.org/BUG-01` | ||
|
||
### AllureLifecycle | ||
[AllureLifecycle](https://github.com/allure-framework/allure-csharp/blob/main/Allure.Commons/AllureLifecycle.cs) class provides methods for test engine events processing. | ||
|
||
Use `AllureLifecycle.Instance` property to access. | ||
|
||
#### Fixture Events | ||
* StartBeforeFixture | ||
* StartAfterFixture | ||
* UpdateFixture | ||
* StopFixture | ||
|
||
#### Testcase Events | ||
* StartTestCase | ||
* UpdateTestCase | ||
* StopTestCase | ||
* WriteTestCase | ||
|
||
### Step Events | ||
* StartStep | ||
* UpdateStep | ||
* StopStep | ||
|
||
#### Attachment Events | ||
* AddAttachment - adds attachment to the current lifecycle executable item | ||
* AddScreenDiff - adds needed artifacts to the test case with given uuid to be used with [screen-diff-plugin](https://github.com/allure-framework/allure2/tree/master/plugins/screen-diff-plugin) | ||
|
||
#### Utility Methods | ||
* CleanupResultDirectory - can be used in test run setup to clean old result files | ||
|
||
### Troubleshooting | ||
... |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,146 @@ | ||
## SpecFlow Adapter [![](http://img.shields.io/nuget/vpre/Allure.SpecFlow.svg?style=flat)](https://www.nuget.org/packages/Allure.SpecFlow) | ||
Currently supports [SpecFlow](http://specflow.org/) v2.1 - 3.1.* | ||
|
||
See [Allure report](https://allure-secondary.z23.web.core.windows.net/) generated from https://github.com/allure-framework/allure-csharp/tree/main/Allure.Features | ||
|
||
Please use corresponding NuGet package version. | ||
### Installation | ||
|
||
Install [Allure.SpecFlow](https://www.nuget.org/packages/Allure.SpecFlow) nuget package according to your Specflow version. | ||
### Configuration | ||
For Specflow 3 please add or update the following section in your specflow.json: | ||
``` | ||
"stepAssemblies": [ | ||
{ "assembly": "Allure.SpecFlowPlugin" } | ||
] | ||
``` | ||
The plugin uses Allure.Commons json configuration extended with custom sections. | ||
### Custom host name | ||
In case if you want to customize host name which is displayed in Allure Timeline section, please configure `allure.title` property in json configuraion file. | ||
#### If you use NUnit | ||
Default value for allure.directory in allureConfig.json is "allure-results", default working directory in NUnit 3.* is the working directory of console runner. If you don't want to place allure results into NUnit default working folder please either set absolute path in allure.config or set working directory for NUnit in your test setup, e.g.: | ||
``` csharp | ||
[OneTimeSetUp] | ||
public void Init() | ||
{ | ||
Environment.CurrentDirectory = Path.GetDirectoryName(GetType().Assembly.Location); | ||
} | ||
``` | ||
### Usage | ||
Just run your SpecFlow scenarios and find `allure-results` folder ready to generate Allure report. | ||
|
||
### Features | ||
#### Grouping | ||
You can structure your scenarios in 3 Allure hierarchies using feature and scenario tags. | ||
Please read [report structure](https://docs.qameta.io/allure/latest/#_report_structure) Allure documentation section for additional details. Hierarchies consist of the following levels: | ||
|
||
**Suites** | ||
* Parent Suite | ||
* * Suite | ||
* * * Sub Suite | ||
|
||
**Behaviors** | ||
* Epic | ||
* * Feature | ||
* * * Story | ||
|
||
**Packages** | ||
* Package | ||
* * Class | ||
* * * Method | ||
|
||
The plugin uses `allure:grouping` configuration section to parse tags with the regular expression. If the expression contains the group, it will be used as hierarchy level name otherwise entire match will be used. E.g: | ||
|
||
`^mytag.*` : any tags starting with `@mytag` will be used for grouping. | ||
|
||
`^type:(ui|api)` : `@ui` or `@api` tags from regex pattern will be used for grouping. | ||
|
||
Check this [config example](https://github.com/allure-framework/allure-csharp/blob/main/Tests.SpecRun/allureConfig.json) as a starting point. | ||
|
||
#### Links | ||
You can add links to your scenarios using tags. Tag and link patterns can be configured in `allureConfig.json` | ||
``` json | ||
{ | ||
"allure": { | ||
"links": [ | ||
"https://myissuetracker.org/{issue}", | ||
"https://mytestmanagementsystem.org?test={tms}" | ||
] | ||
}, | ||
"specflow": { | ||
"links": { | ||
"link": "^link:(.+)", | ||
"issue": "^\\d+", | ||
"tms": "^tms:(\\d+)" | ||
} | ||
} | ||
} | ||
``` | ||
The following scenario | ||
``` cucumber | ||
@123456 @tms:42 @link:http://example.org | ||
Scenario: I do like click on links in Allure report | ||
``` | ||
will have three links in Allure report: | ||
[123456](https://myissuetracker.org/123456), [42](https://mytestmanagementsystem.org?test=tms-42) and http://example.org. | ||
|
||
In case there are links, which are generated during tests, then they can be added in code via AllureLifecycle: | ||
``` c# | ||
AllureLifecycle.UpdateTestCase(testResult.uuid, tc => | ||
{ | ||
tc.links.Add(new Link() | ||
{ | ||
name = "Example link", | ||
url = "http://example.com" | ||
}); | ||
}); | ||
``` | ||
This will show for scenario link with Text: Example link; and url: "http://example.com". | ||
|
||
#### Severity | ||
You can add Severity for your scenarios using tags. It can be configured in `allureConfig.json` | ||
``` json | ||
"labels": { | ||
"severity": "^(normal|blocker|critical|minor|trivial)" | ||
}, | ||
``` | ||
The following scenario | ||
``` cucumber | ||
@critical | ||
Scenario: .... | ||
``` | ||
will set current scenario severity in Allure report as Blocker | ||
|
||
#### Tables conversion | ||
Table arguments in SpecFlow steps can be converted either to step csv-attacments or step parameters in the Allure report. The conversion is configurable in `specflow:stepArguments` config section. | ||
With `specflow:stepArguments:convertToParameters` set to `true` the following table arguments will be represented as parameters: | ||
* one row tables | ||
* two column tables with the headers matching both `specflow:stepArguments:paramNameRegex` and `specflow:stepArguments:paramValueRegex` regular expressions. | ||
|
||
<table> | ||
<th>SpecFlow</th> | ||
<th>Allure</th> | ||
<tr> | ||
<td> | ||
|
||
![](https://github.com/allure-framework/allure-csharp/blob/main/img/wiki-step-all.PNG) | ||
|
||
</td> | ||
<td> | ||
|
||
![](https://github.com/allure-framework/allure-csharp/blob/main/img/allure-step-all.PNG) | ||
|
||
</td> | ||
</tr> | ||
</table> | ||
|
||
#### Attachments | ||
You can add attachments to an Allure report from your step bindings: | ||
```csharp | ||
using Allure.Commons; | ||
... | ||
AllureLifecycle.AddAttachment(path, "Attachment Title"); | ||
// or | ||
AllureLifecycle.AddAttachment("Attachment Title", "application/txt", "path"); | ||
// where "application/txt" - type of your attachment | ||
``` |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
maybe align configuration with https://github.com/allure-framework/allure-js/blob/master/packages/allure-cucumberjs/README.md ?