-
Notifications
You must be signed in to change notification settings - Fork 65
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
* Allure.XUnit steps improvements * Allure.XUnit added support for broken and skipped tests * README.md was sliced to several files according to projects
- Loading branch information
Showing
22 changed files
with
675 additions
and
391 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,77 @@ | ||
| ## Allure.Net.Commons [](https://www.nuget.org/packages/Allure.Net.Commons) | ||
| .Net implementation of [Allure java-commons](https://github.com/allure-framework/allure-java/tree/master/allure-java-commons). | ||
|
|
||
| Can be targeted either by legacy .net 4.5+ or .net standard 2.* projects. | ||
|
|
||
| Use this library to create custom Allure adapters for .Net test frameworks. | ||
|
|
||
| ### Configuration | ||
| Allure lifecycle is configured via json file with default name `allureConfig.json`. NuGet package installs `allureConfig.Template.json` which you can use as an example. There are 2 ways to specify config file location: | ||
| - set ALLURE_CONFIG environment variable to the full path of json config file. This option is preferable for .net core projects which utilize nuget libraries directly from nuget packages folder. See this example of setting it via code: https://github.com/allure-framework/allure-csharp/blob/bdf11bd3e1f41fd1e4a8fd22fa465b90b68e9d3f/Allure.Commons.NetCore.Tests/AllureConfigTests.cs#L13-L15 | ||
|
|
||
| - place `allureConfig.json` to the location of `Allure.Commons.dll`. This option can be used with .net classic projects which copy all referenced package libraries into binary folder. Do not forget to set 'Copy to Output Directory' property to 'Copy always' or 'Copy if newer' in your test project or set it in .csproj: | ||
| ``` | ||
| <ItemGroup> | ||
| <None Update="allureConfig.json"> | ||
| <CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory> | ||
| </None> | ||
| </ItemGroup> | ||
| ``` | ||
| Allure lifecycle will start with default configuration settings if `allureConfig.json` is not found. | ||
|
|
||
| Raw json configuration can be accessed from `AllureLifeCycle.Instance.JsonConfiguration` to extend configuration by adapters. See extension example here: https://github.com/allure-framework/allure-csharp/blob/bdf11bd3e1f41fd1e4a8fd22fa465b90b68e9d3f/Allure.SpecFlowPlugin/PluginHelper.cs#L20-L29 | ||
|
|
||
|
|
||
| Base configuration params are stored in `AllureLifeCycle.Instance.Configuration` | ||
|
|
||
| Allure configuration section is used to setup output directory and link patterns, e.g.: | ||
| ``` | ||
| { | ||
| "allure": { | ||
| "directory": "allure-results", // optional, default value is "allure-results" | ||
| "title": "custom run title", // optional | ||
| "links": //optional | ||
| [ | ||
| "https://example.org/{link}", | ||
| "https://example.org/{issue}", | ||
| "https://example.org/{tms}" | ||
| ] | ||
| } | ||
| } | ||
| ``` | ||
| All | ||
| Link pattern placeholders will be replaced with URL value of corresponding link type, e.g. | ||
|
|
||
| `link(type: "issue", url: "BUG-01") => https://example.org/BUG-01` | ||
|
|
||
| ### AllureLifecycle | ||
| [AllureLifecycle](https://github.com/allure-framework/allure-csharp/blob/main/Allure.Commons/AllureLifecycle.cs) class provides methods for test engine events processing. | ||
|
|
||
| Use `AllureLifecycle.Instance` property to access. | ||
|
|
||
| #### Fixture Events | ||
| * StartBeforeFixture | ||
| * StartAfterFixture | ||
| * UpdateFixture | ||
| * StopFixture | ||
|
|
||
| #### Testcase Events | ||
| * StartTestCase | ||
| * UpdateTestCase | ||
| * StopTestCase | ||
| * WriteTestCase | ||
|
|
||
| ### Step Events | ||
| * StartStep | ||
| * UpdateStep | ||
| * StopStep | ||
|
|
||
| #### Attachment Events | ||
| * AddAttachment - adds attachment to the current lifecycle executable item | ||
| * AddScreenDiff - adds needed artifacts to the test case with given uuid to be used with [screen-diff-plugin](https://github.com/allure-framework/allure2/tree/master/plugins/screen-diff-plugin) | ||
|
|
||
| #### Utility Methods | ||
| * CleanupResultDirectory - can be used in test run setup to clean old result files | ||
|
|
||
| ### Troubleshooting | ||
| ... |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,146 @@ | ||
| ## SpecFlow Adapter [](https://www.nuget.org/packages/Allure.SpecFlow) | ||
| Currently supports [SpecFlow](http://specflow.org/) v2.1 - 3.1.* | ||
|
|
||
| See [Allure report](https://allure-secondary.z23.web.core.windows.net/) generated from https://github.com/allure-framework/allure-csharp/tree/main/Allure.Features | ||
|
|
||
| Please use corresponding NuGet package version. | ||
| ### Installation | ||
|
|
||
| Install [Allure.SpecFlow](https://www.nuget.org/packages/Allure.SpecFlow) nuget package according to your Specflow version. | ||
| ### Configuration | ||
| For Specflow 3 please add or update the following section in your specflow.json: | ||
| ``` | ||
| "stepAssemblies": [ | ||
| { "assembly": "Allure.SpecFlowPlugin" } | ||
| ] | ||
| ``` | ||
| The plugin uses Allure.Commons json configuration extended with custom sections. | ||
| ### Custom host name | ||
| In case if you want to customize host name which is displayed in Allure Timeline section, please configure `allure.title` property in json configuraion file. | ||
| #### If you use NUnit | ||
| Default value for allure.directory in allureConfig.json is "allure-results", default working directory in NUnit 3.* is the working directory of console runner. If you don't want to place allure results into NUnit default working folder please either set absolute path in allure.config or set working directory for NUnit in your test setup, e.g.: | ||
| ``` csharp | ||
| [OneTimeSetUp] | ||
| public void Init() | ||
| { | ||
| Environment.CurrentDirectory = Path.GetDirectoryName(GetType().Assembly.Location); | ||
| } | ||
| ``` | ||
| ### Usage | ||
| Just run your SpecFlow scenarios and find `allure-results` folder ready to generate Allure report. | ||
|
|
||
| ### Features | ||
| #### Grouping | ||
| You can structure your scenarios in 3 Allure hierarchies using feature and scenario tags. | ||
| Please read [report structure](https://docs.qameta.io/allure/latest/#_report_structure) Allure documentation section for additional details. Hierarchies consist of the following levels: | ||
|
|
||
| **Suites** | ||
| * Parent Suite | ||
| * * Suite | ||
| * * * Sub Suite | ||
|
|
||
| **Behaviors** | ||
| * Epic | ||
| * * Feature | ||
| * * * Story | ||
|
|
||
| **Packages** | ||
| * Package | ||
| * * Class | ||
| * * * Method | ||
|
|
||
| The plugin uses `allure:grouping` configuration section to parse tags with the regular expression. If the expression contains the group, it will be used as hierarchy level name otherwise entire match will be used. E.g: | ||
|
|
||
| `^mytag.*` : any tags starting with `@mytag` will be used for grouping. | ||
|
|
||
| `^type:(ui|api)` : `@ui` or `@api` tags from regex pattern will be used for grouping. | ||
|
|
||
| Check this [config example](https://github.com/allure-framework/allure-csharp/blob/main/Tests.SpecRun/allureConfig.json) as a starting point. | ||
|
|
||
| #### Links | ||
| You can add links to your scenarios using tags. Tag and link patterns can be configured in `allureConfig.json` | ||
| ``` json | ||
| { | ||
| "allure": { | ||
| "links": [ | ||
| "https://myissuetracker.org/{issue}", | ||
| "https://mytestmanagementsystem.org?test={tms}" | ||
| ] | ||
| }, | ||
| "specflow": { | ||
| "links": { | ||
| "link": "^link:(.+)", | ||
| "issue": "^\\d+", | ||
| "tms": "^tms:(\\d+)" | ||
| } | ||
| } | ||
| } | ||
| ``` | ||
| The following scenario | ||
| ``` cucumber | ||
| @123456 @tms:42 @link:http://example.org | ||
| Scenario: I do like click on links in Allure report | ||
| ``` | ||
| will have three links in Allure report: | ||
| [123456](https://myissuetracker.org/123456), [42](https://mytestmanagementsystem.org?test=tms-42) and http://example.org. | ||
|
|
||
| In case there are links, which are generated during tests, then they can be added in code via AllureLifecycle: | ||
| ``` c# | ||
| AllureLifecycle.UpdateTestCase(testResult.uuid, tc => | ||
| { | ||
| tc.links.Add(new Link() | ||
| { | ||
| name = "Example link", | ||
| url = "http://example.com" | ||
| }); | ||
| }); | ||
| ``` | ||
| This will show for scenario link with Text: Example link; and url: "http://example.com". | ||
|
|
||
| #### Severity | ||
| You can add Severity for your scenarios using tags. It can be configured in `allureConfig.json` | ||
| ``` json | ||
| "labels": { | ||
| "severity": "^(normal|blocker|critical|minor|trivial)" | ||
| }, | ||
| ``` | ||
| The following scenario | ||
| ``` cucumber | ||
| @critical | ||
| Scenario: .... | ||
| ``` | ||
| will set current scenario severity in Allure report as Blocker | ||
|
|
||
| #### Tables conversion | ||
| Table arguments in SpecFlow steps can be converted either to step csv-attacments or step parameters in the Allure report. The conversion is configurable in `specflow:stepArguments` config section. | ||
| With `specflow:stepArguments:convertToParameters` set to `true` the following table arguments will be represented as parameters: | ||
| * one row tables | ||
| * two column tables with the headers matching both `specflow:stepArguments:paramNameRegex` and `specflow:stepArguments:paramValueRegex` regular expressions. | ||
|
|
||
| <table> | ||
| <th>SpecFlow</th> | ||
| <th>Allure</th> | ||
| <tr> | ||
| <td> | ||
|
|
||
|  | ||
|
|
||
| </td> | ||
| <td> | ||
|
|
||
|  | ||
|
|
||
| </td> | ||
| </tr> | ||
| </table> | ||
|
|
||
| #### Attachments | ||
| You can add attachments to an Allure report from your step bindings: | ||
| ```csharp | ||
| using Allure.Commons; | ||
| ... | ||
| AllureLifecycle.AddAttachment(path, "Attachment Title"); | ||
| // or | ||
| AllureLifecycle.AddAttachment("Attachment Title", "application/txt", "path"); | ||
| // where "application/txt" - type of your attachment | ||
| ``` |
Oops, something went wrong.