Skip to content

Commit

Permalink
Legacy pages moved + structure (#2337)
Browse files Browse the repository at this point in the history
  • Loading branch information
alimogithub committed Mar 2, 2021
1 parent 5cb8970 commit 992065c
Show file tree
Hide file tree
Showing 5 changed files with 23 additions and 18 deletions.
18 changes: 9 additions & 9 deletions docs/Execution/Parallel-Execution.md
Expand Up @@ -2,7 +2,7 @@

SpecFlow scenarios are often automated as integration or system level tests. The system under test (SUT) might have several external dependencies and a more complex internal architecture. The key design question when running the tests in parallel is how the parallel test executions can be isolated from each other.

## Test isolation levels
## Test Isolation Levels

Determining the ideal level of isolation for your automated tests is a tradeoff. The higher the isolation of the parallel tests the smaller the likelihood of conflicts on shared state and dependencies, but at the same time the higher the execution time and amount of resources needed to maintain the isolated environments.

Expand All @@ -13,7 +13,7 @@ Determining the ideal level of isolation for your automated tests is a tradeoff.
| Process | Test threads run in separate processes. | SpecFlow+Runner (Process isolation), VSTest per test assembly |
| Agent | Test threads run on multiple agents. | E.g. VSTest task |

## Parallel scheduling unit
## Parallel Scheduling Unit

Depending on the test isolation level and the used test runner tools you can consider different "units of scheduling" that can run in parallel with each other.
When using SpecFlow we can consider the parallel scheduling on the level of scenarios, features and test assemblies.
Expand Down Expand Up @@ -42,16 +42,16 @@ When using SpecFlow we can consider the parallel scheduling on the level of scen
* You must not use the static context properties of SpecFlow `ScenarioContext.Current`, `FeatureContext.Current` or `ScenarioStepContext.Current` (see further information below).
* You have to configure the test runner to execute the SpecFlow features in parallel with each other (see configuration details below).

### Execution behavior
### Execution Behavior

* `[BeforeTestRun]` and `[AfterTestRun]` hooks (events) are executed only once on the first thread that initializes the framework. Executing tests in the other threads is blocked until the hooks have been fully executed on the first thread.
* All scenarios in a feature must be executed on the **same thread**. See the configuration of the test runners below. This ensures that the `[BeforeFeature]` and `[AfterFeature]` hooks are executed only once for each feature and that the thread has a separate (and isolated) `FeatureContext`.
* Scenarios and their related hooks (Before/After scenario, scenario block, step) are isolated in the different threads during execution and do not block each other. Each thread has a separate (and isolated) `ScenarioContext`.
* The test trace listener (that outputs the scenario execution trace to the console by default) is invoked asynchronously from the multiple threads and the trace messages are queued and passed to the listener in serialized form. If the test trace listener implements `TechTalk.SpecFlow.Tracing.IThreadSafeTraceListener`, the messages are sent directly from the threads.

### NUnit configuration
### NUnit Configuration

By default,[NUnit does not run the tests in parallel](https://docs.nunit.org/articles/nunit/writing-tests/attributes/parallelizable.html).
By default, [NUnit does not run the tests in parallel](https://docs.nunit.org/articles/nunit/writing-tests/attributes/parallelizable.html).
Parallelisation must be configured by setting an assembly-level attribute in the SpecFlow project.

```c#
Expand All @@ -61,9 +61,9 @@ using NUnit.Framework;

***>Note:** SpecFlow does not support scenario level parallelization with NUnit (when scenarios from the same feature execute in parallel). If you configure a higher level NUnit parallelization than "Fixtures" your tests will fail with runtime errors.*

### MSTest configuration
### MSTest Configuration

By default,[MsTest does not run the tests in parallel](https://devblogs.microsoft.com/devops/mstest-v2-in-assembly-parallel-test-execution/).
By default, [MsTest does not run the tests in parallel](https://devblogs.microsoft.com/devops/mstest-v2-in-assembly-parallel-test-execution/).
Parallelisation must be configured by setting an assembly-level attribute in the SpecFlow project.

```c#
Expand All @@ -73,7 +73,7 @@ using Microsoft.VisualStudio.TestTools.UnitTesting;

Note: SpecFlow does not support scenario level parallelization with MsTest (when scenarios from the same feature execute in parallel). If you configure a higher level MsTest parallelization than "ClassLevel" your tests will fail with runtime errors.

### xUnit configuration
### xUnit Configuration

By default xUnit runs all SpecFlow features [in parallel](https://xunit.net/docs/running-tests-in-parallel) with each other. No additional configuration is necessary.

Expand All @@ -97,7 +97,7 @@ If there are no external dependencies or they can be cloned for parallel executi

* You have to use SpecFlow+ Runner with AppDomain or Process isolation.

### Execution behavior
### Execution Behavior

* `[BeforeTestRun]` and `[AfterTestRun]` hooks are executed for each individual test execution thread (AppDomain or process), so you can use them to initialize/reset shared memory.
* Each test thread manages its own enter/exit feature execution workflow. The `[BeforeFeature]` and `[AfterFeature]` hooks may be executed multiple times in different test threads if they run scenarios from the same feature file. The execution of these hooks do not block one another, but the Before/After feature hooks are called in pairs within a single thread (the `[BeforeFeature]` hook of the next scenario is only executed after the `[AfterFeature]` hook of the previous one). Each test thread has a separate (and isolated) `FeatureContext`.
2 changes: 1 addition & 1 deletion docs/Extend/Plugins.md
Expand Up @@ -7,7 +7,7 @@ SpecFlow supports the following types of plugins:

All types of plugins are created in a similar way.

This information only applies to SpecFlow 3. For legacy information on plugins for previous versions, see [Plugins (Legacy)](Plugins-(Legacy).md). With SpecFlow 3, we have changed how you configure which plugins are used. They are no longer configured in your `app.config` (or `specflow.json`).
This information only applies to SpecFlow 3. For legacy information on plugins for previous versions, see [Plugins (Legacy)](https://docs.specflow.org/projects/specflow/en/latest/legacy/plugins-(Legacy).html). With SpecFlow 3, we have changed how you configure which plugins are used. They are no longer configured in your `app.config` (or `specflow.json`).

## Runtime plugins

Expand Down
17 changes: 11 additions & 6 deletions docs/index.rst
Expand Up @@ -123,12 +123,10 @@ Follow our step by step guide to explore, experiment and learn the basics of Spe
Integrations/SpecFlow+Runner-Integration.md
Integrations/MsTest.md
Integrations/NUnit.md
Integrations/xUnit.md

Integrations/xUnit.md
Integrations/Azure-DevOps.md
Integrations/Teamcity-Integration.md
Integrations/Browserstack.md
Integrations/CodedUI.md
Integrations/Browserstack.md
Integrations/Autofac.md
Integrations/Windsor.md

Expand Down Expand Up @@ -160,8 +158,7 @@ Follow our step by step guide to explore, experiment and learn the basics of Spe
:hidden:

Extend/Value-Retriever.md
Extend/Plugins.md
Extend/Plugins-(Legacy).md
Extend/Plugins.md
Extend/Available-Plugins.md
Extend/Available-Containers-&-Registrations.md
Extend/Decorators.md
Expand Down Expand Up @@ -189,6 +186,14 @@ Follow our step by step guide to explore, experiment and learn the basics of Spe
Contribute/potentialProblems.md
Contribute/Coding-Style.md

.. toctree::
:maxdepth: 1
:caption: Legacy
:hidden:

legacy/Plugins-(Legacy).md
legacy/CodedUI.md

.. toctree::
:maxdepth: 1
:caption: Help
Expand Down
2 changes: 1 addition & 1 deletion docs/Integrations/CodedUI.md → docs/legacy/CodedUI.md
@@ -1,4 +1,4 @@
# Coded UI (Legacy)
# Coded UI

## Introduction

Expand Down
@@ -1,4 +1,4 @@
# Plugins (Legacy)
# Plugins

## Introduction

Expand Down

0 comments on commit 992065c

Please sign in to comment.