Skip to content

Commit

Permalink
A few things:
Browse files Browse the repository at this point in the history
- XML doc and comment improvements
- Docs updated for v0.14
- Minor refactoring of XML config reading code
  • Loading branch information
sdcondon committed Feb 1, 2022
1 parent fc037c1 commit ce1e802
Show file tree
Hide file tree
Showing 8 changed files with 43 additions and 50 deletions.
24 changes: 10 additions & 14 deletions README.md
Expand Up @@ -108,41 +108,37 @@ With the VSTest adapter:

For more guidance, please see the [User Guide](docs/user-guide/README.md).

## Plans
## Roadmap

Proper issue tracking would be overkill at this point, so just a bullet list to organise my thoughts:

General ongoing work:
- Take some cues from other frameworks - what am I missing regarding debugging, parallelisation, test attachments, instrumentation, filtering etc?

Specific work, highest priority first:
- *(Done but not yet published - v0.14):* Implementation of test run configuration:
- With the VSTest adapter, test run configuration can be specified with .runsettings
- Test settings that affect individual tests can be overridden via the `UsingConfiguration` builder method.
- Test settings:
- `Parallelise`: Whether or not test execution should be parallelised. Defaults to true. Partitioning not yet suppported.
- `FailedArrangementOutcomeIsSkipped`: A value determining what the test result should be if one of the "Given" clauses throws.
- `ResultNamingStrategy`: Strategy to use to name individual test results.
- *(Jan - v0.14)* More setting stuff:
- For specification of strategy for duration records (which currently makes a "sensible" decision which may not be appropriate in all situations).
- *(Feb - v1.0)* V1 diligence & release
- Get some performance benchmarks in place
- Review abstractions - for flexibility / stability
- Any required "doing it properly" stuff in the test adapter.
- Resolve most or all TODOs
- Split into separate repos
- Separate usage docs
- README in packages
- *(May / Jun - v1.1)* Possible post-v1 additions (after a break to work on other projects):
- Allow for control over parallel partitioning - likely to be trait based (e.g. allow specification of a trait name - all tests with same value won't run in parallel). Also want to allow for by class name and namespace - whether thats treated as a special case or if we hook this into trait system is TBD.
- A little more configurability:
- For specification of strategy for duration records (which currently makes a "sensible" decision which may not be appropriate in all situations).
- Allow for control over parallel partitioning - likely to be trait based (e.g. allow specification of a trait name - all tests with same value won't run in parallel). Also want to allow for by class name and namespace - whether thats treated as a special case or if we hook this into trait system is TBD.
- QoL: Support custom test case labelling - `ToString()` of the prereqs only helpful when this yields something other than the type name. Perhaps `WithResultLabels`? Perhaps somehow support IFormatProviders for test cases (thus making it easy to specify with test settings)? Needs careful thought..
- Basic attachment & output support.
This is likely to require injecting some kind of test context object.
I really want to double-down on the "conventionless"/static nature of FlUnit - i.e. no convention-based ctor parameters.
I really want to double-down on the convention-less/static nature of FlUnit - i.e. no convention-based ctor parameters, all discoverable via IDE method listings etc.
Plan A right now is to introduce some kind of ITestContext as a pre-requisite if needed.
That is, `TestThat.GivenTestContext().And()...When((cxt, ) => ...)`.
This particular approach doesn't allow for test context to be placed inside an anonymous prereq object, though.
Which is perhaps a good thing? But is a mandate for users, rather than a choice.
So, instead (or as well) could allow for cxt to be specified as a parameter of Given delegates (`Given(cxt => ...)`). Hmm, maybe. Still mulling this one over.
More concerning is that it doesn't allow context to be used during pre-req creation.
So, instead (or as well) could allow for cxt to be specified as a parameter of Given delegates (`Given(cxt => ...)`).
Hmm, maybe - this is more complex?
Still mulling this one over.
- *(At some point - v1.2 or later)* Other features:
- Support for async tests?

Expand Down
2 changes: 0 additions & 2 deletions docs/user-guide/test-run-configuration.md
@@ -1,7 +1,5 @@
# Test Run Configuration

*(NB: not yet published - implemented in v0.14+)*

If VSTest is being used, FlUnit test configuration can be provided in a "FlUnit" element in the .runsettings file. See the [annotated example](../../src/Example.TestProject/.runsettings) in the example test project.

Configuration that applies to individual tests can be overridden by individual tests through the use of the `UsingConfiguration` builder method. Configuration overrides can be specified at any point up until the `When` clause is specified.
3 changes: 3 additions & 0 deletions src/FlUnit.Abstractions/ITraitProvider.cs
Expand Up @@ -9,6 +9,9 @@ public interface ITraitProvider
/// <summary>
/// Gets the provided trait.
/// </summary>
/// <remarks>
/// TODO-EXTENSIBILITY: Decide if this should be GetTrait(PropertyInfo) pre-V1.
/// </remarks>
Trait Trait { get; }
}
}
8 changes: 4 additions & 4 deletions src/FlUnit.Abstractions/Test.cs
Expand Up @@ -20,10 +20,10 @@ public abstract class Test
/// Gets a value indicating whether this test has any configuration overrides to apply.
/// </summary>
/// <remarks>
/// NB: While we *could* support configuration overrides with a single method - it'd require either configuration implementation
/// in the FlUnit lib (which I'd rather avoid - configuration is still an execution concern) so that it can be cloned,
/// or would require configuration as a struct (which I definitely don't want to do). So we have a separate prop instead
/// so that the execution logic can determine whether it needs to clone config or not..
/// NB: While we *could* support configuration overrides with a single method - it'd require either a configuration implementation
/// in the FlUnit library (which isn't ideal - configuration is an execution concern) so that it can be cloned, or would require
/// configuration as a struct (which definitely isn't idea). So we have a separate property instead, so that the execution logic
/// can determine whether it needs to clone the configuration or not..
/// </remarks>
public abstract bool HasConfigurationOverrides { get; }

Expand Down
2 changes: 1 addition & 1 deletion src/FlUnit.Adapters.VSTest/ITestContainer.cs
Expand Up @@ -26,7 +26,7 @@ internal interface ITestContainer
void RecordStart();

/// <remarks>
/// TODO: I don't like this method. Its far too close to the VSTest platform (inc all of the assumptions made re what test
/// TODO-MAINTAINABILITY: I don't like this method. Its far too close to the VSTest platform (inc all of the assumptions made re what test
/// result pluralilty means, and how display names are interpreted). Better to talk in FlUnit terms and expect the adapter do
/// more (e.g. RecordCase, RecordAssertion, or similar)..
/// </remarks>
Expand Down
22 changes: 10 additions & 12 deletions src/FlUnit.Adapters.VSTest/TestConfiguration.cs
Expand Up @@ -25,20 +25,18 @@ public static TestConfiguration ReadFromXml(XmlReader reader)
{
TestConfiguration configuration = new TestConfiguration();

if (!reader.TryReadToFirstChildElement())
if (reader.TryReadToFirstChildElement())
{
return configuration;
}

while (reader.NodeType != XmlNodeType.EndElement)
{
if (reader.IsAtElementWithName(nameof(FailedArrangementOutcomeIsSkipped)))
{
reader.TryReadBoolean(b => configuration.FailedArrangementOutcomeIsSkipped = b);
}
else
while (reader.NodeType != XmlNodeType.EndElement)
{
reader.Skip();
if (reader.IsAtElementWithName(nameof(FailedArrangementOutcomeIsSkipped)))
{
reader.TryReadBoolean(b => configuration.FailedArrangementOutcomeIsSkipped = b);
}
else
{
reader.Skip();
}
}
}

Expand Down
2 changes: 1 addition & 1 deletion src/FlUnit.Adapters.VSTest/TestDiscovery.cs
Expand Up @@ -10,7 +10,7 @@ namespace FlUnit.Adapters
internal static class TestDiscovery
{
/// <summary>
/// Finds all of the properties that represent tests in a given loaded assembly - along with the trait providers that should be applied to each.
/// Finds all of the properties that represent tests in a given loaded assembly - along with the traits that are associated with each.
/// </summary>
/// <param name="assembly">The assembly to look in.</param>
/// <param name="runConfiguration">Unused for the moment. Sue me.</param>
Expand Down
30 changes: 14 additions & 16 deletions src/FlUnit.Adapters.VSTest/TestRunConfiguration.cs
Expand Up @@ -63,24 +63,22 @@ public static TestRunConfiguration ReadFromXml(XmlReader reader)
{
TestRunConfiguration configuration = new TestRunConfiguration();

if (!reader.TryReadToFirstChildElement())
if (reader.TryReadToFirstChildElement())
{
return configuration;
}

while (reader.NodeType != XmlNodeType.EndElement)
{
if (reader.IsAtElementWithName(nameof(Parallelise)))
{
reader.TryReadBoolean(b => configuration.Parallelise = b);
}
else if (reader.IsAtElementWithName(nameof(TestConfiguration)))
{
configuration.TestConfiguration = TestConfiguration.ReadFromXml(reader);
}
else
while (reader.NodeType != XmlNodeType.EndElement)
{
reader.Skip();
if (reader.IsAtElementWithName(nameof(Parallelise)))
{
reader.TryReadBoolean(b => configuration.Parallelise = b);
}
else if (reader.IsAtElementWithName(nameof(TestConfiguration)))
{
configuration.TestConfiguration = TestConfiguration.ReadFromXml(reader);
}
else
{
reader.Skip();
}
}
}

Expand Down

0 comments on commit ce1e802

Please sign in to comment.