Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

V4: Allow for arbitrary test group nesting #554

Closed
dlwyatt opened this issue Jun 5, 2016 · 3 comments
Closed

V4: Allow for arbitrary test group nesting #554

dlwyatt opened this issue Jun 5, 2016 · 3 comments
Labels
Milestone

Comments

@dlwyatt
Copy link
Member

dlwyatt commented Jun 5, 2016

Describe / Context become basically aliases for the same command, ideally. Allow any combination of Describe / Context that a test author wants to provide.

This will involve breaking changes to both NUnit XML output and PassThru. (Instead of an array of TestResult objects, we'll likely wind up with a hierarchy of TestGroup / TestResult objects very similar to the structure of NUnit XML.)

The test scripts themselves should also be promoted to first-class citizens, if possible. That way Describe / Context are optional (it'll be valid to just have test cases in a file, if desired), and more importantly we can do BeforeAll / AfterAll blocks that apply to an entire file. (Not sure how easy this will be for Gherkin scenarios; needs investigation.)

Note: Big concern here is functionality of -Tag and -ExcludeTag. Right now they only work on the outermost test groups (Describe). In the new setup, tags will be applicable at any level, because Describes are nestable. Right now, Pester's logic would be pretty stupid about Tag / ExcludeTag. What we'd need is to analyze the structure of the test script ahead of time and come up with a plan for which test groups to run. There are some good ideas on this topic with @redoz on Twitter: https://twitter.com/redoz/status/737728099350106113

dlwyatt added a commit to dlwyatt/Pester that referenced this issue Jun 28, 2016
Working on pester#554 .  Still a ways to go, but at least Mocking and Setup/Teardown work with the new plumbing.

TestResult property still exists on $pester for backward compatibility (which is also why the XML code still works as-is for the moment), but the Describe / Context names are only provided on a best-effort basis.  Anyone using the new ability to nest Describe / Context any way they like should be using the new TestActions hierarchy instead.
@dlwyatt dlwyatt mentioned this issue Jul 1, 2016
@dlwyatt
Copy link
Member Author

dlwyatt commented Jul 1, 2016

This has been partially implemented. Still need to figure out how to handle tags in nested test groups, update the -PassThru object, and write lots of new documentation / tests.

@it-praktyk
Copy link
Contributor

@dlwyatt, can you provide an update? Yeah, I didn't read the code for this.

@nohwnd nohwnd added this to the New runtime milestone Dec 16, 2018
@nohwnd
Copy link
Member

nohwnd commented Mar 24, 2019

Test groups can be nested now, and the result object is hierarchical. Making tests run outside of Describe, and still be able to do all the "infrastructure" setup, and handle teardowns and setups is difficult. Similarly placing After* blocks outiside of describe is problematic in interactive mode, because you don't know which describe is the last on and you cannot rely on AfterAll to be placed at the end.

Right now v5 adds one more block above the Describe to handle multiple describes in the same file, and does not support interactive mode. If the interactive mode is implemented as running the saved file from the code then it would all work well.

I am not sure what about plain unsaved code, and how to run it without being in a file. What is already possible is doing Invoke-Pester -ScriptBlock, so probably something like that.

Overall allowing It blocks ouside of Describe is a lot of work that is not worth it.

Tracking the interactive mode here #1268

The filtering on block or test level works

@nohwnd nohwnd closed this as completed Mar 24, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

3 participants