Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Mark tested which are pended-out or not focused as skipped. #1276

Merged
merged 3 commits into from
Mar 24, 2024

Conversation

younata
Copy link
Member

@younata younata commented Mar 23, 2024

This changes our pending behavior. Now, we will make sure that all tests that Quick skips using pending, xdescribe, xit, xitBehavesLike are marked as skipped in the Xcode test navigator.
Similarly, if you have focused a test using fcontext, fdescribe, fit, fitBehavesLike, then Quick will mark all the other tests as skipped in the Xcode test navigator.

Sadly, this does come with a performance regression. Perhaps in the future, Xcode will expose an API for us to mark tests as skipped in bulk. I filed a request for such an API as FB13696046.

Fixes #332
Fixes #770
Fixes #1093

  • Is this a new feature (Requires minor version bump)?

Basically, if a test is pended out (using pending, xcontext, xdescribe, xit, xitBehavesLike), then the test will be marked as if you had immediately thrown an XCTSkip error.
Similarly, if there is a test that is focused (using fcontext, fdescribe, fit, fitBehavesLike), then all non-focused tests will be marked as if you had immediately thrown an XCTSkip error.
Copy link

1 Warning
⚠️ Big PR

Generated by 🚫 Danger

@younata
Copy link
Member Author

younata commented Mar 23, 2024

Hm, trying this in a larger codebase, specs in classes that are run before the class with the focused spec will also be ran. Per FocusedTests, this is known behavior in the Swift Package version, and has existed for quite some time. Fixing this also doesn't appear to be something we can really do (at least, not at the moment?). So... I'll leave this bug in for now. 😞

Additionally, if there are focused specs in AsyncSpec, then non-focused specs in QuickSpec will still be run. This has likely been a bug since Quick 7 split them apart.

@younata younata enabled auto-merge March 24, 2024 16:54
@younata younata disabled auto-merge March 24, 2024 16:54
@younata younata merged commit ded7c7f into main Mar 24, 2024
14 checks passed
@younata younata deleted the mark_filtered_tests_as_skipped branch March 24, 2024 16:55
cgrindel-self-hosted-renovate bot added a commit to cgrindel/rules_swift_package_manager that referenced this pull request Mar 25, 2024
This PR contains the following updates:

| Package | Update | Change |
|---|---|---|
| [Quick/Quick](https://togithub.com/Quick/Quick) | minor | `from:
"7.4.1"` -> `from: "7.5.0"` |

---

### Release Notes

<details>
<summary>Quick/Quick (Quick/Quick)</summary>

### [`v7.5.0`](https://togithub.com/Quick/Quick/releases/tag/v7.5.0): -
Pending tests are now marked as Skipped in Xcode

[Compare
Source](https://togithub.com/Quick/Quick/compare/v7.4.1...v7.5.0)

#### Highlights

- Any tests which are pended-out (using `pending`, `xdescribe`,
`xcontext`, `xitBehavesLike`, or `xit`), OR tests which are skipped
because they are not focused (that is, the **other** tests when you use
`fdescribe`, `fcontext`, `fitBehavesLike` or `fit`) are now marked as
skipped in Xcode.

#### Automated Changelog

- Bump softprops/action-gh-release from 1 to 2 by
[@&#8203;dependabot](https://togithub.com/dependabot) in
[Quick/Quick#1274
- Mark tested which are pended-out or not focused as skipped. by
[@&#8203;younata](https://togithub.com/younata) in
[Quick/Quick#1276

**Full Changelog**:
Quick/Quick@v7.4.1...v7.5.0

</details>

---

### Configuration

📅 **Schedule**: Branch creation - At any time (no schedule defined),
Automerge - At any time (no schedule defined).

🚦 **Automerge**: Enabled.

♻ **Rebasing**: Whenever PR is behind base branch, or you tick the
rebase/retry checkbox.

👻 **Immortal**: This PR will be recreated if closed unmerged. Get
[config help](https://togithub.com/renovatebot/renovate/discussions) if
that's undesired.

---

- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check
this box

---

This PR has been generated by [Renovate
Bot](https://togithub.com/renovatebot/renovate).

<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNi4xMDkuNCIsInVwZGF0ZWRJblZlciI6IjM2LjEwOS40IiwidGFyZ2V0QnJhbmNoIjoibWFpbiJ9-->

Co-authored-by: Self-hosted Renovate Bot <361546+cgrindel-self-hosted-renovate[bot]@users.noreply.github.enterprise.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
1 participant