[ENH] enable check_estimator and QuickTester.run_tests to work with skip marked pytest tests (issue #2419)
#6233
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
YelenaYY
requested review from
achieveordie,
benHeid,
fkiraly and
yarnabrina
as code owners
fkiraly
requested changes
Mar 29, 2024
There was a problem hiding this comment.
Interesting!
I think that's a suitable solution! Nice!
Questions/comments:
- the test that all is passed is failing. I think it should now check, all skipped or passed? Perhaps, for safety, also check that less than 10% are skipped or similar?
- is importing from
_pytestsafe? That is, isSkippeda public object? Or is it private, and it can change without prior notice?
…sts passed or skipped, and less than 10% are skipped. (PR: [ENH] ensure check_estimator and QuickTester.run_tests work with skipped pytest tests (issue sktime#2419) sktime#6233)
fkiraly
changed the title
check_estimator and QuickTester.run_tests work with skipped pytest tests (issue #2419)
fkiraly
changed the title
check_estimator and QuickTester.run_tests work with skipped pytest tests (issue #2419)check_estimator and QuickTester.run_tests work with skip marked pytest tests (issue #2419)
fkiraly
added
module:tests
fkiraly
changed the title
check_estimator and QuickTester.run_tests work with skip marked pytest tests (issue #2419)check_estimator and QuickTester.run_tests to work with skip marked pytest tests (issue #2419)
geetu040
pushed a commit
to geetu040/sktime
that referenced
this pull request
Jun 4, 2024
…th skip marked `pytest` tests (issue sktime#2419) (sktime#6233) #### Reference Issues/PRs: Fixes sktime#2419. #### What does this implement/fix? Explain your changes. Ensure check_estimator and QuickTester.run_tests work with skipped pytest tests. Added exception handling for NaiveForecaster skip exceptions in QuickTester.run_tests.
This was referenced Jun 13, 2024
fkiraly
added a commit
that referenced
this pull request
Jun 13, 2024
Reverts parts of #6233 which cause `check_estimator` based tests in third party libraries to be always skipped. This is due to `QuickTester.run_tests` propagating the skip conditoin up, affecting the entire test in the 3rd party package instead of just a single `sktime` test.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Reference Issues/PRs:
Fixes #2419.
What does this implement/fix? Explain your changes.
Ensure check_estimator and QuickTester.run_tests work with skipped pytest tests.
Added exception handling for NaiveForecaster skip exceptions in QuickTester.run_tests.
Does your contribution introduce a new dependency? If yes, which one?
No new dependency introduced
What should a reviewer concentrate their feedback on?
Nothing specific.
Following the right conventions.
Did you add any tests for the change?
No tests added
Any other comments?
No
PR checklist
For all contributions
How to: add yourself to the all-contributors file in the
sktimeroot directory (not theCONTRIBUTORS.md). Common badges:code- fixing a bug, or adding code logic.doc- writing or improving documentation or docstrings.bug- reporting or diagnosing a bug (get this pluscodeif you also fixed the bug in the PR).maintenance- CI, test framework, release.See here for full badge reference
maintainerstag - do this if you want to become the owner or maintainer of an estimator you added.See here for further details on the algorithm maintainer role.
For new estimators
docs/source/api_reference/taskname.rst, follow the pattern.Examplessection.python_dependenciestag and ensureddependency isolation, see the estimator dependencies guide.