[MNT] extend CI to newer python versions, deprecate python versions 3.8, 3.9 after EOL, marking further failing tests as xfail

[JATAYU000]

The CI runs only on python versions 3.8 and 3.9 both of which have already reached end of life.

This PR updates the python versions, deprecating any logic that runs tests on python versions 3.8 and 3.9, or scikit-learn versions of that age.

Metadata

Reference Issue: #1544

Depends on #1584 fofr a fix, which should be merged first.

Details

• The test matrix is updated to python versions, 3.10-3.13.
• Further failing tests are skipped using mark.xfail with reason="failures_issue_1544" for all the remaining failed tests (after [BUG] Temporarily fix issue #1544 by marking the failed tests as expected fail. #1572) in issue: [MNT] Test failures in CI and local #1544

[fkiraly]

FYI @JATAYU000, I have merged #1571 into this for simplicity.

This also allows us to track down any further failing tests.

[codecov-commenter]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 52.70%. Comparing base (3380bbb) to head (b1e06ec).
⚠️ Report is 165 commits behind head on main.

❗ There is a different number of reports uploaded between BASE (3380bbb) and HEAD (b1e06ec). Click for more details.

HEAD has 1 upload less than BASE

Flag	BASE (3380bbb)	HEAD (b1e06ec)
	2	1

Additional details and impacted files

@@             Coverage Diff             @@
##             main    #1579       +/-   ##
===========================================
- Coverage   85.24%   52.70%   -32.54%     
===========================================
  Files          38       36        -2     
  Lines        5008     4320      -688     
===========================================
- Hits         4269     2277     -1992     
- Misses        739     2043     +1304     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[fkiraly]

There are still a few failing tests, and also fixture construction errors - could you look at those too, @JATAYU000?

[JATAYU000]

There are still a few failing tests, and also fixture construction errors - could you look at those too, @JATAYU000?

Yes will look into it

[JATAYU000]

@fkiraly #1584 needs to be merged first for fixing

AsertionError
assert False
 +  where False = exists()

Would like your inputs on this fix

[fkiraly]

ok, I will also check whether that resolves the failures here.

[fkiraly]

this fixed the issue, so you should remove the xfails from the fixed tests (see CI runs)

Not in this PR, but in #1584, and I will merge the changes to here as well for more extensive testing.

[JATAYU000]

@fkiraly I am a bit lost on whats happening here, How are the all 48 tests (marked as xfail) passing under sk-only:False ?, #1584 only fixes a few of those

[fkiraly]

A PR by @PGijsbers got merged which seems to fix these.
#1585

Unfortunately, this was unexpected since I thought he was on holiday until Jan - but it seems to be a fix, so we should adapt this PR to the new repository state.

I would suggest to revert the tests that are now fixed, could you quickly do that?

[JATAYU000]

I would suggest to revert the tests that are now fixed, could you quickly do that?

Sure, Thank you

[fkiraly]

@JATAYU000, you still need to deal with the sk-only: False tests

[JATAYU000]

@JATAYU000, you still need to deal with the sk-only: False tests

Yes those tests marks are removed, rest are passing till now the check is still in progress

[JATAYU000]

@JATAYU000, you still need to deal with the sk-only: False tests

@fkiraly Can you point me which tests under sk-only: False are not handled yet? because I have updated all of them as far as I know reference workflow fail.

[fkiraly]

Looks like we have the failing tests under control now.