[MAINT] test install and run windows tests with github actions #3780

Remi-Gau · 2023-06-27T07:13:46Z

Changes proposed in this pull request:

test install in gh actions
run tests for windows in gh actions

github-actions · 2023-06-27T07:14:04Z

👋 @Remi-Gau Thanks for creating a PR!

Until this PR is ready for review, you can include the [WIP] tag in its title, or leave it as a github draft.

Please make sure it is compliant with our contributing guidelines. In particular, be sure it checks the boxes listed below.

PR has an interpretable title.
PR links to Github issue with mention Closes #XXXX (see our documentation on PR structure)
Code is PEP8-compliant (see our documentation on coding style)
Changelog or what's new entry in doc/changes/latest.rst (see our documentation on PR structure)

For new features:

There is at least one unit test per new function / class (see our documentation on testing)
The new feature is demoed in at least one relevant example.

For bug fixes:

There is at least one test that would fail under the original bug conditions.

We will review it as quick as possible, feel free to ping us with questions if needed.

codecov · 2023-06-27T07:29:36Z

Codecov Report

Merging #3780 (03e3c70) into main (c90bf61) will decrease coverage by 0.14%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##             main    #3780      +/-   ##
==========================================
- Coverage   91.65%   91.52%   -0.14%     
==========================================
  Files         139      133       -6     
  Lines       16562    15557    -1005     
  Branches     3229     3229              
==========================================
- Hits        15180    14238     -942     
+ Misses       1379      772     -607     
- Partials        3      547     +544

Flag	Coverage Δ
macos-latest_3.10	`91.44% <ø> (?)`
macos-latest_3.11	`91.44% <ø> (?)`
macos-latest_3.8	`91.40% <ø> (?)`
macos-latest_3.9	`91.40% <ø> (?)`
ubuntu-latest_3.10	`91.44% <ø> (?)`
ubuntu-latest_3.8	`91.40% <ø> (?)`
windows-latest_3.10	`91.38% <ø> (?)`
windows-latest_3.11	`91.38% <ø> (?)`
windows-latest_3.9	`91.35% <ø> (?)`

Flags with carried forward coverage won't be shown. Click here to find out more.

see 81 files with indirect coverage changes

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

Remi-Gau · 2023-06-27T08:17:14Z

OK the errors on windows seem to happen in a non deterministic manner but seem to always be due to

There appear to be 2 leaked folder objects to clean up at shutdown

Windows fatal exception: access violation

Thread 0x00000cc8 (most recent call first):
  File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\kaleido\scopes\base.py", line 149 in _collect_standard_error
  File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\pluggy\_callers.py", line 80 in _multicall
  File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\pluggy\_manager.py", line 112 in _hookexec
  File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\pluggy\_hooks.py", line 433 in __call__
  File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\_pytest\runner.py", line 262 in 
  File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\_pytest\runner.py", line 341 in from_call
  File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\_pytest\runner.py", line 261 in call_runtest_hook
  File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\_pytest\runner.py", line 222 in call_and_report
  File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\_pytest\runner.py", line 133 in runtestprotocol
  File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\_pytest\runner.py", line 114 in pytest_runtest_protocol
  File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\pluggy\_callers.py", line 80 in _multicall
  File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\pluggy\_manager.py", line 112 in _hookexec
  File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\pluggy\_hooks.py", line 433 in __call__
  File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\_pytest\main.py", line 349 in pytest_runtestloop
  File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\pluggy\_callers.py", line 80 in _multicall
  File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\pluggy\_manager.py", line 112 in _hookexec
  File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\pluggy\_hooks.py", line 433 in __call__
  File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\_pytest\main.py", line 324 in _main
  File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\_pytest\main.py", line 270 in wrap_session
  File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\_pytest\main.py", line 317 in pytest_cmdline_main
  File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\pluggy\_callers.py", line 80 in _multicall
  File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\pluggy\_manager.py", line 112 in _hookexec
  File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\pluggy\_hooks.py", line 433 in __call__
  File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\_pytest\config\__init__.py", line 166 in main
  File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\_pytest\config\__init__.py", line 189 in console_main
  File "C:\hostedtoolcache\windows\Python\3.9.13\x64\Scripts\pytest.exe\__main__.py", line 7 in 
  File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\runpy.py", line 87 in _run_code
  File "C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\runpy.py", line 197 in _run_module_as_main
C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\joblib\externals\loky\backend\resource_tracker.py:310: UserWarning: resource_tracker: There appear to be 2 leaked folder objects to clean up at shutdown
  warnings.warn(
C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\joblib\externals\loky\backend\resource_tracker.py:326: UserWarning: resource_tracker: C:\Users\VSSADM~1\AppData\Local\Temp\joblib_memmapping_folder_2448_22a07b0389c94d50b9b0dc10d9b0e882_3c7869e8849444b3aa0abd1a61e63688: FileNotFoundError(2, 'The system cannot find the path specified')
  warnings.warn(f'resource_tracker: {name}: {e!r}')
C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\joblib\externals\loky\backend\resource_tracker.py:326: UserWarning: resource_tracker: C:\Users\VSSADM~1\AppData\Local\Temp\joblib_memmapping_folder_2448_22a07b0389c94d50b9b0dc10d9b0e882_b56149fc0efd42378409b1534b136f5c: FileNotFoundError(2, 'The system cannot find the path specified')
  warnings.warn(f'resource_tracker: {name}: {e!r}')

Remi-Gau · 2023-06-27T08:28:48Z

Seems that the codecov reports are not uploaded when running tests with GH on windows

https://github.com/nilearn/nilearn/actions/runs/5386895758/jobs/9777500494?pr=3780#step:9:37

I don't think this should block this PR, especially that we do not use flags on the code-cov action: we possibly may want to do something like this to have coverage reports for different OS / python

- uses: codecov/codecov-action@v3
  with: 
    flags: ${{ matrix.os }}_${{ matrix.python-version }}

Remi-Gau · 2023-06-27T08:30:19Z

re running failed jobs only

ymzayek · 2023-06-27T08:52:06Z

Seems that the codecov reports are not uploaded when running tests with GH on windows

We can try setting a token (even though it's supposed to be not required for public repos)

Remi-Gau · 2023-06-27T09:24:50Z

re running failed jobs only

OK it seems that re running failed jobs can make them pass, so there is something flaky but I am not yet sure it comes from us.

Remi-Gau · 2023-06-27T11:41:15Z

OK things seem to be green but I am not holding my breath for it to always pass, but at least we are only on GH now.

ymzayek

LGTM thx! Before merging can you update .github/workflows/README.md and add the new installing workflow?

Remi-Gau · 2023-06-27T13:35:17Z

LGTM thx! Before merging can you update .github/workflows/README.md and add the new installing workflow?

good catch: I had totally forgotten about the existence of this README

…rn#3780) * try testing install with gh * try testing on windows * cancel previous test install workflows * rm azure pipeline tests * rm azure badge * update doc

* f stringify * f strings in examples * f string fix * fix f strings * Update pyproject.toml * [MAINT] test install and run windows tests with github actions (#3780) * try testing install with gh * try testing on windows * cancel previous test install workflows * rm azure pipeline tests * rm azure badge * update doc * Update examples/04_glm_first_level/plot_localizer_surface_analysis.py Co-authored-by: bthirion <bertrand.thirion@inria.fr> --------- Co-authored-by: bthirion <bertrand.thirion@inria.fr>

Remi-Gau added 2 commits June 27, 2023 09:10

try testing install with gh

2f7727b

try testing on windows

93f901f

cancel previous test install workflows

5290fe7

ymzayek mentioned this pull request Jun 27, 2023

[REF] Fix more f strings #3776

Merged

Remi-Gau mentioned this pull request Jun 27, 2023

[MAINT] Update config black and pre-commit #3777

Merged

rm azure pipeline tests

d06ae3b

Remi-Gau marked this pull request as ready for review June 27, 2023 09:23

ymzayek approved these changes Jun 27, 2023

View reviewed changes

Remi-Gau added 2 commits June 27, 2023 21:07

rm azure badge

0a35ca0

update doc

03e3c70

Remi-Gau merged commit 7d91e88 into nilearn:main Jun 28, 2023
28 checks passed

Remi-Gau deleted the ci_gh branch June 28, 2023 04:43

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[MAINT] test install and run windows tests with github actions #3780

[MAINT] test install and run windows tests with github actions #3780

Remi-Gau commented Jun 27, 2023

github-actions bot commented Jun 27, 2023 •

edited by Remi-Gau

codecov bot commented Jun 27, 2023 •

edited

Remi-Gau commented Jun 27, 2023

Remi-Gau commented Jun 27, 2023

Remi-Gau commented Jun 27, 2023

ymzayek commented Jun 27, 2023

Remi-Gau commented Jun 27, 2023

Remi-Gau commented Jun 27, 2023

ymzayek left a comment

Remi-Gau commented Jun 27, 2023

[MAINT] test install and run windows tests with github actions #3780

[MAINT] test install and run windows tests with github actions #3780

Conversation

Remi-Gau commented Jun 27, 2023

github-actions bot commented Jun 27, 2023 • edited by Remi-Gau

codecov bot commented Jun 27, 2023 • edited

Codecov Report

Remi-Gau commented Jun 27, 2023

Remi-Gau commented Jun 27, 2023

Remi-Gau commented Jun 27, 2023

ymzayek commented Jun 27, 2023

Remi-Gau commented Jun 27, 2023

Remi-Gau commented Jun 27, 2023

ymzayek left a comment

Choose a reason for hiding this comment

Remi-Gau commented Jun 27, 2023

github-actions bot commented Jun 27, 2023 •

edited by Remi-Gau

codecov bot commented Jun 27, 2023 •

edited