Convert `.travis.yml` to GitHub Actions workflow #7600

jwodder · 2024-05-20T14:54:54Z

Closes #7574.

codecov · 2024-05-20T15:45:49Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 91.65%. Comparing base (06d64fb) to head (a2f9cf5).

❗ Current head a2f9cf5 differs from pull request most recent head fb45559

Please upload reports for the commit fb45559 to get more accurate results.

Additional details and impacted files

@@            Coverage Diff             @@
##            maint    #7600      +/-   ##
==========================================
+ Coverage   91.28%   91.65%   +0.37%     
==========================================
  Files         325      325              
  Lines       43459    43462       +3     
  Branches     5781     5781              
==========================================
+ Hits        39672    39837     +165     
+ Misses       3772     3610     -162     
  Partials       15       15

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

.github/workflows/test.yml

yarikoptic · 2024-05-21T17:35:55Z

on NFS:

=========================== short test summary info ============================
FAILED ../datalad/tests/test_archives.py::test_decompress_file[strip] - AssertionError: assert False
 +  where False = <function exists at 0x7f91c75344d0>('/tmp/nfsmount/datalad-wfvyq6ly/simple-extracted/3.txt')
 +    where <function exists at 0x7f91c75344d0> = op.exists
 +    and   '/tmp/nfsmount/datalad-wfvyq6ly/simple-extracted/3.txt' = <function join at 0x7f91ceba4d40>('/tmp/nfsmount/datalad-wfvyq6ly/simple-extracted', '3.txt')
 +      where <function join at 0x7f91ceba4d40> = op.join
FAILED ../datalad/tests/test_archives.py::test_ExtractedArchive - assert [' |;&%b5{}\'...Йקم๗あ .datc '] == [' |;&%b5{}\'....dbtc /3.txt']
  Right contains one more item: ' |;&%b5{}\'"<>ΔЙקم๗あ .dbtc /3.txt'
  Full diff:
    [
     ' |;&%b5{}\'"<>ΔЙקم๗あ .dbtc / |;&%b5{}\'"<>ΔЙקم๗あ .datc ',
  -  ' |;&%b5{}\'"<>ΔЙקم๗あ .dbtc /3.txt',
    ]
FAILED ../datalad/tests/test_tests_utils_pytest.py::test_with_tempfile_mkdir - assert False
FAILED ../datalad/support/tests/test_annexrepo.py::test_commit_annex_commit_changed[True]
FAILED ../datalad/support/tests/test_annexrepo.py::test_commit_annex_commit_changed[False]
= 5 failed, 441 passed, 40 skipped, 1 xpassed, 6 warnings in 986.59s (0:16:26) =

need to review those tests -- we might have been checking for NFS already but in a "travis specific" way... adjusting now and will push.

evil Russian encoding -- caused Python segfault, restarting now

[gw1] [ 24%] PASSED ../datalad/interface/tests/test_utils.py::test_incorrect_msg_interpolation 
../datalad/interface/tests/test_utils.py::test_custom_result_renderer 
[gw1] [ 24%] PASSED ../datalad/interface/tests/test_utils.py::test_custom_result_renderer 
../datalad/interface/tests/test_utils.py::test_custom_result_summary_renderer 
[gw1] [ 24%] PASSED ../datalad/interface/tests/test_utils.py::test_custom_result_summary_renderer 
Fatal Python error: Segmentation fault

Thread 0x00007f8ffa8be640 (most recent call first):
  File "/tmp/dl-miniconda-hecu530u/lib/python3.8/threading.py", line 306 in wait
  File "/tmp/dl-miniconda-hecu530u/lib/python3.8/threading.py", line 558 in wait
  File "/tmp/dl-miniconda-hecu530u/lib/python3.8/site-packages/tqdm/_monitor.py", line 60 in run
  File "/tmp/dl-miniconda-hecu530u/lib/python3.8/threading.py", line 932 in _bootstrap_inner
  File "/tmp/dl-miniconda-hecu530u/lib/python3.8/threading.py", line 890 in _bootstrap

Thread 0x00007f9012fff640 (most recent call first):
  File "/tmp/dl-miniconda-hecu530u/lib/python3.8/site-packages/execnet/gateway_base.py", line 534 in read
  File "/tmp/dl-miniconda-hecu530u/lib/python3.8/site-packages/execnet/gateway_base.py", line 567 in from_io
  File "/tmp/dl-miniconda-hecu530u/lib/python3.8/site-packages/execnet/gateway_base.py", line 1160 in _thread_receiver
  File "/tmp/dl-miniconda-hecu530u/lib/python3.8/site-packages/execnet/gateway_base.py", line 341 in run
  File "/tmp/dl-miniconda-hecu530u/lib/python3.8/site-packages/execnet/gateway_base.py", line 411 in _perform_spawn

Current thread 0x00007f9013db1740 (most recent call first):
  File "<frozen importlib._bootstrap>", line 219 in _call_with_frames_removed
  File "<frozen importlib._bootstrap_external>", line 1101 in create_module
  File "<frozen importlib._bootstrap>", line 556 in module_from_spec
  File "<frozen importlib._bootstrap>", line 657 in _load_unlocked
  File "<frozen importlib._bootstrap>", line 975 in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 991 in _find_and_load
  File "/tmp/dl-miniconda-hecu530u/lib/python3.8/pdb.py", line 157 in __init__
  File "/tmp/dl-miniconda-hecu530u/lib/python3.8/doctest.py", line 363 in __init__
  File "/tmp/dl-miniconda-hecu530u/lib/python3.8/doctest.py", line 1469 in run
  File "/tmp/dl-miniconda-hecu530u/lib/python3.8/doctest.py", line 1844 in run
  File "/tmp/dl-miniconda-hecu530u/lib/python3.8/site-packages/_pytest/doctest.py", line 304 in runtest
  File "/tmp/dl-miniconda-hecu530u/lib/python3.8/site-packages/_pytest/runner.py", line 173 in pytest_runtest_call
  File "/tmp/dl-miniconda-hecu530u/lib/python3.8/site-packages/pluggy/_callers.py", line 103 in _multicall
  File "/tmp/dl-miniconda-hecu530u/lib/python3.8/site-packages/pluggy/_manager.py", line 120 in _hookexec
  File "/tmp/dl-miniconda-hecu530u/lib/python3.8/site-packages/pluggy/_hooks.py", line 513 in __call__
  File "/tmp/dl-miniconda-hecu530u/lib/python3.8/site-packages/_pytest/runner.py", line 241 in <lambda>
  File "/tmp/dl-miniconda-hecu530u/lib/python3.8/site-packages/_pytest/runner.py", line 341 in from_call
  File "/tmp/dl-miniconda-hecu530u/lib/python3.8/site-packages/_pytest/runner.py", line 240 in call_and_report
  File "/tmp/dl-miniconda-hecu530u/lib/python3.8/site-packages/_pytest/runner.py", line 135 in runtestprotocol
  File "/tmp/dl-miniconda-hecu530u/lib/python3.8/site-packages/_pytest/runner.py", line 116 in pytest_runtest_protocol
  File "/tmp/dl-miniconda-hecu530u/lib/python3.8/site-packages/pluggy/_callers.py", line 103 in _multicall
  File "/tmp/dl-miniconda-hecu530u/lib/python3.8/site-packages/pluggy/_manager.py", line 120 in _hookexec
  File "/tmp/dl-miniconda-hecu530u/lib/python3.8/site-packages/pluggy/_hooks.py", line 513 in __call__
  File "/tmp/dl-miniconda-hecu530u/lib/python3.8/site-packages/xdist/remote.py", line 195 in run_one_test
  File "/tmp/dl-miniconda-hecu530u/lib/python3.8/site-packages/xdist/remote.py", line 174 in pytest_runtestloop
  File "/tmp/dl-miniconda-hecu530u/lib/python3.8/site-packages/pluggy/_callers.py", line 103 in _multicall
  File "/tmp/dl-miniconda-hecu530u/lib/python3.8/site-packages/pluggy/_manager.py", line 120 in _hookexec
  File "/tmp/dl-miniconda-hecu530u/lib/python3.8/site-packages/pluggy/_hooks.py", line 513 in __call__
  File "/tmp/dl-miniconda-hecu530u/lib/python3.8/site-packages/_pytest/main.py", line 339 in _main
  File "/tmp/dl-miniconda-hecu530u/lib/python3.8/site-packages/_pytest/main.py", line 285 in wrap_session
  File "/tmp/dl-miniconda-hecu530u/lib/python3.8/site-packages/_pytest/main.py", line 332 in pytest_cmdline_main
  File "/tmp/dl-miniconda-hecu530u/lib/python3.8/site-packages/pluggy/_callers.py", line 103 in _multicall
  File "/tmp/dl-miniconda-hecu530u/lib/python3.8/site-packages/pluggy/_manager.py", line 120 in _hookexec
  File "/tmp/dl-miniconda-hecu530u/lib/python3.8/site-packages/pluggy/_hooks.py", line 513 in __call__
  File "/tmp/dl-miniconda-hecu530u/lib/python3.8/site-packages/xdist/remote.py", line 393 in <module>
  File "/tmp/dl-miniconda-hecu530u/lib/python3.8/site-packages/execnet/gateway_base.py", line 1291 in executetask
  File "/tmp/dl-miniconda-hecu530u/lib/python3.8/site-packages/execnet/gateway_base.py", line 341 in run
  File "/tmp/dl-miniconda-hecu530u/lib/python3.8/site-packages/execnet/gateway_base.py", line 411 in _perform_spawn
  File "/tmp/dl-miniconda-hecu530u/lib/python3.8/site-packages/execnet/gateway_base.py", line 389 in integrate_as_primary_thread
  File "/tmp/dl-miniconda-hecu530u/lib/python3.8/site-packages/execnet/gateway_base.py", line 1273 in serve
  File "/tmp/dl-miniconda-hecu530u/lib/python3.8/site-packages/execnet/gateway_base.py", line 1806 in serve
  File "<string>", line 8 in <module>
  File "<string>", line 1 in <module>
Error: The operation was canceled.

datalad-next testing is unrelated to these changes -- reported Failing testing of datalad "core" datalad-next#705

yarikoptic · 2024-05-21T19:56:30Z

UTF-8 one keeps segfaulting, I will boost to 3.11 version so we have most fixed python version to debug...

yarikoptic · 2024-05-21T22:35:30Z

nope -- still segfaults , now mentioned that 1 test process segfaults right away

platform linux -- Python 3.11.9, pytest-8.2.1, pluggy-1.5.0 -- /tmp/dl-miniconda-8nv_7dqk/bin/python
cachedir: .pytest_cache
rootdir: /home/runner/work/datalad/datalad
configfile: tox.ini
plugins: cov-5.0.0, xdist-3.6.1, fail-slow-0.5.0
created: 2/2 workers
2 workers [1192 items]
scheduling tests via LoadScheduling
../datalad/__init__.py::datalad.enable_librarymode 
Fatal Python error: Segmentation fault
Thread 0x00007f413ddff[64](https://github.com/datalad/datalad/actions/runs/9181291034/job/25247702054?pr=7600#step:18:65)0 (most recent call first):
  File "/tmp/dl-miniconda-8nv_7dqk/lib/python3.11/site-packages/execnet/gateway_base.py", line 534 in read
  File "/tmp/dl-miniconda-8nv_7dqk/lib/python3.11/site-packages/execnet/gateway_base.py", line 5[67](https://github.com/datalad/datalad/actions/runs/9181291034/job/25247702054?pr=7600#step:18:68) in from_io
  File "/tmp/dl-miniconda-8nv_7dqk/lib/python3.11/site-packages/execnet/gateway_base.py", line 1160 in _thread_receiver
  File "/tmp/dl-miniconda-8nv_7dqk/lib/python3.11/site-packages/execnet/gateway_base.py", line 341 in run
  File "/tmp/dl-miniconda-8nv_7dqk/lib/python3.11/site-packages/execnet/gateway_base.py", line 411 in _perform_spawn
Current thread 0x00007f413effe740 (most recent call first):
  File "<frozen importlib._bootstrap>", line 241 in _call_with_frames_removed

I will now try (as a measure of last resort) to disable parallelization for that run ...

.github/workflows/test.yml

tools/ci/test-jobs.yml

Co-authored-by: Yaroslav Halchenko <debian@onerussian.com>

…ions on NFS Kept a skip where test could have stalled.

Ran into UnboundLocalError: local variable ver referenced before assignment although unclear how/why now since prior run (with a little bit different way to it) did succeed for that matrix run with LC_ALL set to ru_RU.UTF-8

yarikoptic · 2024-05-23T13:44:37Z

ok, we are green and ready to merge (appveyor Windows run stalled / got killed at 1hr mark)! @datalad/developers please chime in! We got a slight boost to coverage by again exploring more code paths and I even had to fix a bug in version checking for rsync.

If no objections, will merge in a few days.

yarikoptic-gitmate · 2024-06-06T21:22:10Z

PR released in 1.0.3

jwodder added semver-tests Changes only affect tests, no impact on version CHANGELOG-missing When a PR's description does not contain a changelog item, yet. labels May 20, 2024

github-actions bot removed the CHANGELOG-missing When a PR's description does not contain a changelog item, yet. label May 20, 2024

jwodder force-pushed the gh-7574 branch from 3e67356 to daee887 Compare May 20, 2024 14:56

jwodder marked this pull request as draft May 20, 2024 14:56

jwodder added the CHANGELOG-missing When a PR's description does not contain a changelog item, yet. label May 20, 2024

github-actions bot removed the CHANGELOG-missing When a PR's description does not contain a changelog item, yet. label May 20, 2024

jwodder force-pushed the gh-7574 branch 2 times, most recently from f4ee4c5 to b11ea16 Compare May 20, 2024 15:45

jwodder force-pushed the gh-7574 branch 2 times, most recently from 68b420c to 10e8795 Compare May 20, 2024 15:54

jwodder marked this pull request as ready for review May 20, 2024 17:05

jwodder requested review from yarikoptic and mih May 20, 2024 17:05

yarikoptic reviewed May 20, 2024

View reviewed changes

.github/workflows/test.yml Outdated Show resolved Hide resolved

yarikoptic force-pushed the gh-7574 branch from 77b5da1 to 645c8f0 Compare May 21, 2024 20:49

yarikoptic reviewed May 21, 2024

View reviewed changes

.github/workflows/test.yml Outdated Show resolved Hide resolved

yarikoptic reviewed May 21, 2024

View reviewed changes

tools/ci/test-jobs.yml Show resolved Hide resolved

yarikoptic force-pushed the gh-7574 branch 4 times, most recently from d9032c5 to 318918e Compare May 21, 2024 23:24

jwodder and others added 4 commits May 21, 2024 19:34

Convert .travis.yml to GitHub Actions workflow

b25faa1

Replace Travis badge with GitHub Actions badge

ae3ff85

[release-action] Autogenerate changelog snippet for PR 7600

22866e5

Fix

64a0577

jwodder and others added 9 commits May 21, 2024 19:34

Properly filter out cron-only jobs

a897d26

Fix

a412876

Append datalad-installer env file to ~/.bashrc

b5c2be4

Fix adding custom Git directory to PATH

8e986bc

Use interactive shell instead of login

7ff5b1f

Fix coverage upload condition

8c7345d

Co-authored-by: Yaroslav Halchenko <debian@onerussian.com>

Adjust/add skips/xfails for the tests which are now failing on GH act…

54abfe9

…ions on NFS Kept a skip where test could have stalled.

BF: Install locales-all if LC_ALL env var is set

9de0f35

Simplify envvar merging

a2f9cf5

jwodder force-pushed the gh-7574 branch from 318918e to a2f9cf5 Compare May 21, 2024 23:34

BF: define var (to None) before it might be assigned

fb45559

Ran into UnboundLocalError: local variable ver referenced before assignment although unclear how/why now since prior run (with a little bit different way to it) did succeed for that matrix run with LC_ALL set to ru_RU.UTF-8

yarikoptic approved these changes May 23, 2024

View reviewed changes

yarikoptic merged commit 3e3e947 into datalad:maint May 28, 2024
30 of 31 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Convert `.travis.yml` to GitHub Actions workflow #7600

Convert `.travis.yml` to GitHub Actions workflow #7600

jwodder commented May 20, 2024

codecov bot commented May 20, 2024 •

edited

yarikoptic commented May 21, 2024

yarikoptic commented May 21, 2024

yarikoptic commented May 21, 2024

yarikoptic commented May 23, 2024

yarikoptic-gitmate commented Jun 6, 2024

Convert .travis.yml to GitHub Actions workflow #7600

Convert .travis.yml to GitHub Actions workflow #7600

Conversation

jwodder commented May 20, 2024

codecov bot commented May 20, 2024 • edited

Codecov Report

yarikoptic commented May 21, 2024

yarikoptic commented May 21, 2024

yarikoptic commented May 21, 2024

yarikoptic commented May 23, 2024

yarikoptic-gitmate commented Jun 6, 2024

Convert `.travis.yml` to GitHub Actions workflow #7600

Convert `.travis.yml` to GitHub Actions workflow #7600

codecov bot commented May 20, 2024 •

edited