-
-
Notifications
You must be signed in to change notification settings - Fork 896
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Make clear every test's status in every CI run #1679
Commits on Sep 25, 2023
-
Instrument workflows to investigate skipped tests
This makes jobs from both test workflows give more information relevant to examining which tests are skipped (and if any tests xfail, those too) in what environments: - Values of os.name and git.util.is_win. - The name of each test that runs, with its status. The latter doesn't increase the output length as much as might be expected, because due to the way the output is handled, the pytest-sugar pretty output format without -v looked like: test/test_actor.py ✓ 0% test/test_actor.py ✓✓ 0% ▏ test/test_actor.py ✓✓✓ 1% ▏ test/test_actor.py ✓✓✓✓ 1% ▏ When instead it was intended to fit on a single line. Still, the current output with -v has extra newlines, increasing length and worsening readability, so it should be improved on if possible.
Configuration menu - View commit details
-
Copy full SHA for 45773c2 - Browse repository at this point
Copy the full SHA 45773c2View commit details -
Show version and platform info in one place
Instead of splitting it in into two places where at least one of the places is highly likely to be missed, this puts it together just before the first steps that makes nontrivial use of the installed packages. Grouping it together, it can't really be shown earlier, because one of the pieces of information is obtained using the git module (to examine that behavior of the code). This also presents the information more clearly. "set -x" makes this easy, so the commands are rewritten to take advantage of it.
Configuration menu - View commit details
-
Copy full SHA for 6fbe511 - Browse repository at this point
Copy the full SHA 6fbe511View commit details -
Configuration menu - View commit details
-
Copy full SHA for bd3307a - Browse repository at this point
Copy the full SHA bd3307aView commit details -
Configuration menu - View commit details
-
Copy full SHA for 680d795 - Browse repository at this point
Copy the full SHA 680d795View commit details -
Keep sugar for local use, but use instafail on CI
There are two benefits of the pytest-sugar plugin: 1. Pretty output. 2. Show details on each failure immediately instead of at the end. The first benefit is effectively local-only, because extra newlines are appearing when it runs on CI, both with and without -v. The second benefit applies both locally and on CI. So this adds the pytest-instafail plugin and uses it on CI to get the second benefit. It is not set up to run automatically, and pytest-sugar still is (though no longer forced), so local testing retains no benefit and we don't have a clash. The name "instafail" refers only to instantly *seeing* failures: it does not cause the pytest runner to stop earlier than otherwise.
Configuration menu - View commit details
-
Copy full SHA for 75cf540 - Browse repository at this point
Copy the full SHA 75cf540View commit details -
Pass -v twice to see full skip reasons
+ Reorder pytest arguments so both workflows are consistent.
Configuration menu - View commit details
-
Copy full SHA for eb56e7b - Browse repository at this point
Copy the full SHA eb56e7bView commit details -
Force pytest color output on CI
While pytest-sugar output gets mangled with extra newlines on CI, colorized output seems to work fine and improves readability.
Configuration menu - View commit details
-
Copy full SHA for 9c7ff1e - Browse repository at this point
Copy the full SHA 9c7ff1eView commit details -
Fix test_blocking_lock_file for cygwin
This permits the longer delay in test_blocking_lock_file--which was already allowed for native Windows--on Cygwin, where it is also needed. That lets the xfail mark for Cygwin be removed. This also updates the comments to avoid implying that the need for the delay is AppVeyor-specific (it seems needed on CI and locally).
Configuration menu - View commit details
-
Copy full SHA for 0eb38bc - Browse repository at this point
Copy the full SHA 0eb38bcView commit details -
Run cygpath tests on Cygwin, not native Windows
They were not running on Cygwin, because git.util.is_win is False on Cygwin. They were running on native Windows, with a number of them always failing; these failures had sometimes been obscured by the --maxfail=10 that had formerly been used (from pyproject.toml). Many of them (not all the same ones) fail on Cygwin, and it might be valuable for cygpath to work on other platforms, especially native Windows. But I think it still makes sense to limit the tests to Cygwin at this time, because all the uses of cygpath in the project are in code that only runs after a check that the platform is Cygwin. Part of that check, as it is implemented, explicitly excludes native Windows (is_win must be false).
Configuration menu - View commit details
-
Copy full SHA for 715dba4 - Browse repository at this point
Copy the full SHA 715dba4View commit details -
Two of the groups of cygpath tests in test_util.py generate tests that fail on Cygwin. There is no easy way to still run, but xfail, just the specific tests that fail, because the groups of tests are generated with `@ddt` parameterization, but neither the unittest nor pytest xfail mechanisms interact with that. If `@pytest.mark.parametrized` were used, this could be done. But that does not work on methods of test classes that derive from unittest.TestCase, including those in this project that indirectly derive from it by deriving from TestBase. The TestBase base class cannot be removed without overhauling many tests, due to fixtures it provides such as rorepo. So this marks too many tests as xfail, but in doing so allows test runs to pass while still exercising and showing status on all the tests, allowing result changes to be observed easily.
Configuration menu - View commit details
-
Copy full SHA for d6a2d28 - Browse repository at this point
Copy the full SHA d6a2d28View commit details -
Run test_commit_msg_hook_success on more systems
This changes a default Windows skip of test_commit_msg_hook_success to an xfail, and makes it more specific, expecting failure only when either bash.exe is unavailable (definitely expected) or when bash.exe is the WSL bash wrapper in System32, which fails for some reason even though it's not at all clear it ought to. This showcases the failures rather than skipping, and also lets the test pass on Windows systems where bash.exe is something else, including the Git Bash bash.exe that native Windows CI would use.
Configuration menu - View commit details
-
Copy full SHA for 881456b - Browse repository at this point
Copy the full SHA 881456bView commit details -
No longer skip test_index_mutation on Cygwin
As it seems to be working now on Cygwin (maybe not native Windows).
Configuration menu - View commit details
-
Copy full SHA for c6a586a - Browse repository at this point
Copy the full SHA c6a586aView commit details -
Report encoding error in test_add_unicode as error
This makes the test explicitly error out, rather than skipping, if it appears the environment doesn't support encoding Unicode filenames. Platforms these days should be capable of that, and reporting it as an error lessens the risk of missing a bug in the test code (that method or a fixture) if one is ever introduced. Erroring out will also make it easier to see the details in the chained UnicodeDecodeError exception. This does not affect the behavior of GitPython itself. It only changes how a test reports an unusual condition that keeps the test\ from being usefully run.
Configuration menu - View commit details
-
Copy full SHA for fc02230 - Browse repository at this point
Copy the full SHA fc02230View commit details -
Configuration menu - View commit details
-
Copy full SHA for 203da23 - Browse repository at this point
Copy the full SHA 203da23View commit details -
Report <2.5.1 in test_linked_worktree_traversal as error
Although GitPython does not require git >=2.5.1 in general, and this does *not* change that, this makes the unavailability of git 2.5.1 or later an error in test_linked_worktree_traversal, where it is needed to exercise that test, rather than skipping the test. A few systems, such as CentOS 7, may have downstream patched versions of git that remain safe to use yet are numbered <2.5.1 and do not have the necesary feature to run this test. But by now, users of those systems likely anticipate that other software would rely on the presence of features added in git 2.5.1, which was released over 7 years ago. As such, I think it is more useful to give an error for that test, so the test's inability to be run on the system is clear, than to automatically skip the test, which is likely to go unnoticed.
Configuration menu - View commit details
-
Copy full SHA for cf5f1dc - Browse repository at this point
Copy the full SHA cf5f1dcView commit details -
Configuration menu - View commit details
-
Copy full SHA for 8923236 - Browse repository at this point
Copy the full SHA 8923236View commit details -
Express known test_depth failure with xfail
Rather than skipping, so it becomes known if the situation changes.
Configuration menu - View commit details
-
Copy full SHA for b198bf1 - Browse repository at this point
Copy the full SHA b198bf1View commit details -
Remove no-effect
@skipIf
on test_untracked_filesIt looked like test_untracked_files was sometimes skipped, and specifically that it would be skipped on Cygwin. But the `@skipIf` on it had the condition: HIDE_WINDOWS_KNOWN_ERRORS and Git.is_cygwin() HIDE_WINDOWS_KNOWN_ERRORS can only ever be true if it is set to a truthy value directly (not an intended use as it's a "constant"), or on native Windows systems: no matter how the environment variable related to it is set, it's only checked if is_win, which is set by checking os.name, which is only "nt" on native Windows systems, not Cygwin. So whenever HIDE_WINDOWS_KNOWN_ERRORS is true Git.is_cygwin() will be false. Thus this condition is never true and the test was never being skipped anyway: it was running and passing on Cygwin.
Configuration menu - View commit details
-
Copy full SHA for cd175a5 - Browse repository at this point
Copy the full SHA cd175a5View commit details -
Make 2 more too-low git version skips into errors
In the tests only, and not in any way affecting the feature set or requirements of GitPython itself. This is similar to, and with the same reasoning as, cf5f1dc.
Configuration menu - View commit details
-
Copy full SHA for f38cc00 - Browse repository at this point
Copy the full SHA f38cc00View commit details -
Update test_root_module Windows skip reason
The current cause of failure is different from what is documented in the skip reason.
Configuration menu - View commit details
-
Copy full SHA for 8fd56e7 - Browse repository at this point
Copy the full SHA 8fd56e7View commit details -
Change test_root_module Windows skip to xfail
And rewrite the reason to give more useful information. (The new reason also doesn't state the exception type, because that is now specified, and checked by pytest, by being passed as "raises".)
Configuration menu - View commit details
-
Copy full SHA for c1798f5 - Browse repository at this point
Copy the full SHA c1798f5View commit details -
Update test_git_submodules_and_add_sm_with_new_commit skip reason
This is working on Cygwin, so that old reason no longer applies. (The test was not being skipped on Cygwin, and was passing.) It is not working on native Windows, due to a PermissionError from attempting to move a file that is still open (which Windows doesn't allow). That may have been the original native Windows skip reason, but the old AppVeyor CI link for it is broken or not public. This makes the reason clear, though maybe I should add more details.
Configuration menu - View commit details
-
Copy full SHA for ba56752 - Browse repository at this point
Copy the full SHA ba56752View commit details -
Change test_git_submodules_and_add_sm_with_new_commit Windows skip to…
… xfail And improve details. The xfail is only for native Windows, not Cygwin (same as the old skip was, and still via checking HIDE_WINDOWS_KNOWN_ERRORS). This change is analogous to the change in c1798f5, but for test_git_submodules_and_add_sm_with_new_commit rather than test_root_module.
Configuration menu - View commit details
-
Copy full SHA for 8704d1b - Browse repository at this point
Copy the full SHA 8704d1bView commit details -
Run the tests in test_tree on Windows
This stops skipping them, as they are now working.
Configuration menu - View commit details
-
Copy full SHA for 1d6abdc - Browse repository at this point
Copy the full SHA 1d6abdcView commit details -
Add missing raises keyword for test_depth xfail
I had forgotten to do this earlier when converting from skip to xfail. Besides consistency with the other uses of xfail in the test suite, the benefit of passing "raises" is that pytest checks that the failure gave the expected exception and makes it a non-expected failure if it didn't.
Configuration menu - View commit details
-
Copy full SHA for 5609faa - Browse repository at this point
Copy the full SHA 5609faaView commit details -
Configuration menu - View commit details
-
Copy full SHA for ed95e8e - Browse repository at this point
Copy the full SHA ed95e8eView commit details -
Configuration menu - View commit details
-
Copy full SHA for ceb4dd3 - Browse repository at this point
Copy the full SHA ceb4dd3View commit details -
Configuration menu - View commit details
-
Copy full SHA for 3276aac - Browse repository at this point
Copy the full SHA 3276aacView commit details -
Try to work in all LF on Cygwin CI
+ Style tweak and comment to clarify the "Limit $PATH" step.
Configuration menu - View commit details
-
Copy full SHA for 5d40976 - Browse repository at this point
Copy the full SHA 5d40976View commit details -
Configuration menu - View commit details
-
Copy full SHA for dda4286 - Browse repository at this point
Copy the full SHA dda4286View commit details
Commits on Sep 26, 2023
-
Remove the recently added "Limit $PATH" step
I had put that step in the Cygwin workflow for purposes of experimentation, and it seemed to make clearer what is going on, but really it does the opposite: it's deceptive because Cygwin uses other logic to set its PATH. So this step is unnecessary and ineffective at doing what it appears to do.
Configuration menu - View commit details
-
Copy full SHA for 3007abc - Browse repository at this point
Copy the full SHA 3007abcView commit details -
Further reduce differences between test workflows
This makes the two CI test workflows more similar in a couple of the remaining ways they differ unnecessarily. This could be extended, and otherwise improved upon, in the future.
Configuration menu - View commit details
-
Copy full SHA for 4860f70 - Browse repository at this point
Copy the full SHA 4860f70View commit details