[BUG]: Windows CI Tests False-Positive of Passing

[adam-grant-hendry]

Description

After reviewing your Actions, it is clear tests are not running on Windows machines. On Windows, the linter and test script does not output any results and takes 0 or 1s to complete, whereas on Linux/MacOS they take >1 minute and output pytest results.

Your .github/workflows/pythonpackage.yml worflow does not specify a shell to use in the Run tests and linters step. Hence, on Windows, it defaults to using Powershell. Thus, what is run (which you can see in this example workflow) is simply:

pwsh -Command ./scripts/test

This does nothing more than ask Windows to attempt to open a file it does not know how to handle, which is easily repeated locally:

Commitizen_Testing_Error.mp4

Cloning the repo, changing the shebang to a portable variant, and running pytest, it is clear many tests are failing on Windows. This was first noticed in #602

https://github.com/commitizen-tools/commitizen/files/9790428/pytest_report.zip

Steps to reproduce

1. Follow the Contributing steps to:
   (a) Fork the repo, install dependencies through poetry, and setup the pre-commit hooks
2. Run pre-commit run --all

Current behavior

Tests are failing silently on GitHub Actions (i.e. CI), yielding false positives on prior commits. Tests fail explicitly when the repo is cloned and tested on Windows, which also prevents developers using Windows to contribute (pre-commit tests fail, preventing pushing to PRs).

Desired behavior

1. Windows tests run on GitHub Actions
2. Windows users can contribute to your repo

If you continue to package for Windows, you need to confirm that your tests are actually running and passing. Your tests are failing at this point.

Screenshots

Please see the Description.

Environment

Commitizen Version: 2.35.0
Python Version: 3.8.10 (tags/v3.8.10:3d8993a, May 3 2021, 11:48:03) [MSC v.1928 64 bit (AMD64)]
Operating System: Windows

Local pytest Report

Adding the pytest run on Windows, first referenced in #602 . This report was generated using the pytest-html plugin.

pytest_report.zip

[Lee-W]

Hi @adam-grant-hendry , thanks so much for the detail report and testing! Without your report we'll not be able to find such issue. However, both Santi and I does not have a Windows machine. Wondering would you be interested in sending us a PR for fixing this issue. I'm happy to cowork with you to see how this can be fixed! (I'm a bit busy these days but will try to reply ASAP)

[adam-grant-hendry]

@Lee-W Thank you for your kindness! It is so greatly appreciated.

Yes, I would be happy to help. There are two things to pursue here. The first is a quick fix and the second addresses the larger issue:

Quick Fix:

1. Specify the default shell as bash in your pythonpackage.yml GitHub Actions workflow.
2. Use --no-verify like you suggested if developing on Windows not in bash

That should get tests running on Windows in your CI.

Long-Term Fix:

1. Add powershell support.

This is going to require fixing several failing tests. One challenge is subprocess in Python on Windows references the environment variable ComSpec to determine which shell to run commands in, which defaults to Command Prompt (C:\WINDOWS\system32\cmd.exe).

This shouldn't be changed because several older Windows OS utilities may rely on running batch scripts that expect the old behavior of Command Prompt, so changing it to PowerShell or Bash could cause problems. Instead, PowerShell should be called out explicitly. Traditional PowerShell (i.e. not PowerShell Core) ships with Windows OS's since Windows 7, and GitHub Actions uses (traditional) PowerShell on Windows by default, so this shouldn't be a problem.

I recommend a wrapper around subprocess as recommended in this realpython.com article and implemented in the dropboxignore GitHub repo.

[woile]

Fixed in pr #605