ci: fix broken tests

[henryiii]

WIP at the moment. These are the 11 Windows PyPy failures:

FAILED tests/test_env.py::test_isolated_env_log - subprocess.CalledProcessErr...
FAILED tests/test_main.py::test_build_package - subprocess.CalledProcessError...
FAILED tests/test_main.py::test_build_package_via_sdist - subprocess.CalledPr...
FAILED tests/test_main.py::test_build_package_via_sdist_cant_build - subproce...
FAILED tests/test_main.py::test_output[via-sdist-isolation] - SystemExit: 1
FAILED tests/test_main.py::test_output[wheel-direct-isolation] - SystemExit: 1
FAILED tests/test_util.py::test_wheel_metadata[True] - subprocess.CalledProce...
FAILED tests/test_util.py::test_wheel_metadata_isolation - subprocess.CalledP...
FAILED tests/test_util.py::test_with_get_requires - subprocess.CalledProcessE...
FAILED tests/test_main.py::test_output_env_subprocess_error[no-color] - asser...
FAILED tests/test_main.py::test_output_env_subprocess_error[color] - assert [...

[henryiii]

The main issue with most of the failures is python -IM ensure-up --upgrade --default-pip is failing with no module named encodings. Our error reporting still needs work, as that error is completely hidden.

[henryiii]

I believe all PyPy problems are due to a single bug, and that bug is getting fixed in https://foss.heptapod.net/pypy/pypy/-/issues/3323 for PyPy 7.3.9, which is planned to be released in a couple of weeks or so. I'll try to test with with a nightly build in a few days if possible. For now, I'm thinking we can xfail strict these tests, and then remove the workarounds once the new PyPy hits.

[FFY00]

Thank you so much for looking into this!

I'll try to test with with a nightly build in a few days if possible.

If it fixes it, we should update the code. We can do that later.

[layday]

What does -a do?

[henryiii]

It summarizes (s)skips, (x)fails, and failures (f, perhaps?). That way you can read a quick summary of everything at the end (like the failures I posted above). Otherwise, you can't easily see what failed, only what was skipped or xfailed! (There's also "A", IIRC, which prints passes too, which is silly for a summary, but failures are important).

FYI, the default is to show failures, but manually specifying it requires an explicit "f", or in this case, "a". I always use "a".

[layday]

Right, that's the value of -r. I would've separated them with a space for clarity, otherwise it looks like you're chaining multiple flags.

[henryiii]

We can do that later.

We have to do that later, I'm quite sure this morning's fix is not in the nightlies yet. :D