ci: annotate pytest failures on GHA

[agoose77]

Closes #2405

[agoose77]

See test_XXXX_failure.py for a test that is deliberately failing to demonstrate this feature.

[jpivarski]

I see, although PyLint doesn't like tests that so obviously would always fail. Perhaps you can trick it like this:

import sys

assert sys.platform == "wacky"

PyLint can't know what all of the platform types are, and that this might actually run on a wacky system someday. We, however, know that it will not. (If you want to be more sure of it, generate a long random name.)

[agoose77]

I see, although PyLint doesn't like tests that so obviously would always fail.

I'm not quite following here — I added these failing tests to trigger a failed run with annotations (on GitHub)

Incidentally, I don't see any pylint warnings about this trivial assertion (either on GHA, or locally)

[jpivarski]

Oh! This was the intention! I had thought these were PyLint failures because PyLint has always been plugged into this system (correlating errors to code in GitHub). Also, PyLint could be smart enough to find out that this assertion would always fail, and that's why I was suggesting a way to fool it. But I misunderstood what was supposed to happen here.

[agoose77]

Ah, I see! Well, if you're happy with this feature, I'll remove the failing test!

[codecov]

Codecov Report

Merging #2448 (c13b710) into main (3d84e05) will not change coverage.
The diff coverage is n/a.

Additional details and impacted files