ci: verify the solution is correct #38
Conversation
Signed-off-by: Frost Ming <me@frostming.com>
frostming
force-pushed
the
ci/test-solution
branch
from
519235c to
7611fcb
Compare
Signed-off-by: Frost Ming <me@frostming.com>
| CHALLENGES_DIR = Path(__file__).parent.parent / "challenges" | ||
| ALL_SOLUTIONS = list(CHALLENGES_DIR.glob("**/solution.py")) | ||
|
|
||
|
|
||
| @pytest.mark.parametrize("solution_file", ALL_SOLUTIONS, ids=lambda x: x.parent.name) | ||
| def test_solution_valid(solution_file: Path): | ||
| with solution_file.open() as f: | ||
| code = f.read() | ||
| result = ChallengeManager._type_check_with_pyright(code) | ||
| assert result.passed, result.stdout |
There was a problem hiding this comment.
I think we should test these cases in parallel to reduce the total time of CI. How about pytest-xdist? 👀
Signed-off-by: Frost Ming <me@frostming.com>
| with solution_file.open() as f: | ||
| code = f.read() | ||
| result = ChallengeManager._type_check_with_pyright(code) | ||
| assert result.passed, result.stdout |
There was a problem hiding this comment.
Curious, is there a need to assert result.stdout?
There was a problem hiding this comment.
This will give the typecheck output when the test fails, which helps up debug the solution.
|
Thanks, I always wanted to add this but don't have time. Overall LGTM. Some new solutions were just added, do you mind do a pull and test them? Appreciate it! |
You can rebase the main onto the pull requests after this is merged. New solutions will be automatically picked up for CI, and the author will know the status. |
I mean they've been added to the codebase but let me merge this and run it myself. |
Signed-off-by: Frost Ming me@frostming.com