Fix judge output parsing edge cases and migrate to polymorphic solution contract #41
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
The test framework uses ast.literal_eval() to parse solution outputs, which converts single numbers like "1" to integers. The judge functions in 7 solutions were only checking for str or list types, causing test failures when outputs were parsed as integers. Fixed solutions: - 0021_merge_two_sorted_lists: Handle int input + fix empty line parsing - 0027_remove_element: Handle int input for k=0 single-line output case - 0075_sort_colors: Handle int input for single-element arrays - 0088_merge_sorted_array: Handle int input + fix empty array parsing - 0283_move_zeroes: Handle int input for single-element arrays - 0876_middle_of_the_linked_list: Handle int input for single-node lists - 0905_sort_array_by_parity: Handle int input for single-element arrays Changes: - Added isinstance(actual, int) checks in all judge functions - Convert single integers to single-element lists: [actual] - Improved input parsing to preserve empty lines correctly - No changes to solution algorithms (all were already correct) Test Results: - All 92 tests pass (7 skipped for problems without static tests) - Static tests: 33/33 passing - Generated tests: 59/59 passing - Combined tests: 99/99 passing Root Cause: The compare.py module uses ast.literal_eval() to parse outputs before passing to JUDGE_FUNC. Single numbers get parsed as int, not str, which the judge functions didn't handle.
BREAKING CHANGE: Remove deprecated wrapper template and --wrapper option - Create docs/solution_contract.md as canonical specification for: - Solution file structure (SOLUTIONS + get_solver() pattern) - SOLUTIONS metadata schema (required: class, method) - Judge/validation contract (JUDGE_FUNC, COMPARE_MODE priority) - Static/dynamic tests and generator requirements - Metadata layers and consistency checklist - Update templates to polymorphic architecture: - template_solution.py: Add SOLUTIONS dict + get_solver() - template_solution_multi.py: Use multiple classes, same method name - Delete template_solution_wrapper.py (deprecated) - Update scripts: - new_problem.bat/sh: Remove --wrapper option - run_tests.bat: Fix argument passing (use %* for all args) - Update documentation: - README.md: Replace wrapper example with polymorphic pattern - README_zh-TW.md: Same updates in Chinese - ARCHITECTURE_MIGRATION.md: Mark Phase 4 complete - Clean up temporary files: - Delete test_debug.py - Delete test_output.py
lufftw
commented
Dec 12, 2025
Owner
Author
lufftw
left a comment
lufftw
left a comment
There was a problem hiding this comment.
Changes are clear and well-structured. LGTM.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
🛠️ Summary: Fix Judge Output Parsing and Complete Polymorphic Contract Migration
The changes improve test robustness and documentation clarity while preserving all existing solution algorithms.
Changes
Judge Output Normalization
The test framework parses solution outputs using
ast.literal_eval(), which converts single-value outputs such as "1" intoint. SeveralJUDGE_FUNCimplementations assumed outputs werestrorlistonly, causing failures in edge cases.Fixes applied:
isinstance(actual, int)handling in affectedJUDGE_FUNCimplementations[actual]Affected solutions:
intinput and fix empty line parsingintinput fork=0single-line outputintinput for single-element arraysintinput and fix empty array parsingintinput for single-element arraysintinput for single-node listsintinput for single-element arraysArchitecture and Contract Updates
Complete the transition from the deprecated wrapper-based solution pattern to the polymorphic solution contract.
Key updates:
docs/solution_contract.mdas the canonical specification for:SOLUTIONS + get_solver())SOLUTIONSmetadata schema (required class and method)JUDGE_FUNC,COMPARE_MODEpriority)template_solution.py: Polymorphic single-solution templatetemplate_solution_multi.py: Multiple classes with shared method name--wrapperCLI optionnew_problem.bat / new_problem.sh: Remove--wrapperoptionrun_tests.bat: Fix argument forwarding using%*README.mdandREADME_zh-TW.mdto reflect the new architectureARCHITECTURE_MIGRATION.mdtest_debug.py,test_output.py)Testing
Root Cause
The compare module parses outputs with
ast.literal_eval()before passing them toJUDGE_FUNC.Numeric outputs are converted to
int, which was not handled by several judge implementations.Backward Compatibility
--wrapperoption are removed (breaking change)SOLUTIONS + get_solver()contract