fix(pipeline): validate test file references in exported tasks#11
Merged
alpha1122x merged 2 commits intomainfrom Feb 17, 2026
Merged
fix(pipeline): validate test file references in exported tasks#11alpha1122x merged 2 commits intomainfrom
alpha1122x merged 2 commits intomainfrom
Conversation
…ation - Fix ep-eaglepoint-ai task: replace TypeScript/Jest tests with Python/pytest tests matching the actual repository language - Fix batocera-linux task: change invalid unittest file-path syntax to pytest - Add extract_test_paths_from_command() and validate_test_file_references() to pipeline.rs for automated test file path validation - Add 12 unit tests for the new validation functions - Update validation_summary.json and benchmark_validation_report.md
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds validation that test file paths referenced in
fail_to_passandpass_to_passcommands actually exist in the exported task directory. Previously, test commands could reference files that were never written to disk, causing silent harness failures.Changes
src/swe/pipeline.rs:export_task_to_diskvalidate_test_file_references()that parses shell commands to extract referenced test file paths and warns when they are missing frommeta.test_filesor the exported tests directoryextract_test_paths_from_command()helper supporting pytest, unittest (dotted module), jest, vitest, and Java test patternsexport_task_to_diskDataset fixes: Corrected test commands and workspace configs for several generated tasks where
fail_to_pass/pass_to_passreferenced non-existent test filesbenchmark_validation_report.md/validation_summary.json: Updated validation status reflecting the fixes