Update with latest commits from johnb i1006_test_scoreboard#2
Merged
clevengr merged 14 commits intoclevengr:i1006_point_scoring_invoke_graderfrom Aug 1, 2025
Merged
Conversation
With the addition of Point Scoring support, it seemed logical (and necessary) to be able to edit/change the score of a judgment. This commit adds that capability.
We can't just use the first test data group and assume it's the root. We have to walk up the getParent() chain until we find the null parent. At that point, we know we're at the top of the tree, and we can use that getRangeMax(). As per Tim deBoer, he suggested we do not send Double.POSITIVE_INFINITY (infinity) if the max is in fact infinity. Rather, just leave the property out of the record. This contradicts the current 2023-06 specification that requires the max_score property.
The value -1 is legal for a submission override if the id is, in fact, 1. 0 is the only illegal value. Not sure why I had it as -1 originally.
The value -1 is legal for a submission override if the id is, in fact, 1. 0 is the only illegal value. Not sure why I had it as -1 originally.
We can't just use the first test data group and assume it's the root. We have to walk up the getParent() chain until we find the null parent. At that point, we know we're at the top of the tree, and we can use that getRangeMax(). As per Tim deBoer, he suggested we do not send Double.POSITIVE_INFINITY (infinity) if the max is in fact infinity. Rather, just leave the property out of the record. This contradicts the current 2023-06 specification that requires the max_score property.
Updated NSA to score correctly for point scoring since the ResultsFile class uses NSA to generate results.tsv and results.csv.
Previously, if a submission received on the CLICS API contained an empty source code (main) file, the submission was rejected. This commit allows empty source to be submitted, which should ultimately cause a Compile Error.
Add a configuration parameter, "submission-throttling", whose default value is "true", to the system.pc2.yaml file. This will provide a means of allowing a contest administrator to disable the throttling mechanism. This is useful during contest playback at high speeds. Remember the submission throttling flag in ContestInformation.
Use Double object instead of primative. This way we don't have to filter and can just use the "only include if non-null" annotation instead of all the extra rtfilter code. Fix copyright date in ProblemDataFiles.
Legacy Grader did not generate proper results for WA/RTE/TLE (non-AC). It was printing them instead of putting them in a string. Added optional debug logging to a specified file.
Fix merge error when throttling was cherry-picked. (ContestImportUtilities)
Import conflict when merging cherry-picked commit fixed.
Add "on_reject" "break" support as per spec. Fix some comments to mention some concerns. Handle null return values on test data group grading. Create grader debug logs for every test data group.
clevengr
approved these changes
Aug 1, 2025
Owner
clevengr
left a comment
clevengr
left a comment
There was a problem hiding this comment.
I did a code review of all 20 changed files; the changes all look reasonable. I'm approving the PR and merging it so that we can do runtime testing with the rest of the point-scoring branch code.
Do not set scoreboard if it's not specify in the YAML config file. The default will be pass-fail if it's not specified. It was being set to "null" which was causing an NPE in the JUnit.
clevengr
merged commit beab98b
into
clevengr:i1006_point_scoring_invoke_grader
2 of 3 checks passed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description of what the PR does
As discussed this addresses the following:
LegacyGrader - handles non-AC cases properly instead of just printing the result, it now return sit.
LegacyGrader - Added Debug file logging
Executable - support on_reject = break
Executable - handle null returns from grader on sub groups
Submission throttling - can disable via setting in system.pc2.yaml
Merged latest changes and fixes to event feed to support Resolver.
Fixes results file generation, and consequently the NSA.
Other small bug fixes.
Issue which the PR addresses
Partial fix to pc2ccs#1006
Environment in which the PR was developed (OS,IDE, Java version, etc.)
Windows 11
Ubuntu 24.04
Precise steps for testing the PR (i.e., how to demonstrate that it works correctly)
N/A