Fetching latest commit…
Cannot retrieve the latest commit at this time.
|Failed to load latest commit information.|
This document explains how the codechecker works. The codechecker start executing from Main.py. This Main program simply waits for a submission with status = "queued" to appear. When this happens, it picks up the submission, changes it to status = "processing" (atomically - many other codechecker threads can be running) and starts processing it. Some terminology about test cases: 1) Testcase - a testcase is an input file for a single run of a submission. 2) Test_group - a collection of one or more testcases that are scored together, associated with a score. Eg: If a program passes all testcases in a test_group it can get the points for that test_group. If it does not pass all testcases, the submission could get no points. 3) Result checker - a program submitted by a problem setter that is used to check the result of a testcase run by a custom algorithm. This program is required only for problems that need a custom result verification algorithm. If the result for a testcase is unique, it is sufficient to use a reference output file and see that the testcase run produces the same output file. Processing consists of first compiling the submission, and generating an executable. Once this is done, the Evaluation module begins. Evaluation consists of testing the submission against a set of test_groups. Each test_group is fetched in order and the testing for the testcases in that test group begins. The submission is executed against each testcase by the Secure Execution module (SE). The Evaluation module also checks the result of each testcase run with either a reference output file for that testcase or a Result_checker program. The Result_checker also runs in the SE module.