Testing

Unit Tests

• CompileCommandUnitTest.java: Verifies the compilation process with valid and invalid Java files, as well as handling non-existent directories.

• GradeUnitTest.java: Ensures that grade allocations and feedback mechanisms operate correctly.

• add more

Performance Testing

To guarantee that the Autograder operates efficiently, especially when handling multiple submissions simultaneously, several performance tests are conducted. These tests measure the system's responsiveness and scalability under various loads.

Performance Test Components

• GradeFactoryPerformanceTest.java: Assesses the time taken to instantiate grade objects from test results.

• FacadePerformanceTest.java: Evaluates the performance of the GradingFacade when processing a large number of submissions.

• PDFPerformanceTest.java: Measures the efficiency of PDF report generation.

Example Performance Test

Below is an example of a performance test for the GradingFacade's processSubmissions method:

public class FacadePerformanceTest {
    private static final PrintStream originalOut = System.out;

    @Test
    public void testFacadePerformance() throws Exception {
        System.setOut(new PrintStream(OutputStream.nullOutputStream()));
        Facade facade = new GradingFacade();
        PerformanceTestResult result = ExecutionTimer.testExecutionTime(
                () -> {
                    facade.processSubmissions("submissions.zip");
                },
                TestConstants.MAX_ALLOWABLE_THRESHOLD_MS,
                "Facade - Process Submissions");
        System.setOut(originalOut);
        assertTrue("Execution of processSubmissions took too long", result.isSuccess());
    }
}

Execution Timer Utility

ExecutionTimer is a microframework designed to measure the execution times of methods or code blocks, enabling precise performance evaluations.ExecutionTimer facilitates the measurement of method execution times, enabling precise performance evaluations:

public class ExecutionTimer {
    public static <T> PerformanceTestResult testExecutionTime(Runnable task, long threshold, String testName) {
        long startTime = System.nanoTime();
        task.run();
        long endTime = System.nanoTime();
        long elapsedTime = (endTime - startTime) / 1_000_000;
        boolean success = elapsedTime <= threshold;
        System.out.println("Execution Time of " + testName + ": " + elapsedTime + " ms");
        return new PerformanceTestResult(testName, success, elapsedTime);
    }
}

Continuous Integration

To maintain high code quality and ensure that all tests are executed consistently, a Continuous Integration (CI) pipeline is integrated into the development workflow. This pipeline automatically runs all unit and performance tests upon each commit, providing immediate feedback and preventing regressions.

CI Pipeline Features

• Automated Testing: Every push to the repository triggers the execution of the full test suite.

• Build Verification: Ensures that all tests pass before changes are merged, maintaining the integrity of the codebase.

• Performance Monitoring: Detects any deviations in performance metrics, enabling prompt optimizations.

Test Coverage

Comprehensive test coverage is achieved through the combination of unit and performance tests, targeting all critical components and interfaces within the Autograder system. Regular assessments and updates to the test suite ensure ongoing reliability and performance.

Tools and Frameworks

• JUnit: Utilized for writing and executing unit tests.

• Maven: Manages project dependencies and builds, facilitating the execution of tests.

• Custom Utilities: Classes like ExecutionTimer and PerformanceTestResult support detailed performance evaluations.

Running the Tests

To execute the test suites, follow the installation and setup instructions provided in the Installation section. Once set up, run the tests using Maven commands:

mvn test

For performance tests, ensure that the appropriate thresholds are configured in the TestConstants class to accurately reflect the performance requirements.

Home