Testing

Getting Started

Test Suite

Unit Tests

The AutoGrader supports full coverage testing across all interfaces using JUnit 4. Unit tests validate individual components in isolation, ensuring they perform as expected.

Below is an example of a unit test for the TestRunner interface:

public class TestRunnerTest {
    @Test
    public void testRunAllTests() {
        TestRunner<?> testRunner = new AssignmentTestRunner();
        List<?> results = testRunner.runAllTests();
        for (var result : results) {
            Assert.assertTrue(result != null);
        }
    }
}

Performance Test

Full coverage testing across the test suite measures the execution time of processes/methods and compares them against the performance limits defined in TestConstants (in ms).

This is achieved using an in-house microframework, ExecutionTimer, developed for precise performance analysis.

public class ExecutionTimer {
    public static <T> PerformanceTestResult testExecutionTime(Runnable task, long threshold, String testName) {
        long startTime = System.nanoTime();
        task.run();
        long endTime = System.nanoTime();
        long elapsedTime = (endTime - startTime) / 1_000_000;
        boolean success = elapsedTime <= threshold;
        System.out.println("Execution Time of " + testName + ": " + elapsedTime + " ms");
        return new PerformanceTestResult(testName, success, elapsedTime);
    }
}

Below is an example performance test for the GradingFacade's processSubmissions method (the system's main module, providing full-coverage testing):

public class FacadePerformanceTest {
    private static final PrintStream originalOut = System.out;

    @Test
    public void testFacadePerformance() throws Exception {
        System.setOut(new PrintStream(OutputStream.nullOutputStream()));
        Facade facade = new GradingFacade();
        PerformanceTestResult result = ExecutionTimer.testExecutionTime(
                () -> {
                    facade.processSubmissions("submissions.zip");
                },
                TestConstants.MAX_ALLOWABLE_THRESHOLD_MS,
                "Facade - Process Submissions");
        System.setOut(originalOut);
        assertTrue("Execution of processSubmissions took too long", result.isSuccess());
    }
}