Skip to content

Home

Jump to bottom
100levelz edited this page Nov 22, 2023 · 53 revisions

Automated Judge System

Introduction

Project Overview

Welcome to our collaborative effort to design, develop, evaluate, and present an automated judge system for evaluating assignments submitted in Object Oriented Programming 1. The primary objective of our project is to streamline the assessment process for student assignments through the development of a Java-based software application.

This application will accept a zipped file, following the naming convention "AssignmentX_Java_Programs.zip," where 'X' represents the assignment number. Within this zipped file, student submissions will adhere to the naming format: "StudentID_Name_AX.zip" and each of these subsequent zipped files will contain .java files from the student’s assignment named after the respective class.

Meet Our Team

  • Hailey Fritz - 816033647
  • Zion Holder - 816033712
  • Jason Balroop - 816033413
  • Khadisha Clarke - 816032790
  • Vanessa Aubin - 816031597
  • Ishmael Jupiter - 816031011

Analysis

Major Requirements and Use Cases

Use Cases

  • Use Case: Submit Assignments
  • Actor: Teacher
  • Description: Use case begins when the teacher submits a zipped folder containing individual zipped folders of student submissions. If the submissions adhere to the naming conventions then they are tested and reports are generated.
  • Use Case: Validate Files
  • Actor: System
  • Description: The system validates the submitted files, checking for correctness, completeness, and adherence to specifications. If the files are validated then they are tested, if not an error is shown.
  • Use Case: Iterate Through Files
  • Actor: System
  • Description: The system opens the initial zipped folder and unzips and checks each student folder within, one at a time.

Target students

The target students of this system, are those that are registered for the course Object-Oriented Programming I.

Design

Design Patterns Used

Mediator Design Pattern for PDF Report Generation

The Mediator design pattern was used in the generation of the PDF document. The Feedback Class acts as the Mediator, managing the interaction between the all the different classes. Each Feedback object represents a response to a test ran when grading a student's assignment. Instead of these classes interacting directly with each other, they interact through the Feedback class, which handles the coordination and arrangement of the data in the final PDF document. This design promotes loose coupling and makes the system easier to understand, maintain, and extend.

Iterator Design Pattern for Traversing Folder Contents

The Iterator design pattern was used for handling the input files. To begin using the platform, the system must accept a zipped folder. This folder contains several individual zipped folders representing the assignment submissions of the students in the Object-Oriented Programming I class. This design pattern is used for iterating through each student's submmision, it allows for each folder to be retrieved without exposing the underlying implementations involved. This also allows for 2 of the 5 SOLID principles to be maintained, namely the Single Responsibility Principle and the Open/Closed Principle.

Template Method Design Pattern for AutoTesting

The Template Method design pattern was used in the testing of student submissions. The TestTemplate class acts as a template for the subsequent concrete test classes, i.e. PassengerTest, LuggageSlipTest, LuggageManifestTest, LuggageManagementSystemTest and FlightTest. The TestTemplate class contains concrete methods that is utilized by all test classes. The template method, runTests() makes a call to the ClassTestSuite class which represents a Test Suite for all class tests, thereby allowing them all to run sequentially. This design pattern allows us to potential duplicate code, long method and large class code smells.

Conformance to SOLID

PDF Report Generation

Single Responsibility Principle (SRP): The addTables method has a single responsibility - to add tables to the document. It does not concern itself with other tasks such as creating the document or saving it.

Open/Closed Principle (OCP): The addTables method is open for extension but closed for modification. More functionality can be added to the method without modifying its existing code. For example, more fields can be added to the Feedback class, and the addTables method will still work without any changes.

Dependency Inversion Principle (DIP): The addTables method depends on the abstraction (Feedback) rather than the details of how feedback is stored and retrieved.

Input Data Handling

Single Responsibility Principle (SRP): The Iterator pattern helps to separate the concerns of iteration and the data structure used to store the student submissions. Managing the traveral of elements within a collection adheres to this principle. It encapsulates the logic for iterating over the collection into a single iterator object, this being FolderIterator. Having a separate class, UnzipUtility, tasked with unzipping the submitted zipped folder and another class for determining if the java files compile, FileCompilationChecker also support this principle.

Open/Closed Principle (OCP): The Iterator pattern also follows the OCP by allowing the addition of new iterators without modifying the underlying collection or the iteration logic. For this sytem one iterator was used, FolderIterator but defferent iterators can be added without changing the existing code that uses the iterators.

AutoGrading

Open/Closed Principle (OCP): The Template Method design pattern adheres to the Open/Closed principle since concrete methods in the TestTemplate class are open for extension in it's subclasses but cannot be modified. Therefore, any subclass that extends the TestTemplate method can use the concrete method's behaviour and further extend it within it's own class.

Class Diagram

Implementation

How to Run

To run this autograder system, staff members, such as the course lecturer or assignment marker, will submit one zipped folder containing several individual zipped folders. These internal zipped files represent each student's submssion and holds their respective java program files. The system then iterates through each submission. When it does this, the student's java files are evaluated to determine the correctness of the program code based on the assignment's specification. After all the files that a student submitted are evaluated, a PDF formatted report is generated to document the test cases that the program successfully passed and those that failed. This report provides relevant and constructive feedback as well as an overall score for the assignment.

Setup Requirements

  • Student submissions must adhere to package configurations i.e. containing the package comp3607project import statement.
  • The zipped folder containing the submissions of all students must not contain an unzipped inner folder. Each individual submission must be easily retrieved upon unzipping this folder.

Testing and Evaluation

Test Cases and Suites

Test Suite

Test Cases

  • TestTemplate

  • PassengerTest

    • init()
    • testPassportNumberField()
    • testFlightNoField()
    • testfirstNameField()
    • testlastNameField()
    • testNumLuggageField()
    • testCabinClassField()
    • testConstructor()
    • assignRandomCabinClassTest()
    • testToString()
    • checkRandom(String fieldName)
    • checkSet(Object expectedValue)

  • LuggageSlipTest

    • LuggageSlipTest()
    • init()
    • testPassengerField()
    • luggageSlipIDCounterField()
    • luggageSlipIDField()
    • labelField()
    • testConstructor()
    • testOverloadedConstrctor()
    • hasOwnerTest()
    • testToString()
    • checkSet(Object expectedValue, Object thisClass)

  • LuggageManifestTest

    • init()
    • testSlips()
    • testCreateLuggageManifest()
    • testAddLuggage()
    • testGetExcessLuggageCost()
    • testToString()
    • testGetExcessLuggageCostByPassenger()

  • FlightTest

    • init()
    • testFlightNo()
    • testDestination()
    • testOrigin()
    • testFlightDate()
    • testManifest()
    • testConstructor()
    • testCheckInLuggage()
    • testPrintLuggageManifest()
    • testToString()
    • testGetAllowedLuggage()

  • LuggageManagementSystemTest

    • init()
    • testSystemRun()
    • testToString()

  • FeedbackReportTest

    • testGenerateDocument()

  • ReportContentTest

    • testAddFeedback()
    • testAddTableHeaders()
    • testAddTables()

  • ReportHeaderTest

    • testAddHeader()

Detailed Documentation Breakdown

Demo Video Link

Clone this wiki locally