Skip to content

Grading Criteria

Jump to bottom
Keshan W edited this page Nov 17, 2024 · 20 revisions

Grading Criteria and Rubric

The grading system assesses student submissions by evaluating four main components: ChatBot, ChatBotPlatform, ChatBotGenerator, and ChatBotSimulation. Each component is allocated a specific number of marks and evaluated for correctness, functionality, and adherence to project requirements.

Grading Rubric

Component Marks
ChatBot 36
ChatBotPlatform 20
ChatBotGenerator 7
ChatBotSimulation 12

Each component is assessed based on weighted test cases, where marks are distributed based on key functional requirements.

Weighting Allocation

Each grading class, such as ChatBotGeneratorGrade, uses an allocateWeightings() method to assign weightings to individual tests within that component. These weightings determine how much each test case contributes to the component’s total score. For example, critical functionality tests for ChatBotGenerator—like checking valid chatbot codes and response accuracy—are given proportionally higher weight.

Feedback Generation

In the current grading system, feedback is simplified to "Pass" or "Fail" results. If a test case passes, it is recorded as "Pass"; if it fails, it is recorded as "Fail" in the PDF report. The feedbackMap in each grading class simply stores the marks earned per test case, making it easy to track test outcomes without custom feedback explanations.

Note: Detailed expectations for each test case are provided in the accompanying PDF to help students understand what was required for each test.

Example Walkthrough: ChatBotGeneratorTest and Marks Allocation

The ChatBotGeneratorTest class demonstrates how the grading and marking process works. Here’s a breakdown of how marks are allocated based on test outcomes:

1. Static Method Check (1 mark)

  • Test: testMethodIsStatic_generateChatBotLLM()
  • Description: Checks if generateChatBotLLM is a static method.
  • Marks Allocation: Passing this test earns 1 mark. If it fails, 0 marks are awarded, and "Fail" is noted in the feedback.

2. Valid Codes (5 marks)

  • Tests:
    • testGenerateChatBotLLM_ReturnsLLaMaForCode1()
    • testGenerateChatBotLLM_ReturnsMistral7BForCode2()
    • testGenerateChatBotLLM_ReturnsBardForCode3()
    • testGenerateChatBotLLM_ReturnsClaudeForCode4()
    • testGenerateChatBotLLM_ReturnsSolarForCode5()
  • Description: Verifies that each valid chatbot code returns the correct chatbot name.
  • Marks Allocation: Each of these methods checks if generateChatBotLLM returns the expected chatbot name. Passing each test awards a portion of the 5 marks (e.g., 1 mark per valid code). Failure results in 0 marks for that test, with "Fail" noted in the feedback.

3. Invalid Codes (1 mark)

  • Test: testGenerateChatBotLLM_ReturnsChatGPT35ForInvalidCodes()
  • Description: Verifies that invalid codes return the default chatbot, "ChatGPT-3.5."
  • Marks Allocation: Passing this test awards 1 mark. If it fails, 0 marks are awarded, and "Fail" is recorded.

Helper Method: assertChatBotResponse

The assertChatBotResponse() method centralizes response assertions for valid codes, streamlining test case management and improving clarity.

Example of Mark Allocation and Feedback

If the ChatBotGeneratorTest runs and the student’s submission passes four out of the five valid code tests, it would receive 4 out of the possible 5 marks for that section. The report would indicate "Pass" for each successful test and "Fail" for the one that failed. These results are mapped in feedbackMap and used to generate a PDF report showing the marks and Pass/Fail status for each test.

This approach gives students clear, objective feedback through Pass/Fail indicators, with detailed expectations in the PDF for further insight.

This rubric and example illustrate the structured, transparent grading process designed to help students understand their progress and focus on improvement.

Clone this wiki locally