3.4 Implementing the Blueprint Validation Module

The goal of this task is to validate the JSON Schema of the blueprint returned by the LLM. If validation fails, automatically feed the error information back to the LLM and ask it to correct, retrying at most 3 times.
After completing this task, you will have a self-correcting blueprint generation pipeline, ensuring that the blueprint received by the subsequent project generator is always valid.

---

Prerequisites

• Completed Task 1: generate_blueprint() in modsmith/llm/client.py can be called normally.
• Completed Task 3: modsmith/blueprint/schema.json and examples are ready.
• .env is configured, the Zhipu API Key is valid, and a free model (such as glm-4-flash) is used to avoid balance issues.

---

Step 1: Update the System Prompt in modsmith/llm/client.py

To make the LLM clear about the required fields for each type from the start, explicitly list the rules in the System Prompt.
Open modsmith/llm/client.py and add the import at the top:

from modsmith.config import MINECRAFT_VERSION, FABRIC_LOADER_VERSION

Find the _build_system_prompt() function, and replace the prompt += """## 规则... part with:

    prompt += """## Rules

1. Output JSON only. Do not include any explanation, comments, or Markdown code block markers.
2. All required fields must be present.
3. The `type` field can only be one of: basic, food, fuel, tool.
4. If the description is unclear, use reasonable defaults.
5. If the `texture` field cannot be determined, use "auto".

## Required fields per type

- **basic**: `id`, `type`, `display_name_en`
- **food**: `id`, `type`, `display_name_en`, `nutrition`, `saturation`, `always_edible`
- **fuel**: `id`, `type`, `display_name_en`, `burn_time`
- **tool**: `id`, `type`, `display_name_en`, `tool_type`, `durability`, `mining_speed`, `attack_damage`

## Suggested defaults

- `saturation`: 0.3
- `always_edible`: true
- `effects`: empty array `[]`
- `texture`: "auto"

## Version requirements (must be strictly followed)

- `minecraft_version` must be "{MINECRAFT_VERSION}"
- `fabric_loader_version` must be "{FABRIC_LOADER_VERSION}"
- Do not use any other version numbers.
"""

Note: This way, the LLM will try to fill in the required fields on the first generation, reducing retry count.

---

Step 2: Create modsmith/blueprint/validator.py

This module provides:

• validate_blueprint(blueprint: dict) -> None: validates the blueprint; raises jsonschema.ValidationError on failure.
• generate_validated_blueprint(user_input: str, max_retries: int = 3) -> dict: calls the LLM to generate a blueprint, feeds back corrections on validation failure, and retries up to the specified number of times.

Write the following complete code:

"""Blueprint validation and LLM-driven correction."""

import json
from pathlib import Path

import jsonschema

from modsmith.llm.client import generate_blueprint

SCHEMA_PATH = Path(__file__).parent / "schema.json"


def _load_schema() -> dict:
    """Load the blueprint JSON Schema."""
    with open(SCHEMA_PATH, "r", encoding="utf-8") as f:
        return json.load(f)


def validate_blueprint(blueprint: dict) -> None:
    """Validate the blueprint against the Schema; raise jsonschema.ValidationError on failure."""
    schema = _load_schema()
    jsonschema.validate(instance=blueprint, schema=schema)


def generate_validated_blueprint(user_input: str, max_retries: int = 3) -> dict:
    """Generate a blueprint and validate it; on failure, feed back to the LLM for correction,
    retrying at most max_retries times.

    Args:
        user_input: The user's natural language description.
        max_retries: Maximum number of retries.

    Returns:
        A validated blueprint dictionary.

    Raises:
        RuntimeError: If a valid blueprint still cannot be generated after max_retries attempts.
    """
    last_error = None
    current_input = user_input

    for attempt in range(1, max_retries + 1):
        try:
            blueprint = generate_blueprint(current_input)
            validate_blueprint(blueprint)
            print(f"✅ Attempt {attempt} successfully generated a valid blueprint.")
            return blueprint
        except jsonschema.ValidationError as e:
            last_error = e
            path = " -> ".join(str(p) for p in e.path) if e.path else "root"
            feedback = f"""The previously generated blueprint is invalid. The error information is as follows:

- Error location: {path}
- Error type: {e.message}

Please correct it strictly according to the following requirements:
1. If `type` is `food`, it must contain `nutrition` (integer), `saturation` (float, e.g. 0.3), `always_edible` (boolean, e.g. true).
2. If `type` is `fuel`, it must contain `burn_time` (integer).
3. If `type` is `tool`, it must contain `tool_type`, `durability`, `mining_speed`, `attack_damage`.
4. If a field is missing, fill in a reasonable default value. For example, `saturation` defaults to 0.3, `always_edible` defaults to true.

Please regenerate a valid blueprint. Output JSON only."""
            print(f"⚠️ Attempt {attempt} failed: {e.message}")
            current_input = f"{user_input}\n\n{feedback}"
        except json.JSONDecodeError as e:
            last_error = e
            feedback = f"The previously returned content is not valid JSON. Error: {e}. Please output JSON only, without any explanation."
            print(f"⚠️ Attempt {attempt} failed: {feedback}")
            current_input = f"{user_input}\n\n{feedback}"

    raise RuntimeError(f"After {max_retries} attempts, a valid blueprint still could not be generated. Last error: {last_error}")

Key improvements:

• The retry feedback directly lists the required fields and suggested defaults for each type.
• Handles jsonschema.ValidationError and json.JSONDecodeError separately.
• On each retry, concatenates the error message and the original input before sending to the LLM.

---

Step 3: Create the test script test_validator.py

Create at the project root:

"""Test the blueprint validation module."""

import json
from modsmith.blueprint.validator import generate_validated_blueprint

result = generate_validated_blueprint("Create an apple that restores 4 hunger points when eaten")
print(json.dumps(result, indent=2, ensure_ascii=False))

---

Step 4: Run the verification

Make sure the virtual environment is activated, then run:

python test_validator.py

Expected result:

• If it succeeds on the first attempt, prints ✅ Attempt 1 successfully generated a valid blueprint. and the blueprint JSON.
• If it fails on the first attempt, prints a warning and retries automatically, usually succeeding on the 2nd or 3rd attempt.
• The output JSON should contain fields such as saturation, always_edible, etc.

Example output:

⚠️ Attempt 1 failed: 'saturation' is a required property
✅ Attempt 2 successfully generated a valid blueprint.
{
  "mod_id": "example-mod",
  ...
  "items": [
    {
      "id": "healing_apple",
      "type": "food",
      "display_name_en": "Healing Apple",
      "display_name_zh": "治愈苹果",
      "nutrition": 4,
      "saturation": 0.3,
      "always_edible": true,
      "texture": "auto"
    }
  ]
}

---

Step 5: Common errors and solutions

Error	Cause	Solution
Still fails after multiple retries	LLM did not understand the feedback	Check whether the feedback is clear, increase max_retries, or switch to a stronger model
jsonschema.ValidationError not caught	Incorrect import path	Confirm import jsonschema is correct and validator.py is under modsmith/blueprint/
RuntimeError raised	Maximum retries reached	Check the last error, whether the Schema is too strict, or adjust the System Prompt
json.JSONDecodeError	LLM output is not JSON	Strengthen the "output JSON only" constraint in the System Prompt, or add cleanup logic

---

Step 6: Task 4 Acceptance Criteria

• The System Prompt in modsmith/llm/client.py has been supplemented with required fields and suggested defaults for each type.
• modsmith/blueprint/validator.py is implemented, containing validate_blueprint and generate_validated_blueprint.
• test_validator.py runs successfully and outputs a valid blueprint.
• When the LLM returns an invalid blueprint, it can automatically retry and eventually return a valid blueprint.
• When the maximum number of retries is reached, raises RuntimeError with the last error information.

After completing this task, your ModSmith will have a self-correcting blueprint generation pipeline. Next, you can proceed to Task 5 (project generator) and Task 6 (Java code generator).