3.8 Implementing the Texture Generator

The goal of this task is to generate a 16x16 pixel texture for each item in the blueprint, where the texture's shape, color, and pattern are determined by the visual field in the blueprint. Additionally, to make textures automatically match the user's mod requirements, this task also requires preliminary modifications to Task 1 (System Prompt) and Task 2 (Schema), so that the LLM proactively infers the visual field when generating a blueprint, instead of waiting for the user to mention textures.

Note: This task includes modifications to the artifacts of previous tasks. This is not about going back to modify old documents, but rather consolidating the necessary adjustments here when implementing Task 8.

Prerequisites

Completed Task 7: modsmith/generator/resources.py has already created the textures/item directory.
Pillow is installed (already declared in pyproject.toml; if not, run pip install pillow).

Step 1: Understand the three dimensions of visual features

Each item's texture is determined by three dimensions:

Dimension	Description	Example
shape	The item's basic outline	`round` (fruit), `blade` (sword), `crystal` (gem)
primary_color	Main color (hex)	`#dc2626` (red)
accent_color	Secondary color (highlight, outline)	`#fca5a5` (light red)
pattern	Overlay texture	`sparkle`, `glow`

shape enum (9 values):

Shape	Applicable type	Example item
`round`	Food, fruit	Apple, berry
`blade`	Tool (sword, dagger)	Sword
`axe`	Tool (axe)	Axe
`pickaxe`	Tool (pickaxe)	Pickaxe
`crystal`	Gem, ore	Ruby, diamond
`ingot`	Metal ingot	Iron ingot, gold ingot
`liquid`	Bottled liquid	Potion, fuel
`organic`	Other food	Bread, meat
`abstract`	Cannot be categorized	Mysterious substance

pattern enum (6 values): none, sparkle, glow, cracks, dots, stripes.

Fallback mechanism: If an item has no visual field (for example, an old blueprint), the procedural generator falls back to the MD5 hash scheme and produces a colored square. Zero breakage for existing pipelines.

Step 2: Preliminary modification — let the LLM proactively infer visual

Problem: If the System Prompt only says "the visual field is required" but the user's description does not include texture information, the LLM may output "visual": {} or omit it entirely, causing the texture generator to fall back to the fallback scheme, resulting in textures that still lack distinctiveness.

Solution: Append a "visual inference" rule to the System Prompt in Task 1, so that the LLM proactively infers suitable visual features based on the item's name, type, and effect. At the same time, in Task 2's Schema, make visual required, using validation to force the LLM to output it every time.

2.1 Modify Task 1's System Prompt

Open _build_system_prompt() in modsmith/llm/client.py, and append the following content after the existing visual feature requirements:

## Visual features must be proactively inferred by you

When users describe a mod, they usually don't mention what the texture looks like. **You must infer a reasonable visual field based on the item's name, type, and effect.** Do not leave it empty, and do not output "auto".

Inference method:

1. **First look at name keywords**:
   - Apple, berry, fruit, bread, meat → shape = round / organic
   - Gem, crystal, diamond, ore, shard → shape = crystal
   - Sword, blade, dagger → shape = blade
   - Axe, pickaxe → shape = axe / pickaxe
   - Potion, plasma, liquid, fuel → shape = liquid
   - Ingot, block, metal → shape = ingot

2. **Then look at effects to infer color**:
   - Healing, regeneration → red family (#dc2626)
   - Speed, night vision → cyan family (#22d3ee)
   - Poison, wither → dark green / purple family (#4d7c0f / #7e22ce)
   - Strength, attack → orange-red family (#ea580c)
   - Luck, fortune → gold family (#f59e0b)
   - Cold, frost → blue-white family (#93c5fd)

3. **Finally look at name adjectives**:
   - Sparkling, glowing → pattern = sparkle
   - Mysterious, shadow → pattern = glow
   - Cracked, ancient → pattern = cracks
   - Pure, smooth → pattern = none

4. **Fallback**: If none of the above can be inferred, use shape = abstract and infer primary_color from the item ID's hash.

Examples:

User input "Create an apple that heals when eaten"
→ Name contains "apple" → shape = round
→ Effect is "heal" → primary_color = #dc2626
→ accent_color = #86efac (the green highlight of the fruit)
→ pattern = sparkle

User input "Make a sparkling diamond"
→ Name contains "diamond" → shape = crystal
→ Name contains "sparkling" → pattern = sparkle
→ primary_color = #22d3ee (the cyan-blue of diamond)

Effect: The LLM will automatically infer suitable shapes and colors for descriptions like "healing apple," "mysterious potion," and "diamond sword," without the user needing to mention the texture in the description.

2.2 Modify Task 2's Schema

Open modsmith/blueprint/schema.json, and find the item object's required array and properties:

Original (visual optional):

"required": ["id", "type", "display_name_en"],
"properties": {
  "id": { ... },
  "type": { ... },
  "visual": {
    "type": "object",
    "properties": {
      "shape": { ... },
      "primary_color": { ... },
      "accent_color": { ... },
      "pattern": { ... },
      "notes": { ... }
    }
  }
}

Change to (visual required, and its internal shape and primary_color also required):

"required": ["id", "type", "display_name_en", "visual"],
"properties": {
  "id": { ... },
  "type": { ... },
  "visual": {
    "type": "object",
    "required": ["shape", "primary_color"],
    "properties": {
      "shape": {
        "type": "string",
        "enum": ["round", "blade", "axe", "pickaxe", "crystal",
                 "ingot", "liquid", "organic", "abstract"]
      },
      "primary_color": {
        "type": "string",
        "pattern": "^#[0-9a-fA-F]{6}$"
      },
      "accent_color": {
        "type": "string",
        "pattern": "^#[0-9a-fA-F]{6}$"
      },
      "pattern": {
        "type": "string",
        "enum": ["none", "sparkle", "glow", "cracks", "dots", "stripes"],
        "default": "none"
      },
      "notes": { "type": "string" }
    }
  }
}

Effect: If the LLM tries to omit visual, Schema validation will fail and trigger a retry until it fills it in. accent_color and pattern remain optional, because they can reasonably be omitted.

Step 3: Rewrite `modsmith/generator/textures.py`

Replace the file content with the following complete code:

"""Generate item textures: draw pixel art of different shapes based on visual features."""

import hashlib
from pathlib import Path

from PIL import Image, ImageDraw


# ============================================================
# Utility functions
# ============================================================

def _hex_to_rgb(hex_color: str) -> tuple[int, int, int]:
    """Convert '#dc2626' to (220, 38, 38)."""
    hex_color = hex_color.lstrip("#")
    return tuple(int(hex_color[i:i+2], 16) for i in (0, 2, 4))


def _color_from_id(item_id: str) -> tuple[int, int, int]:
    """Fallback: generate a deterministic color from the item ID."""
    digest = hashlib.md5(item_id.encode("utf-8")).hexdigest()
    return (int(digest[0:2], 16), int(digest[2:4], 16), int(digest[4:6], 16))


def _get_visual(item: dict) -> dict:
    """Extract visual features from the item definition, using fallbacks when missing.

    - If there is no primary_color, generate one from the item ID's MD5 hash.
    - If there is no accent_color, use a lighter version of the primary color.
    - If there is no shape, use "abstract".
    - If there is no pattern, use "none".
    """
    visual = dict(item.get("visual", {}))

    if not visual.get("primary_color"):
        r, g, b = _color_from_id(item["id"])
        visual["primary_color"] = f"#{r:02x}{g:02x}{b:02x}"

    if not visual.get("accent_color"):
        r, g, b = _hex_to_rgb(visual["primary_color"])
        visual["accent_color"] = (
            f"#{min(r+60,255):02x}{min(g+60,255):02x}{min(b+60,255):02x}"
        )

    visual.setdefault("shape", "abstract")
    visual.setdefault("pattern", "none")
    return visual


# ============================================================
# Shape drawing functions (each returns a 16x16 RGBA Image)
# ============================================================

def _draw_round(size: int, primary: tuple, accent: tuple) -> Image.Image:
    """Round/fruit: a circle with a highlight."""
    img = Image.new("RGBA", (size, size), (0, 0, 0, 0))
    draw = ImageDraw.Draw(img)
    margin = 2
    draw.ellipse(
        [margin, margin, size - margin - 1, size - margin - 1],
        fill=primary, outline=accent,
    )
    draw.ellipse([4, 4, 6, 6], fill=(255, 255, 255, 200))
    return img


def _draw_blade(size: int, primary: tuple, accent: tuple) -> Image.Image:
    """Sword/dagger: diagonal blade + hilt."""
    img = Image.new("RGBA", (size, size), (0, 0, 0, 0))
    draw = ImageDraw.Draw(img)
    draw.line([(4, size - 5), (size - 3, 4)], fill=primary, width=3)
    draw.line([(5, size - 6), (size - 4, 3)], fill=accent, width=1)
    draw.line([(2, size - 3), (5, size - 6)], fill=(101, 67, 33), width=2)
    draw.line([(3, size - 8), (8, size - 3)], fill=accent, width=1)
    return img


def _draw_axe(size: int, primary: tuple, accent: tuple) -> Image.Image:
    """Axe: wooden handle + axe head."""
    img = Image.new("RGBA", (size, size), (0, 0, 0, 0))
    draw = ImageDraw.Draw(img)
    draw.line([(5, size - 2), (size - 5, 4)], fill=(101, 67, 33), width=2)
    draw.polygon([(4, 4), (10, 2), (12, 7), (7, 9)], fill=primary, outline=accent)
    return img


def _draw_pickaxe(size: int, primary: tuple, accent: tuple) -> Image.Image:
    """Pickaxe: wooden handle + pick head."""
    img = Image.new("RGBA", (size, size), (0, 0, 0, 0))
    draw = ImageDraw.Draw(img)
    draw.line([(5, size - 2), (size - 5, 4)], fill=(101, 67, 33), width=2)
    draw.arc([2, 2, size - 4, 10], start=180, end=360, fill=primary, width=2)
    draw.arc([3, 3, size - 5, 9], start=180, end=360, fill=accent, width=1)
    return img


def _draw_crystal(size: int, primary: tuple, accent: tuple) -> Image.Image:
    """Gem/ore: rhombus crystal with facet highlights."""
    img = Image.new("RGBA", (size, size), (0, 0, 0, 0))
    draw = ImageDraw.Draw(img)
    center = size // 2
    draw.polygon([
        (center, 2), (size - 3, center),
        (center, size - 3), (3, center),
    ], fill=primary, outline=accent)
    draw.polygon([
        (center, 3), (center + 3, center), (center, center + 1),
    ], fill=(255, 255, 255, 120))
    return img


def _draw_ingot(size: int, primary: tuple, accent: tuple) -> Image.Image:
    """Metal ingot: a 3D rectangular block."""
    img = Image.new("RGBA", (size, size), (0, 0, 0, 0))
    draw = ImageDraw.Draw(img)
    draw.polygon(
        [(3, 6), (size - 4, 5), (size - 3, 9), (2, 10)],
        fill=accent, outline=primary,
    )
    draw.polygon(
        [(2, 10), (size - 3, 9), (size - 4, size - 3), (3, size - 2)],
        fill=primary, outline=accent,
    )
    return img


def _draw_liquid(size: int, primary: tuple, accent: tuple) -> Image.Image:
    """Bottled liquid: bottle outline + liquid."""
    img = Image.new("RGBA", (size, size), (0, 0, 0, 0))
    draw = ImageDraw.Draw(img)
    draw.rectangle([5, 7, size - 6, size - 3], fill=primary, outline=accent)
    draw.rectangle([7, 3, size - 8, 7], fill=accent)
    draw.rectangle([6, 2, size - 7, 3], fill=(101, 67, 33))
    return img


def _draw_organic(size: int, primary: tuple, accent: tuple) -> Image.Image:
    """Organic food: irregular blocky shape."""
    img = Image.new("RGBA", (size, size), (0, 0, 0, 0))
    draw = ImageDraw.Draw(img)
    draw.ellipse([2, 3, size - 3, size - 2], fill=primary, outline=accent)
    draw.ellipse([4, 5, 7, 8], fill=(255, 255, 255, 100))
    return img


def _draw_abstract(size: int, primary: tuple, accent: tuple) -> Image.Image:
    """Abstract: placeholder square with a dot in the center."""
    img = Image.new("RGBA", (size, size), (0, 0, 0, 0))
    draw = ImageDraw.Draw(img)
    draw.rectangle([1, 1, size - 2, size - 2], fill=primary, outline=accent)
    center = size // 2
    draw.ellipse(
        [center - 2, center - 2, center + 2, center + 2],
        fill=(255, 255, 255, 180),
    )
    return img


_SHAPE_DRAWERS = {
    "round": _draw_round,
    "blade": _draw_blade,
    "axe": _draw_axe,
    "pickaxe": _draw_pickaxe,
    "crystal": _draw_crystal,
    "ingot": _draw_ingot,
    "liquid": _draw_liquid,
    "organic": _draw_organic,
    "abstract": _draw_abstract,
}


# ============================================================
# Pattern overlay
# ============================================================

def _apply_pattern(img: Image.Image, pattern: str, accent: tuple) -> None:
    """Apply a pattern on top of the existing image."""
    draw = ImageDraw.Draw(img)
    size = img.width

    if pattern == "sparkle":
        for x, y in [(4, 4), (size - 6, 5), (5, size - 6)]:
            draw.point((x, y), fill=(255, 255, 255, 255))
            draw.point((x + 1, y), fill=(255, 255, 255, 180))
            draw.point((x, y + 1), fill=(255, 255, 255, 180))

    elif pattern == "glow":
        for i in range(1, 3):
            draw.rectangle(
                [i, i, size - 1 - i, size - 1 - i],
                outline=(accent[0], accent[1], accent[2], 100 - i * 30),
            )

    elif pattern == "cracks":
        draw.line([(3, 5), (7, 9)], fill=accent, width=1)
        draw.line([(7, 9), (5, 13)], fill=accent, width=1)

    elif pattern == "dots":
        for x in range(3, size - 3, 4):
            for y in range(3, size - 3, 4):
                draw.point((x, y), fill=accent)

    elif pattern == "stripes":
        for y in range(3, size - 3, 3):
            draw.line([(2, y), (size - 3, y)], fill=accent, width=1)


# ============================================================
# Entry points
# ============================================================

def generate_texture(item: dict, output_path: Path, size: int = 16) -> None:
    """Generate a single texture based on the item's visual features.

    Args:
        item: Item definition containing id, type, visual, etc.
        output_path: Output PNG path.
        size: Texture side length, default 16.
    """
    visual = _get_visual(item)

    primary = _hex_to_rgb(visual["primary_color"])
    accent = _hex_to_rgb(visual["accent_color"])
    shape = visual["shape"]
    pattern = visual["pattern"]

    drawer = _SHAPE_DRAWERS.get(shape, _draw_abstract)
    img = drawer(size, primary, accent)

    if pattern != "none":
        _apply_pattern(img, pattern, accent)

    output_path.parent.mkdir(parents=True, exist_ok=True)
    img.save(output_path, "PNG")


def generate_all_textures(blueprint: dict, project_dir: Path) -> None:
    """Generate textures for all items in the blueprint.

    Args:
        blueprint: Blueprint dictionary.
        project_dir: Root directory of the generated project.
    """
    mod_id = blueprint["mod_id"]
    textures_dir = (
        project_dir / "src" / "main" / "resources"
        / "assets" / mod_id / "textures" / "item"
    )

    for item in blueprint["items"]:
        output_path = textures_dir / f"{item['id']}.png"
        generate_texture(item, output_path)
        shape = item.get("visual", {}).get("shape", "abstract")
        print(f"  🎨 Generated texture: {output_path.name} ({shape})")

Step 4: Update the example blueprints

In modsmith/blueprint/examples/, add a visual field to each example:

01_basic_item.json:

{
  "id": "ruby",
  "type": "basic",
  "display_name_en": "Ruby",
  "display_name_zh": "红宝石",
  "visual": {
    "shape": "crystal",
    "primary_color": "#dc2626",
    "accent_color": "#fca5a5",
    "pattern": "sparkle"
  },
  "texture": "auto"
}

02_food_item.json:

{
  "id": "healing_apple",
  "type": "food",
  "display_name_en": "Healing Apple",
  "display_name_zh": "治愈苹果",
  "nutrition": 4,
  "saturation": 0.3,
  "always_edible": true,
  "effects": [{"effect": "regeneration", "duration_ticks": 200, "amplifier": 0}],
  "visual": {
    "shape": "round",
    "primary_color": "#dc2626",
    "accent_color": "#86efac",
    "pattern": "sparkle"
  },
  "texture": "auto"
}

03_tool_item.json:

{
  "id": "guidite_sword",
  "type": "tool",
  "display_name_en": "Guidite Sword",
  "display_name_zh": "基迪特剑",
  "tool_type": "sword",
  "durability": 455,
  "mining_speed": 5.0,
  "attack_damage": 1.5,
  "visual": {
    "shape": "blade",
    "primary_color": "#94a3b8",
    "accent_color": "#cbd5e1",
    "pattern": "none"
  },
  "texture": "auto"
}

Step 5: Ensure the project generator calls the texture generator

In modsmith/generator/project.py's generate_project(), generate_all_textures() is already called. Confirm the code exists:

from modsmith.generator.textures import generate_all_textures

def generate_project(blueprint: dict, output_dir: Path) -> None:
    render_project(blueprint, output_dir)
    generate_java_items(blueprint, output_dir)
    generate_resources(blueprint, output_dir)
    generate_all_textures(blueprint, output_dir)
    print(f"✅ Project fully generated at: {output_dir}")

Step 6: Create the test script `test_textures.py`

"""Test the texture generator."""

from pathlib import Path

from modsmith.generator.textures import generate_all_textures

blueprint = {
    "mod_id": "example-mod",
    "items": [
        {
            "id": "ruby",
            "type": "basic",
            "visual": {
                "shape": "crystal",
                "primary_color": "#dc2626",
                "accent_color": "#fca5a5",
                "pattern": "sparkle",
            },
        },
        {
            "id": "healing_apple",
            "type": "food",
            "visual": {
                "shape": "round",
                "primary_color": "#dc2626",
                "accent_color": "#86efac",
                "pattern": "sparkle",
            },
        },
        {
            "id": "guidite_sword",
            "type": "tool",
            "visual": {
                "shape": "blade",
                "primary_color": "#94a3b8",
                "accent_color": "#cbd5e1",
                "pattern": "none",
            },
        },
        {
            "id": "quark_gluon_plasma",
            "type": "basic",
            "visual": {
                "shape": "liquid",
                "primary_color": "#a855f7",
                "accent_color": "#c084fc",
                "pattern": "glow",
            },
        },
        {
            "id": "mystery_dust",
            "type": "basic",
            # Intentionally no visual, to test the fallback
        },
    ],
}

output_dir = Path("./test_textures_output")
generate_all_textures(blueprint, output_dir)
print(f"Textures generated at: {output_dir}")

Run:

python test_textures.py

Expected result: 5 distinctly different textures are generated under test_textures_output/src/main/resources/assets/example-mod/textures/item/.

Step 7: End-to-end test

python test_generator.py

Check the textures:

ls generated_project/src/main/resources/assets/example-mod/textures/item/

Compile verification:

cd generated_project
./gradlew build

In-game verification (optional):

./gradlew runClient

Enter creative mode and check whether each item's texture is distinctive.

Step 8: Verify whether "automatic texture matching" works

Restart the Web service and say in Chat:

"I want to make an apple that heals when eaten"

The 【Requirement Summary】 output by Chat only contains the item description, without visual information. Click "⚡ Summarize Requirements" → "⚡ Generate Mod", then check blueprint.json:

{
  "id": "healing_apple",
  "type": "food",
  "display_name_zh": "治愈苹果",
  "visual": {
    "shape": "round",
    "primary_color": "#dc2626",
    "accent_color": "#86efac",
    "pattern": "sparkle"
  }
}

The generated healing_apple.png should be a red round fruit with a green highlight and sparkle points.

Try another:

"Make a mysterious purple potion"

It should automatically infer:

"visual": {
  "shape": "liquid",
  "primary_color": "#a855f7",
  "accent_color": "#c084fc",
  "pattern": "glow"
}

The user only describes the item and its effect, and the AI automatically matches a suitable texture. This is the effect this task aims to achieve.

Step 9: Common issues

Problem	Cause	Solution
All textures are abstract	LLM did not output `visual.shape`	Check whether the System Prompt modification in Step 2 has been applied
Blueprint validation fails but the LLM keeps not filling in visual	Schema did not set `required`	Check the Schema modification in Step 2
Compilation passes but in-game textures are blurry	Wrong size	Ensure the generated size is 16x16
Shape does not match expectations	Name inference rules do not match	Add more keywords to the inference rules in the System Prompt
Fallback colors are unsatisfactory	MD5 hash randomness	Explicitly provide `primary_color` for the item
Want newly added shapes to take effect too	Only added the drawing function	Also need to register it in `_SHAPE_DRAWERS` and add it to the Schema's `enum`

Step 10: Task 8 Acceptance Criteria

After completing this task, your ModSmith will be able to generate distinctive item textures and automatically match them to user requirements — when the user says "healing apple," the system automatically generates a red round fruit; when the user says "mysterious purple potion," the system automatically generates a purple bottle with a glowing effect.

Future Extension Directions

More shapes: add bow, wand, helmet, boots, bowl, etc.
Dynamic coloring: automatically adjust colors based on item effects (e.g., potion effect → glow).
Multi-layer composition: an item can stack multiple shapes (e.g., "gem + key").
AI image generation: fall back to Zhipu CogView for complex items (requires a new interface; higher cost).
User customization: allow users to upload or choose textures in the Web UI.

Navigation Menu

3.8 Implementing the Texture Generator

Prerequisites

Step 1: Understand the three dimensions of visual features

Step 2: Preliminary modification — let the LLM proactively infer visual

2.1 Modify Task 1's System Prompt

2.2 Modify Task 2's Schema

Step 3: Rewrite `modsmith/generator/textures.py`

Step 4: Update the example blueprints

Step 5: Ensure the project generator calls the texture generator

Step 6: Create the test script `test_textures.py`

Step 7: End-to-end test

Step 8: Verify whether "automatic texture matching" works

Step 9: Common issues

Step 10: Task 8 Acceptance Criteria

Future Extension Directions

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Clone this wiki locally

3.8 Implementing the Texture Generator

Prerequisites

Step 1: Understand the three dimensions of visual features

Step 2: Preliminary modification — let the LLM proactively infer visual

2.1 Modify Task 1's System Prompt

2.2 Modify Task 2's Schema

Step 3: Rewrite modsmith/generator/textures.py

Step 4: Update the example blueprints

Step 5: Ensure the project generator calls the texture generator

Step 6: Create the test script test_textures.py

Step 7: End-to-end test

Step 8: Verify whether "automatic texture matching" works

Step 9: Common issues

Step 10: Task 8 Acceptance Criteria

Future Extension Directions

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Clone this wiki locally

Step 3: Rewrite `modsmith/generator/textures.py`

Step 6: Create the test script `test_textures.py`