Penumbra Mod Generation

Relevant source files

The following files were used as context for generating this wiki page:

Purpose and Scope

This page documents the Penumbra packaging pipeline implemented in clarity/packaging/penumbra.py. The system transforms processed texture assets from the SQLite manifest and scratch directories into structured Penumbra mods. This includes generating meta.json configuration files conforming to Penumbra FileVersion 4, managing single-select tier option groups, enforcing strict ASCII-only naming constraints to prevent game ANSI-codepage loading failures, managing sort-order collection layouts, and handling icon twins and .pmp archive creation.

Sources: [clarity/packaging/penumbra.py:1-9](https://github.com/off-cmd/XIVUpscaler/blob/5c9360ae/clarity/packaging/penumbra.py?plain=1#L1-L9), [tests/test_penumbra_names.py:1-8](https://github.com/off-cmd/XIVUpscaler/blob/5c9360ae/tests/test_penumbra_names.py?plain=1#L1-L8)

Mod Taxonomy and Family Mapping (`MODS` Map)

Texture families generated during the pipeline are aggregated into logical mod packages rather than one-to-one family folders. This groups related assets into cohesive units for end users (e.g., all equipment, accessories, and weapons share a single mod).

The module defines the MODS dictionary, mapping human-readable mod names to tuples of asset families and default priorities [clarity/packaging/penumbra.py:23-42]:

Clarity - Gear: ["equipment", "accessory", "weapon"] (Priority 80) [clarity/packaging/penumbra.py:24]
Clarity - Monsters & Demihumans: ["monster", "demihuman"] (Priority 81) [clarity/packaging/penumbra.py:25-29]
Clarity - Human (skin, faces, tails, ears): ["human-body", "human-face", "human-tail", "human-zear"] (Priority 82) [clarity/packaging/penumbra.py:26-29]
Clarity - World: ["bg", "bg-hou", "bg-ind", "bgcommon", "bgcommon-hou", "bgcommon-mji"] (Priority 83) [clarity/packaging/penumbra.py:33-36]
Clarity - UI & HUD: ["ui-icon", "ui-uld", "ui-other", "common"] (Priority 84) [clarity/packaging/penumbra.py:41]

The ASCII-Only Naming Constraint

Mod names, sort order paths, and internal folders must strictly use ASCII characters [tests/test_penumbra_names.py:1-8](https://github.com/off-cmd/XIVUpscaler/blob/5c9360ae/tests/test_penumbra_names.py?plain=1#L1-L8). Early versions utilized an em-dash (—, Unicode U+2014, bytes E2 80 94) in names like Clarity — Gear. Because the game's native file loader parses paths using the system ANSI code page, this character was misread as Clarity â€” Gear, causing every asset redirect to fail, resulting in invisible characters and client crashes on redraw [clarity/packaging/penumbra.py:16-22].

To maintain backward compatibility with older configurations, LEGACY_NAMES maps old em-dash variants to the canonical ASCII MODS keys [clarity/packaging/penumbra.py:45].

graph TD
  A["Manifest Asset Families"] --> B["mod_for_family()"]
  B --> C["MODS Registry"]
  C -->|Equipment| D["Clarity - Gear"]
  C -->|Monster| E["Clarity - Monsters & Demihumans"]
  C -->|Human| F["Clarity - Human (skin, faces, tails, ears)"]
  C -->|World/BG| G["Clarity - World"]
  C -->|UI/Common| H["Clarity - UI & HUD"]
  
  style B fill:none,stroke:#000,stroke-width:2px
  style C fill:none,stroke:#000,stroke-width:2px

Figure 1: Mapping asset families to Penumbra mod containers via mod_for_family().

Sources: [clarity/packaging/penumbra.py:11-55], [tests/test_penumbra_names.py:1-24]

Tier Option Groups and Fallback Logic (`options_for`)

Tiers produced by the encoding pipeline (native, 2x, 4x) are structured as a single-select option group in Penumbra [clarity/packaging/penumbra.py:1-4].

Because Penumbra's "Single" select option type applies only the files declared within the chosen option and no lower layer, listing solely the textures that literally reached a given tier (e.g., 4x) would cause textures capped by source resolution limitations to vanish when selecting higher qualities [clarity/packaging/penumbra.py:62-78].

The function options_for() resolves this by building cumulative option definitions. Each tier option inherits all assets from lower tiers up to its ceiling:

# Conceptual flow inside options_for(tiers_files)
# For '4x', merge native + 2x + 4x textures so option coverage is complete.

TIER_ORDER: Ordered as ["native", "2x", "4x"] [clarity/packaging/penumbra.py:47].
TIER_LABEL: Mapped as native: "Native (BC7 re-encode)", 2x: "2×", 4x: "4×" [clarity/packaging/penumbra.py:46].

Sources: [clarity/packaging/penumbra.py:46-88]

Meta JSON FileVersion 4 Structure (`write_mod_json`)

Penumbra mods generated by write_mod_json() utilize FileVersion 4 [clarity/packaging/penumbra.py:103-153]. The generated meta.json file contains mod metadata, author tags, and a single-select group containing options for each generated tier.

graph TD
  A["meta.json (FileVersion: 4)"] --> B["Mod Metadata (Name, Author, Version)"]
  A --> C["DefaultData (Files, FileSwaps, Manipulations)"]
  A --> D["Groups (Single-Select Tier Group)"]
  D --> E["Option: Native (BC7 re-encode)"]
  D --> F["Option: 2x"]
  D --> G["Option: 4x"]

  style A fill:none,stroke:#000,stroke-width:2px
  style D fill:none,stroke:#000,stroke-width:2px

Figure 2: Structure of generated meta.json files.

write_mod_json() cleans up legacy configuration files (default_mod.json, group_001_tier.json) if present in the mod directory before writing the updated meta.json [clarity/packaging/penumbra.py:154-159].

Sources: [clarity/packaging/penumbra.py:103-161]

Sort Order, Collection Merging, and Icon Twins

The packaging system also handles integration into XIVPenumbra layouts through collection merging and sort_order.json configuration [clarity/packaging/penumbra.py:6-8].

Folder Hierarchy: Placed under 0 Base upscales (everyone)/Clarity tiers (G4)/ using ASCII-safe path constraints [clarity/packaging/penumbra.py:48], [tests/test_penumbra_names.py:21-40].
Reserved Paths: Specific vanilla texture paths such as dummy textures, white/black references, and skin masks are filtered or handled specially via RESERVED_PATHS to prevent corrupting engine state [clarity/packaging/penumbra.py:90-100].
Icon Twins: Automatically generates parallel redirected paths for UI icons that require upscaled asset twins under interface catalog directories (9 Interface/Icons - upscaled (G6)/...) [tests/test_penumbra_names.py:32-40].