Processing and Upscaling

Processing and Upscaling

Relevant source files

The following files were used as context for generating this wiki page:

• clarity/processing/engine.py

This page provides a high-level overview of the neural upscaling engine and the specialized per-role image processing modules within XIVUpscaler. The system bridges high-performance deep learning super-resolution with game-specific texture constraints (such as unit-length normals and UI alpha channels).

For detailed information, refer to the child pages:

• Inference Engine, Model Slots and Tiling
• Role-Specific Processing Pipelines

---

Processing Architecture Overview

The processing subsystem is responsible for taking decoded texture arrays, dispatching them through appropriate neural network slots or fallback scalers, and applying role-specific image manipulations (such as normal map channel re-mapping or UI color/alpha handling).

graph TD
    subgraph "Natural Language Space: Processing Pipeline"
      A["Texture Source"] --> B["Role Dispatcher"]
      B --> C["Neural Upscaling Engine"]
      C --> D["Role-Specific Post-Processing"]
    end

    subgraph "Code Entity Space: Implementation"
      A1["Tensor / NumPy Array"] --> B1["clarity/processing/roles.py"]
      B1 --> C1["clarity/processing/engine.py:Engine"]
      C1 --> D1["clarity/processing/color.py & normals.py"]
    end

    A --> A1
    B --> B1
    C --> C1
    D --> D1

Loading

Sources: clarity/processing/engine.py:1-154

---

5.1 Inference Engine, Model Slots and Tiling

The neural inference engine is implemented in clarity/processing/engine.py. It provides support for spandrel-loaded ESRGAN-family models, half-precision (fp16) execution on CUDA devices, batched tile inference, automatic CUDA out-of-memory (OOM) recovery, and a Lanczos CPU fallback mode clarity/processing/engine.py:1-88.

Key aspects covered in the child page:

• Model Registries: Configuration of DEFAULT_REGISTRY and RECOMMENDED_REGISTRY mapping logical slots (bc1clean, normal, color, face, skin, hair, ui, mask) to weight files clarity/processing/engine.py:25-47.
• Tiling and Batching: Processing large textures in manageable tiles with configurable overlap padding (tile, pad, tile_batch) clarity/processing/engine.py:53-76.
• Robustness: Dynamic CUDA OOM adaptation and seamless fallback to CPU-based Lanczos filtering when models or hardware capabilities are absent clarity/processing/engine.py:130-142.

For details, see Inference Engine, Model Slots and Tiling.

Sources: clarity/processing/engine.py:1-154

---

5.2 Role-Specific Processing Pipelines

Texture processing in Final Fantasy XIV requires different handling depending on the texture's semantic role in materials. The role dispatcher routes image data to specialized modules that handle specific pixel transformations clarity/processing/engine.py:1-154.

Key aspects covered in the child page:

• Color Pipelines: Handling general diffuse, specular, and base color maps (color.py).
• Normal Maps: Preserving unit-length vectors and managing tangent-space constraints (normals.py).
• Masks and UI: Processing scalar channels, UI elements, and icon sheets with dedicated alpha channel handling (masks.py, ui.py).
• Utilities: Shared image manipulation helpers (utils.py) and dispatch logic (roles.py).

For details, see Role-Specific Processing Pipelines.

Sources: clarity/processing/engine.py:1-154