Getting Started and Configuration

Relevant source files

The following files were used as context for generating this wiki page:

This page details the installation, Python runtime constraints, environment variable configuration, path resolution logic, model registry layout, and external tool dependencies for the clarity pipeline.

Python Version and Installation

The pipeline requires Python 3.11 or 3.12. The runtime ceiling is strictly bound to Python 3.12 because the texture2ddecoder dependency (required for native BC7 decoding) does not publish pre-compiled wheels for Python 3.13 pyproject.toml:24-25.

Installation is managed via uv:

uv sync
uv run clarity where

Alternatively, the package can be installed as a standard distribution via uv pip install xivupscaler or pip install xivupscaler, exposing the clarity CLI command script README.md:43-52.

Sources: README.md:32-52, pyproject.toml:24-37

Path Resolution Architecture (`clarity/paths.py`)

All directory and file locations used throughout the pipeline are defined as defaults in clarity/paths.py and can be overridden by corresponding environment variables.

The project root directory is determined in strict precedence order by _project_root() clarity/paths.py:55-63:

The CLARITY_PROJECT environment variable, if explicitly set.
The parent directory of the clarity package if a pyproject.toml file is present (indicating a source checkout).
The current working directory (os.getcwd()).

graph TD
    Start["Call _project_root()"] --> CheckExplicit{"Environment variable CLARITY_PROJECT set?"}
    CheckExplicit -- Yes --> ReturnExplicit["Return os.path.abspath(explicit)"]
    CheckExplicit -- No --> CheckCheckout{"Package parent has pyproject.toml?"}
    CheckCheckout -- Yes --> ReturnCheckout["Return checkout path (Source Root)"]
    CheckCheckout -- No --> ReturnCWD["Return os.getcwd() (Current Working Directory)"]

    style Start fill:none,stroke:#333,stroke-width:2px
    style CheckExplicit fill:none,stroke:#333,stroke-width:2px
    style CheckCheckout fill:none,stroke:#333,stroke-width:2px
    style ReturnExplicit fill:none,stroke:#333,stroke-width:2px
    style ReturnCheckout fill:none,stroke:#333,stroke-width:2px
    style ReturnCWD fill:none,stroke:#333,stroke-width:2px

Figure 1: Project root resolution flow in clarity/paths.py.

Sources: clarity/paths.py:55-63

Environment Variables and Path Configuration

Every core system path is governed by an environment variable. If unset, it resolves to a structured default relative to PROJECT.

Variable	Default Path	Code Attribute	Description
`CLARITY_DB`	`build-output/manifest.sqlite`	`clarity.paths.DB`	SQLite database storing state, rows, and progression clarity/paths.py:92
`CLARITY_MODELS`	`analysis-specimens/models/`	`clarity.paths.MODELS`	Directory containing external neural network model weights clarity/paths.py:93
`CLARITY_REGISTRY`	`clarity/models/registry.json`	`clarity.paths.REGISTRY`	JSON manifest mapping pipeline slots to weight files clarity/paths.py:94
`CLARITY_TEXCONV`	`vendor-tools/texconv/texconv.exe`	`clarity.paths.TEXCONV`	Microsoft DirectXTex `texconv.exe` executable clarity/paths.py:95-97
`CLARITY_SCRATCH`	`temp-scratch/texconv/`	`clarity.paths.SCRATCH`	Scratch directory for uncompressed RGBA intermediate textures clarity/paths.py:90
`CLARITY_FINGERPRINTS`	`hash-manifests/texture-fingerprints.tsv`	`clarity.paths.FINGERPRINTS`	Tab-separated fingerprint registry for patch change tracking clarity/paths.py:98-101
`CLARITY_PATHLIST`	newest `CurrentPathList*.gz`	`clarity.paths.newest_pathlist()`	ResLogger pathlist file for resolving unnamed textures clarity/paths.py:105-114

Special Consideration: `CLARITY_SCRATCH`

The scratch directory (SCRATCH) is critical for performance and disk health. During upscaling, texio writes uncompressed RGBA DDS scratch files and processed outputs. For a 2048-square source, a 4x upscale pass writes hundreds of megabytes of intermediate data per texture clarity/paths.py:77-89. Setting CLARITY_SCRATCH to a RAM disk or high-end scratch SSD avoids excessive write churn on standard system drives.

graph TD
    EnvironmentVariables["Environment Variables (CLARITY_*)"] --> PathResolutionModule["clarity/paths.py"]
    PathResolutionModule --> DBEntity["DB: manifest.sqlite"]
    PathResolutionModule --> ModelEntity["MODELS: analysis-specimens/models/"]
    PathResolutionModule --> RegEntity["REGISTRY: clarity/models/registry.json"]
    PathResolutionModule --> TexconvEntity["TEXCONV: texconv.exe"]
    PathResolutionModule --> ScratchEntity["SCRATCH: temp-scratch/texconv/"]
    PathResolutionModule --> FPEntity["FINGERPRINTS: texture-fingerprints.tsv"]
    PathResolutionModule --> PathlistEntity["RESLOGGER: newest_pathlist()"]

    style EnvironmentVariables fill:none,stroke:#333,stroke-width:2px
    style PathResolutionModule fill:none,stroke:#333,stroke-width:2px
    style DBEntity fill:none,stroke:#333,stroke-width:2px
    style ModelEntity fill:none,stroke:#333,stroke-width:2px
    style RegEntity fill:none,stroke:#333,stroke-width:2px
    style TexconvEntity fill:none,stroke:#333,stroke-width:2px
    style ScratchEntity fill:none,stroke:#333,stroke-width:2px
    style FPEntity fill:none,stroke:#333,stroke-width:2px
    style PathlistEntity fill:none,stroke:#333,stroke-width:2px

Figure 2: Mapping of environment configuration hooks to internal attributes defined in clarity/paths.py.

Sources: clarity/paths.py:32-132, tests/test_paths.py:1-68

Model Registry (`registry.json`)

The pipeline does not hardcode neural network filenames. Instead, clarity/models/registry.json provides a declarative mapping from logical pipeline processing slots to specific weight filenames README.md:116-121.

{
 "bc1clean": "1x_BC1-smooth2.pth",
 "normal": "4x-Normal-RG0-BC7.pth",
 "normal_bc1": "4x-Normal-RG0-BC1.pth",
 "color": "4x-PBRify_UpscalerV4.pth",
 "color_v3": "4x-PBRify_RPLKSRd_V3.pth",
 "mask": "4x-PBRify_UpscalerSPANV4.pth",
 "face": "4xFaceUpDAT.pth",
 "skin": "x1_ITF_SkinDiffDDS_v1.pth",
 "ui": "4x-UltraSharpV2.safetensors"
}

Model files must be placed in MODELS (CLARITY_MODELS). An alternative registry.json placed alongside the weight files overrides the package-bundled defaults README.md:118-121.

Sources: README.md:116-132, clarity/models/registry.json:1-11

External Tools and Diagnostics

`texconv` Dependency

The pipeline relies on Microsoft's texconv utility (from the DirectXTex repository) for hardware-accelerated BC7/BC3 encoding.

A release build that leverages DirectCompute must be used README.md:38.
Debug builds or missing DirectCompute support cause texconv to silently fall back to single-threaded or CPU-bound loops, severely degrading performance.
Run clarity probe to execute diagnostic checks on texconv and hardware readiness README.md:38.

Pathlist Ingestion

To map anonymous or unindexed game assets, clarity checks analysis-specimens/reslogger/ for ResLogger compressed pathlists (CurrentPathList*.gz) via newest_pathlist() clarity/paths.py:105-114.

Configuration Diagnostics (`clarity where`)

Invoking clarity where invokes clarity.paths.describe(), which formats a status table showing the resolved absolute paths for all core directories and flags missing configuration files or directories clarity/paths.py:117-132.

Sources: README.md:32-42, clarity/paths.py:105-132, tests/test_paths.py:64-68

Home · Repository · Migrated from DeepWiki