Maintenance Commands

Maintenance Commands: requeue, reclassify, fingerprint, audit, modup

Relevant source files

The following files were used as context for generating this wiki page:

Purpose and Scope

This wiki page covers the maintenance command-line interface (CLI) subcommands in XIVUpscaler/clarity: requeue, reclassify, fingerprint, audit, and modup. These utilities allow operators to manage SQLite manifest state (clarity/manifest.py), handle patch deltas across game updates, audit processing constraints, and upscale third-party Penumbra mods via clarity/modtex.py and clarity/uldparts.py without requiring a full database re-enumeration.

Sources: clarity/cli.py:1-172](), clarity/modtex.py:1-172](), `clarity/uldparts.py:1-154]()

1. Requeue Semantics and Patch-Delta Preservation

The requeue maintenance command resets asset rows in the SQLite manifest back to a planned status based on filtering criteria (such as --failed, --skipped, or specific families/roles). A critical requirement of patch-delta handling is that resetting failed or outdated rows must not erase the changed at <version> delta markers tests/test_manifest_requeue.py:1-4. These markers are written by fingerprinting routines when game files change between patches, and they are consumed by clarity run --since to target delta reprocessing.

When requeue executes, it selectively clears failure notes and reset states while retaining any substring starting with changed at in the row's note field, binding parameters securely to prevent SQL injection tests/test_manifest_requeue.py:77-88.

graph TD
    A["OperatorCLIArgs"] --> B["cmd_requeue"]
    B --> C["SQLQueryDatabase"]
    C --> D{"Containschangedatmarker?"}
    D -- Yes --> E["PreserveDeltaMarkerInNote"]
    D -- No --> F["ClearNoteField"]
    E --> G["SetStatusPlanned"]
    F --> G
    G --> H["CommitManifest"]

    subgraph "Files"
    B -.-> I["tests/test_manifest_requeue.py"]
    end

Sources: clarity/cli.py](), tests/test_manifest_requeue.py:1-96]()

2. Reclassification and Fingerprint Diffs

When FFXIV patches release, game assets change their contents or classifications.

Fingerprinting (cmd_fingerprint): Computes or compares hashes of game files against existing manifest records, marking rows that have changed with changed at <version> strings tests/test_manifest_requeue.py:1-4.
Reclassification (cmd_reclassify): Re-runs classification rules (mf.classify()) over existing database entries. If a texture's inferred role or processing family shifts due to policy updates, the manifest is updated in-place.

Sources: clarity/cli.py](), clarity/manifest.py]()

3. Database Auditing (`cmd_audit`)

The audit command checks manifest integrity, path validity, and rule compliance. Crucially, audit queries and validation bounds enforce pipeline invariants directly against code constants rather than hardcoded literals. For example, edge dimension limits check against roles.MAX_EDGE_OUT rather than raw numbers like 8192 tests/test_manifest_requeue.py:90-95.

Key checks performed by cmd_audit:

Verification that source files exist and have valid dimensions.
Detection of orphaned records or unhandled formats.
Validation that output scaling constraints conform to role-specific policy limits.

Sources: clarity/cli.py](), tests/test_manifest_requeue.py:90-96]()

4. Third-Party Mod Upscaling (`modtex.py` & `uldparts.py`)

The modup command and its backing modules (clarity/modtex.py and clarity/uldparts.py) upscale textures contained inside existing third-party Penumbra mod folders without requiring them to be ingested into the main SQLite manifest.

Mod Mirroring and Deduplication (`modtex.py`)

upscale_mod() mirrors a mod directory to <out>/<dir name> (upscaled), copying JSON files and metadata unchanged while replacing .tex files with upscaled twins clarity/modtex.py:1-93.

Security Constraint (contained): The contained() function verifies that all relative paths defined in Penumbra JSON Files maps do not escape the mod root via rooted paths (/path) or directory traversal (..\..) clarity/modtex.py:33-53.
Source-Byte Caching: Identical textures (e.g., icons shipped multiple times across gearsets) are hashed with SHA-256 (hashlib.sha256) and processed exactly once, reusing the encoded result clarity/modtex.py:127-134.

UI Atlas Protection and ULD Parts (`uldparts.py`)

UI textures (ui/uld/*.tex) are atlases packed with no gutters between sprites (e.g., adjacent HUD wheel halves). Upscaling the sheet as a single image causes bleeding across seams. clarity/uldparts.py solves this:

ULD Parsing: Loads ULD definitions using _uldfile(), which dynamically imports uldfile from the ffxiv-rendering knowledge-base tools directory if mounted clarity/uldparts.py:60-89.
Reflect Padding (PAD = 8): Each sprite rectangle is cropped with 8 pixels of reflect padding, upscaled independently, and then cropped back to prevent edge artifacts from contaminating adjacent sprites clarity/uldparts.py:1-24.
Compositing: Parts are composited back into the atlas largest-first to correctly resolve overlapping rectangles clarity/uldparts.py:28-31.

graph TD
    A["ModDirectoryPath"] --> B["mod_files"]
    B --> C["containedCheck"]
    C --> D["HashSourceBytesSHA256"]
    D --> E{"CacheHit?"}
    E -- Yes --> F["WriteCachedTwin"]
    E -- No --> G{"IsUIULDTexture?"}
    G -- Yes --> H["uldparts.build_index"]
    H --> I["ExtractSpriteWithReflectPadding"]
    I --> J["UpscaleRolePipeline"]
    G -- No --> J
    J --> K["EncodeTextureTexconv"]
    K --> L["WriteUpscaledTwin"]

    subgraph "CodeModules"
    B -.-> M["clarity/modtex.py"]
    H -.-> N["clarity/uldparts.py"]
    end

Sources: clarity/modtex.py:1-172](), clarity/uldparts.py:1-154]()

Home · Repository · Migrated from DeepWiki