XGDTool v1.3.0: High-Performance Engine Overhaul

XGDTool v1.3.0 — High-Performance Engine Overhaul

Version 1.3.0 is a comprehensive High-Performance Engine Overhaul by Ashnar2602. Every supported disc and archive format has been refactored for maximum throughput across all available CPU cores and modern NVMe/SSD storage, eliminating micro-I/O bottlenecks, single-threaded compression stalls, and redundant disk re-reads while preserving 100% bit-for-bit format integrity and emulator compatibility.

---

Key Improvements & Optimizations

• High-Throughput 2 MB Chunked I/O (XisoWriter & ImageReader):

  • Replaced legacy 2 KB micro-sector operations with 2 MB sequential batches (CHUNK_SECTORS = 1024).
  • Syscall count for reading and writing standard 8 GB discs reduced by over 99.9% (from ~4,000,000 to ~4,000).
  • In-memory OG Xbox scrub padding zeroing eliminates extraneous disk seeks.
  • Eliminated redundant seekg() repositioning during sequential reads, keeping OS read-ahead caches hot.
  • Upgraded global transfer buffer XGD::BUFFER_SIZE from 64 KB to 2 MB.

• Zero-Overhead Streaming Verification Checksums (ChecksumHelper):

  • CRC32 (zlib), MD5 (OpenSSL), and hardware-accelerated SHA-1 (SHA-NI) are now accumulated in RAM on the fly during image writing.
  • Output verification checksum calculation overhead reduced to 0.0 seconds for ISO creation, eliminating the 8 GB disc re-read entirely.
  • Upgraded fallback reader to a 4 MB streaming buffer operating at RAM speeds (4–6 GB/s) from the OS page cache.

• Lock-Free Multithreaded Compression Engine (CSOWriter & CCIWriter):

  • Replaced single-sector micro-queues, mutex locks, and std::promise heap allocations with a coarse-grained batch coordinator (BatchContext).
  • Worker threads steal 8-sector work slices using lock-free atomic fetch_add, saturating all CPU cores without mutex contention during compression.
  • Reusable preallocated batch buffers eliminate millions of heap allocations.

• High-Speed Multithreaded Zstd Engine (ZARWriter & external/ZArchive):

  • Transformed ZArchive compression from single-threaded into a fully parallel multi-core engine with persistent per-thread ZSTD_CCtx* contexts.
  • 4 MB data block batches are compressed concurrently across all CPU threads and written sequentially to maintain 100% compliance with Cemu/ZArchive specifications, block offset tables, and SHA-256 integrity signatures.
  • Fully hooked up GUI & CLI compression level presets to Zstd (Default = Level 2, Fast = Level 1, Balanced = Level 3, Maximum = Level 6).
  • Confirmed via technical benchmarks that Zstd decompression speed in emulators remains flat and ultra-fast (~2 GB/s) regardless of compression level.

• Contiguous Block I/O & Fast Hashtables (GoDWriter):

  • Replaced 2 KB single-sector reads and writes with contiguous block chunks of up to 816 KB (408 sectors) per syscall via get_contiguous_sectors().
  • Upgraded Sub-Hash Table (SHT) computation to read all 204 blocks (816 KB) in a single file read before hashing in memory.

• High-Performance Chunked Readers (CSOReader, CCIReader, GoDReader):

  • Implemented multi-sector read_sectors across all compressed and container formats.
  • CSOReader and CCIReader now load compressed block spans in single I/O operations and decompress sector slices in parallel across all CPU cores.
  • GoDReader reads contiguous slices of up to 816 KB per syscall, eliminating seek thrashing during extraction or format conversion from GoD.

---

Included Portable Binaries

• XGDTool-GUI.exe: Complete modern GUI application with Dark Mode, Drag & Drop, batch queue manager, and multi-language support.
• XGDTool-CLI.exe: Standalone command-line executable for terminal, scripts, and batch processing.