Skip to content

Zero Copy Storage Architecture

YouDontNeedToKnow edited this page Sep 3, 2026 · 2 revisions

Zero-Copy Android Storage Engine & Performance Architecture

The Zero-Copy Engine introduced in XGDTool completely redefines how disc image conversion operates on mobile devices running modern Android (Android 8.0 through Android 15+ / API 26–35).


The Problem: SAF Cache Duplication in Legacy Mobile Converters

In standard Android applications using the Storage Access Framework (SAF):

  1. Applications do not receive direct file system paths (/storage/emulated/0/...); instead, they receive content URIs (content://...).
  2. When native C++ engines (fopen, std::ifstream, posix_fadvise) require file paths or sequential seek operations, legacy applications are forced to copy the entire input file into the app's internal cache directory (cacheDir), execute the C++ engine on the cached file, and then stream the entire output file back through SAF streams into the user's destination folder.
  3. For an 8 GB Xbox 360 game:
    • Reading requires ~7.5 GB in cache.
    • Writing the converted ISO/CSO requires another ~7.5 GB in cache.
    • Total space required: ~15 to 20 GB of free internal flash storage just to process one single game!
    • In addition to running out of space, writing 15 GB twice thrashes mobile UFS storage controllers and dramatically slows down conversion speeds.

The Solution: Zero-Copy Direct I/O in XGDTool

XGDTool bypasses the cache entirely whenever direct path resolution is available:

graph LR
    A[Source Storage] -->|Direct POSIX Read| B[Native C++ Engine]
    B -->|Direct POSIX Write| C[Target Storage]
    B -.->|0 Bytes Cache| D[RAM Buffers 2MB]
Loading

1. Permission Integration (MANAGE_EXTERNAL_STORAGE)

XGDTool declares MANAGE_EXTERNAL_STORAGE (ACTION_MANAGE_APP_ALL_FILES_ACCESS_PERMISSION), allowing direct POSIX filesystem access across external shared storage, internal storage (/storage/emulated/0), and external MicroSD cards.

2. Canonical SAF Path Resolution (PathUtils.kt)

The custom path resolver inspects SAF document and tree URIs:

  • Standard primary storage URIs (content://.../tree/primary%3AROMs%2FX360) are resolved directly into /storage/emulated/0/ROMs/X360.
  • Secondary SD cards (content://.../tree/1234-5678%3AGames) are resolved to /storage/1234-5678/Games.
  • Raw /storage/ or /sdcard/ file paths are validated directly via File.canRead() and File.canWrite().

3. Native Zero-Copy Execution

When both input and output paths are resolved to direct filesystem paths:

  • Phase 1 (Cache Ingestion) is completely bypassed: 0 bytes copied to cache.
  • Phase 2 (Conversion): Native C++ engine reads directly from the source file and writes directly into the target directory.
  • Phase 3 (SAF Streaming Output) is completely bypassed: The file is already written at its final destination.
  • Real-time MediaStore Indexing: Upon completion, MediaScannerConnection.scanFile() registers the new file immediately so it appears across all file managers and frontend emulators.

Vectorized Hardware-Accelerated CRC32 on ARM64

In v1.4.0, XGDTool integrates ARMv8-A ACLE hardware intrinsics (__crc32d and __crc32b). On modern ARM64 mobile processors (Snapdragon, Dimensity, Tensor), CRC32 calculation reaches 15 to 20 GB/s, completely eliminating checksum bottlenecks during mobile conversions.


Benchmark Comparison

Metric Legacy SAF Cache Mode XGDTool Zero-Copy Mode
Duplicate Storage Overhead 15 – 20 GB 0 Bytes (Zero-Copy)
Input Ingestion Time (8 GB) ~30 – 60 seconds 0.0 seconds (Instant)
Output Export Time (8 GB) ~45 – 90 seconds 0.0 seconds (Instant)
Conversion Throughput ~80 – 150 MB/s Saturates NVMe / UFS 4.0 (~1.2 – 2.5 GB/s)
Flash Wear & Write Cycles ~25 GB written per disc Only the size of the final image written

Clone this wiki locally