RawPipeline Architecture

This document details the architecture of the RawPipeline iOS application, designed to capture unadulterated RAW (Bayer) data directly from the camera sensor, bypassing Apple's standard Image Signal Processor (ISP) pipeline (Deep Fusion, Smart HDR, etc.).

1. High-Level Architecture

The application follows a MVVM (Model-View-ViewModel) pattern:

• View (ContentView.swift):

  • A SwiftUI interface that displays the camera preview, lens controls, and status indicators.
  • Directly observes CameraManager for state updates (e.g., isProcessing, sensorResolution).
  • Handles user interactions and permissions.

• ViewModel / Manager (CameraManager.swift):

  • The core engine responsible for AVCaptureSession management.
  • Handles device discovery, format negotiation, and photo capture.
  • Executes heavy camera operations on a dedicated serial background queue (com.rawpipeline.session) to keep the UI responsive.
  • Publishes state changes to the main thread via @MainActor.

2. Direct Sensor Access (Bypassing Apple ISP)

The primary goal of this application is to retrieve raw sensor data without the computational photography enhancements applied by the Apple ISP (Image Signal Processor).

The Mechanism

The CameraManager achieves this through a specific format selection strategy in configureHighestResFormat(for:):

1. Format Iteration: The app iterates through all available AVCaptureDevice.Formats supported by the hardware.
2. Resolution Priority: formats are sorted by their maximum photo dimensions to ensure the highest fidelity capture.
3. Raw Format Selection (The Bypass):
   • For each format, the app checks photoOutput.availableRawPhotoPixelFormatTypes.
   • Priority 1 (Pure Bayer): It explicitly searches for a pixel format type that is NOT recognized as Apple ProRAW.
     • Standard Apple ProRAW formats are processed linear DNGs (kCVPixelFormatType_420YpCbCr8BiPlanarFullRange wrapped).
     • By selecting formats like kCVPixelFormatType_14Bayer_RGGB (or similar Bayer patterns), we force the system to return the raw data straight from the sensor's analog-to-digital converter, before demosaicing or multi-frame fusion occurs.
   • Priority 2 (Fallback): If no pure Bayer format is available (rare on modern devices for the main sensor, but possible on secondary lenses), it falls back to ProRAW.

Code Highlight

// From CameraManager.swift
if let rawType = tryConfigureFormat(device: device, format: format, photoOutput: photoOutput, enableProRAW: false) {
    // Found a Non-ProRAW (Pure Bayer) format
    selectedFormat = format
    bestRawType = rawType
    usesProRAW = false
    // ...
}

By setting isAppleProRAWEnabled = false and selecting a non-ProRAW pixel format, we effectively disable the computational pipeline.

3. Data Flow

1. Capture Request: User taps the shutter button.
2. Configuration: AVCapturePhotoSettings is configured with the pre-determined rawPixelFormatType (e.g., kCVPixelFormatType_14Bayer_RGGB).
3. Sensor Readout: The sensor captures the image. The ISP performs minimal processing (black level subtraction, defective pixel correction) but skips multi-exposure blending.
4. Data Delivery: The AVCapturePhotoOutput delegate receives a AVCapturePhoto object.
5. DNG Extraction: photo.fileDataRepresentation() is called to generate a standard DNG file containing the raw sensor data and metadata.
6. Direct File Storage:
   • The DNG data is written directly to the app's Documents/CapturedRAWs directory.
   • This bypasses the system PHPhotoLibrary, giving the user direct file access via the iOS Files app or iTunes File Sharing.

4. Components Breakdown

CameraManager

• Session Queue: A dedicated serial queue protects the AVCaptureSession from thread safety issues and UI blocking.
• Lens Switching: Dynamically reconfigures inputs between Ultra Wide (.builtInUltraWideCamera), Wide (.builtInWideAngleCamera), and Telephoto (.builtInTelephotoCamera).
• Error Handling: internal errors are propagated to the UI via the errorMessage published property.

ContentView

• Preview: Uses CameraPreviewView (a UIViewRepresentable wrapping AVCaptureVideoPreviewLayer) to render the live feed.
• Feedback: Provides haptic feedback and visual flash animation during capture.