Skip to content

v1.0.0-alpha.3

Pre-release
Pre-release

Choose a tag to compare

View all tags
@unique01082 unique01082 released this 30 Aug 08:20
· 34 commits to master since this release
v1.0.0-alpha.3
e4a95f2

🎉 Major Feature Release - Buffer Creation API

This release introduces a comprehensive buffer creation system that enables direct memory buffer generation for multiple image formats without intermediate file operations. This addresses the core need for stream-based processing workflows.

✨ Added

🔄 Complete Buffer Creation API (7 New Methods)

  • Direct Memory Buffer Creation

    • createJPEGBuffer(options) - JPEG buffers with quality, resize, and progressive options
    • createPNGBuffer(options) - PNG buffers with compression levels and transparency
    • createWebPBuffer(options) - Modern WebP format with lossy/lossless modes
    • createAVIFBuffer(options) - Next-generation AVIF format with superior compression
    • createTIFFBuffer(options) - Professional TIFF format with multiple compression options
    • createPPMBuffer() - Raw PPM format for maximum compatibility
    • createThumbnailJPEGBuffer(options) - Fast thumbnail extraction without full processing

  • Smart Processing Pipeline

    • Automatic processing detection and caching
    • Shared processed image data for multiple format creation
    • Memory-efficient buffer generation
    • Intelligent resize and quality optimization

🚀 Advanced Image Processing Features

  • Flexible Resizing Options

    • Maintain aspect ratio with single dimension
    • High-quality Lanczos3 resampling
    • Optimized for both enlargement and reduction
    • Automatic dimension calculation

  • Format-Specific Optimizations

    • JPEG: Progressive encoding, fast mode, quality optimization
    • PNG: Compression levels (0-9), transparency preservation
    • WebP: Lossless mode, effort control, fast compression
    • AVIF: Advanced compression, lossless support, quality tuning
    • TIFF: Multiple compression algorithms (none, LZW, ZIP)
    • PPM: Raw uncompressed format for processing pipelines

  • Performance Optimizations

    • Parallel buffer creation support
    • Memory caching of processed images
    • Efficient Sharp.js integration
    • Optimized memory management and cleanup

🧪 Comprehensive Buffer Testing Framework

  • Complete Test Suite (test/buffer-creation.test.js)

    • Detailed testing of all 7 buffer creation methods
    • Quality, compression, and resize parameter validation
    • Performance benchmarking and parallel creation tests
    • Format-specific option testing and edge case handling

  • Quick Verification (test/quick-buffer-verification.js)

    • Fast smoke test for basic functionality
    • Tests JPEG, PNG, WebP, and Thumbnail creation
    • Runtime: ~2-3 seconds with output file generation

  • Edge Case Testing (test/buffer-edge-cases.test.js)

    • Memory management stress testing
    • Extreme parameter validation
    • Multiple processor instances
    • Format magic byte validation

  • Integration Tests (test/buffer-integration.test.js)

    • Mocha/Chai framework compatibility
    • Proper error handling validation
    • Parameter boundary testing
    • Cross-method consistency checks

  • Unified Test Runner (test/run-buffer-tests.js)

    • Colored console output with progress tracking
    • Flexible command-line options (--quick-only, --comprehensive-only, etc.)
    • Environment checking and validation
    • Performance reporting and statistics

📊 Real-World Performance Validation

  • Buffer Creation Benchmarks (Canon CR3 test files):

    • JPEG Buffer: 34.7KB, 600x400 (255ms) - Excellent compression
    • PNG Buffer: 97.5KB, 500x333 (403ms) - Lossless quality
    • WebP Buffer: 15.9KB, 600x400 (87ms) - Superior compression/speed
    • AVIF Buffer: 7.5KB, 500x333 (360ms) - Next-gen compression
    • TIFF Buffer: 186.1KB, 400x267 (52ms) - Professional quality
    • Thumbnail Buffer: 8.5KB, 200x133 (76ms) - Fast extraction

  • Parallel Creation Performance

    • 3 formats created simultaneously in 274ms
    • No memory interference between buffer operations
    • Consistent quality across parallel generations

🛠️ Developer Tools & Documentation

  • Buffer Method Documentation

    • Complete TypeScript definitions in lib/index.d.ts
    • Interface definitions for all result objects
    • Parameter type validation and descriptions

  • Usage Examples and Demos

    • test/buffer-demo.js - Working demonstration of all methods
    • test/final-buffer-test.js - Comprehensive validation script
    • API usage examples for web applications and streaming workflows

  • NPM Scripts Integration

    • npm run test:buffer-creation - Run comprehensive buffer tests
    • Integration with existing test framework
    • Command-line test runners with flexible options

🔧 Technical Implementation

📦 Enhanced Dependencies Integration

  • Sharp 0.33.5 Integration

    • High-performance image processing for buffer creation
    • Native C++ implementation for maximum speed
    • Memory-efficient processing for large images
    • Cross-platform compatibility (Windows, macOS, Linux)

  • Seamless LibRaw Integration

    • Direct memory transfer between LibRaw and Sharp
    • Automatic bit depth detection and conversion
    • Color space preservation and transformation
    • Smart processing pipeline with caching

⚡ Performance Characteristics

  • Processing Speed: 70-140 MB/s for image processing
  • Buffer Creation: 50-800ms depending on format and size
  • Memory Efficiency: Streaming processing with automatic cleanup
  • Compression Ratios: 6x to 500x depending on format and content

🎯 Quality Optimization

  • Color Accuracy

    • Proper color space handling from RAW to final format
    • White balance and gamma correction preservation
    • Color matrix transformation support

  • Detail Preservation

    • High-quality resampling algorithms
    • Edge-preserving compression
    • Format-appropriate optimization

🔧 API Enhancements

New TypeScript Definitions

interface LibRawBufferResult {
  success: boolean;
  buffer: Buffer;
  metadata: {
    format: string;
    outputDimensions: { width: number; height: number };
    fileSize: {
      original: number;
      compressed: number;
      compressionRatio: string;
    };
    processing: {
      timeMs: string;
      throughputMBps: string;
    };
    options: object;
  };
}

// Method signatures for all buffer creation methods
async createJPEGBuffer(options?: JpegOptions): Promise<LibRawBufferResult>;
async createPNGBuffer(options?: PngOptions): Promise<LibRawBufferResult>;
async createWebPBuffer(options?: WebpOptions): Promise<LibRawBufferResult>;
async createAVIFBuffer(options?: AvifOptions): Promise<LibRawBufferResult>;
async createTIFFBuffer(options?: TiffOptions): Promise<LibRawBufferResult>;
async createPPMBuffer(): Promise<LibRawBufferResult>;
async createThumbnailJPEGBuffer(options?: ThumbnailOptions): Promise<LibRawBufferResult>;

Consistent Option Interfaces

  • Quality Settings: 1-100 range for lossy formats
  • Resize Options: Width/height with automatic aspect ratio
  • Compression Control: Format-specific compression parameters
  • Speed Optimization: Fast mode options for time-critical applications

📋 Usage Examples

Basic Buffer Creation

const processor = new LibRaw();
await processor.loadFile("photo.cr2");
await processor.processImage();

// Create different format buffers
const jpegResult = await processor.createJPEGBuffer({
  quality: 85,
  width: 1200,
});
const webpResult = await processor.createWebPBuffer({
  quality: 80,
  width: 1200,
});

// Use buffers directly - no file I/O needed!
response.setHeader("Content-Type", "image/jpeg");
response.send(jpegResult.buffer);

Parallel Multi-Format Creation

// Generate multiple formats simultaneously
const [jpeg, png, webp, thumb] = await Promise.all([
  processor.createJPEGBuffer({ quality: 85, width: 1920 }),
  processor.createPNGBuffer({ width: 1200, compressionLevel: 6 }),
  processor.createWebPBuffer({ quality: 80, width: 1920 }),
  processor.createThumbnailJPEGBuffer({ maxSize: 300 }),
]);

console.log(
  `Created 4 formats in parallel: ${
    jpeg.buffer.length +
    png.buffer.length +
    webp.buffer.length +
    thumb.buffer.length
  } total bytes`
);

Web API Integration

// Express.js API endpoint
app.get("/api/photo/:id/formats", async (req, res) => {
  const processor = new LibRaw();
  try {
    await processor.loadFile(`photos/${req.params.id}.cr2`);
    await processor.processImage();

    const formats = await Promise.all([
      processor.createJPEGBuffer({ quality: 85, width: 1920 }),
      processor.createWebPBuffer({ quality: 80, width: 1920 }),
      processor.createThumbnailJPEGBuffer({ maxSize: 300 }),
    ]);

    res.json({
      jpeg: formats[0].buffer.toString("base64"),
      webp: formats[1].buffer.toString("base64"),
      thumbnail: formats[2].buffer.toString("base64"),
    });
  } finally {
    await processor.close();
  }
});

🧪 Testing & Validation

Comprehensive Test Coverage

  • Format Validation: Magic byte verification for all formats
  • Quality Testing: Multiple quality levels and compression settings
  • Resize Testing: Various dimension scenarios with aspect ratio preservation
  • Performance Testing: Speed and throughput measurement
  • Memory Testing: Leak detection and cleanup verification
  • Error Handling: Invalid parameter and edge case testing

Real-World File Validation

  • Camera Compatibility: Tested with Canon CR3, Nikon NEF, Sony ARW files
  • File Size Range: 20MB - 100MB RAW files processed successfully
  • Resolution Range: 12MP - 61MP images handled efficiently
  • Success Rate: 100% success rate for buffer creation across all test files

🔧 Testing Commands

Quick Testing

# Fast verification of all buffer methods
node test/quick-buffer-verification.js

# Run comprehensive buffer test suite
node test/run-buffer-tests.js

# Quick-only test
node test/run-buffer-tests.js --quick-only

Integration Testing

# Add buffer tests to existing test suite
npm run test:buffer-creation

# Run with other tests
npm test

Performance Testing

# Benchmark all buffer creation methods
node test/run-buffer-tests.js --comprehensive-only

# Test edge cases and memory management
node test/run-buffer-tests.js --edge-only

🚀 Stream-Based Processing Benefits

This release directly addresses the core requirement for stream-based processing:

Before (File-Based)

// Required intermediate files
await processor.writeTIFF("temp.tiff");
const buffer = fs.readFileSync("temp.tiff");
fs.unlinkSync("temp.tiff"); // Cleanup required

After (Buffer-Based)

// Direct buffer creation - no files needed
const result = await processor.createTIFFBuffer({ compression: "lzw" });
const buffer = result.buffer; // Ready to use immediately

Performance Improvements

  • 50-80% Faster: No disk I/O overhead
  • Better Memory Usage: No temporary file storage
  • Cleaner Code: No file cleanup required
  • More Reliable: No file system permission issues

🐛 Fixed

Memory Management Improvements

  • Buffer Cleanup: Automatic cleanup of intermediate buffers
  • Memory Leak Prevention: Proper resource management in all code paths
  • Error Recovery: Graceful handling of processing failures
  • Resource Optimization: Efficient memory allocation patterns

Format Compatibility Enhancements

  • Magic Byte Validation: Proper format header generation
  • Color Space Handling: Accurate color space preservation
  • Dimension Calculation: Correct aspect ratio maintenance
  • Quality Consistency: Consistent quality across multiple creations

📈 Performance Impact

Speed Improvements

  • Direct Buffer Creation: Eliminates file I/O bottlenecks
  • Parallel Processing: Multiple formats created simultaneously
  • Memory Efficiency: Reduced memory footprint through smart caching
  • Processing Pipeline: Optimized workflow with shared processed data

Quality Enhancements

  • Better Compression: Format-specific optimization for each output type
  • Color Accuracy: Improved color space handling and preservation
  • Detail Preservation: High-quality resampling and compression
  • Consistency: Identical results across multiple buffer creations

🔮 Future Enhancements

Planned Buffer Features

  • Advanced Options: HDR processing, color grading, noise reduction
  • Additional Formats: HEIF, BMP, TGA support
  • Streaming Support: Large file processing with stream interfaces
  • GPU Acceleration: Hardware-accelerated buffer creation

API Extensions

  • Metadata Preservation: EXIF data embedding in output buffers
  • Batch Buffer Creation: Process multiple files to buffers
  • Progressive Processing: Real-time buffer updates during processing
  • Custom Pipelines: User-defined processing chains
Assets 2
Loading