Parallel Hashing Performance Upgrade

[pwgit-create]

Parallel Hashing Performance Upgrade, Dependency Update, and Documentation Improvements

Overview

This release introduces a significant performance enhancement to the file integrity scanning engine through the addition of parallel hashing for large files. It also includes dependency updates, scan lifecycle improvements, logging configuration adjustments, and README restructuring.

The primary focus of this change is improving large-file scan performance and reducing redundant I/O operations during multi-algorithm hashing.

---

Key Changes

Hashing Engine Performance Improvement

• Introduced ParallelFileHashHandler to handle large file hashing using parallel execution.

• Refactored FileHashComputer to route large file processing through the parallel hashing implementation.

• Added explicit lifecycle control:

  • initializeParallelHashing() to set up thread pool and handler before scan start
  • shutdownParallelHashProcessor() to properly release resources after scan completion

• Improved fallback handling:

  • OutOfMemoryError now switches to parallel hashing instead of single-threaded big file processing.

Performance Impact

• Parallel hashing provides a major performance improvement in local testing, with 2x+ speedup observed for large file scans.

• For files 1GB and larger, the improvement is especially significant:

  • The file is read once only, regardless of the number of hashing algorithms applied.
  • Previously, each algorithm required separate file reads.
  • Now SHA-256, SHA-3, and BLAKE2b computations are executed in parallel over a single read stream.

• Performance gains scale with file size and are increasingly beneficial in environments with many large files, where I/O reduction becomes the dominant factor.

Scan Service Integration

• Parallel hashing initialization added at scan start for both:

  • scanAllDirectories
  • scanSingleDirectory

• Thread pool shutdown added during scan finalization to prevent resource leaks.

Dependency Update

• Updated algorithm-hash-extraction from 1.2.8 to 1.2.9.

Logging Updates

• Added dedicated logging configuration for:

  • lib.pwss.hash.file_hash_handler.parallel

---

Documentation Updates (README)

• Refactored project title to: File-Integrity Scanner Backend (FIM Engine)
• Improved technical clarity and structure of documentation.
• Expanded explanation of cryptographic hashing and integrity verification.
• Added Related Repositories section for end-user distribution.
• Improved system architecture description and component breakdown.
• Updated setup instructions for developers.

---

Architecture / Behavioral Impact

• Large file processing is now optimized for parallel execution.

• File I/O is reduced significantly when multiple hashing algorithms are used.

• Clear separation of concerns between:

  • Small file hashing (FileHashHandler)
  • Large file parallel hashing (ParallelFileHashHandler)

• Better scalability for environments with high-volume or large-size file systems.

---

Notes

• Parallel hashing must be initialized before scan execution and properly shut down after completion to avoid thread pool leaks.
• Requires algorithm-hash-extraction:1.2.9.

---

Testing

• Verified scan execution for:

  • Single directory scans
  • Full directory scans
  • Large file handling (1GB+ test cases)

• Confirmed:

  • Correct parallel execution of hashing algorithms
  • Proper resource cleanup after scan completion
  • Stable fallback behavior under memory pressure

[lilstiffy]

I really like the addition to the README :D