Skip to content

v0.3.0 - xxHash Performance Upgrade

Choose a tag to compare

View all tags
@Web3NL Web3NL released this 29 May 12:40
· 50 commits to master since this release
v0.3.0
3c5a48d

🚀 Performance Improvement - xxHash Migration

What's New

  • ⚡ Significantly faster duplicate detection using xxHash (XXH3) instead of SHA-256
  • 🔧 Improved performance for large file scans while maintaining accuracy
  • 📦 Updated dependencies for better compatibility

Breaking Changes

  • Hash output format changed from 256-bit to 64-bit (affects hash display only)
  • Internal hash algorithm changed from SHA-256 to xxHash (XXH3)

Features

  • ✅ Safe duplicate detection (read-only operation)
  • ✅ Size-based pre-filtering for efficiency
  • ✅ Recursive directory scanning
  • ✅ Cross-platform compatibility (Windows, macOS, Linux)
  • ✅ Clear duplicate group reporting
  • ✅ Potential space savings calculation

Performance Benefits

xxHash provides significantly faster hashing compared to SHA-256 while maintaining excellent collision resistance for duplicate detection use cases.

Installation

# Download from releases or build from source
git clone https://github.com/Web3NL/file-dedup.git
cd file-dedup
cargo build --release

Usage

# Basic usage
./target/release/file-dedup /path/to/directory

# Multiple paths
./target/release/file-dedup ~/Documents ~/Pictures

# Verbose output
./target/release/file-dedup -v ~/Downloads

Full Changelog: v0.2.0...v0.3.0

Assets 3
Loading