Dupelet

A fast, cross platform duplicate file finder written in Go. Supports Linux and macOS with optimized performance, partial hashing, and portable static binaries.

---

Features

• Cross platform (Linux, macOS)
• Statically linked binaries (CGO_ENABLED=0)
• Multi threaded file scanning and hashing
• Partial hashing for large files
• Supports JSON and CSV reports
• Cache support for incremental scans

---

Installation

Pre-built binaries

Download the latest release from the Releases page.

Build from source

git clone https://github.com/PrivacyAreUs/Dupelet.git
cd Dupelet/src
chmod +x ./build.sh
./build.sh

This will produce statically linked binaries for Linux, and macOS under the ./build directory.

---

Usage

./dupelet -path /path/to/scan -cache ./cache.db -algo blake3 -report-json ./dupes.json

Options

Flag	Description
-path	Directory to scan
-cache	Path to cache file for incremental scanning
-algo	Hashing algorithm: blake3 (default) or fast
-report-json	Output JSON report
-report-csv	Output CSV report
-workers	Number of concurrent workers
-use-partial	Enable partial hashing for large files

---

Examples

Scan a directory and generate a JSON report

./dupelet -path ~/Documents -cache ./dupelet.db -report-json dupes.json

Scan with partial hashing to speed up large file detection

./dupelet -path /mnt/data -use-partial -chunk-size 1048576 -workers 8

---

Contributing

Contributions are welcome! Please open an issue or a pull request with improvements, bug fixes, or new features.

---

License

This project is licensed under the Unlicensed License. See the LICENSE file for details.