HTTPS clone URL
Subversion checkout URL
Find duplicate files on your computer
Fetching latest commit…
Cannot retrieve the latest commit at this time
|Failed to load latest commit information.|
This is a command line Java app to find duplicate files on your computer. You can either build the application from source, or download a pre-compiled version. To build from source, you need Git, a Java JDK, and Maven: git clone https://github.com/caluml/finddups mvn assembly:single install java -jar ~/.m2/repository/finddups/finddups/0.0.1-SNAPSHOT/finddups-0.0.1-SNAPSHOT-jar-with-dependencies.jar 1000000 /home /tmp To just run the supplied version (might be a little out of date): Download https://github.com/caluml/finddups/raw/master/finddups.jar java -jar finddups.jar 1000000 /home /tmp First, it recurses the given directory, building a list of files. Then, it looks for files which have identically sized files. It then reads those identically sized files, comparing them byte for byte until it finds a byte that doesn't match, or it reaches the end of the file. I chose not to use any kind of hashing or CRC system, as that would require reading the entire file, which would be less efficient if a byte near the beginning was different. The app outputs its errors to stderr, and its results to stdout as it goes. Any suggestions/comments to my github username at gmail.com TODO: Time generating two MD5s of two identical files and compare them, with comparing those two files byte by byte. CAVEAT: I have only tested this on Linux, with Java 6.