Skip to content

lunatic-cat/xxhashdir

master
Switch branches/tags
Code

Latest commit

 

Git stats

Files

Permalink
Failed to load latest commit information.
Type
Name
Latest commit message
Commit time
bin
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

xxhashdir

Build Status

Usage

this package does fast filesystem fingerprinting using xxHash

# instead of "find . -type f -exec xxhsum {} \+"
$ ./xxhashdir .
...
880788507839261490    README.md
11541949788444589007  .travis.yml
6467850080536788703   bin/xxhashdir.go
...

typical CLI use:

./xxhashdir dir > before
# modify fs
./xxhashdir dir > after
diff <(sort before) <(sort after) | sort -nk3

Speed

Times faster than find + exec. Digesting xcode-10.2 with >250K files:

Time Cmd
656 sec time find /Applications/Xcode.app -type f -exec xxhsum {} ; > xxhsum.txt
88 sec time find /Applications/Xcode.app -type f -exec xxhsum {} + > xxhsum.txt
45 sec time ./xxhashdir /Applications/Xcode.app > xxhsumdir.txt

Golang api

func Hashdir(root string, out chan Entry)

where

type Entry struct {
    Path   string
    Xxhash uint64
}