Hash to detect potential dupliccates and save memory

[landonjefftaylor]

Use something clever to hash the paths to an integer then discard any we suspect to be duplicates.

[landonjefftaylor]

Add an option to save or delete paths