The key aspects of this challenge is the number of the files and size, for this I used streams of Nodejs to handle big files without need a lot of memory RAM.
Please send the path where you want to search duplicate files
const fileinfo = require("./fileinfo");
fileinfo.getDuplicateFiles("./files", (err, duplicateFiles) => {});
Service will run on port 3000
npm run service
It will return all files duplicates including the path where is located the file.
GET /duplicateFiles
Example Response
["/files/test.txt", "/files/test1.txt", "/files/tmp/test3.txt"]
npm run test