git-url-extractor

checks root of domains in list for the presence of .git folder.
Takes the index and converts it to a list of all files on that url.
Sorted in folders.
usage: python3 extractor.py -f list-of-domains.txt

requirement:

gin: https://github.com/sbp/gin (sudo pip install gin)