Join GitHub today
GitHub is home to over 31 million developers working together to host and review code, manage projects, and build software together.Sign up
A tool for fetching archived URLs (to be rewritten in Go).
Fetching latest commit…
Cannot retrieve the latest commit at this time.
|Type||Name||Latest commit message||Commit time|
|Failed to load latest commit information.|
______ __ / ____/_ ___________ _/ /____ / / / / / / ___/ __ `/ __/ _ \ / /___/ /_/ / / / /_/ / /_/ __/ \____/\__,_/_/ \__,_/\__/\___/ A tool for fetching archived URLs. Initial set-up one-liner This will clone this repository and then move all scripts to /usr/local/bin. $ git clone firstname.lastname@example.org:EdOverflow/curate.git \ && cp curate/curate /usr/local/bin/ \ && echo "You can delete the ./curate/ folder now." Once you have are done with this one-liner, make sure to include your VirusTotal API key in your .bashrc file. The variable should be named 'VIRUS_TOTAL_API_KEY'. Dependencies - curl - jq Usage To fetch archived URLs simply run curate and specify the target host. $ curate <host> $ curate example.com If you want to search for strings straight away in the output you can also specify your search term or regular expression by using the -r flag. $ curate <host> -r <search term> $ curate example.com -r example List of search terms Curate outputs the results into a curate.txt file which allows you to easily grep through the URLs for keywords. Here are some ideas for things to grep for in curate.txt. - admin - password - key - api - hmac - url - path - redir - php - id= It is advisable to exclude images and stylesheets from your searches by piping everything into 'grep -v'. Contributing I welcome contributions from the public. The issue tracker is the preferred channel for bug reports and feature requests. The bug tracker utilises several labels to help organise and identify issues. Whenever submitting a new issue, please use the GitHub issue search first — check if the issue has already been reported. Make sure to run https://github.com/koalaman/shellcheck when you are done editing or create a shell script. This will help maintain clean and uniform code throughout the project. If you would like to submit a patch via email, that is fine too. Just run the following command and send the patch to contact [at] edoverflow [dot] com: $ git format-patch -<n> <SHA1> Legal Disclaimer This project is made for educational and ethical testing purposes only. Usage of this tool for attacking targets without prior mutual consent is illegal. Developers assume no liability and are not responsible for any misuse or damage caused by these scripts. Donations If you would like to support my work, you can use any of the addresses below: Liberapay: https://liberapay.com/EdOverflow Bitcoin: 1E2fZRNrrkCKPnWpKZAsJzByBoyoBURADN Ethereum: 0xe98FC23fB4A8762d700c0354979dA5Db6c29Acc3 License MIT License — Copyright (c) 2018 EdOverflow