pycrawl A Simple multithreaded python crawler (python 2.7) Usage python main.py 'project name' 'url' eg: python main.py 'test project' 'https://eshunsharma.in'