💫 About Me :
Head of Engineering at Jobscan, I lead Jobscan's Engineering Team as we work to provide resources to job seekers to help them optimize their application process and get them hired.
Contact GitHub support about this user’s behavior. Learn more about reporting abuse.
Report abuseHead of Engineering at Jobscan, I lead Jobscan's Engineering Team as we work to provide resources to job seekers to help them optimize their application process and get them hired.
Library to check a url for all possible robots exclusion protocols (robots.txt, http X-Robots-Tag header, meta robots tag, or http status codes)