Skip to content

Adding --ignore-robots flag

Compare
Choose a tag to compare
@adamdehaven adamdehaven released this 22 Oct 14:18
· 11 commits to master since this release
  • Added -i, --ignore-robots flag which will ignore the robots.txt file for the domain.