Skip to content

jevcrawl 0.1.1

Choose a tag to compare

View all tags
@RayenMlayeh RayenMlayeh released this 14 Oct 16:30
· 2 commits to main since this release
v0.1.1
d8b7828

jevcrawl now follows each site's robots.txt by default. Set JEVCRAWL_RESPECT_ROBOTS=0 to turn it off.

Assets 2
Loading