-
Notifications
You must be signed in to change notification settings - Fork 0
utsavsabharwal/Web-Crawlers
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Folders and files
| Name | Name | Last commit message | Last commit date | |
|---|---|---|---|---|
Repository files navigation
dragos is a mature dragon in its mids. Can be used for fast crawling for 50M urls in a one go. penelope is a young dragon. Can be used for very fast crawling for about 5M urls in one go.
About
Trying to develop an optimal high scale web crawler in python
Resources
Stars
Watchers
Forks
Releases
No releases published
Packages 0
No packages published