FlyingMantis is the crawler portion of a full fletched search engine. Developed Hadoop's MapReduce structure, this solution is deployed in a distributed manner on EC2 instances, while storing the files that are crawl on S3s.
-
Notifications
You must be signed in to change notification settings - Fork 0
bill-he/FlyingMantis
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
About
Java Web crawler that uploads html files onto AWS
Resources
Stars
Watchers
Forks
Releases
No releases published
Packages 0
No packages published