Skip to content
/ WebCrawlers Public

Console-based Hotels and Medical sites Web Crawlers developed using Java, Selenium, JSoup. Leveraged several data structures like HashMaps, ArrayList, etc. to store the crawled data.

1 star 0 forks Branches Tags Activity
Star
Notifications You must be signed in to change notification settings

Rudra1402/WebCrawlers

BranchesTags

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

5 Commits
.idea
.idea
 
 
src/main/java
src/main/java
 
 
.gitignore
.gitignore
 
 
Booking.json
Booking.json
 
 
BookingCA.json
BookingCA.json
 
 
Expedia.json
Expedia.json
 
 
ExpediaCA.json
ExpediaCA.json
 
 
Hotels.json
Hotels.json
 
 
HotelsCA.json
HotelsCA.json
 
 
MedicalInfo.csv
MedicalInfo.csv
 
 
MedicalSiteInfo.csv
MedicalSiteInfo.csv
 
 
README.md
README.md
 
 
parameters.json
parameters.json
 
 
pom.xml
pom.xml
 
 

Repository files navigation

WebCrawlers

All the crawlers will work in headless mode.

  1. Crawler.java file crawls "Trivago" and prints all the hotels on the main page.
  2. HotelsCrawler.java file crawls "Hotels" and "Booking" and stores the data in 3 json files, one for each site.
  3. MedicalSiteCrawler.java file crawls "Clevelandclinic" and stores the diseases, their symptoms, diagnosis, and treatments in a csv file.

Note: To run these crawlers you will have to add 2 jar files; 1. selenium-server, 2. gson.

About

Console-based Hotels and Medical sites Web Crawlers developed using Java, Selenium, JSoup. Leveraged several data structures like HashMaps, ArrayList, etc. to store the crawled data.

Topics

java selenium hashmap webcrawler

Resources

Readme
Activity

Stars

1 star

Watchers

1 watching

Forks

0 forks
Report repository

Releases

No releases published

Packages

No packages published

Languages