The Data Scraper project is designed to fetch and extract data from various web sources. The extracted data can be stored in different formats such as CSV, JSON, or directly into databases for further analysis and usage.
- Scrape data from websites
- Parse HTML content using BeautifulSoup
- Store extracted data in CSV or JSON format
- Configurable settings for different websites
- Error handling and logging
- Python 3.7+
- requests
- BeautifulSoup4
- pandas
- selenium
This repository provides data crawled from web:
- Great School: https://www.greatschools.org/
- Realtor: https://www.greatschools.org/
- USnews: https://www.usnews.com/
├── Great School
│ ├── GreatSchool.ipynb
├── Realtor
│ ├── Realtor.ipynb
├──USNews
│ ├── USNews.ipynb
├──requirements.txt
└──README.md
To get started with the project, follow these steps:
-
Clone the repository:
$ git clone https://github.com/toanuitt/Scrape-data.git
-
Navigate to the project directory:
$ cd Scrape-data -
Install the required packages:
$ pip install -r requirements.txt
Contributions are welcome! Please open an issue or submit a pull request for any improvements or bug fixes.
This project is licensed under the MIT License.