This project is a web scraper built using Python and the Scrapy framework. It was designed to scrape data from an online books website, extracting details such as book titles, prices, ratings, and availability. The project follows the tutorial provided by FreeCodeCamp, making it beginner-friendly and easy to understand.
- Scrapes book information: Extracts details like title, price, rating, and stock status.
- Structured data storage: Saves the scraped data in a structured format (e.g., JSON, databases).
- Python 3.x
- Scrapy framework
- Other dependencies (listed in requirements.txt)
Clone the repository:
Copy code
git clone gh repo clone IslamMammadov/FreecodecampScrapingTutorial
cd freecodecampInstall dependencies:
Copy code
pip install -r requirements.txt- Navigate to the project directory:
cd freecodecamp- Run the Scrapy spider:
scrapy crawl bookspider- View or export the scraped data:
JSON: BooksResult.json database file books.db
Project Structure
freecodecamp/: Main Scrapy project folder containing spiders and settings.
spiders/bookspiderr.py: The main spider that scrapes book data.
requirements.txt: Python dependencies for the project.
README.md: Project documentation.