📚 Scrapy Spider: Books to Scrape (with ScrapOps Integration)

🧩 Overview

A professional Scrapy project that scrapes book details (availability,description,price,product_type,rating,tax,title,upc,url) from (https://books.toscrape.com).
Integrated with ScrapOps for monitoring, logging, and performance insights — ideal for showcasing production-ready web scraping pipelines.

---

🚀 Features

• 🔍 Scrapes multiple pages using Scrapy pagination
• 📦 Extracts book data: title, price, stock status, and rating
• 🧠 Integrated with ScrapOps for metrics & observability
• 💾 Exports structured data to CSV or JSON
• ⚙️ Clean modular structure (spiders, items, pipelines, settings)

---

🛠️ Tech Stack

• Python 3.x
• Scrapy
• ScrapOps
• CSV/JSON Output
• (Optional) Python-dotenv for secret management

---

⚙️ How to Run Locally

1️⃣ Clone this repository

git clone https://github.com/adminHarsh7372/books-to-scrape-scrapy.git
cd books-to-scrape-scrapy