ScrapQuest is your go-to solution for effortlessly extracting data from websites. It's a simple yet powerful web scraper built with Next.js, TypeScript, and Puppeteer with Browserless. With ScrapQuest, you can quickly gather valuable information from any website without hassle.
- short video for reference
ScrapQuest.mp4
Frontend Deployed here :- scrap-quest.vercel.app/
Note
Due to some limitations, website functionality is limited on vercel. So setup project locally for complete functionality
- Next.js: A React framework for building server-side rendered and static web applications.
- Puppeteer: A Node.js library for controlling headless Chrome or Chromium, used here for web scraping.
- Browserless: A service that provides headless browser instances for Puppeteer.
- TypeScript: A statically typed superset of JavaScript that adds type definitions for improved code quality and developer productivity.
Follow these steps to set up ScrapQuest locally:
- Node.js (version 18) installed on your machine. You can download it here.
-
Fork the Repository
-
Clone your Fork from GitHub
git clone https://github.com/<your-user-name>/ScrapQuest.git
-
cd ScrapQuest -
docker-compose up
-
Fork the Repository
-
Clone your Fork from GitHub
git clone https://github.com/<your-user-name>/ScrapQuest.git
-
cd ScrapQuest -
npm install
-
npm run dev
- open localhost:3000 on the browser
We welcome contributions from the community to improve ScrapQuest. Whether it's bug fixes, feature enhancements, or documentation improvements, your contributions are valuable.
Before contributing, please take a moment to review our contribution guidelines to ensure a smooth and collaborative process.
We appreciate your interest in contributing to ScrapQuest! Let's make it better together.