This is just a basic scrapping web app made with react, express and puppeteer
So far what I have learned a scraper is a tool that is used to collect data from web pages easily. Basically, it is a research tool. There are lots of advanced scrapers out there but I wanted to know how this thing works so I decided to make one myself. My plan is to document my learning and understanding here. So that in future if I forget or need to remember something I can revisit it.
So, let's talk about the tech I used here. To use a scraper easily anytime one needs to make both frontend and backend both. By that, we don't have to depend on our server or terminal every time we need to scrape anything. I also did that. So this scraper is divided into 2 sections first is the frontend and second is the backend.
- react (The main library)
- Axios (For fetching the data)
- Node
- Express (Backend library)
- Puppeteer (A library to process the data as a headless browser)
Yes a scraper can be made with only vanila Javascript, I used these library to to make it quickly and easily.