Golang Web Scraper

Golang Web Scraper is a tool that allows users to scrape search results from Google for a specific search term, country, and language. The results include the ranking, link, title, and description of each search result.

Project Architecture

Features

• Scrape search results from Google for a specific search term, country, and language
• Results include the ranking, link, title, and description of each search result
• Ability to specify the number of pages and number of results per page
• Option to use a proxy for scraping
• Built-in random user agent feature to prevent bans

Getting Started

1. Clone the repository and navigate to the project directory

git clone https://github.com/yrs147/scraper.git 
cd scraper

2. Build and run the project

go build ./scraper

3. Input the search term, country code, and language code in the prompt
4. Optionally specify the number of pages, number of results per page, and use a proxy
5. The results will be printed in the terminal

Built With

• Go - The programming language used
• goquery - Package for parsing html and xml