Python scraper based on AI

🤖/👨‍🦰 Detect bots/crawlers/spiders using the user agent string

A IOT-Server,The tree root supports devices and data management, and the upper tree supports to combine various business functions by message flow

Amazon products scraper with using of rotating proxies and headless Chrome from ScrapingAnt

API definition, resources and reference implementation of URL Frontiers

🤩 Python Package for Scraping Amazon Product Reviews ✨

Python course for 2nd year NLP students at NRU HSE, 2018-2019

YiraBot: Simplifying Web Scraping for All. A user-friendly tool for developers and enthusiasts, offering command-line ease and Python integration. Ideal for research, SEO, and data collection.

GAMECHANGER Policy Analytics Site Crawlers

Python course for 2nd year NLP students at NRU HSE, 2017-2018

一款强大的大模型微调数据集生成和管理工具。

Detect bots/crawlers/spiders via user-agent string

A Web Crawler developed in Python.

ScrapeGraphAI is a Python-based web-scraping framework that pairs large-language-model reasoning with a graph-style pipeline engine to turn websites (or local XML/HTML/JSON/Markdown files) into structured data with just a handful of lines of code.

Web scraper for collecting product and review data from e-commerce sites using Scraping Bee, AWS, Selenium, and Pandas. Focuses on cost-effective solutions, user-friendly interfaces, and efficient data extraction and analysis.

This is an advanced version of the previously released version of web-crawler

This is a simple web crawler created using python

Web crawlers based on travel information, crawler data from Ctrip, tuniu, qunar.

Web crawlers for Early Irish & Welsh data

A tutorial on Crawling the Web and Scraping off the Useful bits - without actually doing anything!