HyperSel is a Python-based framework for web automation and data scraping. It simplifies crawling, automating interactions, and data extraction from web pages. It is built with various robust tools to handle dynamic and complex web interactions efficiently, offering features such as headless browsing, proxy support, and API sniffing.
- Headless Browsing: Supports headless mode for efficient and faster crawls.
- Proxies: Integration for rotating proxies, ensuring anonymity and bypassing restrictions.
- Playwright and Selenium Support: Utilizes Playwright and Selenium for browser automation, providing flexible tools for interacting with websites.
- Undetected ChromeDriver: Uses undetected ChromeDriver to bypass detection by anti-scraping mechanisms.
- WebDriver Manager: Automatically manages and updates drivers.
- Recursive Crawling: Supports recursive crawling to traverse multiple levels of web pages.
- API Sniffer: Built-in API request sniffer to intercept and extract data from API calls made by websites.
- Regex Tools: Built-in regex utilities for parsing and extracting data from web pages.
- Logging: Comprehensive logging to track scraping sessions and errors.
To set up HyperSel, follow the steps below:
-
Clone the Repository:
git clone https://github.com/foreandr/hyperSel.git
-
Install Dependencies: Navigate to the project directory and install the required Python packages:
cd hyperSel pip install -r req.txt -
Install from PyPI: You can also install HyperSel directly from PyPI using pip:
pip install hyperSel
Once installed, you can import and start using HyperSel in your projects. Example usage:
import hyperSel
# Your web automation code hereFeel free to submit issues and pull requests. Contributions are welcome!
This project is licensed under the MIT License.