Skip to content

πŸŽ‰ v1.0.0 β€” First Public Release

Choose a tag to compare

View all tags
@ccjiao ccjiao released this 15 Jun 06:03
· 2 commits to main since this release
v1.0.0
6c2e8a9

Data Scraper Engine v1.0.0

What's New

  • 🧠 URL Smart Router β€” Drop any URL, auto-detect the data source type
  • πŸ”„ Multi-Protocol β€” HTTP API + WebSocket + Generic Web adapters
  • 🎰 Lottery Data β€” Full historical data for 倧乐透/ζŽ’εˆ—3/δΈƒζ˜Ÿε½©
  • 🎬 Douyin Live β€” Live room data collection (chat, gifts, interactions)
  • 🌐 Generic Web β€” Extract tables, lists, articles from any URL
  • πŸ“Š Structured Output β€” Styled Excel, JSON, CSV
  • πŸ”§ Config-Driven β€” New data sources only need a YAML file

Quick Start

git clone https://github.com/ccjiao/data-scraper-engine.git
cd data-scraper-engine
pip install -r requirements.txt

# One command to scrape!
python scripts/scraper.py scrape "https://www.lottery.gov.cn/kj/kjlb.html?dlt"

Pro Features (Coming Soon)

  • Sentiment analysis (94% accuracy)
  • User profiling & water army detection
  • REST API server for AI tool integration
  • Scheduled tasks & Webhook push
Assets 2
Loading