Skip to content

v0.1.0

Choose a tag to compare

@2913636 2913636 released this 06 Jul 14:18

Initial release of Universal Extractor.

Core

  • 9-layer fallback extraction: Jina → HTTP → Browser DOM → Canvas Hook → CDP → OCR → Vision LLM → Captcha Slider → Font Decrypt
  • Pipeline orchestration engine with search → verify → extract → validate workflow
  • WebLens: search + filter + extract integration engine
  • Cross-validator for multi-source chapter alignment and merging
  • CLI with ue and universal-extractor entry points
  • MCP server for AI agent integration

Captcha & Anti-Bot

  • Captcha solving: slider, font decryption, and API-based solvers
  • Canvas Hook: intercept fillText/strokeText on Canvas elements
  • Font decryption: fontTools cmap parsing + ddddocr glyph OCR

Infrastructure

  • Strategy cache for per-domain capability profiling (automatic best-layer selection)
  • Session manager with persistent browser profiles
  • Proxy manager with auto-rotation and health checking
  • Rate limiter with adaptive throttling
  • Screenshot utilities: viewport capture, dedup, and stitching

Extra

  • JD (Job Description) engine with LLM structuring
  • Crawler with resume support for multi-page sites