Skip to content

CrawlTrove v0.1.0

Choose a tag to compare

View all tags
@chrisamber chrisamber released this 16 Jul 09:44
v0.1.0
e574a07
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

CrawlTrove's first public release.

Highlights

  • HTTP-first scraping with automatic Playwright browser fallback
  • Clean Markdown extraction from web pages, PDFs, EPUBs, and images
  • OCR, crawling, batch scraping, structured extraction, and research jobs
  • Optional PostgreSQL persistence and hybrid retrieval
  • Local dashboard, JSON API, Docker Compose runtime, and health checks

Start

docker compose up --build

Then open http://localhost:8000 or check http://localhost:8000/api/health.

Assets 2
Loading