features

Features

Cross-cutting capabilities that span multiple services in the GroktoCrawl stack.

• Smart scraping pipeline -- three-tier extraction with adapters
• Site adapters -- YouTube, GitHub, Bluesky, Substack handlers
• Search pipeline -- five retrieval modes with semantic search
• Agent research -- autonomous web research with SSE streaming
• Vector indexing -- persistent semantic search with Qdrant
• Scheduled monitors -- change detection via Ofelia cron
• Scrape caching -- freshness-aware result caching in Valkey
• Authentication and security -- API keys, private network protection