Fast Python web crawler for RAG and AI ingestion. Extracts clean Markdown from any site for LLMs and vector stores.
python gemini openai data-extraction structured-data webcrawler ai-agents rag ingestion-pipeline vector-database supabase sitemap-crawler llm pgvector anthropic-claude markdown-extraction
-
Updated
May 16, 2026 - Python