RAG pipeline using phi3.5 and all-minilm-l6-v2

• 1 Fetch Data
  • fetch recent changes on wiki articles from 100 articles of a particular category
  • move fetched data to next step folder : generate queries
  • move step2-4 folders to output-dir of step1 for next part of pipeline

---

TODO:

• add ENV variable that can used in step 1 what category of articles to fetch

---