Karpathy-GPT

Context

This app is a template for using LangChain to build a LLM Q+A assistant from any set of YouTube videos.

We use Karpathy's course on LLMs as an example.

We use LangChain to:

(1) convert YouTube urls to text

(2) feed the text into LangChain auto-evaluator to test different chain parameters

(3) with our chosen parameters, build a vectorstore retriever back-end with FastAPI (deployed to Railway)

(4) stream the generated results (answer and retrieved docs) to a front-end (deployed to Vercel)

---

Step 1: URLs to text

See the notebook in /index folder:

• Uses LangChain's OpenAIWhisperParser to convert urls to text in < 10 lines of code

Step 2: Testing

See the text files in /eval folder:

• Feed the text from step 1 and, optionally, an eval set to the auto-evaluator app
• We can use this to test different parameters (see full README in the repo here)
• Use the UI to run experiments
• Select your best retriever, chain settings (e.g., k, split size, split overlap, etc), LLM, embeddings

Step 3: text to VectorDB

See the notebook in /index folder:

• Split the text from step 1 using parameters you found in step 2
• Upsert the vectors to a VectorDB (e.g., in this example, Pinecone) with metadata
• See this PR / notebook if you want to use locally with a different VectorDB

Step 4: Back-end

See the karpathy_app.py file in /api folder:

• We use LangChain's load_qa_chain with a user specified LLM and prompt (see default_prompt_template)
• Given a question, this will stream answer the text back to front-end and pass the retrieved documents back
• We deploy this FastAPI API to Railway
• See README.md in /api for local testing instructions

Step 5: Front-end

See /nextjs directory for nextJS app:

• This will call the back-end with the query and fetch the documents / answer
• Test the app locally by launching the back-end:

uvicorn karpathy_app:app

• To run front-end locally with you locally running back-end, simply change the source in fetchEventSource here and here to http://localhost:8000/karpathy-docs and http://localhost:8000/karpathy-stream
• To run the front-end locally, run:

npm run dev