This repo is a fork of mayooear's repo - GPT-4 & LangChain which leverages the new GPT-4 api to build a chatGPT-like chatbot for multiple Large PDF files. The goal of my forked repo is to implement some enhancements (mainly ui), refine them, and to hopefully integrate these with mayo's repo.
Mayo's repo is supported by an instructional youtube video and a discord group that I encourage folks to engage in. Last I saw, there were 2000 forks and 10,000 stars for his repo - so it is very popular and deservedly so. As I worked with this code-base, I could see the value in how it combined a front-end framework (next.js) with langChain, but there were changes I needed to make to support my own deployments of it. The code as-is is really a proof-of-concept - especially with the front end components. My enhancements are aimed at supporting custom deployments within a common code base.
- The text areas on the chat page ((title, welcome text, user input placeholder) are very important descriptors since this chatbot can interact with a wide range of pdfs (e.g legal documents, documentation, help files, books, etc.) Currently the text for the chat page elements can only be modified within the code for the main page (
pages/index.tsx
). And these changes would be overwritten by any updates to the repo. My enhancement parameterizes all key page elements and allows users of this repo to specify them in the .env file. - A pull request has been made for this change in Mayo's repo.
- Implemented a simple method for deploying to cloud via fly.io. Fly is a modern and economical provider of application hosting with a CLI that can generate a dockerfile for streamlined deployment. Instructions are provide below.
- The multiple source entries under each AI message created a very expanded message feed that deviated from the clean ChatGPT feed. This upgrade creates a more compact ChatGPT like message feed with fixes to the automated scrolling and maintaining focus on the user input prompt after updates.
The tech stack used in Mayo's repo is well-chosen and essentially unchanged - LangChain, Node, Yarn, Pinecone, Openai, LLM, and Next.js. My only additions are optional additions related to deployment - Fly.io and Docker.
LangChain
- is an app framework for LLMs (large language models) that makes it easier to build scalable AI/LLM apps and chatbots. This open-source initiative started just before the CHatGPT craze and has really taken off. It has both a javascript/typescript and python version. This repo leverages the javascript version. It is important to note that LangChain is not a web-app framework - it is more of a middle-tier framework that has been plugged into a web-framework - like Next.js or Django in the python world.Node.js
- a javascript server environment that LangChain JS runs on in this application. Node allows for a very complete and simple local dev environment.Yarn
- the package manager used in this application - is very helpful for building and maintaining the components for this application.Pinecone
- is a vectorstore for storing embeddings of the PDF content. Pinecone is a cloud SAAS provider with a free tier - so it is great for prototyping. But you will need to move to a paid plan if you need to support multiple indexes. Chroma DB is another option for a vectorstore that I have used. Its advantage is that it is open-source and self-hosted and thus not limited in terms of indexes you can build. But deploying apps with it is more complicated (e.g. multiple docker containers) and not currently an option for this code-base.OpenAI
- leading developer of LLMs and the chatbot ChatGPT. Its API is integrated with the embedding processing for the pdf documents - this is separate from its many LLM models.LLMs
- Large Language models are integrated with any LangChain app and the choice of the model is independent of the embedding process.Next.js
- is a web-app framework built on top of React and manages all the front end web-components to support the chat-bot interface. It is an excellent choice here in that it supports both Client-side and Server-side rendering and has built in routing.Fly.io
andDocker
- fly.io is an app hosting environment with a simple deployment workflow. Docker is the most popular method of containerizing applications for deployment. With these - you can take the local LangChain dev environment and deploy it to the web. Note that .env parameters will need to be added as secrets in the Fly.io environment.
- Node.js - version is 18 or greater.
git clone [github https url]
Install yarn globally (if you haven't already).
npm install yarn -g
Then run:
yarn install
After installation, you should now see a node_modules
folder.
cp .env.example .env
Your .env
file should look like this: (Note this env is expanded from Mayo's version)
OPENAI_API_KEY=
PINECONE_API_KEY=
PINECONE_ENVIRONMENT=
PINECONE_INDEX_NAME=
PINECONE_NAME_SPACE=
# `Optional` - static text on chat page - change if you want new values from defaults
CHAT_PAGE_TITLE=Chat with Your Favorite Author
WELCOME_MESSAGE=Hi, what would you like to learn about this book?
USER_INPUT_PLACEHOLDER=What is the book about??
PFOOTER_URL=
FOOTER_TEXT=
- Visit openai to retrieve API keys and insert them into your
.env
file. - Visit pinecone to create and retrieve your API keys, and also retrieve your environment and index name from the dashboard.
Static text on chat page
- this is the main departure from the original code base - substituting the hardcoded text values with parameters from the .env file and thus outside the code base.insert marked up image
- Modifications to the
utils/makechain.ts
are unchanged from Mayo's repo. In this regard, theQA_PROMPT
andmodelName
are modified manually. I am looking at methods for changing these chains dynamically for testing. Currently this chain from Mayo's repo is built on the Conversational Retrieval QA chain.
This repo can load multiple PDF files
2. Run the script npm run ingest
to 'ingest' and embed your docs. If you run into errors troubleshoot below.
npm run ingest
Once you've verified that the embeddings and content have been successfully added to your Pinecone, you can run npm run dev
to launch the local dev environment, and access localhost:3000
in a web browser to interact with the chat interface.
npm run dev
Fly.io is a low-cost hosting site that can easily configure and deploy an app like this.
Follow these instructions or refer to the full documentation on fly.io
1. Create an account on Fly.io
2. Install flyctl (fly's cli) locally - follow this guide on fly.io
$ fly auth login
$ fly launch
- follow prompts to create an app name and specify a regtion
- Note with this app there is no need to create a database - so chose 'n' when prompted
- this process will create/update 3 files - dockerfile, .dockerignore, fly.toml
Mandatory fields:
$ flyclt secrets set OPENAI_API_KEY=XXXXXX PINECONE_API_KEY=XXXX PINECONE_ENVIRONMENT=XXX PINECONE_INDEX_NAME=XXX
Optional fields:
$ flyclt secrets set CHAT_PAGE_TITLE="Chat With Your Legal Docs" WELCOME_MESSAGE="Hi, ..." USER_INPUT_PLACEHOLDER="What is this...?"
FOOTER_URL=https://twitter.com/... FOOTER_TEXT="Powered by LangChainAI..."
- note you can create them in one command leaving a space between each field=value entrie. For fields with spaces you need to add quotes.
- Default values will be used for any optional field not specified.
$ fly deploy
- You will see the url for your application from the console output - e.g. my-app/.fly.dev/#
In general, keep an eye out in the issues
and discussions
section of this repo for solutions.
General errors
- Make sure you're running the latest Node version. Run
node -v
- Try a different PDF or convert your PDF to text first. It's possible your PDF is corrupted, scanned, or requires OCR to convert to text.
Console.log
theenv
variables and make sure they are exposed.- Make sure you're using the same versions of LangChain and Pinecone as this repo.
- Check that you've created an
.env
file that contains your valid (and working) API keys, environment and index name. - If you change
modelName
inOpenAI
, make sure you have access to the api for the appropriate model. - Make sure you have enough OpenAI credits and a valid card on your billings account.
- Check that you don't have multiple OPENAPI keys in your global environment. If you do, the local
env
file from the project will be overwritten by systemsenv
variable. - If you make changes to the
.env
variables - a restart of the server may be required.
Pinecone errors
- Make sure your pinecone dashboard
environment
andindex
match the ones in the.env
files. - Check that you've set the vector dimensions to
1536
. - Make sure your pinecone namespace is in lowercase.
- Pinecone indexes of users on the Starter(free) plan are deleted after 7 days of inactivity. To prevent this, send an API request to Pinecone to reset the counter before 7 days.
- Retry from scratch with a new Pinecone project, index, and cloned repo.
- Make sure you have OpenAI API credits on your account - the openAI API is required for the ingest process.
This repo is forked from mayooear/gpt4-pdf-chatbot-langchain