LangChain and Streamlit RAG

Demo App on Community Cloud

Quickstart

Setup Python environment

The Python version used when this was developed was 3.10.13

python -mvenv .venv
source .venv/bin/activate
pip install -U pip
pip install -r requirements.txt

If you run into issues related to hnswlib or chroma-hnswlib while installing requirements you may need to install system package for the underlying package.

For example, on Ubuntu 22.04 this was needed before pip install of hnswlib would succeed.

sudo apt install python3-hnswlib

Setup .env file with API tokens needed.

OPENAI_API_KEY="<Put your token here>"
HUGGINGFACEHUB_API_TOKEN="<Put your token here>"

Setup Streamlit app secrets.

1. Set up the .streamlit directory and secrets file.

mkdir .streamlit
touch .streamlit/secrets.toml
chmod 0600 .streamlit/secrets.toml

2. Edit secrets.toml

Either edit secrets.toml in you favorite editor.

OPENAI_API_KEY="<Put your token here>"
HUGGINGFACEHUB_API_TOKEN="<Put your token here>"

Or, you can just reuse .env contents from above.

cat < .env >> .streamlit/secrets.toml

Verify Environment

1. Check that LangChain dependencies are working.

python basic_chain.py

2. Check that Streamlit and dependencies are working.

streamlit run streamlit_app.py

3. Run Individual Example Programs

The most of the Python source files besides streamlit_app.py have a main defined so that you can execute them directly as an example or test.

For example, the main in ensemble.py will use context from an online version of the book The Problems of Philosophy by Bertrand Russell to answer "What are the key problems of philosophy according to Russell?"

python ensemble.py

Split into 313 chunks According to Russell, the key problems of philosophy include the uncertainty of knowledge, the limitations of metaphysical reasoning, and the inability to provide definite answers to fundamental questions. Philosophy aims to diminish the risk of error, but cannot eliminate it entirely due to human fallibility. The value of philosophy lies in its ability to challenge common sense beliefs and lead to the exploration of complex problems.

Usage

As more files are uploaded to the data folder (current compatibility: PDF format ONLY), the data is indexed and upon running, a pipeline is built that creates the embeddings for the given documents. In addition to this, the document can be viewed from the browse data tab inside the web application.

References

Gordon V. Cormack, Charles L A Clarke, and Stefan Buettcher. 2009. Reciprocal rank fusion outperforms condorcet and individual rank learning methods. In Proceedings of the 32nd international ACM SIGIR conference on Research and development in information retrieval (SIGIR '09). Association for Computing Machinery, New York, NY, USA, 758–759. https://doi.org/10.1145/1571941.1572114.

Jiang, A. Q., Sablayrolles, A., Mensch, A., Bamford, C., Singh Chaplot, D., de las Casas, D., … & El Sayed, W. (2023). Mistral 7B. arXiv e-prints, arXiv-2310. https://doi.org/10.48550/arXiv.2310.06825.

Lewis, P., Perez, E., Piktus, A., Petroni, F., Karpukhin, V., Goyal, N., … & Kiela, D. (2020). Retrieval-augmented generation for knowledge-intensive nlp tasks. Advances in Neural Information Processing Systems, 33, 9459–9474.

Liu, N. F., Lin, K., Hewitt, J., Paranjape, A., Bevilacqua, M., Petroni, F., & Liang, P. (2024). Lost in the middle: How language models use long contexts. Transactions of the Association for Computational Linguistics, 12, 157–173.

Robertson, S., & Zaragoza, H. (2009). The probabilistic relevance framework: BM25 and beyond. Foundations and Trends® in Information Retrieval, 3(4), 333–389. https://doi.org/10.1561/1500000019

Thibault Formal, Benjamin Piwowarski, and Stéphane Clinchant. 2021. SPLADE: Sparse Lexical and Expansion Model for First Stage Ranking. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR '21). Association for Computing Machinery, New York, NY, USA, 2288–2292. https://doi.org/10.1145/3404835.3463098.

Tunstall, L., Beeching, E., Lambert, N., Rajani, N., Rasul, K., Belkada, Y., … & Wolf, T. (2023). Zephyr: Direct Distillation of LM Alignment. arXiv e-prints, arXiv-2310. https://doi.org/10.48550/arXiv.2310.16944.

Misc Notes

• There is an issue with newer langchain package versions and streamlit chat history, see langchain-ai/langchain#18834
  • This one reason why a number of dependencies are pinned to specific values.