Groq is a platform for running large language models (LLMs) in a token-based pricing model, without having to manage your own infrastructure. Groq uses LPU Inference Engines, a new type of end-to-end processing unit system that provides fast inference for computationally intensive systems. Sign up for an account at GroqCloud and get an API token, which you'll need for this project.
This repository showcases a simple Streamlit app for running the following open-source text models on Groq:
- Text: Meta Llama 2 70B
- Text: Google Gemma 7B Instruct
- Text: Mixtral 8x7B Instruct
