Project Setup Guide

Create Project Folder and Environment Setup

# Create a new project folder
mkdir <project_folder_name>

# Move into the project folder
cd <project_folder_name>

# Open the folder in VS Code
code .

# Create a new Conda environment with Python 3.10
conda create -p <env_name> python=3.10 -y

# Activate the environment (use full path to the environment)
conda activate <path_of_the_env>

# Install dependencies from requirements.txt
pip install -r requirements.txt

# Initialize Git
git init

# Stage all files
git add .

# Commit changes
git commit -m "<write your commit message>"

# Push to remote (after adding remote origin)
git push

# Cloning the repository

git clone https://github.com/dsaakash/document_portal.git


## Minimum Requirements for the Project

1.  LLm models ## groq(freely) ,openai(paid), claude(paid),huggingface(freely),
ollama(local setup)

2. Embedding Models ## openai,hf,gemini
3. Vector Databases ## In-Memory ,On-Disk,Cloud-Based

## Models and Databases

This section provides a detailed overview of the LLM models, embedding models, and vector databases that can be used in the project.

### Project Structure

-   **Data Ingestion**: Ingest data from various sources.
-   **Data Processing**: Process and clean the ingested data.
-   **Embedding Creation**: Create embeddings using specified models.
-   **Vector Database Storage**: Store embeddings in a vector database.
-   **Querying**: Query the vector database to retrieve relevant information.
-   **Response Generation**: Generate responses using LLM models.


### LLM Models
- **Groq** (Free)
- **OpenAI** (Paid)
- **Gemini** (15 Days Free Access)
- **Claude** (Paid)
- **Hugging Face** (Free)
- **Ollama** (Local Setup)

### Embedding Models
- **OpenAI**
- **Hugging Face**
- **Gemini**

### Vector Databases
- **In-Memory**
- **On-Disk**
- **Cloud-Based**

## API Keys

### GROQ API Key
- [Get your API Key](https://console.groq.com/keys)  
- [Groq Documentation](https://console.groq.com/docs/overview)

### Gemini API Key
- [Get your API Key](https://aistudio.google.com/apikey)  
- [Gemini Documentation](https://ai.google.dev/gemini-api/docs/models)