How can I add AI chat functionality to my application using an LLM API?

[bungakarunanidhi]

🏷️ Discussion Type

Question

💬 Feature/Topic Area

Other

Body

Hi, I’m building an AI application and I need some guidance.

I want to build an AI chat application with:

• A chat UI
• An AI/LLM model
• User login and user accounts
• Memory for each user
• PDF/file upload and document understanding
• A database to store user data and chat history

I would like to understand the best way to structure this project using GitHub and what technologies/services I should use.

What I expected:
I expected to find a recommended architecture or example project that explains how these components can work together.

What I need help with:
What would be the recommended approach for building this application, and are there any existing GitHub repositories or examples I can use as a starting point?

Thank you.

[krishanth7]

Building a Full-Stack AI Chat Application — Architecture and Technology Recommendations

What I am trying to build

I am building a full-stack AI chat application similar in concept to an AI assistant platform.

The application should allow users to:

• Create an account
• Log in securely
• Have private conversations with an AI
• Maintain conversation history
• Upload PDFs and other documents
• Ask questions about uploaded documents
• Have long-term AI memory
• Search previous conversations
• Eventually use web search, tools, agents, images, voice, and other AI capabilities

I am looking for advice on the correct technical architecture and recommended technologies before I start building the application.

---

1. AI Chat Interface

I want to create a modern web-based chat interface.

Users should be able to:

• Create a new conversation
• Send messages to the AI
• Receive streaming AI responses
• Continue previous conversations
• Rename conversations
• Delete conversations
• Search conversation history
• Display user and AI messages separately
• Support Markdown
• Support code blocks
• Upload files
• Eventually support images and other file types

I am considering:

• Next.js
• React
• TypeScript
• Tailwind CSS

Is this a good frontend stack for an AI application?

---

2. AI / LLM Model

I don't want to run a large language model locally initially.

I want to use an API-based LLM provider.

Possible providers include:

• OpenAI
• Google Gemini
• Anthropic
• Other inference providers
• Open-source models through an API

My requirements are:

• Good reasoning
• Good coding ability
• Streaming responses
• Low latency
• Reasonable cost
• Long context support
• Tool/function calling
• Document understanding
• Image understanding eventually

I also want the architecture to allow me to change the AI provider later.

For example:

Frontend
    |
    v
Backend API
    |
    v
AI Abstraction Layer
    |
    +-------- OpenAI
    |
    +-------- Gemini
    |
    +-------- Anthropic
    |
    +-------- Other Models

Would this be a good architecture?

Should I create an abstraction layer so that changing the LLM provider does not require rewriting the entire application?

---

3. User Authentication

Every user should have a separate account.

I need:

• Email/password login
• Google login
• Email verification
• Password reset
• Secure sessions
• User profile
• Account deletion
• User settings
• Subscription/payment support later

I am considering:

• Supabase Auth
• Firebase Authentication
• Auth.js
• Clerk

Which would you recommend for an AI startup MVP?

Security is very important because users will have private conversations and private documents.

I want to guarantee that:

User A
   |
   X
   |
User B's conversations/files

is never possible.

---

4. Database

I need to store:

Users
Conversations
Messages
Files
Documents
Document chunks
Memories
Usage
Subscriptions
Settings

A possible database structure is:

users
    |
    +--- conversations
    |        |
    |        +--- messages
    |
    +--- documents
    |        |
    |        +--- document_chunks
    |
    +--- memories
    |
    +--- usage
    |
    +--- settings

I am considering:

• PostgreSQL
• Supabase PostgreSQL
• Firebase Firestore

I am leaning toward PostgreSQL because the application has many relationships between users, conversations, messages, documents and memories.

Would PostgreSQL be the better choice?

---

5. AI Memory

One of the most important features I want is AI memory.

I don't want to send the entire conversation history to the LLM every time because:

• It increases token usage
• It increases cost
• It increases latency
• Eventually the context window will become a problem

I am thinking about separating memory into different types.

Short-term memory

Recent conversation messages.

User
    |
    v
Current conversation
    |
    v
Recent messages

Long-term memory

Important information about the user.

For example:

The user is building an RAS system.
The user prefers detailed technical explanations.
The user is working on an AI application.

Semantic memory

I am considering storing embeddings for important memories.

For example:

Memory:
"The user is building an RAS system."

Embedding:
[0.0123, -0.0345, 0.0876, ...]

Then when the user asks a question, the system performs semantic search and retrieves relevant memories.

Possible technologies:

• pgvector
• Pinecone
• Qdrant
• Weaviate
• Other vector databases

Would PostgreSQL + pgvector be enough for the initial version?

---

6. PDF / Document Upload

I want users to upload:

• PDF
• DOCX
• TXT
• CSV
• Images eventually

The user should then be able to ask questions about the uploaded document.

For example:

User uploads:
Project_DPR.pdf

User asks:

"What is the total project cost?"

AI answers:

"The total project cost is ₹70 lakh."

Source:
Page 18

I don't want to send the entire document to the LLM for every question.

I am considering a RAG architecture.

---

7. RAG Architecture

My proposed RAG pipeline is:

User uploads PDF
        |
        v
File Storage
        |
        v
PDF Text Extraction
        |
        v
Text Cleaning
        |
        v
Document Chunking
        |
        v
Embedding Generation
        |
        v
Vector Database
        |
        v
Document Ready

Then when the user asks a question:

User Question
        |
        v
Create Query Embedding
        |
        v
Vector Similarity Search
        |
        v
Retrieve Relevant Chunks
        |
        v
Build Context
        |
        v
LLM
        |
        v
Answer

Is this the correct architecture?

---

8. Document Citations

I also want the AI to provide citations.

For example:

According to the uploaded DPR, the total project cost is ₹70 lakh.

Source:
DPR.pdf
Page 18

Therefore, I think every document chunk should contain metadata such as:

document_id
chunk_id
page_number
text
embedding

Is this enough to implement reliable page-level citations?

---

9. File Storage

I need secure storage for uploaded documents.

Possible options:

• Supabase Storage
• Amazon S3
• Cloudflare R2
• Firebase Storage

Requirements:

• Private files
• User-specific access control
• Secure URLs
• File deletion
• Reasonable storage cost
• Large file support

What would you recommend for an MVP?

---

10. Backend Architecture

I am unsure whether I should keep everything inside Next.js or create a separate backend.

Option A

Next.js
    |
    +--- Frontend
    |
    +--- API Routes
    |
    +--- AI Logic
    |
    +--- Database

Option B

Next.js
    |
    | Frontend
    |
    v
FastAPI / Python
    |
    +--- AI
    +--- RAG
    +--- Memory
    +--- Document Processing
    +--- Database

Which architecture would you recommend?

I eventually want to add:

• AI agents
• Tool calling
• Web search
• Image understanding
• Voice
• Background processing
• Document processing
• Autonomous workflows
• Multiple AI providers

I therefore want to avoid an architecture that will need to be completely rebuilt later.

---

11. Python vs TypeScript

I am also trying to decide whether I should use:

Next.js + TypeScript

for everything,

or:

Next.js + TypeScript
+
Python + FastAPI

I understand that Python has a very strong AI/ML ecosystem.

Would using Python for the AI backend be beneficial if I eventually want to add:

• Machine learning
• Computer vision
• AI agents
• Advanced RAG
• Embedding models
• Data processing
• Robotics/AI integrations

Or would this be unnecessary complexity for the first version?

---

12. Background Processing

Large PDFs can take time to process.

I don't want the user's HTTP request to remain open while the document is being processed.

I am considering:

Upload PDF
    |
    v
Create Processing Job
    |
    v
Queue
    |
    v
Worker
    |
    +--- Extract text
    |
    +--- Chunk document
    |
    +--- Generate embeddings
    |
    +--- Store vectors
    |
    v
Document = READY

Possible technologies:

• Redis + BullMQ
• Celery
• Inngest
• Trigger.dev
• Cloud Tasks
• Other queue systems

What would you recommend for an MVP?

---

13. Security

Security is extremely important because users will upload private information.

I want to protect against:

• Unauthorized access
• User-to-user data leaks
• API key exposure
• SQL injection
• XSS
• CSRF
• Malicious file uploads
• Prompt injection
• Unauthorized file downloads
• Database attacks
• Excessive API usage
• Abuse of the AI API

I believe the architecture should be:

Browser
    |
    v
My Backend
    |
    v
LLM Provider

and NOT:

Browser
    |
    v
LLM Provider

because the API key must never be exposed to the browser.

Is this correct?

What other security practices should I implement from the beginning?

---

14. Environment Variables

I plan to store secrets such as:

OPENAI_API_KEY
GEMINI_API_KEY
ANTHROPIC_API_KEY
DATABASE_URL
SUPABASE_SERVICE_ROLE_KEY
STORAGE_SECRET

in environment variables.

For local development:

.env.local

For production:

Hosting Provider Environment Variables

I will not commit secrets to GitHub.

Is this the correct approach?

---

15. API Architecture

I am thinking about having APIs similar to:

/api/auth
/api/chat
/api/conversations
/api/messages
/api/files
/api/documents
/api/memory
/api/search
/api/usage

For example:

POST /api/chat

would receive:

{
  "conversationId": "123",
  "message": "Explain this document"
}

Then the backend would:

Authenticate user
       |
       v
Load conversation
       |
       v
Retrieve relevant memory
       |
       v
Retrieve relevant document chunks
       |
       v
Build LLM context
       |
       v
Call LLM
       |
       v
Stream response
       |
       v
Save message

Is this a reasonable API architecture?

---

16. Recommended Database Structure

I am thinking about something like:

users
----------------
id
email
name
created_at


conversations
----------------
id
user_id
title
created_at
updated_at


messages
----------------
id
conversation_id
role
content
created_at


documents
----------------
id
user_id
filename
storage_path
status
created_at


document_chunks
----------------
id
document_id
page_number
chunk_index
content
embedding


memories
----------------
id
user_id
content
embedding
memory_type
created_at


usage
----------------
id
user_id
tokens
cost
created_at

Would this be a good starting schema?

What important tables or fields am I missing?

---

17. Multi-Tenant Data Isolation

Because multiple users will use the application, I need strong data isolation.

For example:

User A
    |
    +--- Conversation A
    +--- Document A
    +--- Memory A

User B
    |
    +--- Conversation B
    +--- Document B
    +--- Memory B

User A should never be able to query:

Conversation B
Document B
Memory B

I am considering PostgreSQL Row Level Security if I use Supabase.

Would RLS be the correct approach?

---

18. Cost Control

I also want to keep the initial cost low.

The application may have:

• AI API costs
• Database costs
• Storage costs
• Embedding costs
• Hosting costs
• Background processing costs
• Bandwidth costs

I want to prevent one user from consuming unlimited AI resources.

I am considering:

User
   |
   v
Usage Tracking
   |
   +--- Requests
   +--- Input Tokens
   +--- Output Tokens
   +--- Storage
   +--- Documents
   |
   v
Usage Limits

Should I implement usage tracking from the beginning?

---

19. Deployment

For the first production version I am considering:

Frontend:
Next.js + Vercel

Authentication:
Supabase Auth

Database:
Supabase PostgreSQL

Vector Search:
pgvector

File Storage:
Supabase Storage

AI:
OpenAI / Gemini / Anthropic

Background Jobs:
Inngest / Trigger.dev

Monitoring:
Sentry / similar

Would this be a reasonable production MVP architecture?

---

20. Possible Final Architecture

The architecture I currently have in mind is:

                         USER
                           |
                           v
                    ┌──────────────┐
                    │   Next.js    │
                    │   Frontend   │
                    └──────┬───────┘
                           |
                           v
                    ┌──────────────┐
                    │     API      │
                    │    Layer     │
                    └──────┬───────┘
                           |
          ┌────────────────┼────────────────┐
          |                |                |
          v                v                v
     Authentication    Chat Service    File Service
          |                |                |
          v                v                v
      PostgreSQL       AI Service      Object Storage
          |                |
          |                v
          |         ┌──────────────┐
          |         │  AI Provider │
          |         └──────────────┘
          |
          v
      pgvector
          |
          +------- Memories
          |
          +------- Document chunks

For document processing:

PDF
 |
 v
Object Storage
 |
 v
Background Job
 |
 v
Text Extraction
 |
 v
Chunking
 |
 v
Embedding
 |
 v
PostgreSQL + pgvector
 |
 v
READY

For a document question:

User Question
 |
 v
Authentication
 |
 v
Conversation Context
 |
 +---- Recent Messages
 |
 +---- Relevant Memories
 |
 +---- Relevant Document Chunks
 |
 v
Prompt / Context
 |
 v
LLM
 |
 v
Streaming Response
 |
 v
Save Message

---

21. Project Structure

I am considering the following project structure:

my-ai-app/
│
├── app/
│   ├── login/
│   ├── signup/
│   ├── dashboard/
│   ├── chat/
│   ├── settings/
│   └── api/
│       ├── chat/
│       ├── conversations/
│       ├── files/
│       ├── documents/
│       ├── memory/
│       └── usage/
│
├── components/
│   ├── chat/
│   ├── sidebar/
│   ├── file-upload/
│   └── ui/
│
├── lib/
│   ├── ai/
│   ├── auth/
│   ├── database/
│   ├── memory/
│   ├── rag/
│   ├── storage/
│   └── security/
│
├── workers/
│   ├── document-processing/
│   └── embeddings/
│
├── database/
│   ├── migrations/
│   └── schema/
│
├── public/
│
├── tests/
│
├── .env.local
├── package.json
└── README.md

Is this structure reasonable?

---

22. Development Order

I am thinking about developing the application in this order:

Phase 1 — Basic Application

1. Create Next.js project
2. Create UI
3. Create authentication
4. Create database
5. Create basic chat
6. Connect one LLM

Phase 2 — Conversations

7. Save conversations
8. Save messages
9. Conversation history
10. Conversation search

Phase 3 — Documents

11. File upload
12. File storage
13. PDF extraction
14. Document chunking
15. Embeddings
16. pgvector
17. RAG
18. Citations

Phase 4 — Memory

19. Memory extraction
20. Memory storage
21. Semantic memory retrieval
22. Memory management

Phase 5 — Production

23. Rate limiting
24. Usage tracking
25. Security
26. Error handling
27. Logging
28. Monitoring
29. Background processing
30. Deployment

Phase 6 — Advanced AI

31. Tool calling
32. Web search
33. AI agents
34. Image understanding
35. Voice
36. Advanced workflows

Does this development order make sense?

---

23. My Main Questions

I would really appreciate feedback from people who have built production AI applications.

Specifically:

1. Would you use Next.js for this?
2. Would you use PostgreSQL?
3. Would you use pgvector instead of a dedicated vector database initially?
4. Would you use Supabase?
5. Would you use Python/FastAPI or keep the backend in TypeScript?
6. How would you implement long-term memory?
7. How would you implement RAG?
8. How would you process large PDFs?
9. How would you implement document citations?
10. What authentication system would you recommend?
11. What storage system would you recommend?
12. What background-job system would you recommend?
13. What security problems should I consider?
14. What would you change in my proposed architecture?
15. What would you NOT build initially?
16. Are there any good open-source GitHub repositories demonstrating this architecture?

---

24. What I Am Looking For

I am not only looking for a list of technologies.

I want to understand:

• Why each technology should be used
• What problem it solves
• How the components communicate
• How data flows through the application
• How authentication works
• How memory works
• How RAG works
• How documents are processed
• How users are isolated
• How API keys are protected
• How costs are controlled
• How the application can scale later

I am relatively new to building a complete production AI application, so detailed architectural explanations would be very helpful.

If you have built a similar AI application, I would especially appreciate advice about mistakes you made early that I should avoid.

Thank you!