What is Tuwon?

Tuwon is an AI-powered study platform that lets you create custom assessments through natural conversation, then take them in a voice-based quiz guided by a real-time AI proctor. Chat with Tuwon to define your topic, question count, and preferences, then step into an interactive session where the AI reads questions aloud, listens to your answers, and gives instant feedback.

---

Architecture

flowchart TB
    subgraph client [User]
        Browser[Browser]
    end

    subgraph appService [Azure App Service]
        subgraph home [tuwonHome]
            ChatAPI["/api/chat"]
            SessionsAPI["/api/sessions"]
        end
        subgraph session [tuwonSession]
            TokenAPI["/api/token"]
        end
    end

    subgraph azure [Azure Services]
        AzureChat["Azure OpenAI Chat"]
        AzureRealtime["Azure OpenAI Realtime"]
        AzurePG[(Azure PostgreSQL)]
    end

    Browser -->|"1. Chat"| ChatAPI
    ChatAPI --> AzureChat
    Browser -->|"2. Save session"| SessionsAPI
    SessionsAPI --> AzurePG

    Browser -->|"3. Start quiz"| TokenAPI
    TokenAPI -->|"Ephemeral token"| AzureRealtime
    Browser <-->|"4. Voice + function calling"| AzureRealtime

Loading

Flow:

1. tuwonHome – User chats with Tuwon to define topic, questions, and settings. Azure OpenAI generates the assessment JSON.
2. Sessions API – Assessment is stored in Azure PostgreSQL.
3. tuwonSession – User starts a voice quiz. The app fetches an ephemeral token from Azure and connects via WebRTC to the Realtime API.
4. GPT-REALTIME-1.5 – The Realtime model uses function calling to drive the quiz UI (select answer, submit, next question, get progress).

---

Agentic AI

Tuwon uses agentic AI in the Realtime session: the model orchestrates the quiz by calling tools that update the UI and state.

Tool Definitions

Tool	Purpose
select_answer	Visually highlight an answer option when the student indicates their choice
submit_answer	Lock in the answer and return correctness + explanation
next_question	Advance to the next question or end the assessment
get_assessment_progress	Return current score, question index, and time remaining

How It Works

1. System prompt – The model receives detailed instructions (in buildSystemPrompt.ts) to use tools for every answer and transition. It maps user speech (shapes, colors, "true/false") to option IDs.
2. Function calling loop – On response.output_item.done (function_call), the client parses arguments, runs handleToolCall, dispatches to the assessment state machine, and returns JSON to the model.
3. Semantic turn detection – The Realtime API uses semantic_vad with interrupt_response and create_response for natural voice interaction.
4. State machine – Quiz state (current question, score, selected option) is managed in React; tools read and update this state.

The AI never "talks about" actions—it calls tools. The UI only updates when tools are invoked.

---

Azure Services

Tuwon is deployed on Azure App Service. Both tuwonHome and tuwonSession run as separate App Service Web Apps (Linux, Node.js 20).

Service	Usage
Azure App Service	Hosting for tuwonHome (port 3000) and tuwonSession (port 3001)
Azure OpenAI	Chat completions in tuwonHome for conversational session creation
Azure OpenAI Realtime API	Voice quiz in tuwonSession: WebRTC, function calling, ephemeral tokens
Azure Database for PostgreSQL	Session storage (assessment_sessions table)

---

Tech Stack

• tuwonHome: Next.js 15, React 19, TypeScript, pg, openai (Azure)
• tuwonSession: Next.js 15, React 19, TypeScript, Tailwind CSS 4

---

Project Structure

tuwonApp/
├── tuwonHome/          # Conversation + session creation (port 3000)
│   ├── src/
│   │   ├── app/        # Pages, API routes
│   │   ├── lib/        # openai, db
│   │   └── components/
│   └── .env.example
├── tuwonSession/       # Voice quiz (port 3001)
│   ├── src/
│   │   ├── app/        # Pages, API routes
│   │   ├── hooks/      # useRealtimeSession, useAssessment
│   │   └── lib/        # tools, buildSystemPrompt
│   └── .env.example
└── README.md

---

Prerequisites

• Node.js 20 LTS
• Azure OpenAI resource with:
  • Chat deployment (e.g. gpt-4o or gpt-5.4)
  • Realtime deployment (e.g. gpt-realtime-1.5)
• Azure Database for PostgreSQL (or compatible Postgres)

---

Environment Variables

tuwonHome (.env.local)

# Azure OpenAI
AZURE_OPENAI_API_KEY=your-azure-openai-api-key
AZURE_OPENAI_ENDPOINT=https://your-resource.openai.azure.com
AZURE_OPENAI_DEPLOYMENT=gpt-5.4

# Azure PostgreSQL
DATABASE_URL=postgresql://user:password@your-server.postgres.database.azure.com:5432/tuwon?sslmode=require

# tuwonSession URL (for redirects)
NEXT_PUBLIC_TUWON_SESSION_URL=http://localhost:3001

tuwonSession (.env.local)

# Azure OpenAI Realtime
AZURE_OPENAI_API_KEY=your-azure-openai-api-key
AZURE_OPENAI_ENDPOINT=https://your-resource.openai.azure.com
AZURE_OPENAI_REALTIME_DEPLOYMENT=gpt-realtime-1.5

# tuwonHome API (for fetching sessions)
TUWON_HOME_URL=http://localhost:3000

---

Database Schema

Create the assessment_sessions table in your PostgreSQL database:

CREATE TABLE assessment_sessions (
  session_id UUID PRIMARY KEY,
  schema_version VARCHAR(20) NOT NULL,
  status VARCHAR(50) NOT NULL,
  title VARCHAR(500) NOT NULL,
  summary TEXT,
  source_app VARCHAR(100),
  source_model VARCHAR(100),
  conversation_id VARCHAR(255),
  idempotency_key UUID,
  idempotency_payload_hash VARCHAR(64),
  payload_json JSONB NOT NULL,
  created_at TIMESTAMPTZ DEFAULT NOW(),
  updated_at TIMESTAMPTZ DEFAULT NOW()
);

---

Local Setup

1. Clone and install

   cd tuwonApp
   cd tuwonHome && npm install   # or: pnpm install
   cd ../tuwonSession && npm install

2. Configure environment

   • Copy tuwonHome/.env.example to tuwonHome/.env.local
   • Copy tuwonSession/.env.example to tuwonSession/.env.local
   • Fill in Azure and database credentials

3. Create the database table

   Run the SQL above against your PostgreSQL instance.

4. Run both apps

   Terminal 1 (tuwonHome):

   cd tuwonHome && npm run dev   # or: pnpm run dev

   Terminal 2 (tuwonSession):

   cd tuwonSession && npm run dev

5. Open

   • tuwonHome: http://localhost:3000
   • tuwonSession: http://localhost:3001

---

License

MIT