Skip to content
BranchesTags

Repository files navigation

Lively Logo

LISTEN. ADAPT. REMEMBER. ACT.

Python React FastAPI Agora Groq NVIDIA NIM Vercel Render

A real-time voice AI sales agent that listens to every word, adapts its strategy on the fly, remembers full deal context across turns, and acts booking demos, dispatching calendar invites, updating CRMs, and escalating to humans all while the buyer is still on the line.

Powered by the Agora Conversational AI Engine with a custom multi-LLM brain (Groq LPU + NVIDIA NIM).

Features · Architecture · Quick Start · Production Deployment · Demo · Environment Variables


Core Features

Capability What It Does
Real-Time Voice Conversations Sub-300ms round-trip latency with Agora's telecom-grade RTC — natural turn-taking, barge-in, echo cancellation, and noise suppression out of the box.
Adaptive Sales Brain Every utterance is analyzed for buyer intent, scale, budget, timeline, competitor mentions, and authority level. The agent adapts its pitch in real time.
Persistent Deal Memory Additive & mergeable deal state across all turns — nothing is forgotten. Budget changes? User scale updates? Lively merges them with a full change-log.
Autonomous Booking & Email Dispatch Proactively locks in calendar reservations, generates Google Meet links, and dispatches full .ics calendar invites directly to the user's inbox via SMTP.
RAG-Grounded Responses Battlecards, pricing sheets, and competitive intelligence are retrieved in real time so every claim is grounded in facts, not hallucinations.
Live Tool Calling book_meeting(), create_or_update_crm_lead(), escalate_to_human() — the agent takes real-world action while the conversation is unfolding.
Multi-LLM Router Groq LPU for ultra-low-latency voice turns, NVIDIA NIM for complex reasoning, and a built-in sales specialist brain as a zero-downtime fallback.
Resilient Live Telemetry Real-time TTFT, response latency, model distribution, memory diffs, and live transcription streamed via WebSocket with automatic reconnect and polling fallbacks.
Dark / Light Theme Editorial luxury design system with full dark mode and one-click Sun/Moon toggle.

System Architecture

┌────────────────────────────────────────────────────────────────┐
│                 BROWSER  (React / TypeScript)                  │
│                                                                │
│  Sales Cockpit UI       Agora RTC Web SDK       WebSocket      │
│  ┌──────────────┐      ┌──────────────────┐   ┌────────────┐   │
│  │ Transcript   │      │ Publish Mic Audio│   │ Telemetry  │   │
│  │ Deal Stage   │      │ Subscribe Agent  │   │ Stream     │   │
│  │ Battlecards  │      │ Speech           │   │ /ws/{ch}   │   │
│  │ Telemetry    │      └──────── ┬────────┘   └───── ┬─────┘   │
│  └──────────────┘                │                   │         │
└───────────────────────────────── ┼───────────────────┼─────────┘
                                   │                   │
                                   ▼                   ▼
┌────────────────────────────────────────────────────────────────┐
│            AGORA CONVERSATIONAL AI ENGINE                      │
│                                                                │
│  ● Turn-Taking & VAD       ● Barge-In Handling                 │
│  ● Echo Cancellation       ● Noise Suppression                 │
│  ● Streaming STT ──► [Custom LLM Endpoint] ──► TTS             │
└───────────────────────────────┬────────────────────────────────┘
                                │
                                ▼
┌────────────────────────────────────────────────────────────────┐
│            LIVELY INTELLIGENCE CORE  (FastAPI)                 │
│                                                                │
│  ┌──────────────────────────────────────────────────────────┐  │
│  │               /v1/chat/completions                       │  │
│  │  OpenAI-compatible streaming endpoint called by Agora    │  │
│  │                                                          │  │
│  │  1. LISTEN    → Extract intent, budget, competitors      │  │
│  │  2. ADAPT     → RAG retrieval + next-best-action         │  │
│  │  3. REMEMBER  → Additive deal state merge + change-log   │  │
│  │  4. ACT       → Tool-calling & autonomous email dispatch │  │
│  └──────────────────────────────────────────────────────────┘  │
│                                                                │
│  ┌────────────────┐  ┌──────────────┐  ┌───────────────────┐   │
│  │  LLM Router    │  │  Tool Layer  │  │ Telemetry Engine  │   │
│  │                │  │              │  │                   │   │
│  │  Groq LPU      │  │ book_meeting │  │ TTFT / p50 / p95  │   │
│  │  (sub-200ms)   │  │ update_crm   │  │ Model distrib.    │   │
│  │                │  │ escalate     │  │ Memory diffs      │   │
│  │  NVIDIA NIM    │  │ send_invite  │  │ Stage tracking    │   │
│  │  (reasoning)   │  └──────────────┘  └───────────────────┘   │
│  │                │                                            │
│  │  Built-in      │                                            │
│  │  (fallback)    │                                            │
│  └────────────────┘                                            │
│                                                                │
│  ┌──────────────────────────────────────────────────────────┐  │
│  │                   Email & Calendar                       │  │
│  │  SMTP Dispatcher · ICS Calendar Generator · Google Meet  │  │
│  └──────────────────────────────────────────────────────────┘  │
│                                                                │
│  ┌──────────────────────────────────────────────────────────┐  │
│  │                   Storage Layer                          │  │
│  │  SQLite (local dev) / Postgres (production)              │  │
│  │  DealStateTable · EscalationQueue · Analytics            │  │
│  └──────────────────────────────────────────────────────────┘  │
└────────────────────────────────────────────────────────────────┘

Architecture Diagram of Lively

Quick Start (Local Development)

Prerequisites

Requirement Version Download
Python 3.11+ python.org/downloads , Make sure to check "Add Python to PATH" on installation
Node.js 18+ nodejs.org
Git Any git-scm.com

Step 1: Clone the Repository

git clone https://github.com/SujaydRNSIT/Lively.git
cd Lively

Step 2: Install Everything

Double-click install.bat or run from your terminal:

install.bat

This automated script:

  1. Detects Python and creates a .venv virtual environment.
  2. Installs backend dependencies from backend/requirements.txt.
  3. Installs frontend npm packages.
  4. Generates .env from .env.example if missing.
  5. Downloads cloudflared.exe for the local Agora webhook tunnel.

Step 3: Configure API Keys

Open .env and fill in your keys:

# ── Agora Conversational AI ──────────────────────
AGORA_APP_ID=your_agora_app_id
AGORA_APP_CERTIFICATE=your_agora_app_certificate
AGORA_REST_KEY=your_agora_rest_key
AGORA_REST_SECRET=your_agora_rest_secret

# ── LLM Providers ────────────────────────────────
GROQ_API_KEY=gsk_...                      # Primary: low-latency voice turns
NVIDIA_NIM_API_KEY=nvapi-...              # Secondary: complex reasoning

# ── SMTP / Real Email & Calendar Invites ────────
SMTP_HOST=smtp.gmail.com
SMTP_PORT=587
SMTP_USER=yourname@gmail.com
SMTP_PASSWORD=your_16_char_google_app_password
SMTP_FROM="Lively AI <yourname@gmail.com>"
SMTP_FROM_EMAIL=yourname@gmail.com
SMTP_USE_TLS=true

Step 4 — Launch Lively

Double-click run.bat or run:

run.bat

run.bat automatically:

  • Clears any stale processes on ports 8000 & 5173.
  • Launches the Cloudflare tunnel for Agora webhooks.
  • Auto-updates BACKEND_PUBLIC_URL in .env.
  • Boots the FastAPI backend on port 8000.
  • Boots Vite on port 5173 and opens http://localhost:5173 in your browser.

(For separate terminal windows per service, run run_separate.bat instead.)


Production Deployment

Lively is configured for zero-friction cloud deployment:

1. Frontend (Vercel)

The repository includes a ready-to-use vercel.json:

  • Framework: Vite
  • Root Directory: .
  • Build Command: cd frontend && npm run build
  • Output Directory: frontend/dist
  • Rewrites: Automatically routes /api/* and /v1/* to your Render backend.
  • Connect your GitHub repository (/SujaydRNSIT/Lively) in the Vercel Dashboard.

2. Backend (Render)

  • Deploy as a Web Service on Render.
  • Repository: https://github.com//SujaydRNSIT/Lively.git (Branch: main)
  • Environment: Python 3
  • Build Command: pip install -r backend/requirements.txt
  • Start Command: cd backend && uvicorn app.main:app --host 0.0.0.0 --port $PORT
  • Environment Variables:
    • BACKEND_PUBLIC_URL: Your Render service URL (e.g. https://lively-8s3x.onrender.com without a trailing slash). (Lively also automatically detects RENDER_EXTERNAL_URL if omitted).
    • AGORA_APP_ID, AGORA_APP_CERTIFICATE, AGORA_REST_KEY, AGORA_REST_SECRET
    • GROQ_API_KEY
    • SMTP_USER, SMTP_PASSWORD, SMTP_FROM_EMAIL (for direct calendar email dispatches)

Demo Walkthrough

Use the built-in Scripted Demo Harness tab or speak live into your microphone:

Turn Buyer Says Lively Responds System Action
1 "Hi, we're evaluating real-time voice AI. How much does Lively cost?" Quotes Starter ($199/mo) and Growth ($699/mo) tiers with 40–60% TCO advantage over alternatives. RAG retrieval → pricing battlecard
2 "Wait, how do you compare to OpenAI Realtime and Twilio?" (barge-in) Triggers competitive battlecard — highlights Agora SD-RTN (<300ms latency) and freedom to swap custom LLM brains. Competitor detection → battlecard
3 "Actually our team expanded to 80 users and our budget is $100k ARR. I'm the VP of Product." Acknowledges the updates smoothly without repeating old context. Additive memory merge → deal state diff
4 "Can we schedule a live technical walkthrough tomorrow at 2 PM EST?" Confirms the booking, automatically dispatches the Google Meet bridge & calendar invite to the buyer's email, and renders the confirmed meeting card. Autonomous book_meeting() → SMTP dispatch → CRM update

Project Structure

Lively/
│
├── install.bat                 # One-click dependency setup (Python venv, npm, cloudflared)
├── run.bat                     # One-click full system launcher (tunnel + backend + frontend)
├── run_separate.bat            # Launch each service in its own CMD window
├── vercel.json                 # Vercel deployment configuration & API rewrites
├── .env.example                # Environment template with all configurable keys
├── .env                        # Local credentials (git-ignored)
│
├── backend/                     # FastAPI — Lively Intelligence Core
│   ├── app/
│   │   ├── main.py             # FastAPI entry point & router mounting
│   │   ├── config.py           # Pydantic settings with auto Render URL fallback
│   │   ├── api/                # REST & WebSocket route handlers
│   │   │   ├── deal_state.py   # State snapshots, contact capture & reset
│   │   │   ├── llm_proxy.py    # OpenAI-compatible /v1/chat/completions endpoint
│   │   │   └── tools.py        # Calendar booking & CRM sync endpoints
│   │   ├── core/               # Cognitive Sales Engine
│   │   │   ├── deal_state_engine.py   # Additive deal memory & autonomous booking
│   │   │   ├── decision.py            # Next-best-action decision engine
│   │   │   ├── llm_router.py          # Multi-provider LLM routing (Groq → NIM → fallback)
│   │   │   ├── prompts.py            # Persona & autonomous scheduling instructions
│   │   │   ├── rag.py                # RAG retrieval interface
│   │   │   └── tools_defs.py         # Function calling definitions & auto-dispatch
│   │   ├── services/
│   │   │   ├── agora_convo_api.py    # Agora Conversational AI agent lifecycle
│   │   │   ├── email_service.py      # SMTP & ICS calendar invite generator
│   │   │   └── rag_service.py        # Battlecard knowledge base retriever
│   │   └── routers/
│   │       └── telemetry.py          # WebSocket telemetry connection manager
│   ├── requirements.txt        # Core Python dependencies
│   └── tests/                  # Automated test suites
│
├── frontend/                    # React Sales Cockpit — Vite + TypeScript + Tailwind
│   ├── src/
│   │   ├── App.tsx             # Root component with direct WS & polling fallback
│   │   ├── index.css           # Global styles & design system tokens
│   │   ├── components/
│   │   │   ├── Navbar.tsx              # Brand identity, email pill & theme toggle
│   │   │   ├── LiveTranscriptStream.tsx # Real-time conversation turns
│   │   │   ├── ObjectionBattlecards.tsx # Objection detection & response cards
│   │   │   ├── DealStagePipeline.tsx    # Visual 5-stage pipeline tracker
│   │   │   ├── ActionItemsPanel.tsx     # Live telemetry, TTFT & model stats
│   │   │   ├── CalendarEventCard.tsx    # Clean Google Meet link & auto-dispatched status
│   │   │   ├── OutcomeBanner.tsx        # Reserved outcome banner with Reset button
│   │   │   └── VoiceCallHud.tsx         # WebRTC call controls & audio levels
│   │   ├── services/
│   │   │   └── api.ts                  # API client & WebSocket URL resolution
│   │   └── types/                      # TypeScript schemas & state definitions
│   └── public/
│       ├── logo-light.png      # Brand assets
│       └── logo-dark.png
│
└── LIMITATIONS.md              # Architectural notes & future roadmap

Environment Variables Reference

Variable Required Description
AGORA_APP_ID Yes Agora project App ID
AGORA_APP_CERTIFICATE Yes Agora project App Certificate
AGORA_REST_KEY Yes Agora REST API Key
AGORA_REST_SECRET Yes Agora REST API Secret
GROQ_API_KEY Recommended Groq LPU API key — primary low-latency voice turns
GROQ_MODEL — Model name (default: qwen/qwen3.8-27b)
NVIDIA_NIM_API_KEY — NVIDIA NIM API key — complex reasoning fallback
NVIDIA_NIM_MODEL — NIM model (default: meta/llama-3.1-70b-instruct)
BACKEND_PUBLIC_URL Cloud Public URL of the backend (e.g. https://lively-8s3x.onrender.com) without trailing slash
SMTP_HOST Email Outgoing mail server (e.g. smtp.gmail.com)
SMTP_PORT Email Port for TLS encryption (default: 587)
SMTP_USER Email Your authenticated mail account (e.g. you@gmail.com)
SMTP_PASSWORD Email 16-character Google App Password
SMTP_FROM Email Display header ("Lively AI <you@gmail.com>")
SMTP_FROM_EMAIL Email Sender address matching authenticated account
SMTP_USE_TLS Email Enable TLS encryption (default: true)
AGORA_AGENT_VOICE — Voice synthesis model (default: en-US-JennyNeural)
DEBUG — Enable verbose debugging (default: true)

Testing

Run the automated test suite locally:

# Activate virtual environment
.venv\Scripts\activate.bat   # Windows
# source .venv/bin/activate  # macOS / Linux

# Run test suites
python backend/tests/test_phase6_7_8.py
python backend/tests/test_email_and_contact.py

Built for the Agora Conversational AI Hackathon

Made by the Lively team

About

Real-time voice AI sales agent that listens, adapts, remembers customer context, and turns conversations into actionable sales outcomes. Powered by Agora Conversational AI.

lively-psi.vercel.app/

Resources

Readme
Activity

Stars

1 star

Watchers

0 watching

Forks

0 forks
Report repository

Releases

Packages

Contributors

Languages