opal

ai gaming companion powered by claude. plays games with you, talks to you in discord, sees your screen in real-time.

game agnostic — tested on cs:go, valorant, krunker, minecraft, racing games, and more.

what it does

• tft bot: captures screenshots → claude haiku extracts game state → decides what to buy/position/level → clicks for you via pyautogui
• csgo bot: yolo enemy detection + depth estimation + optical flow → navigates maps and shoots via chrome devtools protocol
• discord voice: joins your voice channel → listens via livekit → responds with elevenlabs tts → remembers conversation in markdown
• vision streaming: overshoot sdk streams your screen to claude in real-time so it knows what you're doing

architecture

app/          → react landing page with webgl ribbons and 3d macbook
desktop/      → tauri overlay that floats on your game
dev/src/      → python game bots (tft, csgo, racing, skribbl)
  ├── tft/         → perception/decision/execution pipeline
  ├── csgo/        → vision module + cdp control
  ├── discord-bot/ → livekit voice bridge + memory manager
  └── agent/       → opus voice agent

run

# web
cd app && npm i && npm run dev

# desktop overlay
cd desktop && npm i && npm run tauri dev

# tft bot
cd dev/src && uv run python -m tft

# csgo bot
cd dev/src && uv run python -m csgo

# discord voice (livekit branch)
cd dev/src/discord-bot && python bot.py

tech

claude haiku/sonnet, yolov11, depth anything v2, overshoot real-time vision, livekit, elevenlabs, assemblyai, silero vad, tauri, rust, react, three.js, webgl/glsl, framer motion, chrome devtools protocol, pyautogui, opencv, mss

branches

• main — game bots + landing page
• LiveKit-discord-integration — voice ai companion via discord

---

built for nexhacks 2026