Client Side Tool Calling with the OpenAI WebRTC Realtime API

This project is a Cloudflare Workers app using Hono to relay the OpenAI Realtime API over WebRTC. The main files are just static assets.

Develop

Copy .dev.vars.example to .dev.vars and fill out your OpenAI API Key.

Install your dependencies

npm install

Run local server

npm run dev

Upload your secret

npx wrangler secret put OPENAI_API_KEY

npm run deploy

The hand is a HiWonder AI Hand. AI and I reverse-engineered the mobile app to make it work over Bluetooth, see the code in hand.js