pizzamon

pizzamon is bot to order you some 'za 🍕 over the phone. It leverages Gemini's duplex voice capabilities and some audio processing to interact with pizza places, placing orders based on user input.

This started as me trying to order pizza with a nodered flow for my Wife who calls people all day for her job.

⚠️ Warning: Don't be a jerk and use this on actual pizza places yet! Just call your own phone for now. The duplex voice features are in beta and can lead to awkward interactions. Just remember that this code has real-world consequences okay

✨ Features

This software is still in in development, you may encounter bugs or incomplete code.

• 🗣️ Natural Voice Interaction: Uses Gemini's advanced voice capabilities for realistic conversations.
• 🍕 Automated Ordering: Give it the order, and it handles the phone call.
• 🛡️ Human Handoff: Detects if the call is going south (angry human) and transfers the call to you.
• 🛠️ Developer Friendly: Easy to test locally with ngrok and simple curl commands.

📋 Prerequisites

Note: Batteries not included. You will need to set up your own environment and credentials.

• Python 3.11.12+
• A Twilio Account (Account SID, Auth Token, and a purchased phone number).
• A Google Gemini API Key.
• (optional) ngrok (for exposing your local server to Twilio's webhooks).

🚀 Development

Pro Tip: The best way to test is to call your own phone and pretend to be the pizza place. Try getting angry at the bot to test the transfer logic!

You don't have to be super enthusastic but if you really don't know what to say, go order some pizza yourself to see how it works, and write down your lines. What is fun to test is getting angry at it to make sure the call transfers.

Trigger a Call

curl -X POST http://127.0.0.1:8000/make-call \
-H "Content-Type: application/json" \
-d '{
    "to_number": "+15550000000",
    "webhook_url": "https://YOUR_NGROK_URL.ngrok-free.app/voice"
}'

Roadmap & Todo

• Call Completion Assurance: Ensure calls are explicitly marked as success or fail. If the status is indeterminate, default to failed.
• Improve make-call Logic: Refine how the webhook URL is handled and sent to Twilio.
  • Inject the specific order details as a chat prompt context.
• Support Merge Calling: so you can hear what the robot is doing live
• System Instructions Refinement:
  • Integrate call success/failure tools better into the system instructions.
  • Conversation Flow: Adjust behavior on answer (Say "Hello", wait, then proceed with intro).
  • Order Injection: Ensure the order details are properly injected into the bot's system instructions.
• Navigate Phone Trees: This one has real world consequences and is a bit scary to open-source