Every walk is an adventure.
Sniffari transforms an ordinary dog walk into an AI-generated scavenger hunt filled with real-world discoveries, collectible paw stamps, and humorous field reports voiced directly by your dog.
- Dog Profile & Passport Creation: Enter your dog's name, upload a photo, and customize their energy level (Chill, Playful, Zoomies) and personality archetype (Explorer, Curious, Princess, Chaos, Shy, Foodie).
- Personalized AI Mission Generation: Google Gemini generates 5 safe, engaging, and visually verifiable scavenger hunt missions tailored to the dog's personality and walk duration.
- Multimodal Vision Validation: The owner photographs the discovery along the walk. Google Gemini uses multimodal image understanding to inspect the photo, verify criteria satisfaction, extract structured visual evidence, and return a humorous canine field verdict.
- Collectible Paw Stamps: Successful missions earn animated vintage passport stamps with playful rotation and sound effects.
- AI Adventure Report: Once the adventure is complete, Google Gemini compiles a humorous first-person report written from the dog's point of view detailing the validated discoveries.
- Expressive Dog Narration: ElevenLabs converts the report into expressive spoken audio matched to the dog's personality, accompanied by a custom animated audio player and dog avatar reactions.
- Social Adventure Card: Share the official Sniffari card with human performance ratings and mission badges.
Built for the DEV Weekend Challenge: Dog Days Edition in celebration of our canine companions! Daily dog walks can often feel routine. Sniffari brings playfulness, mindfulness, and shared adventure to every walk, giving dogs a digital passport and an unforgettable AI voice to tell their heroic tales.
Google Gemini is foundational to the core Sniffari loop:
- Personalized Mission Generation (
POST /api/ai/missions): Generates 5 structured scavenger missions adhering to strict safety protocols (no trespassing, no ingestion hazards, visually verifiable). - Multimodal Photo Validation (
POST /api/ai/validate): Gemini inspects the actual photo taken during the walk using multimodal vision, returning structured JSON containing confidence scores, detected evidence tags, and personality-driven canine reactions. - Structured Visual Reasoning: Exposes what Gemini noticed in the photo without exposing chain-of-thought.
- Canine Adventure Report (
POST /api/ai/report): Crafts an 80–150 word first-person narrative based exclusively on the actual validated discoveries.
ElevenLabs gives dogs their own expressive voice:
- Expressive Story Retelling (
POST /api/voice): Transforms the Gemini-generated adventure report into dramatic, high-fidelity spoken audio. - Personality-Tuned Voice Settings: Adjusts stability, similarity boost, and style dynamically based on the dog's personality archetype (Chaos delivers breathless excitement, Princess delivers royal critiques, Explorer delivers confident documentary logs).
- Session Audio Caching: Caches generated speech in memory to ensure fast, seamless replays without redundant API calls.
- Custom Waveform Player: A lightweight custom player featuring sound wave animations and dynamic avatar feedback.
- Frontend: React 19, TypeScript, Tailwind CSS, Lucide React, Canvas Confetti
- Backend: Node.js, Express, Vite, Multer (multipart image processing)
- AI & Vision:
@google/genai(Google Gemini 3.7 Flash) - Voice Synthesis: ElevenLabs Text-to-Speech API
- Storage: LocalStorage persistence with seamless hydration
- Clone the repository:
git clone https://github.com/your-username/sniffari.git
cd sniffari- Install dependencies:
npm install- Configure environment variables in
.env:
GEMINI_API_KEY=your_gemini_api_key
GEMINI_MODEL=gemini-3.7-flash
ELEVENLABS_API_KEY=your_elevenlabs_api_key
ELEVENLABS_VOICE_ID=
ELEVENLABS_MODEL=eleven_v3
NEXT_PUBLIC_APP_URL=http://localhost:3000
NEXT_PUBLIC_DEMO_MODE=true- Run the development server:
npm run dev- Open http://localhost:3000 in your browser.
- Best Use of Google AI: Real-world multimodal photo verification, structured mission generation, and personality-driven story synthesis.
- Best Use of ElevenLabs: Seamless integration turning canine adventure reports into expressive spoken audio with custom playback UI and personality styling.
- Open Sniffari landing page.
- Click Start a Sniffari (or click the 60-Second Demo shortcut to load Mochi with Chaos personality).
- Click Generate Mochi's Sniffari → Gemini generates 5 customized missions.
- Select a mission (e.g. The Yellow Discovery or Tree Investigation).
- Click Take Mission Photo or Choose From Gallery to upload a photo.
- Click Ask Gemini 🪄 → Gemini inspects the photo and displays the animated PAW APPROVED stamp and detected visual evidence.
- Collect stamps for completed missions.
- Click Create Mochi's Adventure Report → Gemini generates the humorous field report and human rating.
- Click Hear Mochi Tell the Story 🔊 → ElevenLabs narrates the adventure in Mochi's voice!