Skip to content

Part 4: Speak — Jobvis, a voice concierge over the observable agent

Latest
Latest

Choose a tag to compare

View all tags
@jamwithai jamwithai released this 13 Aug 10:48
· 67 commits to main since this release
part4.0
cf6bf66

The Observable Job Agent, Part 4: Speak.

Say "find me jobs" and a real search starts. Jobvis tells you it has begun, goes quiet while it runs, then breaks the silence itself to say what it found. Ask it to tailor an application for the second one and a minute later the finished pack is on screen while it reads you the highlights.

The Part 2 grounding contract, carried into a new modality: the browser holds the conversation and forwards every question to Python, so the voice can only say what the LangGraph checkpoint returns. There is nowhere for it to get a fit score it did not measure.

📖 Blog post: https://jamwithai.substack.com/p/build-your-own-voice-agent

What's inside

  • The console (web/): a Next.js static export with a Three.js orb that reacts to the real output spectrum, served by FastAPI. The conversation runs in the browser over WebRTC, which is what buys real barge-in and the browser's own echo cancellation, with no audio library to build
  • api.py: mints short-lived voice tokens, dispatches tool calls, streams events over SSE, and serves the console. Your ElevenLabs key never leaves Python
  • voice/: persona.py declares the agent as code (system prompt, greeting, seven tool schemas), tools.py implements them next to the checkpoint, bridge.py holds the session and run manager, announce.py breaks the silence when a run finishes
  • Seven tools registered under the same names in three places, so a tool added on the agent without a handler fails loudly in one obvious spot
  • make jobvis-agent: idempotent agent creation, so the persona is version-controlled rather than typed into a form
  • Named source failures: a job source that returns nothing now says why (quota exhausted, key rejected, timed out) instead of looking like a quiet day
  • Optional webcam hand control via MediaPipe, off until you set a flag AND click the toggle. No frame leaves the tab

Get started

git clone --branch part4.0 https://github.com/jamwithai/observable-job-agent
cd observable-job-agent
uv sync --all-groups
cp .env.example .env    # add one LLM key (OPENAI_API_KEY, or free via groq:/ollama:)
make test               # 230 tests, no network or keys needed
make app                # http://localhost:7860

The voice console is optional and needs an ElevenLabs key with the Agents Platform (Conversational AI) scopes, plus a voice id you have added under Voices > My Voices:

make jobvis-agent   # prints the agent id, paste it back into .env
make web-build      # npm ci + Next.js static export
make app            # wizard on :7860 AND the console on :8000, one process
Assets 2
Loading