Releases: pjdoland/jf-sebastian
Release list
v2.7.0 - Spotify Voice Control, Drop-in Devices & GPT-5
New Features
-
Spotify voice control: Personalities can now control Spotify playback by voice. The model is given nine
music_*tools (play, pause, resume, skip, previous, volume, and more), resolves Connect speakers by name (exact, alias, substring, or fuzzy match), and speaks a templated confirmation with no second LLM round-trip. The currently-playing track is injected into conversation context so the character can talk about what is on. Auth uses PKCE (no client secret);scripts/spotify_auth.pyand an optionalsetup.shwalkthrough guide first-time login. Opt in per personality (spotify_enabled) and globally (SPOTIFY_ENABLED). Seedocs/SPOTIFY_SETUP.md. -
Drop-in modular output devices: Output devices are now plugin packages that self-register, the same way personalities work. Drop a device package under
jf_sebastian/devices/<name>/and it is auto-discovered; each device owns its own settings and ships its own.envoverlay, loaded automatically when that device is selected. A new optional visual seam (requires_visualplusvisual_*hooks on the device base class) lets a device drive an on-screen renderer; audio-only devices are unaffected. -
GPT-5 family by default: The default conversation model is now
gpt-5.4-mini, with a newGPT_REASONING_EFFORTknob. The engine handles GPT-5 request semantics (max_completion_tokens, notemperature) and falls back cleanly togpt-4o-minifor accounts without GPT-5 access. -
Shimmer input voices for Johnny and Teddy Ruxpin: Both now use the
shimmerTTS voice as the RVC input; their distinct character comes from voice conversion.
Conversation & Audio Pipeline
- System prompt persists across long conversations: The personality's system prompt was stored inside the bounded history deque and got evicted after roughly ten exchanges, causing the character to drift out of character and forget its instructions. It is now pinned outside the history;
MAX_HISTORY_LENGTHbounds only the user/assistant turns. - Per-personality RVC tuning honored:
rvc_filter_radius,rvc_rms_mix_rate, andrvc_protectfrompersonality.yamlare now read and applied (previously silently ignored). RVC is enabled by the presence of a model rather than a redundant flag. - Sentence chunker hardened: Extracted into its own module, now abbreviation-aware with a soft length cap, and no longer splits decimals at the streaming edge (so "1.5 ounces" is spoken as words, not "one point five").
- Debug audio writes are dispatched through a single background writer queue.
Configuration Changes
Audio capture defaults were aligned to the values the project already ships in .env.example, so code, shipped config, and docs now agree:
SAMPLE_RATE=16000 # was 44100; Silero VAD requires 16000, so the old default silently disabled VAD
SILENCE_TIMEOUT=5.0 # was 10.0
SPEECH_END_SILENCE_SECONDS=1.0 # was 1.5If you already set these in .env (the setup.sh path does), you are unaffected. If you relied on the bare code defaults, these now take effect.
New optional environment variables:
GPT_MODEL=gpt-5.4-mini # default; falls back to gpt-4o-mini
GPT_REASONING_EFFORT=low # GPT-5 family only; empty = model default
SPOTIFY_ENABLED=true # global gate for the music tools
SPOTIFY_CLIENT_ID=... # see docs/SPOTIFY_SETUP.md
SPOTIFY_DEFAULT_DEVICE=... # Connect speaker name
SPOTIFY_DEVICE_ALIASES=... # name aliases for speaker resolutionSpotify support installs from the new requirements-spotify.txt (spotipy).
Documentation
- Full fact-check pass across every doc against the source: reconciled the README and Jetson default tables with
settings.py, corrected the Johnny/Teddy voice names, thervc_index_rate(0.5) andrvc_f0_method(harvest) defaults, the wake-word threshold guidance (point atWAKE_WORD_THRESHOLD), and the ARCHITECTURE PPM and eye-control math. Removed references to a nonexistentmodels/directory and nonexistent test files. - Made explicit that RVC voice models are not distributed with this project; you must train or obtain your own
.pth/.indexand place it in the personality folder. Without one, the personality uses its raw OpenAI TTS voice. setup.shadds an optional Spotify install and setup walkthrough and auto-numbers its steps.
Full Changelog: v2.6.0...v2.7.0
v2.6.0 - Silero VAD, Jetson Support & Parallel Filler
New Features
-
Silero VAD replaces WebRTC VAD: The Stage 3 speech-content detector now uses Silero's neural VAD instead of WebRTC. Significantly harder to fool with non-speech sounds (TV chatter, AC hum, distant music) that previously slipped past WebRTC and triggered Whisper hallucinations. Threshold knob is now a probability (0.0-1.0) instead of an aggressiveness level (0-3).
-
Parallel filler + Whisper: The filler phrase no longer waits for the Whisper response before kicking off; both start in parallel as soon as Stages 1-3 validation passes. The filler covers the full transcribe + GPT + TTS latency window instead of just the GPT+TTS portion, which makes the perceived response time noticeably shorter.
-
Layered .env overlays: Settings now compose from three sources (highest precedence first):
personalities/{PERSONALITY}/.env→device_overrides/{OUTPUT_DEVICE_TYPE}/.env→ root.env. Lets you, e.g., set a differentVOICE_GAINfor Squawkers vs Teddy without touching the personality config. Overlay files are.gitignore-d by the existing.envrule. Loaded overlay paths are exposed assettings.LOADED_ENV_OVERLAYSand logged at startup. -
Jetson deployment support: New
docs/JETSON_DEPLOYMENT.mdcovers running on NVIDIA Jetson hardware. RVC behaves better on memory-constrained devices: CUDA allocator pressure reduced, conversion is retried up to 3 times with backoff on transient CUDA OOM, and RVC is re-warmed before each scheduled event so idle-period cold starts don't stall the first audio chunk. -
Streaming wake-word inference:
wake_word.pynow uses streaming inference instead ofpredict_clip, which lowers per-frame latency. Sub-threshold scores in [0.5, threshold) are logged once per second so you can diagnose flakiness: was the score 0.92 (lower threshold a touch) or 0.50 (mic / audio path problem)? -
Jarvis personality: New
jarvispersonality featuring J.A.R.V.I.S. from Iron Man. Uses OpenWakeWord's bundledhey_jarvispre-trained model (no custom training needed), thefableTTS voice through RVC, and a butler-tone system prompt with 30 in-character filler phrases.
Reliability & Audio Pipeline
VOICE_GAINnow applies to RVC-converted audio in addition to the raw TTS path (previously RVC output bypassed the gain stage).- Audio recorder is paused during playback to prevent the toy from waking itself with its own voice.
- Post-playback tail guard added before the mic resumes, so the trailing reverb of the speaker doesn't trigger a new VAD event.
- Output stream buffer bumped to 4096 frames to eliminate underruns on slower hardware.
- PortAudio stream is now actually stopped during recorder pause instead of just dropping frames (was leaking CPU on long IDLE stretches).
- Filler audio is loaded lazily on first use, so startup is faster.
- Debug audio writes are now gated on
SAVE_DEBUG_AUDIO(previously always wrote when debug mode was on).
Documentation
CLAUDE.mdandREADME.mdbrought current with all of the above: Silero VAD references, parallel filler description, RVC tuning knobs (with Jarvis as a worked example), layered env overlays,VAD_THRESHOLDreplacingVAD_AGGRESSIVENESSthroughout. Personality lists trimmed to the seven actually shipped (fred, jarvis, johnny, kitt, leopold, mr_lincoln, teddy_ruxpin) instead of including.gitignore-d local-only installs.docs/JETSON_DEPLOYMENT.mdadded.- Stray
test.pyremoved from the repo root. SERVICE.mdoperational notes now gitignored (per-deployment).
Configuration Changes
Breaking (requires .env update on upgrade):
# Old
VAD_AGGRESSIVENESS=2 # 0-3 (WebRTC)
# New
VAD_THRESHOLD=0.5 # 0.0-1.0 probability (Silero); higher = stricterNew optional env vars:
# Layered overlays are automatic; no flag needed. Optional override files:
# personalities/{PERSONALITY}/.env # highest precedence
# device_overrides/{OUTPUT_DEVICE_TYPE}/.env
# Wake-word near-miss logging is automatic; no flag needed (uses existing LOG_LEVEL).Upgrading:
- Update
.env: renameVAD_AGGRESSIVENESS=2→VAD_THRESHOLD=0.5. Tune if needed: bump higher if Whisper still hallucinates on noise; drop lower if valid speech is being rejected. - (Jetson users only) See
docs/JETSON_DEPLOYMENT.mdfor the recommendedRVC_DEVICE, allocator, and retry settings.
New dependency
silero-vad(replaceswebrtcvad)
v2.5.0 - Pluggable News Headlines
New Features
-
Pluggable news headlines provider: Top headlines are now injected into LLM context so personalities can naturally reference current events ("Did you hear about…", "Speaking of the news…"). New
NewsProviderinterface mirroring the v2.4.0 weather provider pattern with three adapters:- RSS (default) — any RSS or Atom feed via
feedparser. NPR Topics: News is the built-in fallback ifNEWS_RSS_URLis unset, so headlines work out-of-the-box with zero configuration. - Hacker News — free public API, parallel fan-out across items.
- Manual — newline-separated env var for offline / testing (zero network egress).
Cached 30 min, top 5 headlines per turn. Disable entirely with
NEWS_PROVIDER=none. - RSS (default) — any RSS or Atom feed via
Improvements
- Robust RSS handling: HTML markup stripped from titles (e.g.,
<b>BREAKING:</b> Foo— would otherwise be read aloud by TTS). Polite identifyingUser-Agenton every HTTP request, since BBC / NYT / Reuters / Cloudflare-fronted feeds frequently 403 the defaultpython-requests/X.YUA. URL scheme validated (http/https only);allow_redirects=Falseso a misconfigured feed doesn't silently chain through arbitrary hosts.RequestExceptioncleanly returnsNoneper the ABC contract for uniform negative-caching. - Hacker News parallel fetch: top-stories + N item fetches use a shared
requests.Sessionwith aThreadPoolExecutorso cold-miss latency stays bounded even with highNEWS_HEADLINE_LIMIT. Per-item timeout tightened to 2s. Per-itemraise_for_status()added (HN's Firebase API can return Cloudflare HTML on 200 during incidents). - Better startup observability: news provider init log includes the resolved URL/feed name and a
Set NEWS_PROVIDER=none in .env to disablehint, so users can grep one line to see exactly where headlines are coming from. Pre-warm-failure message clarifies that headlines are skipped but the toy itself still works. - Documentation refresh: project structure tree in
README.mdbrought current with the actual filesystem (utils modules, scheduler, supervisor scripts, all tests subdirectories, top-level files likeROADMAP.mdandCLAUDE.md). README.env Settingsnow covers Weather Context, News Headlines, Proactive Scheduler, and Supervisor / Watchdog — four whole categories that were previously undocumented. Architecture > Key Components lists the real-world context provider, proactive scheduler, and process supervisor.tests/README.mdtree refreshed.docs/CREATING_PERSONALITIES.mdnow mentionsscheduled_events.yamland RVC models in the personality directory layout.docs/QUICKSTART.mdgains a §5b for news with feed table, privacy callout, and child-safety guidance.
Configuration
New env vars (all optional; news is on by default with NPR):
#NEWS_PROVIDER=rss # or hackernews / manual / none / auto
#NEWS_RSS_URL=https://feeds.npr.org/1001/rss.xml # NPR by default
#MANUAL_NEWS=Headline one\nHeadline two
#NEWS_HEADLINE_LIMIT=5
#NEWS_CACHE_TTL_MINUTES=30To disable entirely: NEWS_PROVIDER=none.
Upgrading and don't want news? Add NEWS_PROVIDER=none to your existing .env. Existing installs will start fetching NPR headlines at next restart until that line is added.
Privacy note: the news server (e.g., NPR) sees your IP and access cadence; OpenAI sees the headline text and can infer which feed you chose. Use NEWS_PROVIDER=manual or NEWS_PROVIDER=none for zero third-party news egress.
For child-facing personalities (e.g., teddy_ruxpin, fred): NPR Top News may include violence, politics, or other content unsuitable for children. Consider NEWS_PROVIDER=none or a kid-safe RSS feed when running with these personalities.
New dependency
feedparser>=6.0.10(post-6.0 XXE/billion-laughs hardening enabled by default)
v2.4.1 - Fix Scheduled Events Rejected by State Machine
Bug Fixes
- Scheduled events would never fire:
IDLE → SPEAKINGwas missing from the state machine'sVALID_TRANSITIONStable, sotry_transitionrejected every proactive event the scheduler tried to fire. Symptom in the wild: the scheduler loggedFiring scheduled event: …, synthesized TTS via OpenAI (observable in the log), and then silently aborted with no audio playing. NowIDLE → SPEAKINGis allowed because scheduled events legitimately bypass the LISTENING/PROCESSING phases — the speech is server-initiated, not user-initiated. - Misleading log message for scheduled-event CAS failures: previously logged "lost race to another transition" for any CAS failure (including this hardcoded validation rejection). Now reads "could not enter SPEAKING (state=…)" so the cause is more searchable.
Upgrading
No config changes. Pull, restart, and your personalities/<name>/scheduled_events.yaml events will now actually fire at their scheduled times.
v2.4.0 - Pluggable Weather, Process Supervisor & Proactive Scheduling
New Features
- Pluggable weather/context provider: Refactors the hardwired wttr.in fetch into a
WeatherProviderinterface with three adapters — wttr.in (default; free, no API key), Home Assistant (local, no third-party egress), and manual (offline / testing, zero network). Auto-selects whenWEATHER_PROVIDERis unset; existingZIPCODE-only setups keep working unchanged. SetWEATHER_PROVIDER=noneto disable weather entirely. - Process supervisor for unattended deployments: New
scripts/supervisor.pywrapspython -m jf_sebastian.mainwith exponential-backoff restart, watchdog kill of hung children via heartbeat-file staleness, enriched crash reports, and permanent-failure detection (CRITICAL log + extended backoff after N consecutive crashes). Includes launchd plist (macOS) and systemd unit (Linux) templates. Designed for museum exhibits, eldercare companions, kids' rooms — anywhere the toy needs to keep running across PortAudio/RVC crashes. - Proactive scheduler / ambient mode: Personalities can now define proactive utterances in
personalities/<name>/scheduled_events.yaml— morning greetings, bedtime stories, holiday surprises, scheduled reminders. Tiny schedule syntax (HH:MM,HH:MM weekdays,HH:MM YYYY-MM-DD); each event usessay:(verbatim TTS) orprompt:(LLM in character). Events only fire when state is IDLE — never interrupts an in-progress conversation. Quiet-hours suppression with load-time warnings. - ROADMAP.md: Public roadmap synthesized from a seven-persona peer-reviewed codebase audit. Documents Tier 1-3 prioritization plus excluded items, with 70 enhancement suggestions evaluated by Borda count.
Bug Fixes
- Fix
urlsplitraisingValueErroron a malformedHOME_ASSISTANT_URLand breaking the entire weather pipeline (including auto-fallback to wttr) - Fix
_refresh_in_flightflag leakingTruewhen the weather provider became unconfigured mid-refresh, suppressing all future refreshes - Fix Whisper weather cold-miss (first-conversation timeout caused "I can't check the weather" responses)
- Fix numpy int16 overflow
RuntimeWarningin RMS amplitude calculation - Fix race condition in scheduled-event callback that could re-init PyAudio after
audio_player.cleanup()during shutdown - Fix wake-word detector race during scheduled-event TTS synthesis (added atomic
try_transitionCAS to state machine) - Fix
_pause_wake_for_playbackflag set even whenpause()raised, causing missed resume
Improvements
- Local-first weather security: Bearer token to Home Assistant is refused over plain HTTP to non-private hosts (loopback, RFC1918, link-local,
*.localallowed); URL-quoted entity ID;allow_redirects=Falseto prevent token leak via cross-host redirects; defensive.get()-chain parsing for wttr; Celsius detection tolerates"C","celsius", etc. - Defense-in-depth supervision: launchd/systemd watch the supervisor; supervisor watches the child; heartbeat thread (
utils/heartbeat.py) reports liveness so even hung-PROCESSING is detected and killed viaos.killpgon the child's process group (kills ffmpeg / RVC subprocesses, not just the immediate child) - Crash report enrichment: PID, personality, ran_for, heartbeat age at exit, hostname, Python version, last 100 log lines; pruned to most recent N (default 200) so disk can't fill on a permanent-failure loop
- Rotating logs:
jf_sebastian.logandsupervisor.loguseRotatingFileHandler(10MB × 5) so unattended deployments can't fill disk - Atomic state transitions: New
StateMachine.try_transition(expected, target, trigger)compare-and-swap closes TOCTOU windows for any caller that needs to gate on current state;try_transitionandtransition_toshare a single body via_apply_transition_locked - Cooperative shutdown: SIGTERM/SIGINT use cooperative
_running = Falseinstead ofsys.exit()from a signal handler;SystemExit(0)from handler when SIGTERM arrives during__init__so the supervisor doesn't have to escalate to SIGKILL - Hobbyist-friendly schedule UX:
weekdays/weekendsaliases, friendlier error messages for typos ("empty weekday token"instead of crypticunknown weekday ''), past-dated one-shot warnings at load, items listed in scheduler startup log so users can verify their event loaded - Updated wake word models for Fred, Johnny, KITT, Leopold
- Documentation: New "Running Unattended" section in README; QUICKSTART covers all three weather providers + scheduled events; ARCHITECTURE diagram now shows supervisor + heartbeat layers; CLAUDE.md catalogs the new modules
Configuration
New env vars (all optional, sensible defaults):
# Weather provider (auto-selects from configured vars if unset)
#WEATHER_PROVIDER=wttr # or homeassistant / manual / none / auto
#HOME_ASSISTANT_URL=http://homeassistant.local:8123
#HOME_ASSISTANT_TOKEN=...
#HOME_ASSISTANT_WEATHER_ENTITY=weather.home
#MANUAL_WEATHER=Sunny and 72F
# Proactive scheduler (per-personality scheduled_events.yaml)
SCHEDULER_ENABLED=true
#QUIET_HOURS_START=22:00
#QUIET_HOURS_END=07:00
# Supervisor / watchdog (only relevant when running scripts/supervisor.py)
#HEARTBEAT_FILE=/tmp/jf_sebastian.heartbeat
#WATCHDOG_TIMEOUT=60.0
#RESTART_BACKOFF_INITIAL=1.0
#RESTART_BACKOFF_MAX=60.0
#CRASH_REPORT_DIR=./crash_reports/See .env.example for the full annotated list.
v2.3.0 - Real-World Context, Cross-Platform & Stability
New Features
- Real-world context for conversations: LLM now receives current date/time and weather (via wttr.in) so personalities can naturally answer questions like "What time is it?" — configure with
ZIPCODEin.env - Headless output device: New
headlessdevice type for computer-only playback without hardware - GPT-5 model compatibility: Automatic parameter adaptation for GPT-5 models (token limits, temperature constraints)
- Multi-stage silence detection: 4-stage audio validation pipeline prevents Whisper hallucinations and unnecessary API calls
- Cross-platform support: Linux/Jetson compatibility for audio, GPU detection, and setup
- Fred (Mister Rogers) personality: New personality added
Bug Fixes
- Fix race condition in audio recorder when
stop_recordingcalled from callback thread - Fix race condition in audio recorder continuous mode
- Fix RVC crash when
vc_singlereturns tuple with None audio - Fix RVC audio playing too fast by propagating actual sample rate
- Fix echo suppression getting stuck when skipping LISTENING state
Improvements
- Codebase cleanup: Fixed bugs, deduplicated code (Squawkers as HeadlessDevice subclass, AudioPlayer helper extraction), removed dead code (unused mock classes, commented-out blocks)
- Efficiency: Cached pyphen dictionary in PPMGenerator, capped state machine transition history
- RVC reliability: Device validation, fail-fast on permanent errors, Jetson-specific installation support
- Updated wake word models for fred, johnny, kitt, leopold, mr_lincoln, teddy_ruxpin
- Added CLAUDE.md for Claude Code guidance
v2.2.0 - Teddy Ruxpin Personality & Audio Optimizations
New Features
Teddy Ruxpin Personality
- Added complete Teddy Ruxpin personality (classic 1980s storytelling bear from Grundo)
- Character with friends Grubby, Princess Aruzia, and Newton Gimmick
- 30 adventure-themed filler phrases about crystals, treasures, and exploring
- Wake word: "Hey, Teddy Ruxpin"
- Warm, friendly Echo voice with childlike wonder
- RVC voice conversion support for authentic Teddy Ruxpin voice
Performance Improvements
Audio Processing Optimizations
- Reduced macOS audio stream delay from 2.0s to 0.5s
- Saves 1.5 seconds between filler and response playback
- Significantly improves conversation flow
RVC Processing Optimizations
- Optimized RVC settings for 30-40% faster voice conversion:
- Disabled pitch median filtering (filter_radius: 0)
- Reduced volume envelope mixing (rms_mix_rate: 0.1)
- Reduced consonant protection (protect: 0.2)
- Applied to K.I.T.T. and Teddy Ruxpin personalities
- Combined optimizations save approximately 3.8 seconds per interaction
Code Organization
Repository Improvements
- Moved RVC optimization script to scripts/benchmark_rvc.py
- Cleaned up personality documentation (removed redundant READMEs)
- Updated personalities README with Teddy Ruxpin configuration examples
Technical Details
The RVC optimizations maintain voice quality while significantly improving processing speed. The settings are tuned for the pm (Parselmouth) pitch detection method which offers the best balance of speed and quality for real-time animatronic applications.
v2.1.1 - Improved RMS Detection
Improvements
Better Silence Detection
- Peak RMS Detection: Switched from average RMS to peak RMS using 100ms sliding windows
- Prevents silence from dragging down amplitude measurements
- Detects speech even when surrounded by quiet audio
- More accurate filtering of background noise
Enhanced Logging
- Added detailed RMS logging with 📊 indicator for easier threshold tuning
- Shows both peak RMS value and current threshold for each audio capture
Configuration Updates
- Updated
MIN_AUDIO_RMSthreshold from 800 to 60 (appropriate for peak RMS measurements) - Removed all hard-coded RMS values - now uses
settings.MIN_AUDIO_RMSthroughout
Transcription Filtering
- Added "bye" to meaningless transcription filter to reduce false positives
- Reduces erroneous wake-up triggers from background noise
Technical Details
The new peak RMS algorithm:
- Divides audio into 100ms windows with 50% overlap
- Calculates RMS for each window
- Returns the maximum RMS value found
This approach is much more effective at detecting actual speech in buffers that contain both speech and silence, significantly improving the accuracy of the silence filtering system.
v2.1.0 - Gapless Audio & Continuous Conversation
Major Features
🎵 Gapless Audio Playback
- Implemented persistent stream sessions for seamless multi-chunk audio
- Filler audio flows directly into response chunks with no gaps
- Works with both Teddy Ruxpin (44.1kHz) and Squawkers McCaw (48kHz)
💬 Continuous Conversation Mode
- Multi-turn dialogues without requiring wake word for each turn
- Natural back-and-forth conversation flow
- Configurable conversation timeout
⚡ RVC Warmup
- Pre-loads RVC models during startup
- Eliminates first-use delay for voice conversion
- Faster initial responses
🚗 New Kitt Personality
- K.I.T.T. from Knight Rider with RVC voice conversion
- Custom voice model integration
- Complete with character-appropriate responses
⚙️ Configurable Filler Audio
- New ENABLE_FILLER_AUDIO environment variable
- Option to disable filler phrases for faster responses
- Maintains backwards compatibility
Improvements
- macOS Audio Fixes: Resolved audio playback hangs and stream timeout issues
- RVC Index Feature: Enabled index_rate for better voice quality (fixed faiss/PyTorch OpenMP conflict)
- Transcript Validation: Prevents responses to empty or meaningless speech (um, uh, etc.)
- Audio Recorder: Enhanced reliability for continuous conversation mode
- Documentation: Updated setup and configuration guides
Bug Fixes
- Fixed CoreAudio stream abandonment strategy on macOS
- Resolved audio recorder restart issues in continuous mode
- Fixed multiple playback timing and buffer management issues
- Corrected buffer drain timing to prevent stream close timeouts
Technical Details
New APIs:
AudioPlayer.start_playback_session()AudioPlayer.write_session_chunk()AudioPlayer.end_playback_session()
Backwards Compatibility:
- All changes are backwards compatible
- Existing
play_stereo()API maintained for single-shot playback - No breaking changes to configuration or personality files
Full Changelog: v2.0.0...v2.1.0
v2.0.0 - Major Setup & Stability Improvements
J.F. Sebastian v2.0.0
Major release with improved setup experience, Python 3.10 enforcement for RVC compatibility, enhanced audio playback stability, and new personality support.
🚀 Setup & Installation Improvements
Python 3.10 Enforcement & Auto-Installation
- Auto-detect Python 3.10: Automatically finds and uses Python 3.10 if available on system
- Auto-install Python 3.10: Offers to install via pyenv or Homebrew if not found
- Smart venv management: Detects and offers to recreate venv if using wrong Python version
- RVC compatibility: Enforces Python 3.10.x specifically (3.11+ not compatible with RVC)
- Version file: Added
.python-versionfor automatic version selection with pyenv
Integrated RVC Installation
- One-step setup: RVC installation now integrated into main setup script
- Automatic pip management: Handles pip downgrade to 24.0, RVC installation, and pip upgrade automatically
- Interactive prompts: Optional RVC installation during setup (defaults to No)
- Verification testing: Automatically tests RVC import after installation
Enhanced Setup Experience
- Optional filler generation: Filler audio generation now optional during setup (saves 2-3 minutes)
- Improved error messages: Clear guidance when Python version requirements not met
- Better documentation: Comprehensive RVC installation guide with troubleshooting
🎯 Audio Playback Stability
Sequential Queue Architecture
- Eliminated race conditions: Restructured audio system to use single sequential playback queue
- Seamless transitions: Filler audio flows directly into response chunks without gaps
- Interruptible playback: Audio now writes in chunks, allowing graceful interruption
- No more hangs: Fixed blocking PyAudio calls that prevented playback interruption
State Management Improvements
- Prevent premature transitions: System now waits for all audio chunks to complete before transitioning to IDLE
- Recovery system fixes: Recovery timeout disabled during active sequential playback
- Extended timeouts: Increased PROCESSING timeout from 15s to 30s to accommodate normal processing
- Better state tracking: Added
_sequential_playback_activeflag for proper state coordination
Performance Optimizations
- Pre-loaded filler audio: All filler audio loaded at startup (eliminates 14s conversation pauses)
- Faster audio loading: Switched to soundfile library for faster WAV parsing
- Capture post-wake audio: Records audio immediately after wake word to prevent missing first words
- Optimized transitions: Seamless filler-to-response transitions without delays
🤖 New Features
K.I.T.T. Personality
- Knight Rider's AI: Added K.I.T.T. personality from the classic TV series
- Custom wake word: Trained "Hey Kitt" wake word model with 90s F1 score
- Optimized speech: K.I.T.T. speech rate adjusted to 0.92 for better character accuracy
RVC Voice Conversion Support
- Custom voices: Full support for Retrieval-based Voice Conversion (RVC)
- Streaming compatible: RVC integrated with streaming TTS pipeline
- Comprehensive docs: Added detailed RVC setup and configuration guide
- Pip compatibility workaround: Documented and automated pip 24.0 workaround for RVC dependencies
🏗️ Architecture Improvements
Modular Device System
- Device architecture: Refactored to support multiple device types (Teddy Ruxpin, Squawkers McCaw)
- Device-specific audio: Each device can have custom filler audio and PPM configurations
- Extensible design: Easy to add new animatronic devices
Wake Word Improvements
- Better detection: Fixed wake word detection during active interactions
- Improved models: Updated wake word models with 90s+ F1 scores
- Leopold model update: Refreshed Leopold personality wake word
🐛 Bug Fixes
- Fixed AttributeError in state machine (use
stateinstead ofcurrent_state) - Fixed filler playback timeout and flag management issues
- Fixed missing import for
find_audio_device_by_name - Resolved git tracking issues with personality configurations
- Fixed wake word filename references after model updates
📦 Configuration Changes
- El Rey personality: Excluded from version control (private personality)
- Updated configs: Refreshed personality configurations for Leopold and K.I.T.T.
- Python version: All documentation updated to specify Python 3.10.x requirement
Breaking Changes
- Python 3.10 required: Python 3.11+ no longer supported due to RVC dependency requirements
- Setup script changes: Setup now includes RVC installation step (step 6/12)
- Filler generation: Now optional during setup (was previously automatic)
Upgrade Notes
If upgrading from v1.0.0:
- Ensure you're using Python 3.10.x (setup script will help with this)
- Re-run
./setup.shto recreate venv with correct Python version if needed - RVC users: Setup will offer to install RVC dependencies automatically
- Regenerate filler audio if you skipped during setup:
python scripts/generate_fillers.py