Skip to content

v2.7.0 - Spotify Voice Control, Drop-in Devices & GPT-5

Latest
Latest

Choose a tag to compare

View all tags
@pjdoland pjdoland released this 22 Jun 12:33
v2.7.0
b4a3839

New Features

  • Spotify voice control: Personalities can now control Spotify playback by voice. The model is given nine music_* tools (play, pause, resume, skip, previous, volume, and more), resolves Connect speakers by name (exact, alias, substring, or fuzzy match), and speaks a templated confirmation with no second LLM round-trip. The currently-playing track is injected into conversation context so the character can talk about what is on. Auth uses PKCE (no client secret); scripts/spotify_auth.py and an optional setup.sh walkthrough guide first-time login. Opt in per personality (spotify_enabled) and globally (SPOTIFY_ENABLED). See docs/SPOTIFY_SETUP.md.

  • Drop-in modular output devices: Output devices are now plugin packages that self-register, the same way personalities work. Drop a device package under jf_sebastian/devices/<name>/ and it is auto-discovered; each device owns its own settings and ships its own .env overlay, loaded automatically when that device is selected. A new optional visual seam (requires_visual plus visual_* hooks on the device base class) lets a device drive an on-screen renderer; audio-only devices are unaffected.

  • GPT-5 family by default: The default conversation model is now gpt-5.4-mini, with a new GPT_REASONING_EFFORT knob. The engine handles GPT-5 request semantics (max_completion_tokens, no temperature) and falls back cleanly to gpt-4o-mini for accounts without GPT-5 access.

  • Shimmer input voices for Johnny and Teddy Ruxpin: Both now use the shimmer TTS voice as the RVC input; their distinct character comes from voice conversion.

Conversation & Audio Pipeline

  • System prompt persists across long conversations: The personality's system prompt was stored inside the bounded history deque and got evicted after roughly ten exchanges, causing the character to drift out of character and forget its instructions. It is now pinned outside the history; MAX_HISTORY_LENGTH bounds only the user/assistant turns.
  • Per-personality RVC tuning honored: rvc_filter_radius, rvc_rms_mix_rate, and rvc_protect from personality.yaml are now read and applied (previously silently ignored). RVC is enabled by the presence of a model rather than a redundant flag.
  • Sentence chunker hardened: Extracted into its own module, now abbreviation-aware with a soft length cap, and no longer splits decimals at the streaming edge (so "1.5 ounces" is spoken as words, not "one point five").
  • Debug audio writes are dispatched through a single background writer queue.

Configuration Changes

Audio capture defaults were aligned to the values the project already ships in .env.example, so code, shipped config, and docs now agree:

SAMPLE_RATE=16000                 # was 44100; Silero VAD requires 16000, so the old default silently disabled VAD
SILENCE_TIMEOUT=5.0               # was 10.0
SPEECH_END_SILENCE_SECONDS=1.0    # was 1.5

If you already set these in .env (the setup.sh path does), you are unaffected. If you relied on the bare code defaults, these now take effect.

New optional environment variables:

GPT_MODEL=gpt-5.4-mini            # default; falls back to gpt-4o-mini
GPT_REASONING_EFFORT=low          # GPT-5 family only; empty = model default

SPOTIFY_ENABLED=true              # global gate for the music tools
SPOTIFY_CLIENT_ID=...             # see docs/SPOTIFY_SETUP.md
SPOTIFY_DEFAULT_DEVICE=...        # Connect speaker name
SPOTIFY_DEVICE_ALIASES=...        # name aliases for speaker resolution

Spotify support installs from the new requirements-spotify.txt (spotipy).

Documentation

  • Full fact-check pass across every doc against the source: reconciled the README and Jetson default tables with settings.py, corrected the Johnny/Teddy voice names, the rvc_index_rate (0.5) and rvc_f0_method (harvest) defaults, the wake-word threshold guidance (point at WAKE_WORD_THRESHOLD), and the ARCHITECTURE PPM and eye-control math. Removed references to a nonexistent models/ directory and nonexistent test files.
  • Made explicit that RVC voice models are not distributed with this project; you must train or obtain your own .pth/.index and place it in the personality folder. Without one, the personality uses its raw OpenAI TTS voice.
  • setup.sh adds an optional Spotify install and setup walkthrough and auto-numbers its steps.

Full Changelog: v2.6.0...v2.7.0

Assets 2
Loading