PyJippety

PyJippety is a desktop voice assistant for Python.

It listens for a local wake word with Porcupine, records a follow-up prompt, sends that prompt to OpenAI for transcription and response generation, and can speak the answer back through OpenAI speech.

The project is built to work in two modes:

• normal desktop use through a small GUI
• developer use through a simple Python package and CLI

What It Does

• local wake-word detection with Porcupine
• microphone capture for the spoken prompt
• OpenAI transcription with fallback model support
• OpenAI chat responses
• OpenAI speech output with fallback model support
• a local wake chime when the wake word is detected
• lightweight local memory for saved notes and recent exchanges
• local action plugins for simple desktop tasks
• desktop control panel for settings, profiles, logs, transcripts, typed testing, memory review, and history
• optional tray/menu bar indicator for background operation

Install

PyJippety now ships with a cross-platform Python installer.

Linux or macOS:

python3 install-pyjippety.py

Windows:

py install-pyjippety.py

The installer will:

• create an isolated app environment in your user data directory
• create a persistent config file in your user config directory
• install a launcher script for your platform
• copy the app logo into the installed app directory
• add a Linux desktop entry when running on Linux

Typical install locations:

• Linux app data: ~/.local/share/pyjippety
• Linux config: ~/.config/pyjippety/.env
• macOS app data/config: ~/Library/Application Support/PyJippety
• Windows app data: %LOCALAPPDATA%\PyJippety
• Windows config: %APPDATA%\PyJippety\.env

System requirements by platform:

• Linux: python3-tk, PortAudio development/runtime packages, and a supported tray backend are commonly required
• macOS: use a Python build with Tk included; Homebrew PortAudio is commonly needed for PyAudio
• Windows: use a standard Python.org install with Tk included; a working build of PyAudio is still required

Linux example:

sudo apt update
sudo apt install portaudio19-dev python3-tk
python3 install-pyjippety.py

First Run

Open the app, go to the Setup tab, and fill in:

• OpenAI API key
• Picovoice access key

Then:

1. Save settings.
2. Start voice mode.
3. Say the wake word.
4. Speak your prompt.
5. If you want clarification, ask a quick follow-up right after the reply. You do not need the wake word again until the follow-up window closes.

If you want to test the assistant without using the wake word, use the typed request box in the Use tab.

Useful desktop shortcuts:

• F8 toggles voice mode
• Ctrl+Space triggers push-to-talk
• Escape interrupts current output

If tray support is available, closing the window hides PyJippety to the system tray or menu bar instead of exiting. You can reopen it from the tray icon.

The app keeps spoken output focused on actual replies. Wake-word detection uses a short local chime instead of speaking status messages back to you.

If you want the assistant to remember something, say or type:

remember that I prefer short answers

You can also ask:

what do you remember about me

Small convenience features built into the UI:

• Repeat: speak the last answer again
• Copy answer: copy the last answer to the clipboard
• Clear view: clear the current transcript/answer area without touching memory
• Wake cue: play the configured wake sound
• Mute speech: keep replies visible in the UI without speaking them out loud
• Open config folder: jump to the profile/config storage location
• Open logs folder: jump to the local log directory
• Export profile: save the current profile as a zip archive
• Recent prompts: quickly reuse recent spoken or typed requests

Custom Wake Word

PyJippety supports either:

• a built-in Porcupine keyword
• a custom Porcupine wake-word model (.ppn)

Built-in keyword

The quickest path is to use one of Porcupine’s built-in keywords in the Setup tab:

1. Leave Custom keyword file empty.
2. Set Built-in keyword to the word you want to use.
3. Set Wake word label to the same word or phrase you want the UI to display.
4. Save settings.

Custom .ppn wake word

If you want a custom wake word such as computer, create a Porcupine keyword model and point PyJippety at it.

Typical flow:

1. Go to the Picovoice Console.
2. Create a custom Porcupine wake word.
3. Download the generated .ppn model for your platform.
4. Save that file somewhere stable on your machine.
5. In the Setup tab:
   • set Custom keyword file to the path of the .ppn file
   • set Wake word label to the phrase you want PyJippety to show in the UI
6. Save settings.

Notes:

• If Custom keyword file is set, it takes priority over the built-in keyword field.
• .ppn files are local assets and are ignored by Git in this repo by default.
• If PyJippety says it cannot find the keyword file, open Setup and either clear the field or point it to the correct file again.

Everyday Use

Typical voice flow:

1. Open PyJippety.
2. Click Start.
3. Wait for the app to enter listening mode.
4. Say the wake word.
5. Ask your question.
6. Listen for the reply.
7. If needed, ask a follow-up question immediately after the reply.

Examples:

porcupine what time is it

porcupine remember that I like concise answers

porcupine open website github.com

porcupine what do you remember about me

Useful interaction modes:

• Start: enable wake-word listening
• Push to talk: ask one question without the wake word
• Ask: type a request instead of speaking it
• Send transcript: resend the last recognized speech after editing it
• Interrupt: stop current playback
• Sleep: stop active listening without quitting the app

If tray support is available:

• closing the window keeps the app running in the background
• you can reopen it from the tray or menu bar icon

Developer Setup

If you want to run the project directly from source:

python -m venv .venv
source .venv/bin/activate
pip install -e .

You can also install from requirements.txt:

pip install -r requirements.txt

If you want a project-local config file during development:

cp .env.example .env

Run the desktop UI:

pyjippety-ui

Run the voice loop from the terminal:

pyjippety

Run one cycle only:

pyjippety --once

If you are running from source without installing entrypoints:

PYTHONPATH=src python -m pyjippety.gui
PYTHONPATH=src python -m pyjippety.bot

How It Works

The runtime pipeline is intentionally simple:

1. Porcupine listens locally for a wake word.
2. Once triggered, the app records a single spoken prompt.
3. The prompt audio is sent to OpenAI transcription.
4. The transcript is sent to an OpenAI chat model.
5. The reply is spoken with OpenAI speech, or printed if speech is disabled or unavailable.
6. The app keeps listening for a small number of follow-up questions before returning to wake-word mode.

Wake-word detection stays on-device. The transcription, response, and speech steps use the OpenAI API.

The app also has lightweight local actions for requests such as checking the time, checking the date, opening a website, and going to sleep. When safe tool mode is enabled, side-effect actions stay blocked unless you turn that protection off.

Interface

The desktop app is split into two focused areas:

• Use
  • start or stop voice mode
  • use push-to-talk
  • type requests directly
  • inspect and resend the last transcript
  • inspect live activity logs
  • keep memory notes and recent activity nearby without leaving the main work area
• Setup
  • manage profiles and personality presets
  • edit the common settings in a compact basic view
  • reveal model fallbacks, timing values, safe tool mode, idle timeout, memory limits, and prompt tuning through Show advanced settings

The goal is to feel like a small desktop utility, not a chat dashboard.

Configuration

Most users should manage settings from the desktop UI. The app writes them to:

~/.config/pyjippety/.env

For manual editing, the main settings are:

• OPENAI_API_KEY
• PICOVOICE_ACCESS_KEY
• ASSISTANT_WAKE_WORD
• ASSISTANT_PORCUPINE_KEYWORD
• ASSISTANT_PORCUPINE_KEYWORD_PATH
• ASSISTANT_DISPLAY_NAME
• ASSISTANT_CHAT_MODEL
• ASSISTANT_TRANSCRIPTION_MODEL
• ASSISTANT_TRANSCRIPTION_FALLBACK_MODELS
• ASSISTANT_TTS_ENABLED
• ASSISTANT_MUTE_SPEECH
• ASSISTANT_TTS_MODEL
• ASSISTANT_TTS_FALLBACK_MODELS
• ASSISTANT_TTS_VOICE
• ASSISTANT_TTS_SPEED
• ASSISTANT_SYSTEM_PROMPT
• ASSISTANT_TTS_INSTRUCTIONS
• ASSISTANT_MEMORY_ENABLED
• ASSISTANT_MEMORY_TURN_LIMIT
• ASSISTANT_MEMORY_FACT_LIMIT
• ASSISTANT_LOW_VERBOSITY
• ASSISTANT_LISTEN_TIMEOUT
• ASSISTANT_PHRASE_TIME_LIMIT
• ASSISTANT_FOLLOW_UP_ENABLED
• ASSISTANT_FOLLOW_UP_TURN_LIMIT
• ASSISTANT_FOLLOW_UP_TIMEOUT
• ASSISTANT_START_HIDDEN
• ASSISTANT_CHIME_VOLUME
• ASSISTANT_SAFE_TOOL_MODE
• ASSISTANT_IDLE_TIMEOUT_SECONDS
• ASSISTANT_AMBIENT_ADJUST_SECONDS
• ASSISTANT_ENERGY_THRESHOLD
• ASSISTANT_AUDIO_DEVICE_INDEX
• ASSISTANT_EXIT_WORDS

Notes:

• If you use a custom wake word such as computer, point ASSISTANT_PORCUPINE_KEYWORD_PATH to a Picovoice .ppn file.
• If your OpenAI project cannot access the default transcription or speech models, the app will try configured fallback models.
• If speech is disabled or all speech models fail, replies stay visible in the UI log.
• Memory is stored locally in the user config area and reused as context for later prompts when memory is enabled.
• Profiles store their own settings, memory file, and action history under ~/.config/pyjippety/profiles/<profile-name>/.
• Idle timeout stops voice mode after inactivity and puts the app into a sleeping state until you start it again.

Bundle and Package Artifacts

To build a desktop bundle:

Linux or macOS:

python3 build-pyjippety-bundle.py

Windows:

py build-pyjippety-bundle.py

To build release artifacts for the current platform:

Linux or macOS:

python3 package-pyjippety.py

Windows:

py package-pyjippety.py

Current packaging targets:

• Windows: .exe and .zip
• macOS: .dmg and .tar.gz
• Linux: .deb, .tar.gz, and .AppImage
• Linux: .flatpak when Flatpak tooling is available

Artifacts are written to:

release/

GitHub Automation

The repository includes GitHub Actions workflows for CI and releases:

• CI
  • runs unit tests on Linux, macOS, and Windows
  • installs the platform-specific system dependencies needed for the current Python/audio stack
  • compiles the Python sources to catch syntax regressions
• Release
  • triggers on tags that start with v
  • builds desktop bundles for Linux, macOS, and Windows
  • packages release artifacts for each platform
  • builds Python sdist and wheel artifacts
  • publishes all bundled artifacts to a GitHub Release

Flatpak builds are generated during releases too. The packaging script vendors the required Python wheels and installs the Flatpak runtime/SDK in CI so the Flatpak build can run without depending on network access inside the Flatpak build sandbox.

Flathub publishing is still a separate submission/review workflow through flathub/flathub; GitHub Releases can publish .flatpak artifacts, but that does not by itself publish the app on Flathub.

Typical release flow:

git tag v0.1.0
git push origin v0.1.0

That tag will trigger the release workflow and publish the generated artifacts to GitHub Releases.

Testing

PYTHONPATH=src python -m unittest discover -s tests
python -m compileall src tests

Troubleshooting

ALSA warnings on Linux

Some Linux systems print ALSA warnings even when the app still works. These messages are often noisy rather than fatal.

OpenAI model access errors

If you see 403 Forbidden or model_not_found, your OpenAI project likely does not have access to the configured model. Change the primary model in the UI or set a compatible fallback list.

No spoken output

If speech fails, the app will fall back to text output in the log. Check the speech model settings and your OpenAI project access.

Wake word does not trigger

Check:

• your Picovoice access key
• the chosen built-in keyword
• the custom .ppn path if you are using one
• your microphone device selection

macOS notes

• Tkinter support depends on the Python distribution you installed.
• If PyAudio fails to build, install PortAudio first, for example with Homebrew.

Windows notes

• Use py install-pyjippety.py if python is not on PATH.
• If PyAudio installation fails, install a compatible wheel for your Python version or use a Python environment known to work with PyAudio.

Project Layout

src/pyjippety/
  bot.py            CLI entrypoint
  actions.py        lightweight local command plugins
  config.py         config parsing and env loading
  memory.py         local memory storage and memory-aware prompting
  runtime.py        assistant orchestration
  integrations.py   OpenAI, Porcupine, audio, and wake chime adapters
  gui.py            desktop frontend
  controller.py     GUI control flow and side-effect orchestration
  profile_store.py  settings/history persistence by profile
  ui_shared.py      shared UI schema and constants
  views.py          Tk layout and widget construction
tests/
  test_bot.py       assistant loop tests
  test_memory.py    memory behavior tests
install-pyjippety.py
install-pyjippety.sh
build-pyjippety-bundle.py
build-pyjippety-bundle.sh

Current Scope

This is a phrase-based assistant, not a full duplex realtime conversation agent.

That keeps the app simpler to install, easier to reason about, and easier to package as a desktop utility.