Repository navigation
Tuxistant 0.3.0, a preview
Pre-releaseA preview. Most of what is new here has run only in tests and in evaluations against a real model, not yet on the owner's own desktop. Mail search was asked four times there: two answers, and two turns cut short. Web search, facts, your own skills, media, timers and Home Assistant have not yet been tried by voice on a real desktop, and Home Assistant has never been used with a real home.
Three changes reach what 0.2.0 already did:
- Every Fn hold is now turned into words first, and a separate model call decides whether it is a question of fact, before the voice model hears it. Apart from the four mail questions, which may have gone this way, it has not run on a real desktop.
- Tapo lights, which worked in 0.2.0, now run in the skill sandbox. That needs
bwrapandprlimit, an x86_64 or aarch64 computer, and Linux 6.12 or newer with Landlock on. Lights needed none of this in 0.2.0, so without it they stop working after the update. As a skill with the network, they also now ask for a click when the conversation has already read something private, such as your calendar or a file, which 0.2.0 never did. They have not been tried on a real desktop since the move. - The daemon now starts at login on every install, not only with the Fn helper. That changes nothing for an install with the Fn helper, and has not been tried without one.
After updating, restart the daemon with systemctl --user restart tuxistant, or log out and back in. Until then the old version keeps running. For only what has been seen working, stay on 0.2.0.
How well the model picks the right tool with all of this offered is measured in docs/TESTING.md, with what it still gets wrong, such as "Open Spotify" pressing play.
- Your own skills, with code. A skill can bring its own program, which runs in a bubblewrap sandbox and may ask for the network and nothing else. Put one in
~/.local/share/tuxistant/skills/and it loads, changes and goes while Tux runs, with no restart. docs/SKILLS.md shows how to write one. Tapo lights now run this way, and find their bulbs again with a broadcast the first time after updating. - Home Assistant. "Turn off the kitchen light", "Is the front door locked?" and "Set the heating to 21". It controls and reads lights, plugs, fans, heating, blinds, locks, scenes and scripts by the names they have in Home Assistant. Unlocking, opening, activating a scene and running a script show a card first. Add the address and a long-lived access token in Settings. Until both are set the model isn't offered it, so nothing changes for anyone without one. The token is never handed back, and a new address clears it.
- Timers and reminders. "Set a pasta timer for 8 minutes", "Remind me at 5pm to call mum" and "How long is left on the pasta?". Tux says it out loud when it is due and leaves a notification, and they are kept through a restart. On a locked screen there is only the notification, and the words come if you unlock within an hour for a timer, or a day for a reminder. After that only the notification remains. "Chime for timers" in Settings, which starts off, adds a short sound. The daemon now starts at login on every install, with or without the Fn helper, as a reminder can't be said if nothing is running.
- Media. "Pause the music", "Skip this song", "What's playing?", "Turn it up" and "Volume 40". It reaches players through MPRIS, which Spotify, VLC and browsers speak, though so far it has only driven stand-in players. The volume never goes above 100 percent.
- Mail search. "When did I book the flights?" or "What did my landlord say about the rent?" is answered from the mail Thunderbird keeps on this computer, with who sent it and when. It is off until you turn on "Search my mail" in Settings. Mail goes nowhere but your own server. Junk and Trash are skipped, by Thunderbird's own mark or by their name. Sign-in codes and passwords are hidden by the words around them and their shape, and docs/ROADMAP.md lists what gets past both. An answer from a sender who is new to you comes with a warning.
- Facts from what's written down. A question of general knowledge asked on an Fn hold is answered from Wikipedia, then from web search, with no click. Tux says it doesn't know when neither clearly answers, instead of guessing. It repeats what a page says, even when the page is wrong, and in the last facts evaluation one of 54 hard questions got a wrong answer. Sums are worked out on this computer. "Look up facts on Wikipedia" in Settings, which starts on, turns the lookup off.
- Web search and research. "Search the web for flights to Lisbon" reads out the top results, and "Research electric bikes" reads a few pages and tells you what they say, naming its sources. Both need a SearXNG server of your own with JSON output turned on, which Settings can point at.
Fixed
Most of these fix code that is new in 0.3.0. They were found in tests and review before this release, not in use. Only the entries on lights, the calendar messages in Settings, and the key that the evaluations, the voice check and the demo video send change what 0.2.0 did.
- A timer or reminder whose words couldn't be spoken, for example with the speech server down, is no longer marked missed at once. It stays pending and is tried again after one, two and four minutes, and only then counts as missed. The error shows once for each try. Once some of the words were heard it counts as said, so it is never said twice.
- The timer chime plays to its end however the words fail, such as a refused connection or a timeout, not only on a server error. A key press still cuts it.
- A timer or reminder whose words were cut by the screen locking before any was heard is no longer recorded as said. It is said after the unlock, late if it has to be.
- One timer waiting to be tried again after its words failed no longer holds back the ones due after it. They are said on time.
- If the timer scheduler ever fails, it is logged and starts again, so timers keep working for the rest of the session.
- In the daemon as built, the timer chime would never have played on a locked screen, and a timer that came due while the screen was unlocked would have been marked missed instead of said. A setting was read by a name that was lost when the daemon's start-up code was split in two. A test caught it before any timer ran in use.
- If the screen-lock check fails while it is being set up, speech stays locked, so a timer shows a notification and is not said aloud on what may be a locked screen.
- A damaged
timers.jsonholding an absurd id or time loses only that row. A due time in the last days of the year 9999 no longer stops every timer listing. - A saved id counter at its very top no longer hands out ids that the next start drops.
- Lights are offered to the model only once a TP-Link email and password are in Settings.
- A Home Assistant switch or helper whose name or id says alarm, siren, lock, security, arm, disarm or deadbolt, or a garage, door or gate that isn't also a light, lamp, bulb, heater or fan, now waits for a click whichever way it is switched. The name and the id are judged apart, and joined spellings such as Doorlock, Frontdoor and Garageopener count. A long name is judged past the 40 characters the card shows, and a card is shown if either the shown or the longer name says so.
- Questions about the music playing, timers and the home that use words such as my, our, this song, timer, time left, locked, front door or the heating, like "How much time is left?", "Is the kettle on?" or the temperature in the kitchen, now go the normal way without being sorted, so the sorter doesn't send them to Wikipedia, though the voice model can still pick its lookup for one. One worded without any of these, such as "Who's at the door?", still depends on the sorter: in the last measured run it called "What colour is the lamp?" a fact once, which sends the question to Wikipedia.
- Without Evolution Data Server's Python bindings, Settings now names the packages to install, evolution-data-server and gir1.2-ecal-2.0, in the same words the voice uses, instead of suggesting an Online Account.
- Settings no longer tells you to add an Online Account when it can't list your calendars, for example after a timeout or with a daemon that hasn't restarted since an update. It says the listing failed instead.
- The evaluations and the live protocol test send the key only to a server named for the run, or to one saved in Settings that is neither the default nor on this computer. They share one set of stand-ins, which now also blocks timer notifications and never starts a code skill's process.
- The voice check and the demo video now check the server before they send the key, like the evaluations. A demo video with no server the key may go to comes out silent. A server on this computer is refused unless it is named for the run: a loopback address in any spelling, such as 127.1, 2130706433 or ::ffff:127.0.0.1, a localhost name such as ip6-localhost, the machine's own name, and any address its network interfaces hold. Nothing is looked up, so another name that
/etc/hostsgives this computer is not caught. The end-to-end harness now needsTUXISTANT_SERVERas well. - Text from a code skill can no longer carry chat-template delimiters to the model. An answer or an error loses its
<and>, and a skill whose name, description, version, action description or params text holds them is refused with a reason. ./uninstall.sh --purgenow also removes~/.local/state/tuxistant, where code skills keep what they save in/state, such as the addresses of the bulbs the lights found. Without--purgeit is kept, and the uninstaller says so.- A new test,
tests/test_names.py, fails the suite on any name a function reads as a global that nothing defines. It reads every file undertuxistant/andtools/without running them, so it would have caught the timer chime bug above.
It needs what 0.2.0 needed: GNOME Shell 50 on Wayland, Python 3.14, and a LocalAI server of your own with an 8 GB NVIDIA GPU, whose speech-to-text model (whisper-large-turbo-q5_0) every Fn hold now uses, not only dictation. Tapo lights and your own skills also need the skill sandbox: bwrap and prlimit on an x86_64 or aarch64 computer, and Linux 6.12 or newer with Landlock on for lights and any skill that uses the network. Media needs players that speak MPRIS and wpctl, which comes with WirePlumber. Home Assistant needs its address and a long-lived access token. Mail search needs the search index Thunderbird keeps on this computer. Web search, research and the web fallback for facts need a SearXNG server of your own with JSON output turned on, which the server kit doesn't set up.
Known limits are listed in docs/ROADMAP.md, and what the security reviews found and did not cover is in docs/SECURITY.md. Install and update steps are in the README.