Skip to content

Keys Ghost Machine v0.1.0 — powered by TensorFold 1.1.4

Choose a tag to compare

View all tags
@drowzeys drowzeys released this 25 Oct 01:12
· 15 commits to main since this release
v0.1.0
9bd9cf6

Superseded by v0.2.0 — please use v0.2.0 (fixes Drama 3 long-line stalls with the silence guard on the DGX Spark and stall resume, the deafness after fast short replies, Drama 3 503 "speech output capacity is full" errors, Bolt repetition loops and interrupted replies in the history; adds the Kokoro-82M Lite voice). Docker: ghcr.io/drowzeys/keys-ghost-machine-audio-visual-agent:0.2.0.

Keys Ghost Machine v0.1.0: a hands-free audio/visual agent, powered by TensorFold

Meet Boo: talk to it from your phone, show it things through your camera, and let it hand real work to a tool-using agent, with no keyboard and no terminal. It runs entirely on your own hardware, on a DGX Spark or an Apple silicon Mac, and you reach it from anywhere on your private Tailscale network.

▶ 46-second demo: https://github.com/drowzeys/keys-ghost-machine-audio-visual-agent/blob/main/docs/media/promo-45s.mp4

What it does

  • Speech in, speech out: Parakeet v3 hears you, Nemotron 3.5 Bolt answers, Drama 3 speaks, starting about half a second after you stop talking on a Mac.
  • Sees through your camera: Cosmos3-Edge describes what's in front of you, reads text, and tells your left from your right on a selfie camera; the hand tracker counts fingers exactly.
  • Live facts in seconds: questions about the president, the weather or the news are looked up on the web in the same turn.
  • Real work in the background: "ask Hermes…" hands a task to a tool-using agent, which asks you out loud before running anything that changes your machine.
  • Remembers: camera captions every 1 s to 15 min while the camera is on, kept 1 day to 1 week.
  • Yours to shape: name it, pick one of 14 voices (Vesper is the default), and turn on the Futuristic effect.

Measured (TensorFold 1.1.4)

Apple silicon Mac DGX Spark
Voice to voice 0.47–0.60 s 0.55–0.77 s
Camera answer (first token) 79 ms 103 ms
Voice starts (Vesper) 122 ms 121 ms

Requirements

  • Mac: Apple silicon with 96 GB unified memory minimum (128 GB+ recommended).
  • DGX Spark: one Spark (128 GB), about 64 GB free.
  • Disk: about 50 GB for the models.

Get it

Built on the work of many people: NVIDIA (Bolt, Cosmos3-Edge, Parakeet), Fish Audio (Drama 3) and plotarmordev's NVFP4 build, Ash Hart (TensorFold), tonyd2wild (Eyes of Kai), Nous Research (Hermes), the qwen-audio-agent design, Silero, MediaPipe, Three.js and more. The full list is in the Credits section of the README. These are unofficial community builds of TensorFold, not official releases.

Questions and bug reports: GitHub Issues.

Build details

Prebuilt TensorFold builds and a Docker image for Boo, built on TensorFold 1.1.4.

These are unofficial community builds of TensorFold by drowzeys (Apache-2.0), not official TensorFold releases. Model weights are not included.

Assets

File Platform Contents
keys-boo-tensorfold-1.1.4-macos-arm64.tar.gz Apple silicon, macOS 26+ One tensorfold-native that serves Nemotron 3.5 Bolt (with libbolt.dylib), Drama 3 NVFP4 with named voices and the voice-prefix cache, and Cosmos3-Edge on Metal
keys-boo-tensorfold-1.1.4-linux-aarch64.tar.gz DGX Spark (GB10), CUDA 13 tensorfold-cosmos3 (Cosmos3-Edge on CUDA, 1.1.4) and tensorfold-drama3-nvfp4 (plotarmordev's NVFP4 Drama engine + voice-prefix cache). Bolt uses the stock TensorFold 1.1.4 release

Each archive has README.md (requirements and run commands), CHANGES.md (every commit on top of upstream), LICENSE, NOTICE, THIRD_PARTY_NOTICES.md, LICENSES/ and SHA256SUMS. Check the archive against its .sha256 file.

Docker (DGX Spark)

docker pull ghcr.io/drowzeys/keys-ghost-machine-audio-visual-agent:0.1.0
docker run --gpus all --rm -p 8095:8095 \
  -v ~/models/bolt:/models/bolt -v ~/models/drama:/models/drama -v ~/models/cosmos3:/models/cosmos3 \
  -v ~/models/hf-cache:/models/hf-cache -e KAI=1 \
  ghcr.io/drowzeys/keys-ghost-machine-audio-visual-agent:0.1.0

See docker/README.md.

Source

https://github.com/drowzeys/TensorFold branches bolt-metal-114, drama3-metal-voices-114, cosmos3-metal-114, cosmos3-cuda-114, drama3-cuda-voices.

Verified

  • macOS (M5 Ultra): Bolt answers greedy chat identically to the source build; Drama 3 audio is byte-identical to the live build; Cosmos3-Edge serves text and images.
  • DGX Spark: Drama 3 audio is byte-identical to the live Spark build; Cosmos3-Edge gives the same answers as the live server and matches the transformers reference (cosine 0.9998).

Credits

TensorFold (Ash Hart, Apache-2.0); the NVFP4 Drama 3 engine and checkpoint by plotarmordev; Nemotron 3.5 Bolt, Cosmos3-Edge and Parakeet v3 by NVIDIA; Drama 3 by Fish Audio (Fish Audio's licence applies to Drama 3 and its outputs); Eyes of Kai by tonyd2wild. Full list in CREDITS.md.

Assets 6
Loading