ElfDeckAI 1.1.3

Fixes from an adversarial review of the 1.0.x/1.1.x line, checked against
1.1.2 (several review findings were already fixed by 1.1.2 and are not
listed here).

Fixed

• ElfMouse: a duplicate down after a moveto switched devices orphaned
  the button on the first device — held forgot its fd, so neither up
  nor release could ever release it. The first device is now released
  when a held button is re-pressed on the other device.
• Stuck input when the MCP client disconnects: the server exits on stdin
  EOF without releasing anything, leaving a down held forever. It now
  sends release/reset to all three daemons on the way out.
• Daemons now trap SIGHUP as well as SIGTERM/SIGINT, so a daemon started
  with elfkey & over SSH cleans up (and releases) when the session
  closes instead of dying mid-hold.

Added

• wait <ms> command in all three daemons and a wait MCP tool
  (0–5000 ms): timing between inputs is now expressible. Queued taps
  otherwise land back-to-back, and a frame-polled game can merge them
  into one continuous hold.

Changed

• README Safety model: the audit log now warns that typed secrets
  (passwords, one line per keystroke) land in the journal, and a new
  "Manipulated callers" paragraph states plainly that the validation
  layer covers hallucinating callers, not an agent manipulated by
  on-screen content.

ElfDeckAI 1.1.2

Added

• One-line PC setup: setup-pc.ps1 (Windows) and setup-pc.sh (macOS,
  Linux) create a dedicated SSH key, copy it to the Deck, check the
  passwordless connection, and add ElfDeckAI to Claude Desktop's config.
  They work with Decks that only accept keys, including when the PC's
  ssh-agent holds a key restricted to one command.

Changed

• README opens with a four-step "Start here" for Steam Deck owners; the
  technical description follows it. The Deck installer and
  AI_SETUP_GUIDE.md point to the PC setup scripts, and SSH is enabled with
  sudo.

ElfDeckAI 1.1.1

Fixed

• Gamepad in Steam Deck Gaming Mode: games that ignored ElfPad were only
  reading Player 1, which Steam gives to the Deck's own controls. The
  pad_press and pad_down tool descriptions and AI_SETUP_GUIDE.md now
  tell agents how to make ElfPad Player 1 through Steam's Reorder
  Controllers (verified in Bastion).

Added

• scripts/probe-gamepad.py, which shows which of Steam's virtual pads
  carries ElfPad's input.

ElfDeckAI 1.1.0

Added

• mouse_click_at and mouse_move_to MCP tools: click or point at (x, y)
  in the pixels of the most recent screenshot. Backed by a new absolute
  pointer device, "ElfMouse Absolute", and the ElfMouse command
  moveto <x> <y> [<width> <height>].
• Screenshots larger than 1280 px on their long side are scaled down
  before sending (ELFDECKAI_SCREENSHOT_MAX sets the limit; 0 disables).
  The screenshot result now also states the image size.
• Steam Deck Gaming Mode screenshots, through gamescope's gamescopectl.
  Used first whenever gamescope is running; the Desktop Mode tools remain
  the fallback.
• ROADMAP.md, and scripts/probe-gaming-mode.sh for checking Gaming Mode
  support on a Deck.

Changed

• Screenshots are captured as uncompressed PPM where the capture tool
  supports it, falling back to PNG.
• The installer also checks for the ElfMouse Absolute device.
• The server re-reads the graphical session's variables before every
  screenshot, so switching between Desktop Mode and Gaming Mode no longer
  needs a server restart.

ElfDeckAI 1.0.7

Fixed

• Daemons no longer exit when a FIFO line contains bytes that are not
  valid UTF-8; the line is refused like any other malformed command.
• The KWin D-Bus screenshot fallback now deletes the temporary file KWin
  writes to /tmp.
• The installer's troubleshooting hint names the correct systemd unit.
• Setup documentation now requires key-based SSH login: an MCP client
  launching ssh cannot answer a password prompt.

Changed

• The MCP server negotiates the protocol version (2025-06-18,
  2025-03-26 or 2024-11-05) instead of always answering 2024-11-05.
• Python 3.9 or newer is required (3.8 is end-of-life).
• The package version is defined once, in elfdeckai_mcp.__version__.
• Rewritten README and module documentation.

Added

• MCP Registry metadata (server.json package entry and README marker).
• Continuous integration and a release workflow that publishes to PyPI
  and the MCP Registry and creates the GitHub Release. It runs on a
  version tag, or from the Actions tab, in which case it creates the tag.
• glama.json listing the maintainer for Glama.

v1.0.6 — AI eyes: screenshot tool + one-paste Deck setup

What this is

ElfDeckAI gives an AI agent hands and eyes on a Linux machine — built for the Steam Deck. Three zero-dependency daemons create a virtual keyboard (ElfKey), gamepad (ElfPad) and mouse (ElfMouse) through Linux uinput, and a small MCP server exposes them — plus screenshots — to any MCP-capable AI client (Claude Desktop, Claude Code, Codex, …) over stdio, typically via SSH. The AI sees the screen, presses buttons, reads the result, and plays: turn-based games, menus, dialogue, setup flows. Pure Python standard library, no packages to install on the Deck side beyond Python itself.

Install

On a Steam Deck (the one-paste setup) — switch to Desktop Mode, open Konsole, paste:

curl -fsSL https://raw.githubusercontent.com/Garthnesss/ElfDeckAI/main/install-on-deck.sh | sh

It installs everything, starts the three daemons as user services, verifies the virtual devices exist, and prints the exact MCP config block for your PC — including the Deck's current IP address. Then turn on SSH on the Deck (passwd once, systemctl enable --now sshd) and add that config block to your AI client on your PC. Full walkthrough, written so you can hand it to your AI and let it explain everything: AI_SETUP_GUIDE.md.

Anywhere else (PC side or generic Linux):

pip install elfdeckai

Also on PyPI: https://pypi.org/project/elfdeckai/ — provides the elfkey, elfpad, elfmouse and elfdeckai-mcp commands.

What's new in 1.0.6

• Screenshots work when the MCP server is launched over SSH. The server now imports the graphical-session environment (WAYLAND_DISPLAY / DISPLAY / session bus) from the systemd user manager when it's missing — previously, capture tools crashed in SSH sessions because they had no display to talk to. Found and fixed during a live session on a real Deck.
• Installer: detects the Deck's IP address via ip (SteamOS has no hostname command in a minimal shell), so the printed MCP config contains your real address instead of a placeholder.
• AI setup guide: new troubleshooting entry — if a Steam game ignores the gamepad (Steam Input can hand the controller slot to the Deck's built-in pad), drive the keyboard tools instead; some games treat the keyboard as primary input.

Also new since the last GitHub release (from 1.0.5)

• screenshot tool (19 MCP tools total). Returns a PNG of the screen of the machine the server runs on, as MCP image content — the AI literally sees your Deck. Capture tries, in order: Spectacle (KDE), grim, scrot, ImageMagick, and the KWin D-Bus interface; if all fail, the error lists what each one said.
• install-on-deck.sh — the one-paste installer described above. Works even on a stock Deck with no pip: it falls back to downloading the single-file programs directly and wiring up user services.
• AI_SETUP_GUIDE.md — a setup guide written for an AI reader: hand the file to your assistant and it walks you through both sides (Deck + PC), verification, honest limits, troubleshooting and uninstall.

Verified on real hardware

This release was proven end-to-end on an actual Steam Deck, through the exact path a new user takes: the one-paste installer (including its no-pip fallback — a stock Deck has no pip), MCP-over-SSH from a second machine, a live screenshot of a running game (Dragon Quest VII, Desktop Mode), and inputs driven through the tools — the party was walked across the room and into an NPC conversation, confirming the see → act → see loop. 151 automated tests pass.

Requirements & honest limits

• Linux with /dev/uinput (the included udev rule handles permissions on non-Deck machines; the Deck allows it out of the box). Python 3.10+, standard library only. 64-bit.
• Screenshots need a graphical session: on the Deck that means Desktop Mode. Gaming Mode vision is not supported yet.
• The control loop runs at screenshot speed — a second or two per look-and-act cycle. Perfect for turn-based, menu-driven and dialogue-heavy games; not for twitch action.
• Everything is local-network by design: the daemons only listen on Unix FIFOs, and the MCP server reaches your PC only through your own SSH connection.

MIT licensed. Issues and PRs welcome.

v1.0.4 — 360-true ElfPad, audit hardening, on PyPI

Everything since v1.0.1 — two external-audit fix packs, one hardware-verified gamepad fix, and PyPI.

Install

pip install elfdeckai

ElfDeckAI is now on PyPI. The git route keeps working too.

ElfPad: analog-only triggers (hardware-verified)

ElfPad no longer declares digital BTN_TL2/BTN_TR2 bits. A real Xbox 360 pad
doesn't have them, and declaring them shifted Select/Start/Guide/L3/R3 two
button indices off the stock layout for index-based consumers (SDL's 360
mapping, joydev). press / down / up on l2/r2 now drive the analog
trigger axes, exactly like the real pad. Confirmed with evtest against a
real 360 pad on a Steam Deck, then fixed and re-verified on the same Deck.
A unit test pins the button set to the real pad's codes.

MCP server: strict validation (1.0.2)

Tool arguments are validated against the advertised schemas before anything
touches a daemon — unknown or missing arguments, wrong types (a bool is not
an int), out-of-range values and non-object params are rejected with proper
JSON-RPC errors. Request lines are bounded at 1 MiB with draining, and
NaN/Infinity literals are refused.

Audit fix pack (1.0.3)

• Each command now emits as a single atomic evdev frame (one SYN per
  command) in all three daemons.
• Held state is tracked before events are emitted, so cleanup can never
  miss a stuck key or button.
• Single-instance lock per daemon, and a clear error when /dev/uinput
  can't be opened instead of a traceback.
• Daemons log accepted commands to stderr (journalctl / your service log).
• A tools/call sent as a notification is executed but never answered, per
  the JSON-RPC spec.
• The udev rule is now 70-elfdeckai-uinput.rules — uaccess tags are
  consumed earlier in boot than the old 99- name ran.
• ElfMouse has its own USB product ID instead of sharing ElfKey's.

Notes

148 tests pass. ElfKey, ElfPad and ElfMouse have all run on a real Steam
Deck; the Pokémon story in the README was driven by ElfKey before Pad and
Mouse existed.

v1.0.1 — ElfMouse + audit hardening

The set is complete, and the MCP server is hardened.

New since v1.0.0:

• ElfMouse: the third daemon — virtual mouse with relative moves, timed glides, click/drag (down/move/up) and wheel + horizontal scroll, with the same bounded, never-stuck safety design as ElfKey and ElfPad
• Run it yourself: pip install git+https://github.com/Garthnesss/ElfDeckAI installs console commands — elfkey, elfpad, elfmouse, elfdeckai-mcp (verified end-to-end in a clean environment)
• MCP hardening (from an external audit, fixes verified and regression-tested): a malformed or non-object JSON line can no longer kill the server (proper JSON-RPC protocol errors instead); FIFO failures report the real cause (not running / busy / permission / broken pipe) instead of always claiming the daemon is down; advertised tool schemas now carry the daemons' actual enums and bounds
• New release command in ElfKey/ElfMouse and key_release / mouse_release tools (18 total): any session can unstick everything a previous caller left held
• License and package metadata now credit the author by name: Garth Elfstrom

138 tests. Standard library only. MIT.

v1.0.0 — first release

First release.

• ElfKey: virtual keyboard on /dev/uinput, driven by one-line FIFO commands
• ElfPad: virtual gamepad (Xbox-class — buttons, analog triggers, 2 sticks, D-pad)
• elfdeckai_mcp.py: zero-dependency MCP server exposing both as agent tools
• Safety-first design for LLM callers: strict input validation, bounded holds, nothing ever left stuck, owner-only FIFO
• Proven live: an LLM played Pokémon FireRed on a real Steam Deck through ElfKey (see notes/driving.md)

91 tests. Standard library only. MIT.