Move the mouse with your hand.
Hold your hand up to the webcam and the cursor goes where you point. Tap your thumb and finger together to click.
Linux (X11 and Wayland) · Windows · macOS
MotionKey watches one hand through MediaPipe and turns it into pointer movement, clicks, drags and scrolling. It all happens on your machine — the model is downloaded once and nothing is sent anywhere after that.
| Pose | What happens |
|---|---|
| Hand held up | The cursor follows your hand |
| Tap thumb + index together | Left click |
| Pinch thumb + index and hold | Click and drag |
| Tap thumb + middle together | Right click |
| Tap thumb + ring together | Middle click |
| Raise index + middle, move your hand | Scroll, up/down and sideways |
| Make a fist | Cursor freezes where it is |
| Hold that fist ~1.5 s | Pause tracking, and again to resume |
Turn any of them off, and nudge the pinch sensitivity, the click/drag threshold and the scroll speed until they suit your hand.
Grab a build from the releases page — Python and everything else is bundled, so there is nothing to set up first.
| System | Installer | Or portable |
|---|---|---|
| Windows 10/11 | …-windows-x86_64-setup.exe |
…-windows-x86_64.zip |
| macOS (Apple silicon) | …-macos-arm64.dmg |
…-macos-arm64.tar.gz |
| Debian, Ubuntu, Mint | motionkey_…_amd64.deb |
…-linux-x86_64.tar.gz |
| Any other Linux | …-linux-x86_64.AppImage |
it is the portable one |
None of the builds are code signed, so Windows SmartScreen and macOS Gatekeeper will both want a word the first time. The release notes give the exact clicks.
Intel Macs need the source install below — MediaPipe stopped publishing x86_64 macOS wheels at 0.10.22.
To run from source instead:
git clone https://github.com/timeofthewolf/MotionKey
cd MotionKey
uv sync # or: pip install -e .
uv run motionkeyThe hand landmark model (~8 MB) downloads by itself the first time you start tracking and is cached from then on.
If something misbehaves, motionkey --selftest loads every component, runs one
inference and tells you which piece is missing.
MotionKey creates a virtual mouse through /dev/uinput. That is the only
approach that works under Wayland, and it works fine on X11 too.
The .deb installs the necessary udev rule for you, and it takes effect at your
next login. With the AppImage or the tarball, do it once by hand:
echo 'KERNEL=="uinput", MODE="0660", GROUP="input", OPTIONS+="static_node=uinput", TAG+="uaccess"' \
| sudo tee /etc/udev/rules.d/60-motionkey-uinput.rules
sudo udevadm control --reload && sudo udevadm triggerSome distributions already hand your session an ACL on /dev/uinput, in which
case there is nothing to do. If it still will not open, add yourself to the
input group (sudo usermod -aG input $USER) and log back in.
Windows needs nothing. macOS asks for camera access on first launch, and you have to grant Accessibility permission by hand before it can move the pointer: System Settings → Privacy & Security → Accessibility.
Open the app, flip Tracking on, hold your hand up. That is the whole setup — the default Direct mapping needs no calibration.
- Direct — a box in the middle of the camera view maps onto the screen. Works immediately. Shrink the active area if you would rather move your hand less.
- Calibrated — records a perspective transform, which is what you want when the camera sits off to one side of the screen rather than above it. Click Calibrate…, then pinch at each of the nine dots and hold still.
- Relative — trackpad style: your hand nudges the cursor from wherever it already is. Best for precision work and very large desktops.
Smoothing and Responsiveness tune the 1€ filter behind the cursor. It smooths hard while your hand hovers and gets out of the way when you move quickly, so you get neither jitter nor lag — but if you want it steadier or snappier, those are the two dials.
The virtual mouse sends relative movement by default, so MotionKey has to keep track of where it thinks the pointer is. Grab a physical mouse at the same time and the two disagree. Press Re-sync pointer, or switch Pointer mode to Absolute under Setup, which cannot drift at all. Absolute mode is not the default because it relies on your compositor mapping an absolute pointing device across the whole layout, and not all of them do.
Run motionkey --diagnose. It walks you through five poses and prints what
MediaPipe actually reports for your hand — jitter, hand speed, pinch ratios,
per-finger straightness — next to the thresholds in use, so a bad fit is
something you can see rather than guess at. Add --diagnose-out samples.json
to keep the raw landmarks.
The numbers map straight onto settings: the pinch section suggests a Pinch sensitivity value, and the jitter and speed figures tell you whether Smoothing is doing too much or too little.
motionkey --diagnose # measure your hand against the pose thresholds
motionkey --selftest # check every component loads
motionkey --list-cameras # show detected cameras
motionkey --camera 2 # pick a camera for this run
motionkey --backend uinput # force an input backend
motionkey --start # begin tracking immediately
motionkey --paths # print config file locations
motionkey --reset-config # restore defaultsOn Windows use motionkey-cli.exe for these — the windowed build has no
console to print to.
Settings and calibration live in the usual per-platform place:
~/.config/motionkey on Linux, %APPDATA%\MotionKey on Windows,
~/Library/Application Support/MotionKey on macOS. motionkey --paths prints
the exact locations. Everything is editable in the app, so you should never
need to open the files.
camera ──► capture thread ──► newest frame only ──► MediaPipe ──► HandData
│
gesture state machine ◄───────┘
│
1€ filter ──► pointer mapping ──► input backend ──► OS
Capture and inference run on separate threads with a single-slot mailbox between them. When inference cannot keep up, the extra frames are dropped rather than queued, so latency stays at roughly one inference instead of creeping upwards — which is the thing you actually feel when steering a cursor by hand.
Finger poses come from joint angles on MediaPipe's metric world landmarks, and pinch distances are divided by the hand's own size. One set of thresholds therefore holds whatever the distance from the camera, whichever way the hand is turned, and whoever's hand it is.
uv sync --extra dev
uv run pytestThe tests build synthetic hand poses (tests/hands.py), so the gesture logic,
the mapping and the configuration are all covered without a camera.
The screenshots and GIFs above are generated the same way:
uv run python tools/make_media.py docs/mediaThat drives the real window offscreen with synthetic hands and a recording input backend, so it never opens a camera or touches your pointer. The camera image in the preview is drawn from the same landmarks — everything layered over it is the real app rendering real state.
.github/workflows/release.yml builds every download when a v* tag is
pushed, installs and runs each installer on a clean runner before publishing,
and attaches the lot to a GitHub release with checksums. PyInstaller freezes
the interpreter and native libraries of the machine it runs on, so each
platform has to be built on its own runner — there is no cross-compiling. CI
installs with uv sync --locked so a release contains the same dependency
versions the tests ran against.
To build for the platform you are on:
uv sync --group packaging
uv run pyinstaller packaging/motionkey.spec --distpath dist --noconfirm
./dist/MotionKey/MotionKey --selftest
packaging/linux/make_deb.sh dist/MotionKey 0.3.0 artifacts
packaging/linux/make_appimage.sh dist/MotionKey 0.3.0 artifacts| Package | What it does |
|---|---|
mediapipe |
Hand landmark detection |
opencv-python |
Camera capture and homography |
PyQt6 |
The GUI |
numpy |
Numerical helpers |
pynput |
Pointer backend for Windows, macOS and X11 |
evdev |
Virtual mouse on Linux, and the only thing that works on Wayland |
python-xlib |
Reads the true cursor position on X11 |
MIT licensed.