LOLPingYourAI

Ping anywhere on your desktop. Your AI teammate gets the context.

Hold ⌥ Option, drag, release on a signal. A League-style ping sends the spot you pointed at, the window it belongs to and what the signal means to your Codex, Claude or DSH teammate. It pings back, explains, works in that window, and stops the moment you call Danger.

English · 简体中文 · Download · ▶ Demo video · Architecture · Verification

_{🔊 Full demo with sound (47 s): English · 中文}

Writing "in the PDF on the right, second equation, why divide by √d?" takes longer than the question itself. In League you'd ping it. LOLPingYourAI brings that to your desktop. One gesture hands your AI teammate three things at once:

• Where: a 512×512 square around the pointer, mapped back to real screen coordinates.
• What: the window it belongs to (app, title, bounds, a stable window ID).
• Intent: the signal's instruction, such as explain, help finish this, stop, or check back in 30 minutes.

The teammate is your existing Codex, Claude or DSH harness, with its own model, account, tools and memory. LOLPingYourAI adds no planner and no separate history service. It supplies the ping, a bundled skill and a narrow set of desktop tools.

Point, don't describe The screenshot, window and intent arrive as structured data, not as text typed into whichever terminal has focus.	Recovers what you meant When a ping is ambiguous, the skill searches relevant sessions, project files and memory first. It asks you one focused question only if that isn't enough.	Interrupt any time Danger and Retreat revoke desktop actions at once. "Stopped" means the harness actually confirmed the cancellation.
It pings you back Replies show up on screen as reverse pings: on my way, all in, ?. They play one at a time, across displays, without moving your cursor.	Wards that come back Ward a window and the teammate looks again once, 30 minutes later. The visit survives restarts and sleep, and never runs twice.	Your wheel, your emotes Rebind slots and messages, import or crop emotes, or ask the teammate to draw one. Send a Weak! and it may talk back.

• The app opens to a dedicated chat home with full replies, a fixed composer and task status. Use the Settings button for configuration; switching back keeps your draft.
• You can also type directly on the chat home to start a conversation. Text-only chat needs no desktop capture permissions; ping a region when you want to share visual context.
• A small reply card appears near the target without requesting focus. Expand, copy, ask a follow-up or answer a clarification there.
• Agent pings and emotes stay visible for 6 seconds by default. Adjust Signals & Sound → AI reply duration (3–15 seconds). Replies wait for both the preceding visual and sound to finish; sounds play once.
• Live status uses LoL icons: received → understanding → working / waiting for you → done. A completion reaction plays through the existing serial visual-and-sound queue.
• Normal pings queue behind active work; Danger/Retreat interrupt promptly; All In switches targets. Ward/clear-ward never cancel the active task, and reactions append to its conversation.
• First-run setup checks the harness and macOS permissions, then offers an observation-only test. Settings have Teammate / Signals / Emotes / Wards tabs and an English/Chinese switch. Wards show a countdown, cancel and visit-now controls.
• Selecting a harness loads its own model catalog. Model and fast-mode choices are saved per harness; fast mode is off by default, resets to off when changing models, and only appears enabled when the backend advertises support. Changes apply next turn.
• Reasoning defaults to 0, the model's lowest supported level. 1 chooses the next level; larger indices clamp to the model's maximum. The effective level is displayed after a turn starts. DSH defaults to thinking on: its indices count enabled levels, with a separate off switch.
• DeepSeek Harness (DSH) is available through ACP. See installation, free text API setup and verification limits.

v0.1.4 brings the chat homepage, upstream interaction improvements and exploration feedback to both desktop installers. Download assets appear when the tagged release passes CI.

• A custom keyboard trigger still types when tapped; holding it while dragging uses it for the wheel. Releasing the trigger still cancels an open wheel.
• Settings → Wheel → Wheel size adjusts both wheels from 300–900 logical pixels (default 600). This is separate from the 512-pixel AI focus region and ping artwork size.
• In Settings → Emotes, import an image, then use its speaker button to add MP3 / OGG / WAV, up to 1 MiB and 5 seconds. Use the same button to remove the sound; click the tile to preview. Sounds stay on your computer and respect mute and the emote-sound switch.
• Optional Upside-down Missing (¿) is in the signal pool; drag it into a slot. It uses the Question instruction, including your Question customization. Enemy Vision now has the same red artwork on the wheel and desktop.
• Windows overlays leave the last 1 DIP of the display uncovered and refresh their topmost position before new content, including queued agent replies. macOS overlays stay out of Mission Control; replayed clicks retain double/triple-click counts, and settings recover after Dock restore. The current macOS source requires macOS 13+ on Apple Silicon.

v0.1.4 brings the chat homepage, upstream interaction improvements and exploration feedback to both desktop installers. Download assets appear when the tagged release passes CI.

	Signal	Your teammate…
	Question	explores and explains the selected region; uses relevant page navigation and native context search.
	Assist	inspects progress and helps finish. It recovers context first and asks if your intent is still unclear.
	Danger	interrupts, looks at a fresh screenshot and your correction, then reassesses.
	On my way	expands the target window and improves readability with reversible view changes.
	Ward	comes back once in 30 minutes, to observe and report only.
	All in	shifts its work to this target.
	Weak!	gets a playful complaint. A light comeback is allowed, and nothing is undone automatically.
	Retreat	stops and hands control back.
	Nice / Celebrate	keeps going, or celebrates a finished task.
	Clear ward	cancels this window's pending visits.

The original eight-slot layout is kept. Extra signals are in the slot picker: Push, Attention here, Look here. Every signal's message and behavior (observe, explore, work, correct, stop, ward, emote) can be edited under AI Teammate in settings. The skill reads the current configuration on every ping.

sequenceDiagram
  actor You
  participant App as LOLPingYourAI
  participant AI as Codex / Claude
  You->>App: ⌥ drag, release on "?"
  Note over App: capture the 512×512 square<br/>and the window identity
  App->>AI: signal meaning + screenshot + context
  AI->>App: MCP get_context, screenshot
  AI->>App: MCP reply_ping "on my way"
  App-->>You: reverse ping on screen + answer
  You->>App: Assist ping on unfinished work
  AI->>App: MCP desktop_action (that window only)
  You->>App: Danger!
  App->>AI: revoke actions, interrupt turn
  AI-->>App: cancellation confirmed

The native helper inherited from lolPing handles press, drag, release and cancel. A confirmed ping is captured and passed to a persistent teammate session. The same harness thread is resumed between pings and after app restarts, with no app-imposed idle expiry. Context management stays with the harness. Your messages, signals, agent replies and reverse pings are saved locally and shown in the chat window; use “Load earlier records” for older entries. The agent then works through a local MCP server: get_context, screenshot, desktop_action, reply_ping, save_handoff and ward. submit_icon is available only to a separate, explicitly requested icon-generation job. One action gate serializes desktop operations, and a correction invalidates every older one. Details are in docs/architecture.md.

v0.1.4 brings the chat homepage, upstream interaction improvements and exploration feedback to both desktop installers. Download assets appear when the tagged release passes CI.

1. Install. Download LOLPingYourAI-0.1.4-arm64.dmg from Releases and drag the app to Applications. Quit the original lolPing first: only one global input helper should run at a time.
2. Connect a teammate. Install and sign in to Codex CLI (codex login). The CLI bundled with the Codex desktop app is found automatically. Claude is experimental (see Platform status).
3. Grant permissions. Open the app, then enable Accessibility and Screen Recording for LOLPingYourAI in System Settings → Privacy & Security and restart it. AI Teammate in settings shows both checks.
4. Ping. Hover anything, hold ⌥ Option + left mouse, drag, and release on ?. ⌃ Control + drag opens the emote wheel.
5. Take control back with Retreat or ⌃ Control ⌥ Option Esc (Ctrl Alt Esc on Windows).

Optional: paste a goal / progress / next step once under AI Teammate → Task handoff. Later pings continue from it.

Download LOLPingYourAI-Setup-0.1.4.exe from v0.1.4. Install per user, connect a locally signed-in Codex CLI, and check the trigger key in Settings. Quit other lolPing/LOLPingYourAI instances first. The installer is unsigned.

The app discovers the Windows Codex desktop executable and standard npm installations. If needed, set LOLPINGAI_CODEX to the absolute codex.exe path before launching. Emergency stop is Ctrl+Alt+Esc. See Windows build, tests and limitations.

What was actually exercised, and what wasn't, is recorded in docs/verification.md.

• Local only. The MCP server talks to the app over loopback with a random per-launch token. Browser-origin requests are refused. Captures, sessions, wards and tokens stay in ~/Library/Application Support/LOLPingYourAI (%APPDATA%/LOLPingYourAI on Windows) and never belong in Git.
• Scoped actions. Question uses navigation-only exploration; explicit observe-only mappings and ward visits cannot click or type. The window is re-resolved before every action, coordinates go stale when it moves, and a closed, locked or ambiguous target fails visibly.
• Data to your provider. The square and the instruction go to the harness and model you chose, under your existing account.
• Claude's tools. Only read-only Read, Grep and Glob plus the desktop tools are exposed. A pre-tool hook refuses credential folders, key files, whole-home sweeps and every other tool, even ones your global Claude settings allow.
• Not a sandbox. These controls protect the backend this project ships. Extra plugins you add to a harness are outside them.

Defaults: focus square 512×512 logical pixels, ward delay 0.5 h. Change them in settings or from a terminal while the app runs:

node ai/cli.mjs help
node ai/cli.mjs doctor
node ai/cli.mjs status
node ai/cli.mjs ping '{"signal":"missing","point":{"x":640,"y":400}}'
node ai/cli.mjs config '{"regionSize":512,"wardHours":0.5}'
node ai/cli.mjs bind '{"harness":"codex","summary":"Goal, progress, next step"}'
node ai/cli.mjs stop

lolpingai is also the package bin. Where shell quoting differs, use --file payload.json. The CLI talks only to the running local app. doctor tells SDK availability apart from authenticated live operation. A per-signal example config is in examples/config.json.

Import PNG, JPEG, GIF or WebP under Emotes. Create an emote lets you crop to a square, keep transparency or remove a flat background, and preview before adding. You can also describe one, optionally with a reference image. The teammate uses an image tool if it has one, otherwise it draws an original SVG, and the preview labels which method it used. Imported emotes can carry their own instructions.

The bundled ping textures, sounds and 44 animated emotes are the League of Legends assets that upstream lolPing 0.5.1 ships. A local pack can override them, and Restore LoL pack removes the override.

Requirements: Node 22+, pnpm 10 and Xcode command-line tools on macOS. On Windows: Visual Studio 2022 or later C++ build tools (C++20) and the .NET Framework C# compiler.

git clone https://github.com/ChristianYang37/LOLPingYourAI.git
cd LOLPingYourAI
pnpm install --frozen-lockfile --ignore-scripts
node node_modules/electron/install.js
pnpm build:helper && pnpm build:desktop
pnpm dev

pnpm typecheck
pnpm test:helper                                # C++ input helper
pnpm test                                       # Vitest + AI bridge tests
pnpm test:acceptance                            # Portable acceptance-tool protocol tests
pnpm test:performance                           # Local performance gates; no API calls
pnpm build && pnpm exec electron tools/check-assets.cjs
pnpm exec electron tools/check-agent-queue.cjs  # real overlay + Web Audio queue
pnpm exec electron tools/test-board.cjs         # disposable window for manual tests
pnpm dist:mac                                   # Apple Silicon DMG in release/
pnpm dist                                       # Windows x64 installer (run on Windows)

• Run it on Windows and report what breaks.
• Repeat the signed-in Claude acceptance script with the integrated reply card and effort settings.
• Physical multi-monitor, mixed-DPI and sleep/lock testing.
• Developer ID signing and notarization for the macOS build.
• Integrating native Computer Use backends with the shared cancellation gate.

Thanks to @csxq0605 for the Claude acceptance investigation and reusable test scripts in PR #8. The findings informed the fixes in PR #11, and the adapted scripts now live in tools/acceptance/.

Thanks to @fzh000543 for publishing an early Windows portable build and contributing its packaging configuration and usage documentation in PR #7.

Derived from darrenprx/lolPing 0.5.1, which provides the input helper, radial wheels, overlays, settings and emote import. Copyright © 2026 Danrui Meng, MIT. Code in this repository is MIT (LICENSE).

League of Legends artwork, animations and audio © Riot Games, Inc. They are not covered by the MIT license; see THIRD_PARTY_NOTICES.md. This is an independent fan project, not endorsed by Riot Games, OpenAI or Anthropic.

Connection & diagnostics is off by default. Enable it explicitly in Settings → AI Teammate when troubleshooting. With it off, raw protocol events are not collected; chat and ping history remain available. See performance testing for CI scope, budgets and local measurements.

The chat shows the request capture and latest agent view, with Markdown tables, fenced code and LaTeX math. A generated working emote stays beside streaming output during active desktop tasks. Nearby repeated question marks add tone to the same request when the target identity matches; emotional emotes invite playful replies without starting edits. See interaction behavior and validation.

Contributing and PR workflow · Release procedure