Skip to content

Clippy Vision v2.1.0

Choose a tag to compare

View all tags
@protocorn protocorn released this 13 Oct 14:59
· 43 commits to main since this release
v2.1.0
79e8ff5

Clippy Vision v2.1.0 Release Notes

Overview

This release builds on Clippy Vision v2.0.0 with experimental Linux support, in-app chat, expanded private-window detection, OS capture opt-outs, MCP safety labeling, and unsigned Windows/Linux packaging.

Version: v2.1.0

Baseline: Clippy Vision v2.0.0 (commit 8271941, October 9, 2026)

Included Platforms: Windows NSIS installer and Linux AppImage (unsigned). macOS installers are excluded (opt-in workflow only).

Verification labels:

  • CI-only — unit/integration tests in GitHub Actions
  • Xvfb launch — packaged Linux AppImage under Xvfb in .github/workflows/release.yml
  • Windows silent install — packaged NSIS installer on windows-latest in that workflow
  • real machine — a physical desktop; v2.1.0 has not been tested on one

This release was CI-tested only. There has been no real-machine testing for v2.1.0. The Windows installer and Linux AppImage are unsigned.

What the release dry-run proves (and does not)

The PR dry-run of .github/workflows/release.yml is the maintainer go/no-go bar for packaging:

Intended to prove: the packaged Windows installer and Linux AppImage start, and — after first-run setup is already complete — the Electron main process spawns the Python backend and GET /health returns HTTP 200.

Does not prove: the interactive setup wizard, Ollama, a real chat/embedding model, or real hardware/desktops (GNOME, KDE, Win32 UI Automation on a personal PC, etc.).

CI bypasses used to reach /health (not shipped to users):

  • setup_complete.json is pre-seeded at Electron userData (the SETUP_FLAG path in electron-ui/electron/lib/paths.js: <userData>/setup_complete.json, not <userData>/data/). That skips the wizard so main.js runs preflight and api.startServer().
  • llm_config.json in <userData>/data/ sets base_url to http://127.0.0.1:11555 so usesManagedOllama() is false.
  • scripts/ci_llm_stub.py answers GET / and GET /api/tags for the seeded chat_model (test-model).
  • Python 3.12 plus requirements.txt are installed on the runner; CLIPPY_PYTHON points the packaged app at that interpreter. Python is not inside the installer or AppImage.

/health on the release build: both packaging jobs green (CI run 38286025883).

  • build-linux (CI job 114934494105): GET http://127.0.0.1:39367/health → 200 {"status":"ok"} at 10s. USER_DATA=/home/runner/.config/Clippy Vision.
  • build-windows (CI job 114934493911): GET http://127.0.0.1:61677/health → 200 {"status":"ok"} at 3s. USER_DATA=C:\Users\runneradmin\AppData\Roaming\Clippy Vision.

New Features

1. Experimental Linux support

Verification: CI-only for capture/AT-SPI/setup-check tests. Packaged AppImage Xvfb launch in CI run 38286025883: /health 200 {"status":"ok"}. Not real machine.

What the Linux code path implements:

  • Window metadata (xdotool), process names (psutil), clipboard (wl-paste / xclip / xsel), idle (xprintidle / xdotool)
  • X11 screenshots via mss with focused-window crop and a blank-frame guard
  • Wayland session detection: compositor is logged, capture is skipped (no blank JPEGs stored)
  • AT-SPI accessibility text — CI-only; not tried on a real desktop
  • Setup wizard OS check accepts Linux — CI-only; the wizard itself is not driven in the dry-run

Known Linux gaps:

  • No native Wayland screenshot capture (xdg-desktop-portal not implemented)
  • No browser URL or private-window detection on Linux
  • AppImage is unsigned

Installation (Linux):

  1. Install a system Python 3.11+ (see Python runtime below). Python is not in the AppImage.
  2. Download ClippyVision-Linux-x86_64-2.1.0.AppImage
  3. chmod +x ClippyVision-Linux-x86_64-2.1.0.AppImage
  4. Optional capture helpers: xdotool, x11-utils, xprintidle, xclip/xsel/wl-clipboard
  5. Run the AppImage. Complete the setup wizard on first launch (the dry-run does not exercise that wizard).

2. Firefox, Opera, and Vivaldi private window detection

Verification: CI-only (mocked accessibility trees). Not real machine.

Browser Windows macOS Linux Verification
Chrome yes yes no inherited from v2.0.0; not re-tested on a real machine
Edge yes yes no inherited from v2.0.0
Brave yes yes no inherited from v2.0.0
Firefox yes (CI-only) untested no CI-only; macOS AppleScript is attempted, not run on a real Mac
Opera yes (CI-only) untested no CI-only; macOS AppleScript is attempted, not run on a real Mac
Vivaldi yes no no CI-only
Safari no no — not implemented

3. In-app chat

Verification: CI-only (backend unit tests; frontend not launched against Ollama). Not real machine.

  • Backend: /chat and /chat/stream with recent-activity context, private-window filtering, conversation persistence
  • Frontend: chat panel, streaming bubbles, conversation drawer
  • Requires a working local model via Ollama in real use. CI does not start Ollama.

4. Setup wizard: live Ollama download progress

Verification: CI-only (stream parser unit tests). The wizard UI is not driven in CI.

5. Untrusted captured text in MCP replies

Verification: CI-only. Screen-derived content in MCP tool replies and in-app chat is wrapped in nonce-delimited untrusted blocks.

6. OS-level window capture opt-outs

Verification: CI-only (mocked APIs). Not real machine.

When the foreground window asks not to be captured, Clippy skips screenshot, OCR, accessibility text, and clipboard content and records at most a minimal redacted event.

  • Windows: GetWindowDisplayAffinity (WDA_EXCLUDEFROMCAPTURE / WDA_MONITOR)
  • macOS: frontmost layer-0 window sharing state
  • Linux: no standard per-window API; process-name fallback only (password-manager executable names)

Detection errors fail open (allow capture). Browser private-window detection is separate.

7. CI and packaging

Verification: workflows run on every PR.

  • pytest on Ubuntu and Windows (CI-only), including a deflake of the Xvfb screenshot test
  • Unsigned Windows NSIS + Linux AppImage on tag/dispatch, with a PR dry-run when packaging files change

8. MCP setup guides and templates

  • Claude Desktop and LM Studio guides
  • VS Code MCP setup guide (@Afrin-banu-A) — documentation; author marked the steps as not verified on a real VS Code install
  • GitHub issue/PR templates

Python runtime (not bundled)

A system Python 3.11 or later is required. Python is not bundled in the Windows installer or the Linux AppImage.

requirements.txt does not pin a Python version. The packaged app does not contain an interpreter.

Resolution (electron-ui/electron/lib/paths.js): CLIPPY_PYTHON if set, else python3 (non-Windows) or python (Windows) from PATH.

CI unit tests and the release dry-run use Python 3.12 (.github/workflows/tests.yml).

When Python is missing, the setup wizard (electron-ui/electron/lib/setup.js) does this:

  • Preflight copy in electron-ui/electron/lib/setup.js now says "Clippy needs Python 3.11+ on PATH." The published v2.1.0 installer and AppImage were not rebuilt, so those binaries still ship the older "Python 3.9+" strings. The HTML step subtitle in electron-ui/src/setup.html still says "Checking for Python 3.9+" (updated on main after this build).
  • The check is python --version succeeding, not a parsed 3.11 minimum. Python 3.9 is accepted.
  • Windows: winget install Python.Python.3.11 --silent. If winget fails: "Could not install Python. Please install manually from python.org". If install succeeds but PATH does not: "Python installed but not on PATH. Restart required."
  • macOS and Linux: tries Homebrew brew install python when brew --prefix works. If that does not yield a working interpreter: log "Install Python 3.11+ from python.org or Homebrew, then retry." and step error "Install Python 3.11+ and make sure it is on PATH." Linux is not installed via apt by the wizard.

To force an interpreter (PowerShell):

$env:CLIPPY_PYTHON = "C:\path\to\python.exe"

Linux/macOS:

export CLIPPY_PYTHON=/path/to/venv/bin/python3

Bug Fixes

  • Linux PID → process name — CI-only
  • pytest platform markers
  • Linux setup wizard OS/VRAM check — CI-only
  • Xvfb screenshot test flake — CI-only
  • VS Code one-click Connect wrote mcp.json to ~/AppData/Roaming/Code/User/mcp.json on macOS/Linux — CI-only (mocked platform/env/home path tests). Now uses %APPDATA%\Code\User\mcp.json (Windows), ~/Library/Application Support/Code/User/mcp.json (macOS), $XDG_CONFIG_HOME or ~/.config/Code/User/mcp.json (Linux). Not real machine.

Changes

  • Roadmap / deferred signed installers
  • Lazy Windows/macOS imports

Known issues

Setup wizard can't install Ollama on Linux (and is limited on macOS)

In these 2.1.0 binaries the setup wizard installs Ollama (and Python) automatically only on Windows, via winget. On macOS it only tries Homebrew. On Linux it has no installer path and stops at the Ollama step. Workaround: install Ollama yourself (https://ollama.com/download) and a system Python 3.11+, then re-run setup. A fix is in progress on main and is not included in these 2.1.0 binaries.

Private-browsing docs overstate coverage

v2.1.0 ships without waiting for issue 16. Several docs claim more private-browsing protection than the code was tested for, and they contradict each other:

  • docs/usage.md says Firefox, Opera, and Vivaldi private windows "are covered on Windows", then two lines later says "Firefox, Opera, and Safari are not covered." The first sentence treats the new detectors as finished product coverage. Those extra detectors are CI-only (mocked accessibility trees). Nobody opened a real Firefox, Opera, or Vivaldi private window.
  • PROJECT_VISION.md says Chrome, Edge, Brave, Firefox, Opera, and Vivaldi private windows "are blacked out on Windows" as if that were verified. Firefox/Opera/Vivaldi on Windows have no real machine test.
  • core/private_windows.py still documents only Chrome, Edge, or Brave.
  • What the code actually does: Windows tries Chrome, Edge, Brave, Opera, Vivaldi, and Firefox; macOS AppleScript is attempted for Chrome, Edge, Brave, Firefox, and Opera (Firefox/Opera untested on a Mac; Vivaldi is not in the macOS map); Safari is not covered; Linux always returns false.
  • Pause capture when a private window must not be stored.

Python 3.11+ is not bundled

Install a system Python 3.11+ before first run. The NSIS installer and AppImage do not ship an interpreter. Set CLIPPY_PYTHON to pick one; otherwise the app uses python (Windows) or python3 (Linux/macOS) from PATH. See Python runtime (not bundled) above.

Unsigned installers

All installers are unsigned.

  • Windows: SmartScreen will warn ("Windows protected your PC"). Choose More info → Run anyway.
  • Linux AppImage: run chmod +x ClippyVision-Linux-x86_64-2.1.0.AppImage before the first launch.
  • macOS (not in this release): Gatekeeper block; right-click → Open, or xattr -d com.apple.quarantine.

Verification labels for every feature above are CI-only, Xvfb launch, Windows silent install, or real machine. There has been no real machine testing for v2.1.0.

Known Limitations

Unsigned installers

See Unsigned installers under Known issues. Signing is deferred (cost).

First-run setup

On a real machine the backend starts only after the wizard writes setup_complete.json. The dry-run pre-seeds that file; it does not test the wizard.

Linux

See Linux section above. Not real machine.

Release Assets

  1. ClippyVision-Windows-Setup-2.1.0.exe (unsigned) — Windows silent install in CI run 38286025883: /health 200 {"status":"ok"}. Setup pre-seeded, Ollama stubbed. Not real machine.
  2. ClippyVision-Linux-x86_64-2.1.0.AppImage (unsigned) — Xvfb launch in the same run: /health 200 {"status":"ok"}. Same bypasses. Not real machine.

macOS installers are not included.

Links

Credits

Credit for Clippy Vision design goes to protocorn (Sahil Chordia) and contributors.

Assets 4
Loading