Skip to content

v2.0.0 — OCR Service, Window Occlusion, Batch Sequencing

Latest

Choose a tag to compare

@amichail-1 amichail-1 released this 11 Mar 02:29

What's New in v2.0.0

New Features

  • Window Occlusion Detection — Grid-based analysis showing which windows are truly visible (visibility %) and which are hidden behind others
  • Desktop Region Detection — Flood-fill algorithm to find uncovered screen areas
  • Shared OcrService — Centralized OCR with automatic dark theme enhancement (color inversion + contrast boost) — single-pass
  • PrintWindow API — Capture window content even when obscured by other windows
  • click_element OCR Fallback — UIAutomation first, then OCR for dark themes, web apps, iframes
  • run_sequence — Batch multiple UI actions (click, type, paste, hotkey, wait, focus) in a single MCP call
  • click_menu_item — Navigate parent > child menus with smooth mouse movement
  • DPI Awareness — Per-monitor DPI for correct coordinates on multi-monitor setups with mixed scaling
  • Embedded AI Instructions — Server sends tool usage guidelines on MCP connection via ServerInstructions

Improvements

  • Dark theme OCR: 0 → 37 lines detected on draw.io dark UI
  • OCR matching: exact > starts/ends with > contains, prefer closest length
  • Token efficiency: OCR-first workflow uses ~200 tokens vs ~5000 for screenshots

45+ Tools

Vision, interaction, batch actions, mouse/keyboard, window management, screenshots, OCR — full desktop control from a single .NET 8 executable.