Click on any text using just your keyboard. uniclick
takes a
screenshot of your desktop, runs optical character recognition (OCR)
on it, then displays narrowing UI that allows you to select a word. Once
your target word is selected, you cursor will be moved to its center.
uniclick will click or double-click on request. It's similar in spirit to
keynav.
Here's a demo:
The UI works by overlaying dark boxes over each word the OCR engine has
detected. You build up a search string by typing letters or numbers (all
other characters are ignored). As you type, uniclick will narrow the words
it highlights to just those that start with your search string. Pressing
<backspace>
will delete the last character you typed. Once there are less
than 5 words highlighted, you can also press <tab>
to cycle between them.
The selected word is surrounded by an extra border. Once you have selected
the word you want, type <enter>
to finish. The cursor will be moved to
the center of that word.
If you want to give up/cancel, press <esc>
at any time.
Note: because uniclick uses OCR to find the text, it can take a little
while to startup (approx. 5 seconds on my machine). This is it's primary
drawback. Also, whilst tesseract tries it's best, there will be occasionnal
errors. For example, if you've typed unicl
and the word uniclick
stops being
highlighted, it's probably because the l
has been misread. Try pressing
<backspace>
then 1
and seeing if it stays highlighted.
If you discover a method to recude startup time to, please let me know.
It expects to be inside an x11 session with a compositor running. I've only tested this on Ubuntu 19.04 running i3 and compton.
System requirements:
- tesseract
- xdotool
- scrot
- python3
Python requirements are in requirements.txt.
This could do with a proper installation method, but for now I do the following:
- Ensure the system requirements are installed (via
apt
for me). - Clone the repo:
$ git clone https://github.com/dj311/uniclick.git && cd uniclick
. - Make and enter virtual env:
$ python3 -m venv .venv && source .venv/bin/activate
. - Install Python dependencies
$ pip install -r requirements.txt
. - Make a wrapper script
$ touch ~/.local/bin/uniclick && chmod u+x ~/.local/bin/uniclick
with something like this inside:#!/usr/bin/env bash <path-to-uniclick-repo>/.venv/bin/python <path-to-uniclick-repo>/uniclick.py "$@"
Just run python3 uniclick.py
, or uniclick
if you followed the steps above. Add --click
or --double-click
if you want it to click at the end.
I have it bound to super+c
in my i3 config, like this:
bindsym $mod+c exec uniclick
bindsym $mod+control+c exec uniclick --click
bindsym $mod+shift+c exec uniclick --double-click
a. Narrowing can take advantage of common OCR errors. E.g searches for unicl
could
include unic1ick
.
b. Try different metrics for the search algorithm. Maybe some kind of edit distance
for ranking. That could even take into account OCR errors as in a.
c. Try to speed up OCR.
d. ???
- This gist was vital for me getting the overlay window to work.
- This stack overflow answer pointed me towards using the composite overlay window which made the ui reliable enough to actually be used.
- tesseract, pyocr, xdotool, pillow, python-xlib, etc for doing the hard work..