aircalc

This is under review.

aircalc has been downloaded by 20,387 downloads worldwide as of Feb.24, 2023.

aircalc is a Python program based on two state-of-the-art libraries including hand gesture recognition library using mediapipe and optical character recognition library using tesseract. aircalc can be easilly installed by pip command (PyPi). This short program is made for education by showing how to use two state-of-the-art libraries. aircalc has an error correction function for correcting hand-drawn images in order to achieve perfect image recognition by tesseract.

The paper on aircalc is under submission. If the paper is accepted, the source code will be disclosed with detailed explanations.

The hand gesture recognition allows you to draw a math expression and it will be automatically calculated. The answer will be posted on the screen.

Writing letters with a pen on paper is very different from drawing letters in the air.

Drawing "-" minus operator and "+" operator in the air are extremely difficult so that these operators in the current system are replaced by "W" and "P" respectively.

When drawing letters with fingers in the air, letters that are difficult to recognize or write in the air need to be replaced with letters that can be accurately recognized by artificial intelligence. For example, replace the number "1" with "L" in the air.

How to install necessary libraries

$ pip install pytesseract

For Windows users, you should also install the latest tesseract

https://github.com/UB-Mannheim/tesseract/wiki

And add tesseract.exe of Tesseract-OCR directory PATH in .profile or .bashrc.

$ pip install mediapipe

Finally install aircalc

$ pip install aircalc

$ pip install aircalc --force-reinstall --no-cache-dir --no-binary :all:

How to run aircalc

aircalc is a program for drawing a math expression in the air for possible calculation.

aircalc is based on two open source libraries including mediapipe and tesseract.

There are six states of five fingers recognized by mediapipe library: 0-finger, 1-finger, 2-finger, 3-finger, 4-finger, and 5-finger respectively.

A pen of index finger tip is used for drawing an expression 
by fingers. 

0-finger can move the pen without drawing. 
1-finger can draw lines in the air. 
2-finger can move the pen without drawing. 
3-finger can delete the last touches of drawn letters for correction.
4-finger can call tesseract for transforming the hand-writing 
images to the digital text for possible calculation. 
For several seconds, 4-finger can terminate the program 
for showing the answer of the hand-drawn expression.
5-finger can move the pen without drawing. 

Continuous 4-finger state can terminate and exit this program.

0-finger or 5-finger is equivalent to 2-finger.

The saved picture is tranformed into digital text using the state-of-the-art optical character recognition.

Writing letters with a pen on paper is very different from drawing letters in the air.

Of the 0 to 9 digits, 1 is the least recognizable number.

Drawing "L" in the air represents "1".

"S" or "5" in the air represents "5".

"P" in the air represents "+" plus operator.

"W" or "-" represents "-" minus operator.

"V" in the air represents "/" division operator.

"M" in the air represents "*" multiplication operator.

"&" in the air represents "**" exponential operator.

Drawing two letters "a" and "A" in the air represents the sqrt() function. Therefore, the string "a13A" or "aL3A" represents sqrt(13).

$ aircalc

https://youtu.be/med_jrFTMPA

  sqrt(6)*2=?    

         1-3=?                10+2=?  

        4-5-3=?            

[

        2-3/5=?   

[

        34*5=?     

[

 2**8=?

[

      2&9V3 --> 2**9/3        

[

       aLLAV3=? -> sqrt(11)/3 

[