A demonstration of document QR Code/text scanning using Tesseract and opencv
This demonstration first builds a jpg version of the following pdf document:
https://www.va.gov/vaforms/va/pdf/VA0730b.pdf
This jpg version is then used to display a web form along with a QR code for a unique document reference.
Once the web form is then printed, filled out by hand and scanned as a jpg, it is then processed to extract the printed/handwritten text along with the text associated with the QR code.
git clone https://github.com/RamSailopal/Doc-Scan.git
cd Doc-Scan
docker-compose up
The script pdfconvert/convert.py is used to generate the initial jpeg.
The web form can then be viewed in a browser by navigating to:
http:dockerserveraddress:8080?ref=testref
Where testref is the reference to be translated into a QR code.
Once the document is filled and scanned, the resulting jpeg is then used to output text using the script pdfscan/scan.py
This demonstration takes the following jpg:
https://github.com/RamSailopal/Doc-Scan/blob/main/pdfscan/FilledOut.jpg
It then processes the file to generate the following text
https://github.com/RamSailopal/Doc-Scan/blob/main/pdfscan/pdfscanout.txt
The initial web form had to be scaled out to display on one page and this effected the quality of the jpeg and subsequently the OCR results. Printed text was fine, but hand written text proved difficult to process acccuratly. QR codes were not processed at all.
As a comparison, A "screen grab" of part of the web form was taken and then the mouse used to add text (as if it were a pen). The resulting jpg can be viewed here:
https://github.com/RamSailopal/Doc-Scan/blob/main/pdfscan/doc-out1.png
The processed output can be seen here:
https://github.com/RamSailopal/Doc-Scan/blob/main/pdfscan/pdfscanout1.txt
With original scaling and no loss of quality, the QR code is processed correctly as well as the printed text. The mouse written text is again "patchy"
Once a form is printed, filled and scanned, add it to the pdfscan folder. Once this has been done run:
docker exec -it pdfscan /bin/bash -c 'cd /home/pdfscan && python3 scan1.py <imagefilename> > <nameofoutputfile>'
i.e.
docker exec -it pdfscan /bin/bash -c 'cd /home/pdfscan && python3 scan1.py scannedimage.jpg > outputtext.txt'
The output data will then be available to be viewed in the file pdfscan/outputtext.txt
In terms of hand written text, tesseract can be improved with "training" - https://tesseract-ocr.github.io/tessdoc/tess4/TrainingTesseract-4.00.html
Tesseract - https://tesseract-ocr.github.io/
Python Tesseract - https://pypi.org/project/pytesseract/
OpenCV QR Code detection - https://docs.opencv.org/4.x/de/dc3/classcv_1_1QRCodeDetector.html
Web QR Code generator - https://github.com/kazuhikoarase/qrcode-generator