UTRNet: High-Resolution Urdu Text Recognition

Official Implementation of the paper "UTRNet: High-Resolution Urdu Text Recognition In Printed Documents"

The Poster:

Using This Repository

Environment

• Python 3.7
• Pytorch 1.9.1+cu111
• Torchvision 0.10.1+cu111
• CUDA 11.4

Installation

1. Clone the repository

git clone https://github.com/abdur75648/UTRNet-High-Resolution-Urdu-Text-Recognition.git

2. Install the requirements

conda create -n urdu_ocr python=3.7
conda activate urdu_ocr
pip3 install -r requirements.txt -f https://download.pytorch.org/whl/torch_stable.html

Running the code

1. Training

python3 train.py --train_data path/to/LMDB/data/folder/train/ --valid_data path/to/LMDB/data/folder/val/ --FeatureExtraction HRNet --SequenceModeling DBiLSTM --Prediction CTC --exp_name UTRNet-Large --num_epochs 100 --batch_size 8

2. Testing

CUDA_VISIBLE_DEVICES=0 python test.py --eval_data path/to/LMDB/data/folder/test/ --FeatureExtraction HRNet --SequenceModeling DBiLSTM --Prediction CTC --saved_model saved_models/UTRNet-Large/best_norm_ED.pth

3. Character-wise Accuracy Testing

• To create character-wise accuracy table in a CSV file, run the following command

CUDA_VISIBLE_DEVICES=0 python3 char_test.py --eval_data path/to/LMDB/data/folder/test/ --FeatureExtraction HRNet --SequenceModeling DBiLSTM --Prediction CTC  --saved_model saved_models/UTRNet-Large/best_norm_ED.pth

• Visualize the result by running char_test_vis

4. Reading individual images

• To read a single image, run the following command

CUDA_VISIBLE_DEVICES=0 python3 read.py --image_path path/to/image.png --FeatureExtraction HRNet --SequenceModeling DBiLSTM --Prediction CTC  --saved_model saved_models/UTRNet-Large/best_norm_ED.pth

5. Visualisation of Salency Maps

• To visualize the salency maps for an input image, run the following command

python3 vis_salency.py --FeatureExtraction HRNet --SequenceModeling DBiLSTM --Prediction CTC --saved_model saved_models/UTRNet-Large/best_norm_ED.pth --vis_dir vis_feature_maps --image_path path/to/image.pngE

Dataset

1. Create your own lmdb dataset

pip3 install fire
python create_lmdb_dataset.py --inputPath data/ --gtFile data/gt.txt --outputPath result/train

The structure of data folder as below.

data
├── gt.txt
└── test
    ├── word_1.png
    ├── word_2.png
    ├── word_3.png
    └── ...

At this time, gt.txt should be {imagepath}\t{label}\n
For example

test/word_1.png label1
test/word_2.png label2
test/word_3.png label3
...

Downloads

Trained Models

1. UTRNet-Large
2. UTRNet-Small

Datasets

1. UTRSet-Real
2. UTRSet-Synth
3. IIITH (Updated) (Original)
4. UPTI (Source)
5. UrduDoc - Will be made available subject to the execution of a no-cost license agreement. Please contact the authors for the same.

Text Detection (Supplementary)

The text detection inference code & model based on ContourNet is here. As mentioned in the paper, it may be integrated with UTRNet for a combined text detection+recognition and hence an end-to-end Urdu OCR.

Synthetic Data Generation using Urdu-Synth (Supplementary)

The UTRSet-Synth dataset was generated using a custom-designed robust synthetic data generation module - Urdu Synth.

End-To-End Urdu OCR Webtool

This tool was developed by integrating the UTRNet (https://abdur75648.github.io/UTRNe) with a text detection model (YoloV8 finetuned on UrduDoc) for end-to-end Urdu OCR.

The application is deployed on Hugging Face Spaces and is available for a live demo. You can access it here. If you prefer to run it locally, you can clone its repository and follow the instructions given there - Repo.

Note: This version of the application uses a YoloV8 model for text detection. The original version of UTRNet uses ContourNet for this purpose. However, due to deployment issues, we have opted for YoloV8 in this demo. While YoloV8 is as accurate as ContourNet, it offers the advantages of faster processing and greater efficiency.

Updates

• 01/01/21 - Project Initiated
• 21/11/22 - Abstract accepted at WIDAFIL-ICFHR 2022
• 12/12/22 - Repository Created
• 20/12/22 - Results Updated
• 19/04/23 - Paper accepted at ICDAR 2023
• 23/08/23 - Poster presentation at ICDAR 2023
• 31/08/23 - Webtool made available
• 31/01/24 - Updated Webtool (with YoloV8) made available via HuggingFace here

Acknowledgements

• This repository is based on deep-text-recognition-benchmark
• We acknowledge the Rekhta Foundation and the personal collections of Arjumand Ara for the scanned images and Noor Fatima and Mohammed Usman for the dataset and the manual transcription.
• We also thank all the members of the Computer Vision Group, IIT Delhi for their support and guidance.

Contact

• Abdur Rahman
• Prof. Arjun Ghosh
• Prof. Chetan Arora

Note

This is an official repository of the project. The copyright of the dataset, code & models belongs to the authors. They are for research purposes only and must not be used for any other purpose without the author's explicit permission.

Citation

If you use the code/dataset, please cite the following paper:

@InProceedings{10.1007/978-3-031-41734-4_19,
		author="Rahman, Abdur
		and Ghosh, Arjun
		and Arora, Chetan",
		editor="Fink, Gernot A.
		and Jain, Rajiv
		and Kise, Koichi
		and Zanibbi, Richard",
		title="UTRNet: High-Resolution Urdu Text Recognition in Printed Documents",
		booktitle="Document Analysis and Recognition - ICDAR 2023",
		year="2023",
		publisher="Springer Nature Switzerland",
		address="Cham",
		pages="305--324",
		isbn="978-3-031-41734-4",
		doi="https://doi.org/10.1007/978-3-031-41734-4_19"
}

License

. This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License for Noncommercial (academic & research) purposes only and must not be used for any other purpose without the author's explicit permission.