TiGeSR: Restore Text First, Enhance Image Later

[CVPR 2026] Restore Text First, Enhance Image Later: Two-Stage Scene Text Image Super-Resolution with Glyph Structure Guidance

Minxing Luo^1,2, · Linlong Fan^2, · Qiushi Wang^2,3 · Ge Wu¹ · Yiyan Luo² · Yuhang Yu² · Jinwei Chen² · Yaxing Wang^1,4 · Qingnan Fan^2,† · Jian Yang^1,5,†

¹ PCA Lab, VCIP, College of Computer Science, Nankai University · ² vivo BlueImage Lab, vivo Mobile Communication Co., Ltd. · ³ SDS, CUHK, Shenzhen · ⁴ NKIARI, Shenzhen Futian · ⁵ PCA Lab, School of Intelligence Science and Technology, Nanjing University

TIGER (Text-Image Guided supEr-Resolution) first restores text glyph structure, then uses the restored glyph prior to guide full-image enhancement. It targets scene-text images where both character fidelity and natural scene detail matter.

News

• 2026.07 — Code, inference, training, evaluation, UZ-ST alignment utilities, and model links are released.
• 2026.06 — TiGeSR is accepted to CVPR 2026.

Overview

The first stage reconstructs legible glyph structures from a low-quality scene image. The second stage conditions an image enhancer on both the input and the restored glyph mask, retaining realistic image content while protecting text readability.

Dependencies and installation

Tested on Linux with NVIDIA GPUs. Create a Python environment, install a PyTorch build compatible with your CUDA version from pytorch.org, then install the project dependencies:

git clone git@github.com:OpenVeraTeam/TiGeSR.git
cd TiGeSR

conda create -n tiger python=3.11 -y
conda activate tiger
pip install -r requirements.txt

Download the Stable Diffusion 3.5 Medium files and place them in weights/stable-diffusion-3.5-medium/.

Pre-trained models

We provide two internally matched checkpoint sets in the TiGeSR Hugging Face model repository:

• UZ-ST: stage 1 and stage 2 trained on UZ-ST.
• RealCE: stage 1 and stage 2 trained on RealCE; its stage-1 model is the pruned RoPE variant.

Download exactly one matching set. Do not mix stage-1 and stage-2 weights across sets.

# UZ-ST-trained checkpoints
hf download mxluocv/TiGeSR --include "uzst/**" --local-dir weights

# RealCE-trained checkpoints
hf download mxluocv/TiGeSR --include "realce/**" --local-dir weights

The expected layout is:

weights/
├── uzst/
│   ├── stage1/
│   │   ├── model.pth
│   │   └── encoder.pth
│   └── stage2/model.pkl
├── realce/
│   ├── stage1/
│   │   ├── model.pth
│   │   └── encoder.pth
│   └── stage2/model.pkl
└── stable-diffusion-3.5-medium/

Use configs/stage1.yaml with weights/uzst/. Use configs/realce_stage1.yaml with weights/realce/; this is the released pruned RoPE configuration (RoPE2Out_mask_pruned_config_hybridwEncoder.yaml).

⚡ Inference

Run the full TIGER pipeline

Run from the repository root. Without --anno_path, TIGER runs PP-OCRv5 internally. The following UZ-ST command is the default example:

python scripts/inference.py \
  --test_folder /path/to/lr_images \
  --output_dir outputs/uzst_demo \
  --difftsr_config configs/stage1.yaml \
  --idm_ckpt_path weights/uzst/stage1/model.pth \
  --enc_path weights/uzst/stage1/encoder.pth \
  --ctrlnet_ckpt_path weights/uzst/stage2/model.pkl \
  --timestep 150 --ctrl_layers 6 --enable_tiling

For RealCE, only switch to the matching checkpoint paths and pruned stage-1 config:

python scripts/inference.py \
  --test_folder /path/to/lr_images \
  --output_dir outputs/realce_demo \
  --difftsr_config configs/realce_stage1.yaml \
  --idm_ckpt_path weights/realce/stage1/model.pth \
  --enc_path weights/realce/stage1/encoder.pth \
  --ctrlnet_ckpt_path weights/realce/stage2/model.pkl \
  --timestep 150 --ctrl_layers 6 --enable_tiling

Use your own PP-OCRv5 annotations

Create reusable annotations from the integrated OCR path:

python scripts/make_ocr_annotations.py \
  --input-dir /path/to/lr_images \
  --output-dir /path/to/annotations

You may also supply manually corrected annotations or annotations produced by PaddleOCR / PP-OCRv5 or PaddleOCR2Pytorch. Use one file per image at <annotation_dir>/<image-stem>_res.json, then add --anno_path /path/to/annotations to inference.

{
  "doc_preprocessor_res": {"angle": 0},
  "dt_polys": [[[x1, y1], [x2, y2], [x3, y3], [x4, y4]]],
  "rec_texts": ["recognized text"],
  "rec_scores": [0.99],
  "textline_orientation_angles": [0]
}

dt_polys must be in pixels of the original input image. rec_scores and textline_orientation_angles are optional. TIGER supplies defaults of 1.0 and 0 when absent, and scales polygons internally when tiling is enabled.

Train

The launchers preserve the original experiment hyperparameters. Replace only the dataset and output paths in the scripts before launching.

# Stage 1, pruned RoPE (RealCE configuration)
bash scripts/train_stage1.sh

# Stage 1, unpruned RoPE (UZ-ST configuration)
bash scripts/train_stage1_unpruned.sh

# Stage 2, glyph-guided image enhancement
bash scripts/train_stage2.sh

Stage 1 accepts a comma-separated --train_folders list. For stage 2, prepare paired folders following the released RealCE convention: LR images in 13mm_aligned_1024/, targets in 52mm_aligned_1024/, and glyph masks in mask_52mm_patchedx4/. Select configs/stage1.yaml for UZ-ST or configs/realce_stage1.yaml for RealCE; stage 2 consumes stage-1 glyph guidance rather than this YAML directly.

Evaluation

Prediction and ground-truth filenames must share the same image stem. Auxiliary *_mask.png and *_subsr.png files are ignored.

python scripts/evaluate.py \
  --pred-dir outputs/uzst_demo \
  --gt-dir /path/to/ground_truth

For OCR accuracy (OCR-A), pass a JSON map from filename to expected text:

python scripts/evaluate.py \
  --pred-dir outputs/uzst_demo \
  --gt-dir /path/to/ground_truth \
  --labels-json /path/to/labels.json

Data preparation utilities

Synthetic degradation

data/deg_mine.py implements the synthetic degradation used in our experiments: two-stage Real-ESRGAN-style blur, noise, JPEG compression, and bicubic upsampling. It writes low-quality images to lr_normal_blur_kernel/ under the dataset root.

python data/deg_mine.py --root_path /path/to/rendered_dataset

UZ-ST alignment and color correction

tools/uzst_alignment/ contains the UZ-ST preparation utilities used for SIFT/MAGSAC alignment, LoFTR, RIPE, and color fixing.

pip install -r tools/uzst_alignment/requirements.txt

See tools/uzst_alignment/README.md for script roles, the optional RIPE dependency, and path configuration notes.

Acknowledgements

This release builds on the open-source ecosystems of Diffusers, PaddleOCR, and their dependencies. We thank their authors and contributors.

License

The code is released under the Apache License 2.0. Third-party model weights and datasets remain subject to their respective licenses and terms of use.

Citation

If you find TiGeSR useful for your research, please cite:

@inproceedings{luo2026restore,
  title={Restore text first, enhance image later: Two-stage scene text image super-resolution with glyph structure guidance},
  author={Luo, Minxing and Fan, Linlong and Wang, Qiushi and Wu, Ge and Luo, Yiyan and Yu, Yuhang and Chen, Jinwei and Wang, Yaxing and Fan, Qingnan and Yang, Jian},
  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  pages={30553--30563},
  year={2026}
}