-
Notifications
You must be signed in to change notification settings - Fork 738
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
* add dbnet++ * fix docstring * fix some param names * fix * fix docstring * add init * add doc; remove configs * add dbnet++ to readme * fix readme * update config * update readme * update readme * update ocr.py * update metafile.yml * update readme * update readme * move to dbnetpp * fix paths * fix head level * fix typo * update demo.md * Update configs/textdet/dbnetpp/README.md Co-authored-by: Tong Gao <gaotongxiao@gmail.com> * fix typo * fix link
- Loading branch information
1 parent
b4678eb
commit fbc138d
Showing
14 changed files
with
320 additions
and
13 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,28 @@ | ||
model = dict( | ||
type='DBNet', | ||
backbone=dict( | ||
type='mmdet.ResNet', | ||
depth=50, | ||
num_stages=4, | ||
out_indices=(0, 1, 2, 3), | ||
frozen_stages=-1, | ||
norm_cfg=dict(type='BN', requires_grad=True), | ||
norm_eval=False, | ||
style='pytorch', | ||
dcn=dict(type='DCNv2', deform_groups=1, fallback_on_stride=False), | ||
init_cfg=dict(type='Pretrained', checkpoint='torchvision://resnet50'), | ||
stage_with_dcn=(False, True, True, True)), | ||
neck=dict( | ||
type='FPNC', | ||
in_channels=[256, 512, 1024, 2048], | ||
lateral_channels=256, | ||
asf_cfg=dict(attention_type='ScaleChannelSpatial')), | ||
bbox_head=dict( | ||
type='DBHead', | ||
in_channels=256, | ||
loss=dict(type='DBLoss', alpha=5.0, beta=10.0, bbce_loss=True), | ||
postprocessor=dict( | ||
type='DBPostprocessor', text_repr_type='quad', | ||
epsilon_ratio=0.002)), | ||
train_cfg=None, | ||
test_cfg=None) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,33 @@ | ||
# DBNetpp | ||
|
||
> [Real-Time Scene Text Detection with Differentiable Binarization and Adaptive Scale Fusion](https://arxiv.org/abs/2202.10304) | ||
<!-- [ALGORITHM] --> | ||
|
||
## Abstract | ||
|
||
Recently, segmentation-based scene text detection methods have drawn extensive attention in the scene text detection field, because of their superiority in detecting the text instances of arbitrary shapes and extreme aspect ratios, profiting from the pixel-level descriptions. However, the vast majority of the existing segmentation-based approaches are limited to their complex post-processing algorithms and the scale robustness of their segmentation models, where the post-processing algorithms are not only isolated to the model optimization but also time-consuming and the scale robustness is usually strengthened by fusing multi-scale feature maps directly. In this paper, we propose a Differentiable Binarization (DB) module that integrates the binarization process, one of the most important steps in the post-processing procedure, into a segmentation network. Optimized along with the proposed DB module, the segmentation network can produce more accurate results, which enhances the accuracy of text detection with a simple pipeline. Furthermore, an efficient Adaptive Scale Fusion (ASF) module is proposed to improve the scale robustness by fusing features of different scales adaptively. By incorporating the proposed DB and ASF with the segmentation network, our proposed scene text detector consistently achieves state-of-the-art results, in terms of both detection accuracy and speed, on five standard benchmarks. | ||
|
||
<div align=center> | ||
<img src="https://user-images.githubusercontent.com/45810070/166850828-f1e48c25-4a0f-429d-ae54-6997ed25c062.png"/> | ||
</div> | ||
|
||
## Results and models | ||
|
||
### ICDAR2015 | ||
|
||
| Method | Pretrained Model | Training set | Test set | #epochs | Test size | Recall | Precision | Hmean | Download | | ||
| :---------------------------------------------------------------------------: | :------------------------------------------------------------------------------------------------------------------------: | :-------------: | :------------: | :-----: | :-------: | :----: | :-------: | :---: | :-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------: | | ||
| [DBNetpp_r50dcn](/configs/textdet/dbnetpp/dbnetpp_r50dcnv2_fpnc_1200e_icdar2015.py) | [Synthtext](/configs/textdet/dbnetpp/dbnetpp_r50dcnv2_fpnc_100k_iter_synthtext.py) ([model](https://download.openmmlab.com/mmocr/textdet/dbnet/dbnetpp_r50dcnv2_fpnc_100k_iter_synthtext-20220502-db297554.pth) \| [log](https://download.openmmlab.com/mmocr/textdet/dbnet/dbnetpp_r50dcnv2_fpnc_100k_iter_synthtext-20220502-db297554.log.json))| ICDAR2015 Train | ICDAR2015 Test | 1200 | 1024 | 0.822 | 0.901 | 0.860 | [model](https://download.openmmlab.com/mmocr/textdet/dbnet/dbnetpp_r50dcnv2_fpnc_1200e_icdar2015-20220502-d7a76fff.pth) \| [log](https://download.openmmlab.com/mmocr/textdet/dbnet/dbnetpp_r50dcnv2_fpnc_1200e_icdar2015-20220502-d7a76fff.log.json) | | ||
|
||
## Citation | ||
|
||
```bibtex | ||
@article{liao2022real, | ||
title={Real-Time Scene Text Detection with Differentiable Binarization and Adaptive Scale Fusion}, | ||
author={Liao, Minghui and Zou, Zhisheng and Wan, Zhaoyi and Yao, Cong and Bai, Xiang}, | ||
journal={IEEE Transactions on Pattern Analysis and Machine Intelligence}, | ||
year={2022}, | ||
publisher={IEEE} | ||
} | ||
``` |
33 changes: 33 additions & 0 deletions
33
configs/textdet/dbnetpp/dbnetpp_r50dcnv2_fpnc_100k_iter_synthtext.py
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,33 @@ | ||
_base_ = [ | ||
'../../_base_/default_runtime.py', | ||
'../../_base_/schedules/schedule_sgd_100k_iters.py', | ||
'../../_base_/det_models/dbnetpp_r50dcnv2_fpnc.py', | ||
'../../_base_/det_datasets/synthtext.py', | ||
'../../_base_/det_pipelines/dbnet_pipeline.py' | ||
] | ||
|
||
train_list = {{_base_.train_list}} | ||
test_list = {{_base_.test_list}} | ||
|
||
train_pipeline_r50dcnv2 = {{_base_.train_pipeline_r50dcnv2}} | ||
test_pipeline_4068_1024 = {{_base_.test_pipeline_4068_1024}} | ||
|
||
data = dict( | ||
samples_per_gpu=16, | ||
workers_per_gpu=8, | ||
val_dataloader=dict(samples_per_gpu=1), | ||
test_dataloader=dict(samples_per_gpu=1), | ||
train=dict( | ||
type='UniformConcatDataset', | ||
datasets=train_list, | ||
pipeline=train_pipeline_r50dcnv2), | ||
val=dict( | ||
type='UniformConcatDataset', | ||
datasets=test_list, | ||
pipeline=test_pipeline_4068_1024), | ||
test=dict( | ||
type='UniformConcatDataset', | ||
datasets=test_list, | ||
pipeline=test_pipeline_4068_1024)) | ||
|
||
evaluation = dict(interval=200000, metric='hmean-iou') # do not evaluate |
39 changes: 39 additions & 0 deletions
39
configs/textdet/dbnetpp/dbnetpp_r50dcnv2_fpnc_1200e_icdar2015.py
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,39 @@ | ||
_base_ = [ | ||
'../../_base_/default_runtime.py', | ||
'../../_base_/schedules/schedule_sgd_1200e.py', | ||
'../../_base_/det_models/dbnetpp_r50dcnv2_fpnc.py', | ||
'../../_base_/det_datasets/icdar2015.py', | ||
'../../_base_/det_pipelines/dbnet_pipeline.py' | ||
] | ||
|
||
train_list = {{_base_.train_list}} | ||
test_list = {{_base_.test_list}} | ||
|
||
train_pipeline_r50dcnv2 = {{_base_.train_pipeline_r50dcnv2}} | ||
test_pipeline_4068_1024 = {{_base_.test_pipeline_4068_1024}} | ||
|
||
load_from = 'checkpoints/textdet/dbnetpp/res50dcnv2_synthtext.pth' | ||
|
||
data = dict( | ||
samples_per_gpu=32, | ||
workers_per_gpu=8, | ||
val_dataloader=dict(samples_per_gpu=1), | ||
test_dataloader=dict(samples_per_gpu=1), | ||
train=dict( | ||
type='UniformConcatDataset', | ||
datasets=train_list, | ||
pipeline=train_pipeline_r50dcnv2), | ||
val=dict( | ||
type='UniformConcatDataset', | ||
datasets=test_list, | ||
pipeline=test_pipeline_4068_1024), | ||
test=dict( | ||
type='UniformConcatDataset', | ||
datasets=test_list, | ||
pipeline=test_pipeline_4068_1024)) | ||
|
||
evaluation = dict( | ||
interval=100, | ||
metric='hmean-iou', | ||
save_best='0_hmean-iou:hmean', | ||
rule='greater') |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,28 @@ | ||
Collections: | ||
- Name: DBNetpp | ||
Metadata: | ||
Training Data: ICDAR2015 | ||
Training Techniques: | ||
- SGD with Momentum | ||
- Weight Decay | ||
Training Resources: 1x Nvidia A100 | ||
Architecture: | ||
- ResNet | ||
- FPNC | ||
Paper: | ||
URL: https://arxiv.org/abs/2202.10304 | ||
Title: 'Real-Time Scene Text Detection with Differentiable Binarization and Adaptive Scale Fusion' | ||
README: configs/textdet/dbnetpp/README.md | ||
|
||
Models: | ||
- Name: dbnetpp_r50dcnv2_fpnc_1200e_icdar2015.py | ||
In Collection: DBNetpp | ||
Config: configs/textdet/dbnetpp/dbnetpp_r50dcnv2_fpnc_1200e_icdar2015.py | ||
Metadata: | ||
Training Data: ICDAR2015 | ||
Results: | ||
- Task: Text Detection | ||
Dataset: ICDAR2015 | ||
Metrics: | ||
hmean-iou: 0.860 | ||
Weights: https://download.openmmlab.com/mmocr/textdet/dbnet/dbnetpp_r50dcnv2_fpnc_1200e_icdar2015-20220502-d7a76fff.pth |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.