[Model] Add MASTER #807

JiaquanYe · 2022-03-01T16:52:45Z

This PR is for adding text recognition algorithm 'MASTER' code in mmocr repository.

Motivation

Develop text recognition algorithm 'MASTER' with mmocr framework.

Modification

Implement MASTER components in mmocr framework
Implement MASTER train config in mmocr

Checklist

Before PR:

I have run pre-commit and get some Failed，it is necessary to fix this?
yapf.....................................................................Failed

hook id: yapf
files were modified by this hook

docformatter.............................................................Failed

hook id: docformatter
files were modified by this hook

And I do unit test, and get some error info, which is unrelated with MASTER code.
====================================================== short test summary info ======================================================
FAILED tests/test_apis/test_single_gpu_test.py::test_single_gpu_test_recog[../configs/textrecog/sar/sar_r31_parallel_decoder_academic .py]
FAILED tests/test_apis/test_single_gpu_test.py::test_single_gpu_test_recog[../configs/textrecog/crnn/crnn_academic_dataset.py] - R...
FAILED tests/test_apis/test_single_gpu_test.py::test_single_gpu_test_recog[../configs/textrecog/seg/seg_r31_1by16_fpnocr_academic.py]
FAILED tests/test_apis/test_single_gpu_test.py::test_single_gpu_test_det[../configs/textdet/psenet/psenet_r50_fpnf_600e_icdar2017.py]
FAILED tests/test_apis/test_single_gpu_test.py::test_single_gpu_test_kie[../configs/kie/sdmgr/sdmgr_unet16_60e_wildreceipt.py] - R...
FAILED tests/test_apis/test_single_gpu_test.py::test_single_gpu_test_kie_novisual[../configs/kie/sdmgr/sdmgr_novisual_60e_wildreceipt .py]

codecov · 2022-03-02T01:40:27Z

Codecov Report

Merging #807 (e0bfdf2) into main (fbc138d) will decrease coverage by 0.02%.
The diff coverage is 82.16%.

❗ Current head e0bfdf2 differs from pull request most recent head 8f19d14. Consider uploading reports for the commit 8f19d14 to get more accurate results

@@            Coverage Diff             @@
##             main     #807      +/-   ##
==========================================
- Coverage   83.77%   83.74%   -0.03%     
==========================================
  Files         168      170       +2     
  Lines       11202    11359     +157     
  Branches     1724     1743      +19     
==========================================
+ Hits         9384     9513     +129     
- Misses       1449     1468      +19     
- Partials      369      378       +9

Flag	Coverage Δ
unittests	`83.74% <82.16%> (-0.03%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
mmocr/utils/ocr.py	`77.36% <ø> (ø)`
mmocr/models/textrecog/plugins/common.py	`67.14% <62.29%> (-32.86%)`	⬇️
mmocr/models/textrecog/decoders/master_decoder.py	`94.56% <94.56%> (ø)`
mmocr/models/textrecog/recognizer/master.py	`100.00% <100.00%> (ø)`

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 137713f...8f19d14. Read the comment docs.

Mountchicken · 2022-03-10T03:31:31Z

Hi JiaquanYe!
Thank you for your great contribution👍. Sorry for the late reply, we will review your PR in the following days.

mmocr/models/textrecog/backbones/resnet_master.py

mmocr/models/textrecog/convertors/master.py

mmocr/models/textrecog/decoders/master_decoder.py

mmocr/models/textrecog/encoders/positional_encoder.py

mmocr/models/textrecog/layers/context_block.py

configs/_base_/recog_models/master.py

mmocr/models/textrecog/encoders/positional_encoder.py

mmocr/models/textrecog/decoders/master_decoder.py

mmocr/models/textrecog/recognizer/master.py

gaotongxiao · 2022-04-02T03:44:15Z

Hi @JiaquanYe , do you still have time to refactor the code? If not, it's totally fine and we can help you with the rest of the process.

JiaquanYe · 2022-04-02T04:03:18Z

I have refactored some code. And I will finish it in this vocation. Sorry about refactoring late. 发自我的iPhone

…

------------------ Original ------------------ From: Tong Gao ***@***.***> Date: Sat,Apr 2,2022 11:44 AM To: open-mmlab/mmocr ***@***.***> Cc: Jianquan Ye ***@***.***>, Mention ***@***.***> Subject: Re: [open-mmlab/mmocr] [Model] Add MASTER (PR #807) Hi @JiaquanYe , do you still have time to refactor the code? If not, it's totally fine and we can help you with the rest of the process. — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.Message ID: ***@***.***>

gaotongxiao · 2022-04-02T06:02:42Z

@JiaquanYe Awesome!

gaotongxiao · 2022-04-06T03:01:07Z

BTW, please install pre-commit hooks and follow the guide to pass the lint test. They help you format your code :) Remember if you have run into an error saying 'gem' is not found, just install Ruby 2.7 as said in the guide.

JiaquanYe · 2022-04-22T01:19:48Z

@JiaquanYe The training process is still continuing and so far it looks like it is possible to reproduce the accuracy of the original paper, so I'll help you do the rest of the work. Thank you for your contribution！❤.

OK. Thanks for your help during this time! @Mountchicken @gaotongxiao

JiaquanYe · 2022-04-22T01:22:29Z

I accidentally click the close PR button, now I reopen now...

gaotongxiao

Things remain to be done except for the comments:

@Mountchicken @xinke-wang We can wrap up this PR together in the following days.

configs/_base_/schedules/schedule_adadelta_5e.py

configs/_base_/schedules/schedule_adam_step_5e.py

configs/_base_/schedules/schedule_adam_step_6e.py

configs/textrecog/master/master_academic.py

mmocr/models/common/modules/transformer_module.py

mmocr/models/textrecog/plugins/common.py

tests/data/ocr_toy_dataset/debug_label.txt

mmocr/models/textrecog/layers/context_block.py

mmocr/models/textrecog/losses/ce_loss.py

gaotongxiao · 2022-04-22T07:27:25Z

Hi @JiaquanYe , one more question: Does this implementation uses any code from any other open-source project? If so, please let us know so that we can cite them as well :)

JiaquanYe · 2022-04-22T07:52:17Z

Hi @JiaquanYe , one more question: Does this implementation uses any code from any other open-source project? If so, please let us know so that we can cite them as well :)

I have refered to this two repositories MASTER-pytorch , MASTER-TF when I develop MASTER-mmocr. I think you can cite these two repositories.

gaotongxiao · 2022-04-22T14:41:08Z

@JiaquanYe OKay!

…test for it, remove MasterLoss

mmocr/utils/ocr.py

configs/_base_/schedules/schedule_adam_step_12e.py

configs/textrecog/master/README.md

xinke-wang · 2022-05-05T07:39:57Z

Need to add MASTER to the docs/en/model_summary.md

…into test

Mountchicken · 2022-05-05T07:57:21Z

Need to add MASTER to the docs/en/model_summary.md

Done.

* fix open-mmlab#794: add MASTER * fix conflict add MASTER * fix conflict add MASTER * fix conflict add MASTER * fix conflict add MASTER * fix conflict add MASTER * fix conflict add MASTER * fix conflict add MASTER * Fix linting * after git rebase main * after git rebase main * fix conflict add MASTER * fix conflict add MASTER * after git rebase main * fix conflict add MASTER * fix conflict add MASTER * fix conflict add MASTER * after git rebase main * add GCAModule to plugins * coexist master and master_old * fix merge mmocr 0.5.0 conflict * fix lint error * update * [fix] remove remains in __init__ * [update] update code in review * update readme for master * Add docstr to MasterDecoder, refined MasterDecoder, remove MASTERLoss * Unify the output length of MasterDecoder in train and test mode; add test for it, remove MasterLoss * update readme * update * update metafile,README,demo/README,config,ocr.py * Update mmocr/utils/ocr.py * update Co-authored-by: gaotongxiao <gaotongxiao@gmail.com> Co-authored-by: Mountchicken <mountchicken@outlook.com>

JiaquanYe added 8 commits February 27, 2022 13:41

fix open-mmlab#794: add MASTER

ccf1b6f

fix conflict add MASTER

9f47122

fix conflict add MASTER

d63c4e4

fix conflict add MASTER

9c01466

fix conflict add MASTER

263899d

fix conflict add MASTER

570905c

fix conflict add MASTER

d028ec8

fix conflict add MASTER

0b15c26

gaotongxiao changed the title ~~Feature/iss 794~~ [Model] Add MASTER Mar 2, 2022

gaotongxiao requested review from gaotongxiao, Mountchicken and xinke-wang March 8, 2022 12:08

Mountchicken reviewed Mar 16, 2022

View reviewed changes

gaotongxiao reviewed Mar 16, 2022

View reviewed changes

Fix linting

248decd

JiaquanYe force-pushed the feature/iss_794 branch from 6bd640e to 248decd Compare March 27, 2022 16:46

Merge branch 'main' into feature/iss_794

66fa5f8

JiaquanYe added 7 commits April 10, 2022 10:23

after git rebase main

0461300

after git rebase main

c58fd7c

fix conflict add MASTER

7161d3a

fix conflict add MASTER

651b6bd

after git rebase main

8f42bac

fix conflict add MASTER

a3bec83

fix conflict add MASTER

88d219b

JiaquanYe reopened this Apr 22, 2022

[fix] remove remains in __init__

fc6e824

gaotongxiao reviewed Apr 22, 2022

View reviewed changes

[update] update code in review

12c7259

update readme for master

447131a

gaotongxiao and others added 6 commits April 23, 2022 23:40

Add docstr to MasterDecoder, refined MasterDecoder, remove MASTERLoss

f9d1ae1

Unify the output length of MasterDecoder in train and test mode; add …

0c6f651

…test for it, remove MasterLoss

update readme

6111201

Merge remote-tracking branch 'origin/main' into test

c60f3d2

update

d18ecda

update metafile,README,demo/README,config,ocr.py

eb9a1c1

gaotongxiao reviewed May 5, 2022

View reviewed changes

mmocr/utils/ocr.py Outdated Show resolved Hide resolved

Update mmocr/utils/ocr.py

0b4aa98

gaotongxiao approved these changes May 5, 2022

View reviewed changes

xinke-wang reviewed May 5, 2022

View reviewed changes

configs/_base_/schedules/schedule_adam_step_12e.py Show resolved Hide resolved

xinke-wang reviewed May 5, 2022

View reviewed changes

configs/textrecog/master/README.md Outdated Show resolved Hide resolved

Mountchicken added 3 commits May 5, 2022 15:44

Merge remote-tracking branch 'origin/main' into test

001ac48

update

3dca667

Merge branch 'feature/iss_794' of https://github.com/JiaquanYe/mmocr …

8f19d14

…into test

xinke-wang approved these changes May 5, 2022

View reviewed changes

gaotongxiao merged commit 061d353 into open-mmlab:main May 5, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Model] Add MASTER #807

[Model] Add MASTER #807

JiaquanYe commented Mar 1, 2022

codecov bot commented Mar 2, 2022 •

edited

Loading

Mountchicken commented Mar 10, 2022

gaotongxiao commented Apr 2, 2022

JiaquanYe commented Apr 2, 2022 via email

gaotongxiao commented Apr 2, 2022

gaotongxiao commented Apr 6, 2022

JiaquanYe commented Apr 22, 2022

JiaquanYe commented Apr 22, 2022

gaotongxiao left a comment •

edited by Mountchicken

Loading

gaotongxiao commented Apr 22, 2022

JiaquanYe commented Apr 22, 2022

gaotongxiao commented Apr 22, 2022

xinke-wang commented May 5, 2022

Mountchicken commented May 5, 2022

[Model] Add MASTER #807

[Model] Add MASTER #807

Conversation

JiaquanYe commented Mar 1, 2022

Motivation

Modification

Checklist

codecov bot commented Mar 2, 2022 • edited Loading

Codecov Report

Mountchicken commented Mar 10, 2022

gaotongxiao commented Apr 2, 2022

JiaquanYe commented Apr 2, 2022 via email

gaotongxiao commented Apr 2, 2022

gaotongxiao commented Apr 6, 2022

JiaquanYe commented Apr 22, 2022

JiaquanYe commented Apr 22, 2022

gaotongxiao left a comment • edited by Mountchicken Loading

Choose a reason for hiding this comment

gaotongxiao commented Apr 22, 2022

JiaquanYe commented Apr 22, 2022

gaotongxiao commented Apr 22, 2022

xinke-wang commented May 5, 2022

Mountchicken commented May 5, 2022

codecov bot commented Mar 2, 2022 •

edited

Loading

gaotongxiao left a comment •

edited by Mountchicken

Loading