feat: add MASTER in models.recognition module #300
Conversation
charlesmindee
added
type: enhancement
Codecov Report
@@ Coverage Diff @@
## main #300 +/- ##
==========================================
- Coverage 96.11% 93.98% -2.14%
==========================================
Files 48 50 +2
Lines 2135 2326 +191
==========================================
+ Hits 2052 2186 +134
- Misses 83 140 +57
Flags with carried forward coverage won't be shown. Click here to find out more.
Continue to review full report at Codecov.
|
| name='transform' | ||
| ) | ||
|
|
||
| @tf.function |
There was a problem hiding this comment.
curious: what does the decorator add?
There was a problem hiding this comment.
It disables eager mode to improve efficiency. However I don't think if it is better in our case to use it.
doctr/models/recognition/master.py
Outdated
| final_logits = tf.zeros(shape=(B, max_len - 1, self.vocab_size + 1), dtype=tf.float32) # don't fgt EOS | ||
| # max_len = len + 2 | ||
| for i in range(self.max_length - 1): | ||
| tf.autograph.experimental.set_loop_options( |
There was a problem hiding this comment.
I'm not familiar with set_loop_options, what does it change?
There was a problem hiding this comment.
Without enforcing the shape with this function (as they did in the official implementation) I had an issue before but it seems to pass now, so let's remove it
|
I added the model in the landing page and the readme, the entry in the models page of the doc will appears in the next PR when model loss & wrapping function will be available. |
This PR implements the MASTER model, following the official TF implementation.
This model (based on a transformer decoder, and a resnet/Global Context block encoder) achieved very impressive performances compared to SAR and others, at ICDAR 2021.
The decoder used is exactly the one presented in "Attention is all you need", therefore it is implemented in a recognition.transformer.py. The MASTER is implemented in recognition.master.py.
Any feedback is welcome!
Closes #215