[Fix] add ignore keys if classes differ - KIE training

[felixdittrich92]

This PR:

• make it possible to use --pretrained if you use a pretrained det model and train on a KIE dataset
• does nothing change for "normal" det training and inference
• same way we have handled it for recognition on differing vocabs

Any feedback is welcome 🤗

[codecov]

Codecov Report

Merging #1271 (d2ef624) into main (f113530) will increase coverage by 0.00%.
Report is 1 commits behind head on main.
The diff coverage is 100.00%.

@@           Coverage Diff           @@
##             main    #1271   +/-   ##
=======================================
  Coverage   95.75%   95.75%           
=======================================
  Files         154      154           
  Lines        6901     6903    +2     
=======================================
+ Hits         6608     6610    +2     
  Misses        293      293           

Flag	Coverage Δ
unittests	95.75% <100.00%> (+<0.01%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files Changed	Coverage Δ
...s/detection/differentiable_binarization/pytorch.py	97.79% <100.00%> (+0.01%)	⬆️
doctr/models/detection/linknet/pytorch.py	98.19% <100.00%> (+0.01%)	⬆️

... and 1 file with indirect coverage changes

[odulcy-mindee]

Can you elaborate ?

[felixT2K]

Yes, just like with the recognition models, if we want to fine-tune a pre-trained model on another vocab we have to reset (resp. resize embeddings) the classifier head as well as the embeddings, otherwise it leads to a shape mismatch.
We have the same thing now when training on a KIE data set, since the number of classes varies, before that in normal detection training it is always one class (text) so it didn't matter.

[felixT2K]

This will reset the Conv"head" which makes it also possible to use '--pretrained' for KIE detection training without an impact on the normal detection training and inference otherwise '--pretrained' raises a shape mismatch while init the model with the pretrained checkpoint

[felixT2K]

as example for crnn:

if pretrained:
        # The number of classes is not the same as the number of classes in the pretrained model =>
        # remove the last layer weights
        _ignore_keys = ignore_keys if _cfg["vocab"] != default_cfgs[arch]["vocab"] else None
        load_pretrained_params(model, _cfg["url"], ignore_keys=_ignore_keys)
        
 ignore_keys=["linear.weight", "linear.bias"],

[odulcy-mindee]

Super clear, thanks