Table is not getting identified completely after training.

[Nikhilsonawane07]

I am trying to fine tune table structure recognition model on 10000 financial tables. I am resuming my training from TATR-v1.1-All weights. I have done annotations for all six classes : table, row, column, column headers, span cell, row headers. I have also performed postprocessing on annotated bboxes to align them properly with table. Using padding of 30 for training table images. When I am running inference pipeline with newly trained weights, tables are not fully getting detected. It is detecting table from line below Table column headers as shown in fig. Rows, columns and other things are getting identified correctly. What could be the issue? How can i improve my model?
@bsmock Can you please help

[srivatsan-sridhar99]

Hey @Nikhilsonawane07 ,
I am also trying to fine tune the TSR model (microsoft/table-structure-recognition-v1.1-all) using a proprietary annotated dataset of financial tables with just table row and table column as the classes. The results are extremely bad.

I used the fine tuning notebook mentioned in https://github.com/NielsRogge/Transformers-Tutorials/tree/master/DETR. How exactly did you fine tune the model, using the script in this repo or the ones provided by hugging face?

Also, does the retrained model 'microsoft/table-structure-recognition-v1.1-all' also require table images and bounding boxes to have some sort of padding for fine tuning? is this just applicable for the entire table boundary or even for the cell boundaries? my cell boundaries are tightly cropped around the text.