Re: detectron2 not predicting results

[deeptigoyal]

Hi,

I am trying to predict titles, tables and text in an input image.

I used the command :

python demo/demo.py --config-file configs/DLA_mask_rcnn_R_101_FPN_3x.yaml --input "1.png" --output "./testt.png" --confidence-threshold 0.5 --opts MODEL.WEIGHTS "detectron2://COCO-InstanceSegmentation/mask_rcnn_R_101_FPN_3x/138205316/model_final_a3ec72.pkl" MODEL.DEVICE cpu

The command runs with the following logs and returns true at the end.

Config './configs/DLA_mask_rcnn_R_101_FPN_3x.yaml' has no VERSION. Assuming it to be compatible with latest v2.
'roi_heads.box_predictor.cls_score.weight' has shape (81, 1024) in the checkpoint but (6, 1024) in the model! Skipped.
'roi_heads.box_predictor.cls_score.bias' has shape (81,) in the checkpoint but (6,) in the model! Skipped.
'roi_heads.box_predictor.bbox_pred.weight' has shape (320, 1024) in the checkpoint but (20, 1024) in the model! Skipped.
'roi_heads.box_predictor.bbox_pred.bias' has shape (320,) in the checkpoint but (20,) in the model! Skipped.
'roi_heads.mask_head.predictor.weight' has shape (80, 256, 1, 1) in the checkpoint but (5, 256, 1, 1) in the model! Skipped.
'roi_heads.mask_head.predictor.bias' has shape (80,) in the checkpoint but (5,) in the model! Skipped.

However, the output file remains unchanged with no bounding boxes or labels getting predicted. Can anyone guide why it's happening so.

Kindly help.

Thanks!

[hpanwar08]

You are not using correct model weights. Download the weights from the link in the README file.

[deeptigoyal]

Thanks for your quick response. Will look at it again and revert

[deeptigoyal]

Not yet able to resolve ;( .

I downloaded model and save it in a trainedmodels folder and run comand:

python demo/demo.py --config-file configs/DLA_mask_rcnn_R_101_FPN_3x.yaml --input "1.png" --output "./testt.png" --confidence-threshold 0.5 --opts MODEL.WEIGHTS "./trainedmodels/R-101.pkl" MODEL.DEVICE cpu

Got log as:
04/23 08:33:47 d2.checkpoint.c2_model_loading]: Some model parameters are not in the checkpoint:
backbone.fpn_lateral2.{bias, weight}
backbone.fpn_lateral3.{bias, weight}
backbone.fpn_lateral4.{bias, weight}
backbone.fpn_lateral5.{bias, weight}
backbone.fpn_output2.{bias, weight}
backbone.fpn_output3.{bias, weight}
backbone.fpn_output4.{bias, weight}
backbone.fpn_output5.{bias, weight}
proposal_generator.anchor_generator.cell_anchors.{0, 1, 2, 3, 4}
proposal_generator.rpn_head.anchor_deltas.{bias, weight}
proposal_generator.rpn_head.conv.{bias, weight}
proposal_generator.rpn_head.objectness_logits.{bias, weight}
roi_heads.box_head.fc1.{bias, weight}
roi_heads.box_head.fc2.{bias, weight}
roi_heads.box_predictor.bbox_pred.{bias, weight}
roi_heads.box_predictor.cls_score.{bias, weight}
roi_heads.mask_head.deconv.{bias, weight}
roi_heads.mask_head.mask_fcn1.{bias, weight}
roi_heads.mask_head.mask_fcn2.{bias, weight}
roi_heads.mask_head.mask_fcn3.{bias, weight}
roi_heads.mask_head.mask_fcn4.{bias, weight}
roi_heads.mask_head.predictor.{bias, weight}
[04/23 08:33:47 d2.checkpoint.c2_model_loading]: The checkpoint contains parameters not used by the model:
fc1000_b
fc1000_w
0%| | 0/1 [00:00<?, ?it/s]/pytorch/torch/csrc/utils/python_arg_parser.cpp:756: UserWarning: This overload of nonzero is deprecated:
nonzero(Tensor input, *, Tensor out)
Consider using one of the following signatures instead:
nonzero(Tensor input, *, bool as_tuple)
[04/23 08:33:49 detectron2]: 1.png: detected 0 instances in 1.72s

Please help......

[hpanwar08]

Try downloading "model_final_trimmed.pth" from this link https://www.dropbox.com/sh/wgt9skz67usliei/AAD9n6qbsyMz1Y3CwpZpHXCpa?dl=0

[hpanwar08]

All the models are in dropbox as of now. You could ask your network admin to download this file for you.

[pollyMath]

hi, there, first of all, what u have done is really cool and thanks for sharing! :)
second, I am quite a newbie to all of this, so I was wondering why if I run the inference with the model weights given above but with different config
I get visual boxes for config DLA_mask_rcnn_R_101_FPN_3x.yaml
but not for DLA_mask_rcnn_X_101_32x8d_FPN_3x.yaml which I saw as given as an example in another comment?
can you explain what is the difference? I am using a jpg from PubLayNet test data as input

[nisha23june]

Hi, i am also new to all this. I too ran the command:
python demo/demo.py --config-file configs/DLA_mask_rcnn_R_101_FPN_3x.yaml --input "nis.png" --output "./out.png" --confidence-threshold 0.5 --opts MODEL.WEIGHTS "./tm/R-101.pkl" MODEL.DEVICE cpu

But got error log. As i cannot access dropbox can you please help me from this link which model i should download for document analysis.
https://github.com/hpanwar08/detectron2/blob/master/MODEL_ZOO.md

Your help will be appreciated. Thanks. Kindly help

[hpanwar08]

hi, there, first of all, what u have done is really cool and thanks for sharing! :)
second, I am quite a newbie to all of this, so I was wondering why if I run the inference with the model weights given above but with different config
I get visual boxes for config DLA_mask_rcnn_R_101_FPN_3x.yaml
but not for DLA_mask_rcnn_X_101_32x8d_FPN_3x.yaml which I saw as given as an example in another comment?
can you explain what is the difference? I am using a jpg from PubLayNet test data as input

Thanks,
I am not sure if I understand your question correctly. The models that are provided by facebook (original detectron2) are trained on real images (coco dataset). These models will not give you the correct instances in text image. The config files in original detectron2 contains parameters and training config for these models.

Whereas this repo contains custom trained models that are specifically used for document layout analysis. Also config files DLA_* should be used only with these models and not with original detectron2 models.

There a 3 custom trained models as of now. Mask RCNN Resnet50, Resnet101 and Resnext101 and each of these have a associated config file.

[hpanwar08]

Hi, i am also new to all this. I too ran the command:
python demo/demo.py --config-file configs/DLA_mask_rcnn_R_101_FPN_3x.yaml --input "nis.png" --output "./out.png" --confidence-threshold 0.5 --opts MODEL.WEIGHTS "./tm/R-101.pkl" MODEL.DEVICE cpu

But got error log. As i cannot access dropbox can you please help me from this link which model i should download for document analysis.
https://github.com/hpanwar08/detectron2/blob/master/MODEL_ZOO.md

Your help will be appreciated. Thanks. Kindly help

You have to download the weights from the first table in https://github.com/hpanwar08/detectron2/blob/master/README.md, but these models are in dropbox. I don't have any other cloud storage. If you have any, I could copy one of the model there.

[deeptigoyal]

thanks @hpanwar08 ...it's working :) .

Thanks a lot!

[nisha23june]

Thanks @hpanwar08. I am able to download the model.

[xchhmanong]

Thanks @hpanwar08. I am able to download the model.

can you share the model to baidu netpan. because i can't download it from the dropbox.