MaskRCNN training - Issue with pytorch lightning vs pytorch

[purnasai-soulpageit]

Model output is not the same compared to Pytorch and pytorch_lightning

We have used PyTorch detection model maskrcnn_50_fpn model in PyTorch and in PyTorch lightning to perform instance segmentation of Weapon&Knife with Same data, Data loaders, Epcohs and Environment. Framework is the only difference here.

after training for the same number of epochs(40), we can see loss following the same pattern in both PyTorch and lightning, but the prediction of the model differs a lot. Predictions from Both frameworks are below and in notebooks as well.

Pytorch prediction

Pytorch-lighning prediction

To Reproduce

Use Below notebooks to reproduce

1. Pytorch_lightning instance segmentation: here
2. Pytorch instance segmentation: here
3. sample_data: here

Expected behavior

Expected to have Prediction results same in Pytorch and in Pytorch lightning

Environment

• CUDA:
  - GPU:
  - Tesla T4
  - available: True
  - version: 10.2
• Packages:
  - numpy: 1.20.3
  - pyTorch_debug: False
  - pyTorch_version: 1.8.0
  - pytorch-lightning: 1.5.8
  - tqdm: 4.62.3
• System:
  - OS: Linux
  - architecture:
  - 64bit
  -
  - processor: x86_64
  - python: 3.7.10
  - version: added test model to do also #64~18.04.1-Ubuntu SMP Fri Dec 3 17:59:13 UTC 2021

cc @awaelchli @ananthsub @ninginthecloud @rohitgr7

[rohitgr7]

hey @purnasai-soulpageit !

can you try:

def image_prediction(...):
    ...
    model = InstanceSegment.load_from_checkpoint(model_weight, classes=classes)
    ...

instead of:

def image_prediction(...):
    ...
    model=InstanceSegment(classes)
    model.load_from_checkpoint(model_weight)
    ...

[purnasai-soulpageit]

hi @rohitgr7, Thanks for the suggestion. Made the changes. Prediction looks better now. adding prediction below.

So How did just changing these 2 lines made a difference. aren't both ways meant to do the same?.
And what other changes can I make to improve them better?
Thank you

[rohitgr7]

hey @purnasai-soulpageit !!

So How did just changing these 2 lines made a difference. aren't both ways meant to do the same?.

check this PR: #11467

So How did just changing these 2 lines made a difference. aren't both ways meant to do the same?.

that you have to check whether the prediction code is same on both sides or not and whether the finally trained model leads some similar metrics on both sides using some test/validation set.
I'll have a quick look over the weekend too.