Is it possible to use the Detectron2 API to load a pretrained model?

[ok-ad]

I want to run inference on an image using a pretrained model given in this project and I want to use the detectron2 API to load the model. Is it something possible?

Here's what I tried:

1. I tried to run the Run a pre-trained detectron2 model section in the Getting Started notebook of detectron2.
2. When obtaining the config, I've changed the code to below:

URL = "ade20k-150/maskformer_R50_bs16_160k.yaml"
cfg = get_cfg()
# add project-specific config (e.g., TensorMask) here if you're not running a model in detectron2's core library
cfg.merge_from_file(model_zoo.get_config_file(URL))
cfg.MODEL.ROI_HEADS.SCORE_THRESH_TEST = 0.5  # set threshold for this model
# Find a model from detectron2's model zoo. You can use the https://dl.fbaipublicfiles... url as well
cfg.MODEL.WEIGHTS = model_zoo.get_checkpoint_url(URL)
        

It seems like the models aren't available in the model zoo:

  ---------------------------------------------------------------------------
  RuntimeError                              Traceback (most recent call last)
  <ipython-input-9-f6671c809590> in <module>()
        1 cfg = get_cfg()
        2 # add project-specific config (e.g., TensorMask) here if you're not running a model in detectron2's core library
  ----> 3 cfg.merge_from_file(model_zoo.get_config_file(URL))
        4 cfg.MODEL.ROI_HEADS.SCORE_THRESH_TEST = 0.5  # set threshold for this model
        5 # Find a model from detectron2's model zoo. You can use the https://dl.fbaipublicfiles... url as well
  
  /usr/local/lib/python3.7/dist-packages/detectron2/model_zoo/model_zoo.py in get_config_file(config_path)
      117     )
      118     if not os.path.exists(cfg_file):
  --> 119         raise RuntimeError("{} not available in Model Zoo!".format(config_path))
      120     return cfg_file
      121 
  
  RuntimeError: ade20k-150/maskformer_R50_bs16_160k.yaml not available in Model Zoo!

If this isn't possible, what is the way we can load a pretrained model in the user code?

[bowenc0221]

You can simply replace it with

cfg = get_cfg()
# add project-specific config (e.g., TensorMask) here if you're not running a model in detectron2's core library
cfg.merge_from_file("configs/ade20k-150/maskformer_R50_bs16_160k.yaml")
cfg.MODEL.ROI_HEADS.SCORE_THRESH_TEST = 0.5  # set threshold for this model
# Find a model from detectron2's model zoo. You can use the https://dl.fbaipublicfiles... url as well
cfg.MODEL.WEIGHTS = "https://dl.fbaipublicfiles.com/maskformer/semantic-ade20k/maskformer_R50_bs16_160k/model_final_d8dbeb.pkl"
predictor = DefaultPredictor(cfg)
outputs = predictor(im)

You can find links to all pre-trained weights under https://github.com/facebookresearch/MaskFormer/blob/master/MODEL_ZOO.md

[ok-ad]

Thank you @bowenc0221 ! I have tried your snippet, but it complained about not being able to find the config file. Does this repo need to be downloaded to the local machine and change some PATH variable to get that working?

cfg = get_cfg()
# add project-specific config (e.g., TensorMask) here if you're not running a model in detectron2's core library
cfg.merge_from_file("configs/ade20k-150/maskformer_R50_bs16_160k.yaml")

Returned:

---------------------------------------------------------------------------
AssertionError                            Traceback (most recent call last)
<ipython-input-15-998bf0b622e5> in <module>()
      1 cfg = get_cfg()
      2 # add project-specific config (e.g., TensorMask) here if you're not running a model in detectron2's core library
----> 3 cfg.merge_from_file("configs/ade20k-150/maskformer_R50_bs16_160k.yaml")
      4 cfg.MODEL.ROI_HEADS.SCORE_THRESH_TEST = 0.5  # set threshold for this model
      5 # Find a model from detectron2's model zoo. You can use the https://dl.fbaipublicfiles... url as well

/usr/local/lib/python3.7/dist-packages/detectron2/config/config.py in merge_from_file(self, cfg_filename, allow_unsafe)
     28     # Note that the default value of allow_unsafe is changed to True
     29     def merge_from_file(self, cfg_filename: str, allow_unsafe: bool = True) -> None:
---> 30         assert PathManager.isfile(cfg_filename), f"Config file '{cfg_filename}' does not exist!"
     31         loaded_cfg = self.load_yaml_with_base(cfg_filename, allow_unsafe=allow_unsafe)
     32         loaded_cfg = type(self)(loaded_cfg)

AssertionError: Config file 'configs/ade20k-150/maskformer_R50_bs16_160k.yaml' does not exist!

[bowenc0221]

Yes, you will need to download the config file, which you can find here: https://github.com/facebookresearch/MaskFormer/tree/master/configs

You will also need to import mask_former, otherwise, models are not properly registered.

[ok-ad]

@bowenc0221 I think you've referred to something from the mask_former directory. What should the import look like?

[bowenc0221]

I would suggest you check the demo file: https://github.com/facebookresearch/MaskFormer/blob/master/demo/demo.py