Testing Baselines on personal dataset

[giulia95]

❓ Questions and Help

Hi,
I'm working on the Hateful Memes project; I'm trying to use the proposed pretrained models to make predictions on a personal dataset.
I've been able to reproduce the results using the documentation and the given dataset, but I'd like to make predictions on my own dataset.
How can I adjust mmf_predict command to run on my dataset? I've been using the command with the following structure:

mmf_predict config=<REPLACE_WITH_BASELINE_CONFIG> model=<REPLACE_WITH_MODEL_KEY> dataset=hateful_memes \
run_type=test checkpoint.resume_zoo=<REPLACE_WITH_PRETRAINED_ZOO_KEY> checkpoint.resume_pretrained=False

Do I have to make some preprocessing on my dataset?
At the moment my dataset is composed by a set of images and a jsonl file with the same structure of the HM ones:
{"id":8291,"img":"img\/08291.png","label":1,"text":"white people is this a shooting range"} {"id":46971,"img":"img\/46971.png","label":1,"text":"bravery at its finest"}

[apsdehal]

Hi,
You will have to update dataset_config.hateful_memes.annotations.test[0] to point to your annotation file. If the images have changed and you are using feature based models (such as VisualBERT), you will also have to generate features for those files using feature extractor scripts.

References:

• Config file for Hateful Memes: https://github.com/facebookresearch/mmf/blob/master/mmf/configs/datasets/hateful_memes/defaults.yaml#L28
• Documentation on extracting features: https://mmf.sh/docs/tutorials/image_feature_extraction

[giulia95]

Hi,
Thank you for your help.
I've been trying to execute the following command on Colab to extract features for my images.
python mmf/mmf/tools/scripts/features/extract_features_vmb.py --model_name=X-152 --image_dir=<FOLDER_PATH_TO_DATASET> --output_folder=<OUTPUT_FOLDER>
but I got an error:

Traceback (most recent call last):
  File "mmf/tools/scripts/features/extract_features_vmb.py", line 21, in <module>
    from maskrcnn_benchmark.utils.model_serialization import load_state_dict
  File "/content/vqa-maskrcnn-benchmark/maskrcnn_benchmark/utils/model_serialization.py", line 7, in <module>
    from maskrcnn_benchmark.utils.imports import import_file
  File "/content/vqa-maskrcnn-benchmark/maskrcnn_benchmark/utils/imports.py", line 4, in <module>
    if torch._six.PY3:
AttributeError: module 'torch._six' has no attribute 'PY3'

I think I have installed all the requirements; here's my code:

# maskrcnn_benchmark and coco api dependencies
!pip install ninja yacs cython matplotlib

# install pycocotools
%cd /content
!git clone https://github.com/cocodataset/cocoapi.git
%cd cocoapi/PythonAPI
!python setup.py build_ext install

# Install maskrcnn-benchmark to extract detectron features
%cd /content
!git clone https://gitlab.com/vedanuj/vqa-maskrcnn-benchmark.git
%cd /content/vqa-maskrcnn-benchmark
# Compile custom layers and build mask-rcnn backbone
!python setup.py build develop

%cd /content/
%rm -rf mmf
!git clone https://github.com/facebookresearch/mmf.git mmf
%cd /content/mmf
!sed -i '/torch/d' requirements.txt
!pip install -e .

!python mmf/tools/scripts/features/extract_features_vmb.py --model_name=X-152 --image_dir=/content/drive/MyDrive/Dataset  --output_folder=/content/drive/MyDrive/

[shivgodhia]

@giulia95 just modify the code in /content/vqa-maskrcnn-benchmark/maskrcnn_benchmark/utils/imports.py, change PY3 to PY37

@apsdehal the features using that script don't seem to be exactly the correct ones, they're not the same as what you get when you extract from detection.lmdb