Facing issues with MMF config for running the VilBERT pretrained model on Hateful Meme dataset.

[AnjumJ123]

I am trying to reproduce the code for running VilBERT on hateful meme dataset, but the exiting code needs to be modified to point to the new data source for hateful meme challenge data and then be linked in the code.

Facebook released a webpage (https://hatefulmemeschallenge.com/) where the dataset can be downloaded. Would you be able to change the notebook slightly to reproduce the code? I am facing challenges in mmf_convert_hm to get the hateful meme data to be converted into the MMF format and the image, .jsonl files to be moved to the corresponding folders as the repo expects. Facing challenge in terms of ensuring the MMF pre-requisites after downloading the datasets and the changes that need to be made if any in the .yaml and config files to make the code to work.

[AnjumJ123]

Not sure why it is looking for dev.jsonl when that is not one of the .jsonl files in the hateful_memes dataset. Any suggestions to address this error?

'!mmf_run config="projects/visual_bert/configs/hateful_memes/from_coco.yaml"
model=visual_bert
dataset=hateful_memes
run_type=train_val
training.log_interval=200
training.max_updates=22000
training.batch_size=64
training.evaluation_interval=200
training.tensorboard=True
training.checkpoint_interval=200
checkpoint.resume_pretrained=True
checkpoint.resume_zoo=visual_bert.pretrained.coco
dataset_config.hateful_memes.annotations.train[0]="hateful_memes/defaults/annotations/train.jsonl"
dataset_config.hateful_memes.annotations.val[0]="hateful_memes/defaults/annotations/dev_unseen.jsonl"
dataset_config.hateful_memes.annotations.test[0]="hateful_memes/defaults/annotations/test_unseen.jsonl"'

Error:

You can disable this warning by setting the environment variable OC_DISABLE_DOT_ACCESS_WARNING=1
warnings.warn(message=msg, category=UserWarning)
Overriding option config to projects/visual_bert/configs/hateful_memes/from_coco.yaml
Overriding option model to visual_bert
Overriding option datasets to hateful_memes
Overriding option run_type to train_val
Overriding option training.log_interval to 200
Overriding option training.max_updates to 22000
Overriding option training.batch_size to 64
Overriding option training.evaluation_interval to 200
Overriding option training.tensorboard to True
Overriding option training.checkpoint_interval to 200
Overriding option checkpoint.resume_pretrained to True
Overriding option checkpoint.resume_zoo to visual_bert.pretrained.coco
Using seed 30549397
Logging to: ./save/logs/train_2022-04-20T16:33:30.log
Downloading features.tar.gz: 100% 8.44G/8.44G [06:24<00:00, 22.0MB/s]
Traceback (most recent call last):
File "/usr/local/bin/mmf_run", line 8, in
sys.exit(run())
File "/usr/local/lib/python3.7/dist-packages/mmf_cli/run.py", line 111, in run
main(configuration, predict=predict)
File "/usr/local/lib/python3.7/dist-packages/mmf_cli/run.py", line 40, in main
trainer.load()
File "/usr/local/lib/python3.7/dist-packages/mmf/trainers/base_trainer.py", line 59, in load
self.load_datasets()
File "/usr/local/lib/python3.7/dist-packages/mmf/trainers/base_trainer.py", line 83, in load_datasets
self.dataset_loader.load_datasets()
File "/usr/local/lib/python3.7/dist-packages/mmf/common/dataset_loader.py", line 18, in load_datasets
self.val_dataset.load(self.config)
File "/usr/local/lib/python3.7/dist-packages/mmf/datasets/multi_dataset_loader.py", line 114, in load
self.build_datasets(config)
File "/usr/local/lib/python3.7/dist-packages/mmf/datasets/multi_dataset_loader.py", line 131, in build_datasets
dataset_instance = build_dataset(dataset, dataset_config, self.dataset_type)
File "/usr/local/lib/python3.7/dist-packages/mmf/utils/build.py", line 106, in build_dataset
dataset = builder_instance.load_dataset(config, dataset_type)
File "/usr/local/lib/python3.7/dist-packages/mmf/datasets/base_dataset_builder.py", line 96, in load_dataset
dataset = self.load(config, dataset_type, *args, **kwargs)
File "/usr/local/lib/python3.7/dist-packages/mmf/datasets/builders/hateful_memes/builder.py", line 39, in load
self.dataset = super().load(config, dataset_type, *args, **kwargs)
File "/usr/local/lib/python3.7/dist-packages/mmf/datasets/mmf_dataset_builder.py", line 141, in load
dataset = dataset_class(config, dataset_type, imdb_idx)
File "/usr/local/lib/python3.7/dist-packages/mmf/datasets/builders/hateful_memes/dataset.py", line 19, in init
super().init(dataset_name, config, *args, **kwargs)
File "/usr/local/lib/python3.7/dist-packages/mmf/datasets/mmf_dataset.py", line 25, in init
self.annotation_db = self._build_annotation_db()
File "/usr/local/lib/python3.7/dist-packages/mmf/datasets/mmf_dataset.py", line 39, in _build_annotation_db
return AnnotationDatabase(self.config, annotation_path)
File "/usr/local/lib/python3.7/dist-packages/mmf/datasets/databases/annotation_database.py", line 24, in init
self._load_annotation_db(path)
File "/usr/local/lib/python3.7/dist-packages/mmf/datasets/databases/annotation_database.py", line 32, in _load_annotation_db
self._load_jsonl(path)
File "/usr/local/lib/python3.7/dist-packages/mmf/datasets/databases/annotation_database.py", line 39, in _load_jsonl
with PathManager.open(path, "r") as f:
File "/usr/local/lib/python3.7/dist-packages/mmf/utils/file_io.py", line 45, in open
newline=newline,
FileNotFoundError: [Errno 2] No such file or directory: '/root/.cache/torch/mmf/data/datasets/hateful_memes/defaults/annotations/dev.jsonl'