Some doubts regarding executing the scripts

[architlatkar27]

I was following the instructions on this page regarding how to use the builtin mscoco dataset for image captioning. I have doubts on the following points:

1. In which directory should i have my images and annotations?
2. I was trying to run the tools/create_feats.py script for converting the karpathy_train_resnet101_faster_rcnn_genome.tsv.0 into .npz format. however I am running out of space on colab pro. What could be the reason for this and is there any fix available?

[architlatkar27]

Another error that i got while running train_net.py
RuntimeError: NCCL error in: /pytorch/torch/lib/c10d/ProcessGroupNCCL.cpp:911, invalid usage, NCCL version 2.7.8
ncclInvalidUsage: This usually reflects invalid usage of NCCL library (such as too many async ops, too many collectives at once, mixing streams in a group, etc).

[YehLi]

I was following the instructions on this page regarding how to use the builtin mscoco dataset for image captioning. I have doubts on the following points:

1. In which directory should i have my images and annotations?
2. I was trying to run the tools/create_feats.py script for converting the karpathy_train_resnet101_faster_rcnn_genome.tsv.0 into .npz format. however I am running out of space on colab pro. What could be the reason for this and is there any fix available?

1. You can follow the structure:
   open_source_dataset/
   mscoco_dataset/
   annotations files
   features/
   up_down/*.npz
   xmodaler/
   configs/
   xmodaler/
   tools/
   train_net.py
2. Please try the following script:
   python2 tools/create_feats.py --infeats bottom_up_tsv --outfolder ./mscoco/feature/up_down_10_100

[YehLi]

Another error that i got while running train_net.py
RuntimeError: NCCL error in: /pytorch/torch/lib/c10d/ProcessGroupNCCL.cpp:911, invalid usage, NCCL version 2.7.8
ncclInvalidUsage: This usually reflects invalid usage of NCCL library (such as too many async ops, too many collectives at once, mixing streams in a group, etc).

Which pytorch/cuda/driver versions do you use? I test on torch 1.9.0, CUDA 10.2, driver Version: 440.95.01 on P40/V100 and there is no error.

[architlatkar27]

Hey for that, i downgraded pytorch to 1.6.0 and it worked properly

[architlatkar27]

FileNotFoundError: [Errno 2] No such file or directory: '../open_source_dataset/mscoco_dataset/features/mil/resnet101_mil.pkl'

Where can i find this file? and what exactly does it contain?

[YehLi]

The features have been uploaded to the mscoco_dataset folder (https://drive.google.com/drive/folders/16rCskpeG5ci02YiE8uR6P8AimV6WIkcB?usp=sharing).

[architlatkar27]

Thanks a lot

[architlatkar27]

Hi even this file is missing -
../open_source_dataset/mscoco_dataset/features/global_feat/resnet101_pool5.pkl'

[YehLi]

The file has been uploaded.

[architlatkar27]

Hi thanks to the two files you uploaded, i was able to run the train_net for lstm-A.
However, i am a bit confused as the whole script ran within few seconds without leaving any errors. Here is a snapshot of the last part of the output -

So what exactly does this script do?
Also in this snapshot, it says that it failed to import detectron2. Is it a requirement for image captioning?

Thank for helping out

[YehLi]

1. The first output is about loading data for training and evaluation.
2. detectron2 is not required for image captioning and other tasks. You can ignore the message.

[architlatkar27]

Hi, thanks for getting back.
How exactly do we put it to training then? If i run train_net.py it just loads the data and exits.

[YehLi]

merge the issue into #21