The evaluation of referring expression

[Jiang15]

Hi,

I followed the instruction of evaluation for referring expression on COCO dataset train2014. But when I passed the args for test "!python run_with_submitit.py --dataset_config configs/refcoco.json --batch_size 4 --resume https://zenodo.org/record/4721981/files/refcoco_resnet101_checkpoint.pth --ngpus 1 --nodes 1 --ema --test --test_type testA", I didn't get any result of precision or recall, only:
"Start training
Training time 0:00:00
submitit INFO (2021-07-17 11:30:17,492) - Job completed successfully"

I also downloaded the coco dataset of val2014 and test2014 but I am not sure if I need to use that because it gave me error when I pass these dataset.

Thanks a lot in advance!

Best,

[alcinos]

Hello! Thanks for your interest in MDETR. Could you please add the --eval flag and retry? Le sam. 17 juil. 2021 à 07:48, Wei Jiang ***@***.***> a écrit :

…

Hi, I followed the instruction of evaluation for referring expression on COCO dataset train2014. But when I passed the args for test "!python run_with_submitit.py --dataset_config configs/refcoco.json --batch_size 4 --resume https://zenodo.org/record/4721981/files/refcoco_resnet101_checkpoint.pth --ngpus 1 --nodes 1 --ema --test --test_type testA", I didn't get any result of precision or recall, only: "Start training Training time 0:00:00 submitit INFO (2021-07-17 11:30:17,492) - Job completed successfully" I also downloaded the coco dataset of val2014 and test2014 but I am not sure if I need to use that because it gave me error when I pass these dataset. Thanks a lot in advance! Best, — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#18>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABYYC4GMCY434ZUC25A6TNTTYFU2FANCNFSM5AQ74VXQ> .

[Jiang15]

Hello! Thanks for your interest in MDETR. Could you please add the --eval flag and retry? Le sam. 17 juil. 2021 à 07:48, Wei Jiang @.***> a écrit :
…
Hi, I followed the instruction of evaluation for referring expression on COCO dataset train2014. But when I passed the args for test "!python run_with_submitit.py --dataset_config configs/refcoco.json --batch_size 4 --resume https://zenodo.org/record/4721981/files/refcoco_resnet101_checkpoint.pth --ngpus 1 --nodes 1 --ema --test --test_type testA", I didn't get any result of precision or recall, only: "Start training Training time 0:00:00 submitit INFO (2021-07-17 11:30:17,492) - Job completed successfully" I also downloaded the coco dataset of val2014 and test2014 but I am not sure if I need to use that because it gave me error when I pass these dataset. Thanks a lot in advance! Best, — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#18>, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABYYC4GMCY434ZUC25A6TNTTYFU2FANCNFSM5AQ74VXQ .

Thanks a lot! Problem solved.

[Jiang15]

Hello! Thanks for your interest in MDETR. Could you please add the --eval flag and retry? Le sam. 17 juil. 2021 à 07:48, Wei Jiang @.***> a écrit :
…
Hi, I followed the instruction of evaluation for referring expression on COCO dataset train2014. But when I passed the args for test "!python run_with_submitit.py --dataset_config configs/refcoco.json --batch_size 4 --resume https://zenodo.org/record/4721981/files/refcoco_resnet101_checkpoint.pth --ngpus 1 --nodes 1 --ema --test --test_type testA", I didn't get any result of precision or recall, only: "Start training Training time 0:00:00 submitit INFO (2021-07-17 11:30:17,492) - Job completed successfully" I also downloaded the coco dataset of val2014 and test2014 but I am not sure if I need to use that because it gave me error when I pass these dataset. Thanks a lot in advance! Best, — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#18>, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABYYC4GMCY434ZUC25A6TNTTYFU2FANCNFSM5AQ74VXQ .

May I ask if you split the train2014 training dataset into train, val and test? Because I saw only the train2014 is downloaded but not val2014 and test2014 for evaluation.

[ashkamath]

Hi! The Refexp datasets have train val and test all taken from the train split of coco 2014 (we did not design these splits). But we did have to be careful while constructing our pre training dataset to avoid this overlap. So you only need train 2014 for what you’re trying to run right now!

On Sun, Jul 18, 2021 at 6:30 AM Wei Jiang ***@***.***> wrote: Hello! Thanks for your interest in MDETR. Could you please add the --eval flag and retry? Le sam. 17 juil. 2021 à 07:48, Wei Jiang *@*.***> a écrit : … <#m_-3136810570433280254_> Hi, I followed the instruction of evaluation for referring expression on COCO dataset train2014. But when I passed the args for test "!python run_with_submitit.py --dataset_config configs/refcoco.json --batch_size 4 --resume https://zenodo.org/record/4721981/files/refcoco_resnet101_checkpoint.pth --ngpus 1 --nodes 1 --ema --test --test_type testA", I didn't get any result of precision or recall, only: "Start training Training time 0:00:00 submitit INFO (2021-07-17 11:30:17,492) - Job completed successfully" I also downloaded the coco dataset of val2014 and test2014 but I am not sure if I need to use that because it gave me error when I pass these dataset. Thanks a lot in advance! Best, — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub < #18 <#18>>, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABYYC4GMCY434ZUC25A6TNTTYFU2FANCNFSM5AQ74VXQ . May I ask if you split the train2014 training dataset into train, val and test? Because I saw only the train2014 is downloaded but not val2014 and test2014 for evaluation. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#18 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ADQZNM56HUDY2WP5H3ENNC3TYKUKVANCNFSM5AQ74VXQ> .

-- Aishwarya Kamath PhD Student New York University

[Jiang15]

train val and test all taken from the train split of coco 2014 (we did not design these splits). But we did have to be careful while constructing our pre training dataset to avoid this overlap.

Great. Thanks for answering!