Trying to replicate an experiment

[geobao]

Hi there,

I train by leaving the defaults. Specifically, I used:

./tools/dist_train.sh configs/fcosrbox/fcosr_rx50_32x4d_fpn_3x_dota10_single.py 0,1,2,3 --seed 129 --deterministic

Then I measure mAP by executing:

python tools/test.py configs/fcosrbox/fcosr_rx50_32x4d_fpn_3x_dota10_single.py work_dirs/DOTA10/FCOSR-M/FCOSR_rx50_32x4d_fpn_3x_dota10_single/latest.pth --eval mAP

My result is:

npos num: 0
{'iou_50': {'mAp': 0.0, 'detail': {'plane': 0.0, 'baseball-diamond': 0.0, 'bridge': 0.0, 'ground-track-field': 0.0, 'small-vehicle': 0.0, 'large-vehicle': 0.0, 'ship': 0.0, 'tennis-court': 0.0, 'basketball-court': 0.0, 'storage-tank': 0.0, 'soccer-ball-field': 0.0, 'roundabout': 0.0, 'harbor': 0.0, 'swimming-pool': 0.0, 'helicopter': 0.0}}}

What am I missing? I have tried with 3 seeds: 98 , 8 and 129. With the same result

[geobao]

After further investigation, here is what I have found:

• The default config uses "trainval" for training as well as for validation. Then it uses the original DOTA "testing" for testing.
• In my case, I used the original dota "train" partition for training with the intention to use the validation partition for testing. However, I wrongly cropped the validation partition by calling the function "DOTA2COCO_ONLYIMG" within "DOTA_devkit/prepare_dota.py"
• After fixing this mistake, I get meaningful mAP values.

Following the default configuration, the testing points to a json in coco format that has no annotations (because the testing partition in dota does not provide annotations). How is then the performance of the model checked? Is it checked against the DOTA server? I don't see any code or reference about that. Any tips are welcome
This is the testing partition used in default config

[geobao]

After further investigation I trained with trainval and the testing results ready for pushing to dota testing server are empty for some categories. Why is that? Is that expected?
I am referring to categories bridge and basketball-court. you can see the 0Bytes for these files:

Did you encounter this issue?

[geobao]

Using the full trainval partition for training and testing on the test partition will result in successful (complete) out files. Thanks!