evaluation during training

[seacloud-0420]

Does this code support evaluation during training? If I want to evaluate while training, what should I do?

[csuhan]

Yes.

• change the following settings to val set
  

  s2anet/configs/dota/s2anet_r50_fpn_1x_dota.py

  Lines 133 to 135 in d287096

  	evaluation = dict(
  	gt_dir='data/dota/test/labelTxt/', # change it to valset for offline validation
  	imagesetfile='data/dota/test/test.txt')

• add --validate to your training command
  

  s2anet/tools/train.py

  Line 27 in d287096

  '--validate',

[seacloud-0420]

Thank you for your answer, I have changed it as you said, but the training appears: NoteImplementedError:Built-in validation is not implemented yet in not_distributed training. Use distributed training or test.py and *eval.py scripts instead.
Also, I made my dataset according to the format of the DOTA dataset, the object in my dataset is relatively large and it is not suitable for split, I did not split my dataset. I got an error when I tested my own dataset, as shown in the image below. Is this code not suitable for the uncut DOTA data format, if so how do I need to change it?

[csuhan]

Ohh I'm sorry. I checked again, and this codebase does not support eval during training. It just support testing-then-evaluation (in one command) if you setup the evaluation config.

If the images in your dataset is small and do not need to split, you can comment L26-27

s2anet/mmdet/datasets/dota.py

Lines 20 to 34 in d287096

	def evaluate(self, results, work_dir=None, gt_dir=None, imagesetfile=None):
	dst_raw_path = osp.join(work_dir, 'results_before_nms')
	dst_merge_path = osp.join(work_dir, 'results_after_nms')
	mmcv.mkdir_or_exist(dst_raw_path)
	mmcv.mkdir_or_exist(dst_merge_path)
	print('Saving results to {}'.format(dst_raw_path))
	self.result_to_txt(results, dst_raw_path)
	print('Merge results to {}'.format(dst_merge_path))
	mergebypoly(dst_raw_path, dst_merge_path)
	print('Start evaluation')
	detpath = osp.join(dst_merge_path, '{:s}.txt')
	annopath = osp.join(gt_dir, '{:s}.txt')

Besides, you many need to change something in your result files, see this issue