[feature request] evaluate mIoU, accu, FID every N iteration

[wasd96040501]

First, thanks for your amazing work!
In your paper, mIoU, accu, FID are evaluated to make quantitative comparison to related work.
And it seems that these evaluation codes are missing from this version of code?
I think it will be better evaluating and log (or print) these value when we training the model.
For those who what to develop a new method based on your codes (like me), the mIoU, acccu, FID curves will intuitively show us whether our work is promising or not.
Thanks!

[taesungp]

Hello,

Unfortunately, we do not plan to include code for evaluation metrics because it was not written by us.

For FID, we used this code repo pretty much out of the box. https://github.com/mseitzer/pytorch-fid

For semantic segmentation score, we specified in paper which semantic segmentation network we used for evaluation.

[dgrid]

@taesungp
Hi, thanks for your great work!
I have a question.

For FID, especially on COCO-Stuff did you split dataset into train and val and evaluate FID with 5,000 images in val, or just use 50,000 images in train?

[ShihuaHuang95]

@dgrid Have you figured out which one is used in the FID calculation? @taesungp Would you like to make this clear? Many thanks!