evaluate code

[2448845600]

Could you publish the evaluate code? I think this repo is a good pipeline for few-shot segmentation. Thanks! @HKUSTCV

[Balabala-Hong]

I have some questions below:

1. In your experiments, how many times did you split the trainval classes(760) to train(520) and val(240)?
2. How do you evaluate your model on test classes? For example, is your one shot experiment result for a specific class calculated with each one of ten images as support image ?
   So can you provide your split classes of train classes(520) and val classes(240)? With your classes split result and evaluation methods, a fair comparision can be conducted for people. @HKUSTCV

[ZezhouCheng]

It is also confusing for me how the train set and validation set are used during training and the exact procedure to evaluate the model. From training.py, there is no procedure to conduct model selection or hyperparameter searching based on the validation set. Perhaps it is in the evaluation code which is not released. Could you provide the evaluation methods for a fair comparison?

[SMHendryx]

I've added evaluation code in this PR: #13

[jzsherlock4869]

@SMHendryx Have you evaluate the author published model (in readme.md) https://drive.google.com/open?id=1Vk0Pq8vOZrfrDtCISMcJmAQnt9jkXfPn in one-shot setting? I have tested on my own evalutation script, and also applied your PR #13 script by setting args.k=1, both experiments give the mean IoU of ~ 0.80, which is ~0.07 higher than the paper reported (0.73). Could you please kindly let me know your reproduction of the one-shot setting?

[lilyandluc]

Could you publish the evaluate code? I think this repo is a good pipeline for few-shot segmentation. Thanks! @HKUSTCV

have you got that evaluation code?

[lilyandluc]

@SMHendryx Have you evaluate the author published model (in readme.md) https://drive.google.com/open?id=1Vk0Pq8vOZrfrDtCISMcJmAQnt9jkXfPn in one-shot setting? I have tested on my own evalutation script, and also applied your PR #13 script by setting args.k=1, both experiments give the mean IoU of ~ 0.80, which is ~0.07 higher than the paper reported (0.73). Could you please kindly let me know your reproduction of the one-shot setting?

Would you please share that evaluation code with me ?