Loss Calculation for Validation

[hilmicanguven]

Can anybody tell me where we compute validation (evaluation) loss exactly? I saw that we have evaluator class and make predictions there. This class returns "image", "label", "prediciton", but I didn't see that where we compute dice loss for evaluation. Thank you for your responses.

[Nic-Ma]

Hi @hilmicanguven ,

Thanks for your interest here.
If you are using MONAI evaluator, you can refer to this example:
https://github.com/Project-MONAI/tutorials/blob/master/modules/engines/unet_evaluation_dict.py#L108
If you are writing a pure PyTorch program, please refer to:
https://github.com/Project-MONAI/tutorials/blob/master/3d_segmentation/torch/unet_evaluation_dict.py#L93

Thanks.

[hilmicanguven]

Sir @Nic-Ma , thank you first.
I try to implement that tutorial : https://github.com/Project-MONAI/tutorials/tree/master/3d_segmentation/challenge_baseline
Therefore, I use monai library and its evaluator class. I think I use source code of monai library. In order to edit this evaluator, I add monai to my directory and I want to edit this. In my case, I want to change my validation arguments. But I can't find where we exactly calculate dice loss in evaluation. (In training, we use dice loss in run_net.py and I can simply change it, but in evaluation, I can't find we we call the exact function to calculate loss)

[Nic-Ma]

May I know what's the evaluation program you refer to?

Thanks.

[hilmicanguven]

Of course sir,
The code I mentioned, run_net.py, uses two class (trainer and evaluator) and you can find here: https://github.com/Project-MONAI/MONAI/tree/dev/monai/engines
The code use trainer.py to train, and evaluator.py to validation. Exact class for evaluation is this:

MONAI/monai/engines/evaluator.py

Line 126 in c1e06f5

class SupervisedEvaluator(Evaluator):

But still I can't find where we calculate evaluation loss.
Thank you

[Nic-Ma]

This line set the metrics for validation:
https://github.com/Project-MONAI/tutorials/blob/master/3d_segmentation/challenge_baseline/run_net.py#L188

Thanks.