Have question about the decision network

[bill2239]

Hi,
Thanks for the great work, but I have question about the decision network and how it help with segmentation.
According to your paper, segmentation network is trained separately. When decision network is trained, weights for segmentation network are frozen. So I am wondering if decision network help segmentation network at all or it just do classification. Thanks

[onlyNata]

You can understand it as attention mechanism. Semantic segmentation can focus on small defects, but direct classification can't.

[bill2239]

I thought the ultimate goal is producing a good segmentation mask, so I am wondering what the purpose of the decision network in this scenario. Maybe I missed something in there.

[onlyNata]

Because the result of semantic segmentation is not good enough, it can not distinguish background and defect accurately(the defect too small and low number of samples). then decision network refine the results(using multi-label classified algorith ).

[bill2239]

That's where the question is, the decision network is trained with segmentation network's weight frozen. So how training the decision network would improve segmentation result in this way.

[onlyNata]

OK, I see.

"I thought the ultimate goal is producing a good segmentation mask, ..."

No,this is a classification prediction issue.Semantic segmentation is an auxiliary means not ultimate goal. In other words, semantic segmentation is not necessary if the defects are large enough.
the decision network would not improve segmentation result ( a better mask),but reuse the seg-feature(train with segmentation network's weight frozen) can make the classification prediction more
accurate.

[skokec]

Hi,

thanks @onlyNata for explained it quite well.

The purpose is NOT segmentation but image level classification. Segmentation is only used as auxiliary step to get good features for classification, since classification will have issues due to small sample sizes limitations that often happens in industrial applications.

In our on-going work, we are doing end-to-end learning, i.e., both segmentation and classification at the same time, but even then it proved better that decision net has minimal or no influence on learning of the segmentation layers.

Best,
Domen