Use of masks and Highpass filter

[arshagarwal]

I would like to understand the intuition and improvement attained by using masks and w_hpf as in the following lines:

1. At solver.py Line 106:
   masks = nets.fan.get_heatmap(x_real) if args.w_hpf > 0 else None
2. At mode.py Line 182-185:

if (masks is not None) and (x.size(2) in [32, 64, 128]):
                mask = masks[0] if x.size(2) in [32] else masks[1]
                mask = F.interpolate(mask, size=x.size(2), mode='bilinear')
                x = x + self.hpf(mask * cache[x.size(2)])

PS: I think both stargan and stargan v2 are impeccable researches.

[yunjey]

@arshagarwal Sorry for a late reply. The purpose of introducing masks is to accurately preserve face landmarks (i.e., eyes, nose) of a source image during translation. The hyperparameter w_hpf controls the strength of the preservation. If you increase the value of w_hpf, then the generator preserve the source landmarks more precisely, but it would decrease the reflection of reference styles.

[arshagarwal]

Understood, thanks for the answer. @yunjey

[leonardodora]

@arshagarwal Sorry for a late reply. The purpose of introducing masks is to accurately preserve face landmarks (i.e., eyes, nose) of a source image during translation. The hyperparameter w_hpf controls the strength of the preservation. If you increase the value of w_hpf, then the generator preserve the source landmarks more precisely, but it would decrease the reflection of reference styles.

hello, I want to konw that is w_hpf range from 0.0 to 1.0(like 0.2, 0.5...) ?

[yunjey]

@leonardodora You can set w_hpf higher than 1 for strong source preservation.