Questions about the Attention-based Generator

[taylover-pei]

Congratulations! You have done a great job.
But I have a question about the Attention-based Generator. This paper said that the generator outputs two masks, a color mask C and attention mask A. And you gave the following formula:

So, I'm wondering how did you come up with this formula? I mean, how do you explain this formula?
Looking forward to your apply.

[donydchen]

This is somehow a common trick when playing with attention map / attention mask. From my point of view,

• Intuitively attention mask enables some specific areas where facial muscle changed to get more focus, applying it to the color mask () can generate images with clear dynamic area and unclear static area.
• After that, what left is to enhance the static area, which should be similar between the generated image and the original real image. So we can enhance the static area (basically it refers to background area) in the original real image () and merge it to the above to get final result ().

You can understand it more clearly with the help of Sec. 4.1_Generator and Sec. 6.5 of the original paper.

However, the above is what the paper tells, while this project actually implements it in a different (or wrong?) way. You can check out Issue #21 for more details.