Alternative way to concatenate AU conditions with input image

[aiueogawa]

@albertpumarola
Hi, I have a question about a way to incorporate the AU conditions into an input.

Your paper says that desired AU conditions (expression) are originally a N-length vector which has a normalized activation value between 0 and 1 respectively, and are concatenated to input as additional channels of input by expanding them into the same size as that of input image.
I think this expansion is for the input to be compatible with the first convolutional layer.
I wonder what is the most reasonable way to construct an input with non-image-like (scalar or vector) conditions.

A possible alternative way to do it is concatenating AU conditions as a vector with image unrolled into a vector and replace the first convolutional layer with a fully-connected layer.
How do you think of it?

[aiueogawa]

A possible alternative way to do it is concatenating AU conditions as a vector with image unrolled into a vector and replace the first convolutional layer with a fully-connected layer.

This produces much many parameters especially when image is high resolution, so it seems not so good.

[albertpumarola]

There exists a large literature on image conditional generation (from noise+conditioning) check them out, you should find some useful insights on conditional representation for non-image like inputs.

[aiueogawa]

@albertpumarola
As you say, there is a lot of ways for image conditional generation.
Why did you choose the aforementioned way?
Did you follow the way adopted in pix2pix?

[albertpumarola]

On my previous paper I checked a bunch of methods and concatenating had the best tradeoff performance-overhead.