Questions about the implementation of GLU

[sanaperiod]

This is my first issue post, so I apologize if this is not the right place to discuss what I'm posting.

I have a question about the GLU used in layers such as ResidualBlock in "athena/layers/commons.py".
In the ResidualBlock implementation, the output tensor of the same convolution layer is used for the main signal and the gate signal of the GLU. Is this a mistake in the implementation? Or my misunderstanding?

   def call(self, x, c):
        x = self.conv_1(x)
        x = self.cin_1(x, c)
        x = gated_linear_layer(inputs=x, gates=x)

If my understanding is correct, the tensor given to the GLU is the output of different convolutional layer instances with the same structure.

   def call(self, x, c):
        x_main = self.conv_1_main(x)
        x_main = self.cin_1_main(x_main, c)
        x_gate = self.conv_1_gate(x)
        x_gate = self.cin_1_gate(x_gate, c)
        x = gated_linear_layer(inputs=x_main, gates=x_gate)

I'm a beginner, so I apologize if I'm wrong.

[hoyden]

@sanaperiod Your implementation is right！

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[stale]

This issue is closed. You can also re-open it if needed.