Confused about the variable k in the code.

[chilynn]

"k = 4 if n_in != out_size else 3".
I notice that many places involve the variable k which sometimes be 4 and sometimes be 3. It seems that it is related to the different layer. I think the k is related to the 3 different W matrix in the paper which are W, W_f, W_r. Could you please explain about it? Thanks.

[taolei87]

@chilynn This question is related to #12 .

When the input and output dimension do not match, we add one more linear transformation in the highway connection:
h'[t] = r[t] * h[t] + (1-r[t]) * (Wx[t])

This matrix multiplication (mm) is batched together with other mm. that said, this (Wx) is being put into U. So there will be four mms (k==4) instead of three (k==3).