Why adding bias to the forget gate?

[AzizCode92]

supercell/supercell.py

Line 216 in 063b01e

new_c = c*tf.sigmoid(f+self.forget_bias) + tf.sigmoid(i)*g

The code implementation didn't correspond exactly to the equation we have in the layer normalization paper.
I also have doubts about normalizing all the gates, so for example, the forget gate will never be equal to zero du to the shift we add.
Isn't more logic to just keep the gates as they are and then just normalize cell state?

Thank you

[AzizCode92]

found the reason why