typo/bug in pooler?

[N0r9st]

Probably enumerate(self.blocks, 1) ?

in attentive_pooler.py

for layer_id, layer in enumerate(1, self.blocks):
        rescale(layer.attn.proj.weight.data, layer_id + 1)
        rescale(layer.mlp.fc2.weight.data, layer_id + 1)

[MidoAssran]

Yes that's correct, thanks for the catch. Will update that!

In the released configs we only use one block so the issue does not appear.