double forward in goal gpt

[Howuhh]

Hi! I noticed one more not straightforward thing in goal conditioned version of GPT.

Here:

trajectory-transformer/trajectory/models/transformers.py

Lines 288 to 295 in e0b5f12

	gx = torch.cat([goal_embeddings, x], dim=1)
	gx = self.blocks(gx)
	x = gx[:, self.observation_dim:]
	#### /goal
	x = self.blocks(x)
	## [ B x T x embedding_dim ]
	x = self.ln_f(x)

After you append goal embeddings to the main sequence, you do self.blocks twice. Is that how it's intended to work? Shouldn't one time be enough, since all embeddings will have all needed information about the goal due to the attention mechanism.

[jannerm]

Good catch! commit fix