example.py does not work

[SowreshMS]

RuntimeError Traceback (most recent call last)
in <cell line: 25>()
24 #test forward pass
25 with torch.no_grad():
---> 26 output = model(x)
27 print(f"Output shape: {output.shape}") # expected (batch_size, seq_Len)
28

4 frames
in apply_rotary_pos_emb(x, sin, cos, scale)
33 sin, cos = map(lambda t: duplicate_interleave(t * scale), (sin, cos))
34 # einsum notation for lambda t: repeat(t[offset:x.shape[1]+offset,:], "n d -> () n () (d j)", j=2)
---> 35 return (x * cos) + (rotate_every_two(x) * sin)
36
37

RuntimeError: The size of tensor a (512) must match the size of tensor b (64) at non-singleton dimension 2

Upvote & Fund

• We're using Polar.sh so you can upvote and help fund this issue.
• We receive the funding once the issue is completed & confirmed by you.
• Thank you in advance for helping prioritize & fund our backlog.

[kyegomez]

@SowreshMS Hey please excuse me for this, i was experimenting, it should work now.

[kyegomez]

@SowreshMS Hey please excuse me, it works now try again!