Repository navigation
Appendix C errata #268
daveyburke
started this conversation in
General
Replies: 2 comments 1 reply
|
Hi, that's a good catch I think, your pattern does matches for 8 dims, (x0, x4), (x1,x5) etc... For the position, I believe Sebastian wanted to emphasize on the difference between the 2 variants with a simplified notation but you're right about the rigorous notation. Note that the RoPE code is accurate, it's this extra insert not in the .ipynb, it's a book thing only. @rasbt ICYMI on p.344 |
1 reply
|
Looks good to me!
…On Sat, Sep 26, 2026 at 8:06 AM Sebastian Raschka ***@***.***> wrote:
Good catch @daveyburke <https://github.com/daveyburke> and I am sorry
about the typo on pg 344. I messed up there (probably copy & paste error).
In the meantime, you can find a correct version in the code comments here:
https://github.com/rasbt/reasoning-from-scratch/blob/1c0b9b17f80d335a9e7f73ca4738486bfa49fa9f/reasoning_from_scratch/qwen3.py#L224-L271
—
Reply to this email directly, view it on GitHub
<#268?email_source=notifications&email_token=AHOFS3I4PR2MJ3VS4CMBOCT5Q7LPTA5CNFSNUABIM5UWIORPF5TWS5BNNB2WEL2ENFZWG5LTONUW63SDN5WW2ZLOOQXTCOBWGE2DAOBVUZZGKYLTN5XKO3LFNZ2GS33OUVSXMZLOOSWGM33PORSXEX3DNRUWG2Y#discussioncomment-18614085>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AHOFS3PVAVPLGLJKKPCDVYL5Q7LPTAVCNFSNUABIKJSXA33TNF2G64TZHM4TIMRWHE3DIOJXHNCGS43DOVZXG2LPNY5TCMBYGYYTIMBUUF3AE>
.
Triage notifications, keep track of coding agent tasks and review pull
requests on the go with GitHub Mobile for iOS
<https://github.com/notifications/mobile/ios/AHOFS3N7J3AEI6WTGVPIDCD5Q7LPTA5CNFSNUABIM5UWIORPF5TWS5BNNB2WEL2ENFZWG5LTONUW63SDN5WW2ZLOOQXTCOBWGE2DAOBVUZZGKYLTN5XKO3LFNZ2GS33OUVSXMZLOOSVGM33PORSXEX3JN5ZQ>
and Android
<https://github.com/notifications/mobile/android/AHOFS3MT53H4DUZVHPEVWU35Q7LPTA5CNFSNUABIM5UWIORPF5TWS5BNNB2WEL2ENFZWG5LTONUW63SDN5WW2ZLOOQXTCOBWGE2DAOBVUZZGKYLTN5XKO3LFNZ2GS33OUVSXMZLOOSXGM33PORSXEX3BNZSHE33JMQ>.
Download it today!
You are receiving this because you were mentioned.Message ID:
<rasbt/reasoning-from-scratch/repo-discussions/268/comments/18614085@
github.com>
|
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
In the box "RoPE implementation variants", unless I misunderstood, the rotation matrix for the split halves is incorrect (looks like copy/paste error - shows the interleaved form). For 8 dims it would be:
[cos(theta) 0 0 0 -sin(theta) ...
0 cos(theta) 0 0 0 -sin(theta)...
0 0 cos(theta) ...
...
]
Prob too in the weeds, but for both matrices the angle is m * theta_i where m is seq position and i is the feature pair index.
Oh and maybe there is meant to be a "2)" in front of "The interleaved"?
All reactions