Repository navigation
Looped / Latent Reasoning Chapter? #266
Closed
thomas-ferraz
started this conversation in
Ideas
Replies: 1 comment 1 reply
|
Thanks for the note! Oh, this "In Progress" was just a leftover that I forgot to clean up. Regarding looped transformer / reasoning, that would be out of scope here because it would require training a new model with a new architecture from scratch. I.e., it's not simply something that can be added to an existing architecture (without pre-training from scratch after the mod, the model would do very badly). I do have a blog post about that though if you are interested: GPT-6 Astra, Looped Transformers, and Hidden Reasoning |
1 reply
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
I see the Table of Contents still marks "In Progress". Do you plan or consider adding a chapter about latent reasoning? In particular the looped reasoning that many claimed to be the approach behind GPT-6-Astra? It seems natural after talking about distillation for efficient reasoning. Thanks
All reactions