We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
The Grokking demo is currently not executing properly in Colab.
Describe the bug The Grokking demo currently fails execution
Code example The block...
print(loss_fn(all_logits, labels)) # This bugged on models not fully trained
Generate the following error....
RuntimeError Traceback (most recent call last) [<ipython-input-88-6591fc338d6e>](https://xoc33mq2olp-496ff2e9c6d22116-0-colab.googleusercontent.com/outputframe.html?vrz=colab_20240412-060119_RC00_624124077#) in <cell line: 1>() ----> 1 print(loss_fn(all_logits, labels)) # This bugged on models not fully trained [<ipython-input-18-45ad8e4e85fe>](https://xoc33mq2olp-496ff2e9c6d22116-0-colab.googleusercontent.com/outputframe.html?vrz=colab_20240412-060119_RC00_624124077#) in loss_fn(logits, labels) 4 logits = logits.to(torch.float64) 5 log_probs = logits.log_softmax(dim=-1) ----> 6 correct_log_probs = log_probs.gather(dim=-1, index=labels[:, None])[:, 0] 7 return -correct_log_probs.mean() 8 train_logits = model(train_data) RuntimeError: Size does not match at dimension 0 expected index [12769, 1] to be smaller than self [113, 113] apart from dimension 1
Skipping this block allows the demo to run for another block, but after that the demo errors out due to a memory error.
System Info Describe the characteristic of your environment:
Additional context
The text was updated successfully, but these errors were encountered:
I can work on this today.
Sorry, something went wrong.
No branches or pull requests
The Grokking demo is currently not executing properly in Colab.
Describe the bug
The Grokking demo currently fails execution
Code example
The block...
Generate the following error....
Skipping this block allows the demo to run for another block, but after that the demo errors out due to a memory error.
System Info
Describe the characteristic of your environment:
Additional context
Checklist
The text was updated successfully, but these errors were encountered: