New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Phi model does not produce output on M3 #55
Comments
This issue is related to #51 |
Does this still happen if you pull main branch? I believe for others this has been fixed. It may be the same issue with M3 that llama is facing |
I'm fairly certian the problem is the softmax kernel producing inf on your machine, which makes the logits come out NaN, and triggers the blank token to be outputted, which is why you see no output at all. I will be revisiting the softmax kernel today or tomorrow to fix this |
I pulled the main branch right now, and the problem persists. Thank you so much @jafioti ! |
Currently, I can't extract an output by running the phi3 example:
The text was updated successfully, but these errors were encountered: