Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Use full-sized GPT-J #107

Closed
jeremyfowers opened this issue Feb 9, 2024 · 0 comments · Fixed by #108
Closed

Use full-sized GPT-J #107

jeremyfowers opened this issue Feb 9, 2024 · 0 comments · Fixed by #108
Assignees
@jeremyfowers
Labels
bug Something isn't working model Pertaining to the models corpus //models/

Comments

@jeremyfowers
Copy link
Collaborator

Problem: Our GPT-J model uses a "tiny" checkpoint that wasn't produced by the original author.

Solution: Replace the GPT-J checkpoint with the real 6B param version.

First observed by @pcolange

cc @danielholanda
@jeremyfowers jeremyfowers added bug Something isn't working model Pertaining to the models corpus //models/ labels Feb 9, 2024
@jeremyfowers jeremyfowers self-assigned this Feb 9, 2024
@jeremyfowers jeremyfowers mentioned this issue Feb 9, 2024
Merged
@jeremyfowers jeremyfowers linked a pull request Feb 9, 2024 that will close this issue
Use the real checkpoint for GPT-J so that it will be 6B params #108
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@jeremyfowers jeremyfowers

Labels
bug Something isn't working model Pertaining to the models corpus //models/
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Use the real checkpoint for GPT-J so that it will be 6B params onnx/turnkeyml
1 participant
@jeremyfowers