Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: nanoGPT example #255

Merged
merged 13 commits into from
May 23, 2023
Merged

feat: nanoGPT example #255

merged 13 commits into from
May 23, 2023

Conversation

alexander-camuto
Copy link
Collaborator

@alexander-camuto alexander-camuto commented May 23, 2023

Here we add nanoGPT as an example to the repo. We leverage divide-by rather than multiply-by rescaling to accomodate large model depth and chained linear layers.

to run nanoGPT mock test:

cargo test --release --verbose tests::large_mock_::large_tests_1_expects

to run the full test:

cargo test --release --verbose tests::large_kzg_prove_and_verify_::large_tests_1_expects

thanks to @biancaganescu for helping make this happen :)

@alexander-camuto alexander-camuto changed the title refactor: divide-by rescaling feat: nanoGPT example May 23, 2023
@alexander-camuto alexander-camuto marked this pull request as ready for review May 23, 2023 13:38
@alexander-camuto alexander-camuto merged commit 66fa25b into main May 23, 2023
19 checks passed
@alexander-camuto alexander-camuto deleted the ac/switch-rescaling-strategy branch June 16, 2023 22:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant