Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: Add Nested quantization check #3916

Merged
merged 5 commits into from
Jan 29, 2024
Merged

Conversation

jeffkinnison
Copy link
Contributor

Using LLMEncoder with 8-bit quantization can lead to the same error addressed by #3606. This happens because the quantization check does not account for quantization config nested in the encoder block. This adds a utility for retrieving all quantization information from a config, including accounting for multiple quantized encoders, and updates the trainer to use this utility.

Copy link

github-actions bot commented Jan 25, 2024

Unit Test Results

  6 files  ±0    6 suites  ±0   14m 14s ⏱️ -2s
12 tests ±0    9 ✔️ ±0    3 💤 ±0  0 ±0 
60 runs  ±0  42 ✔️ ±0  18 💤 ±0  0 ±0 

Results for commit 318aa19. ± Comparison against base commit 9bb89c6.

♻️ This comment has been updated with latest results.

@jeffkinnison jeffkinnison merged commit 0a24d0a into master Jan 29, 2024
18 checks passed
@jeffkinnison jeffkinnison deleted the nested-quantization-check branch January 29, 2024 19:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants