Skip to content

Fix gguf cuda ut #1975

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
XuehaoSun merged 4 commits into from
Jul 1, 2026
Merged

Fix gguf cuda ut #1975

Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
42b8399
fix auto_scheme gguf format wrong bits if use tiny model
n1ck-guo Jun 25, 2026
0c837df
fix gguf cuda ut fail
n1ck-guo Jul 1, 2026
df9a144
correct merge
n1ck-guo Jul 1, 2026
dda75b7
Merge branch 'main' into fix_gguf_cuda_ut
n1ck-guo Jul 1, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion test/test_cuda/algorithms/test_auto_scheme.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -72,7 +72,8 @@ def test_gguf(self):
model, layer_config = ar.quantize()
avg_bits, _ = compute_avg_bits_for_model(model, ignore_scale_zp_bits=True)
print(avg_bits)
assert target_bits - 0.1 < avg_bits <= target_bits + 1e-3
# Due to the tiny model and embedding, the actual number of bits of gguf format will be larger than the target bits.
assert target_bits - 0.1 < avg_bits <= target_bits + 0.3

def test_shared_layers(self, tiny_opt_model_path):
model_name = tiny_opt_model_path
Expand Down