Skip to content

fix few bugs cause less accuracy of gguf format#1960

Merged
n1ck-guo merged 6 commits into
mainfrom
henggu/fix_gguf_acc
Jul 2, 2026
Merged

fix few bugs cause less accuracy of gguf format#1960
n1ck-guo merged 6 commits into
mainfrom
henggu/fix_gguf_acc

Conversation

@n1ck-guo

@n1ck-guo n1ck-guo commented Jun 29, 2026

Copy link
Copy Markdown
Contributor

Description

Please briefly describe your main changes, the motivation.

Type of Change

Bug fix

Related Issues

Fixes or relates to #

Checklist Before Submitting

  • My code has been tested locally.
  • Documentation has been updated as needed.
  • New or updated tests are included where applicable.
  • The CUDA CI has passed. You can trigger it by commenting /azp run Unit-Test-CUDA-AutoRound.

--options "gguf:q2_k_s,gguf:q4_k_s" --target_bits 3.5 --ignore_scale_zp_bits

gpqa_diamond math_500
qwen3-8b f16 0.4394 0.786
gguf format 0.4545 0.77

Signed-off-by: n1ck-guo <heng.guo@intel.com>
Comment thread auto_round/compressors/base.py Outdated
Comment thread auto_round/data_type/gguf.py Outdated
@chensuyue

Copy link
Copy Markdown
Contributor

/azp run Unit-Test-CUDA-AutoRound

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines successfully started running 1 pipeline(s).

Signed-off-by: n1ck-guo <heng.guo@intel.com>
@chensuyue

Copy link
Copy Markdown
Contributor

/azp run Unit-Test-CUDA-AutoRound

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines successfully started running 1 pipeline(s).

@chensuyue

Copy link
Copy Markdown
Contributor

/azp run Unit-Test-CUDA-AutoRound

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines successfully started running 1 pipeline(s).

n1ck-guo added 2 commits July 2, 2026 00:43
Signed-off-by: n1ck-guo <heng.guo@intel.com>
@chensuyue

Copy link
Copy Markdown
Contributor

/azp run Unit-Test-CUDA-AutoRound

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines successfully started running 1 pipeline(s).

@n1ck-guo n1ck-guo requested a review from wenhuach21 July 2, 2026 02:43
@chensuyue chensuyue added this to the 0.14.0 milestone Jul 2, 2026
Comment thread auto_round/formats.py
Comment thread auto_round/data_type/gguf.py
@wenhuach21 wenhuach21 self-requested a review July 2, 2026 03:17
Comment thread auto_round/algorithms/quantization/base.py
Signed-off-by: n1ck-guo <heng.guo@intel.com>
@chensuyue

Copy link
Copy Markdown
Contributor

/azp run Unit-Test-CUDA-AutoRound

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines successfully started running 1 pipeline(s).

@n1ck-guo n1ck-guo merged commit e4404be into main Jul 2, 2026
47 checks passed
@n1ck-guo n1ck-guo deleted the henggu/fix_gguf_acc branch July 2, 2026 08:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants