Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix bug caused by 'groupsize' vs 'group_size' and change all code to use 'group_size' consistently #58

Merged

Conversation

TheBloke
Copy link
Contributor

@TheBloke TheBloke commented May 5, 2023

Today I found a bug in the quantisation code caused by #43 . in auto_gptq/modeling/_utils.py it uses groupsize but code calling it passes group_size.

I fixed that, and then thought I should update the whole repo to be consistent. I think it would be great if one name was used consistently throughout the repo. This will help avoid bugs and confusion. And 'group_size' seems to be the best name to use, as that's what's used in quantize_config.json.

With this PR merged into faster-llama, all *.py files use 'group_size' . There are no longer any references to 'groupsize'.

Before:

tomj@Eddie ~/src/AutoGPTQ (faster-llama)$ find . -name "*.py" -exec grep -o "group_size" {} + | wc -l
      38
tomj@Eddie ~/src/AutoGPTQ (faster-llama)$ find . -name "*.py" -exec grep -o "groupsize" {} + | wc -l
      54

After:

tomj@Eddie ~/src/TheBloke_AutoGPTQ (TheBloke_faster-llama_groupsize_fix)$ find . -name "*.py" -exec grep -o "group_size" {} + | wc -l
      92
tomj@Eddie ~/src/TheBloke_AutoGPTQ (TheBloke_faster-llama_groupsize_fix)$ find . -name "*.py" -exec grep -o "groupsize" {} + | wc -l
       0

Note: I only changed .py files, not any CUDA kernels. I don't know CUDA code and don't want to risk touching anything which I don't understand, in case there any implications I don't know about.

I have done tests of Triton and CUDA inference, and Triton and CUDA quantisation, and all seems OK.

Hope this change is OK with you guys, @PanQiWei @qwopqwop200 ?

…er than 'groupsize' due to dependencies in other files.
…ched any CUDA kernels in case there's any complexity there I don't understand
@PanQiWei
Copy link
Collaborator

PanQiWei commented May 5, 2023

Thank you very much for standardize the spell of 'groupsize' to 'group_size'! ❤️

@PanQiWei PanQiWei merged commit 7c33fa2 into AutoGPTQ:faster-llama May 5, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants