Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement GPTQ quantization #467

Open
wants to merge 14 commits into
base: master
Choose a base branch
from
Open

Implement GPTQ quantization #467

wants to merge 14 commits into from

Conversation

EricLBuehler
Copy link
Owner

This PR adds GPTQ quantization (paper here) support.

Refs: #418, #448.

Copy link

github-actions bot commented Jun 23, 2024

Code Metrics Report
  ===============================================================================
 Language            Files        Lines         Code     Comments       Blanks
===============================================================================
 Dockerfile              1           34           25            0            9
 Happy                   1          442          369            0           73
 JSON                    9           21           21            0            0
 Python                 32         1256         1075           37          144
 TOML                   16          444          403            1           40
-------------------------------------------------------------------------------
 Jupyter Notebooks       1            0            0            0            0
 |- Markdown             1           60           30           22            8
 |- Python               1           96           87            1            8
 (Total)                            156          117           23           16
-------------------------------------------------------------------------------
 Markdown               18         1321            0          982          339
 |- BASH                 5          100           97            0            3
 |- Python               6          122          110            0           12
 |- Rust                 3          151          135            6           10
 (Total)                           1694          342          988          364
-------------------------------------------------------------------------------
 Rust                  119        36639        33095          672         2872
 |- Markdown            59          664           13          614           37
 (Total)                          37303        33108         1286         2909
===============================================================================
 Total                 198        40157        34988         1692         3477
===============================================================================
  

@EricLBuehler EricLBuehler added the new feature New feature or request label Jun 24, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
new feature New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant