Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add support for CPU Inference #1496

Merged
merged 1 commit into from
Oct 31, 2023
Merged

Add support for CPU Inference #1496

merged 1 commit into from
Oct 31, 2023

Conversation

vivekkhandelwal1
Copy link
Contributor

Refer: AutoGPTQ/AutoGPTQ#385

Signed-Off By: Vivek Khandelwal vivek@nod-labs.com

@vivekkhandelwal1 vivekkhandelwal1 mentioned this pull request Oct 31, 2023
Merged
@vivekkhandelwal1
Copy link
Contributor Author

@fxmarty @SunMarc can you please review this PR?

fxmarty
fxmarty approved these changes Oct 31, 2023
View reviewed changes
Copy link
Collaborator

@fxmarty fxmarty left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@fxmarty
Copy link
Collaborator

fxmarty commented Oct 31, 2023

Can you change the requirements to put AutoGPTQ>=0.5.0?

@vivekkhandelwal1
Copy link
Contributor Author

Can you change the requirements to put AutoGPTQ>=0.5.0?

The transformer's requirements?

@fxmarty
Copy link
Collaborator

fxmarty commented Oct 31, 2023

Nevermind. There's actually no auto-gptq in setup.py. You can change AUTOGPTQ_MINIMUM_VERSION in utils/import_utils.py

@vivekkhandelwal1
Add support for CPU Inference
78342d1 
Refer: AutoGPTQ/AutoGPTQ#385

Signed-Off By: Vivek Khandelwal <vivek@nod-labs.com>
@vivekkhandelwal1
Copy link
Contributor Author

@fxmarty I have updated the PR with the required changes.

fxmarty
fxmarty approved these changes Oct 31, 2023
View reviewed changes
Copy link
Collaborator

@fxmarty fxmarty left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! I will merge it once the corresponding PR in AutoGPTQ is merged :)

@fxmarty fxmarty merged commit 9a9a7bf into huggingface:main Oct 31, 2023
2 checks passed
@vivekkhandelwal1 vivekkhandelwal1 deleted the cpu-inference branch October 31, 2023 11:51
@vivekkhandelwal1 vivekkhandelwal1 mentioned this pull request Oct 31, 2023
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@fxmarty fxmarty fxmarty approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
2 participants
@vivekkhandelwal1 @fxmarty