Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BLD: add autoawq in setup #1190

Merged
merged 2 commits into from
Mar 27, 2024
Merged

BLD: add autoawq in setup #1190

merged 2 commits into from
Mar 27, 2024

Conversation

utopia2077
Copy link
Contributor

Support autoawq.
Otherwise, when inferring the AWQ quantization model, it will error that dependencies are missing.

@XprobeBot XprobeBot added the enhancement New feature or request label Mar 26, 2024
@XprobeBot XprobeBot added this to the v0.9.5 milestone Mar 26, 2024
@qinxuye
Copy link
Contributor

qinxuye commented Mar 26, 2024

I think you can add the autoawq to ALL and transformers.

Refer to auto-gptq

auto-gptq ; sys_platform!='darwin'

and

auto-gptq

Then when pip install "xinference[all]" it will be installed, so you don't need to modify the dockerfile.

@utopia2077
Copy link
Contributor Author

Modified as above.
Installing dependencies inside a Docker container is very inconvenient, so it is recommended to add it.

setup.cfg Outdated Show resolved Hide resolved
@qinxuye qinxuye changed the title ENH: support autoawq BLD: add autoawq in setup Mar 27, 2024
@XprobeBot XprobeBot added build and removed enhancement New feature or request labels Mar 27, 2024
Copy link
Contributor

@qinxuye qinxuye left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@qinxuye qinxuye merged commit 189531d into xorbitsai:main Mar 27, 2024
12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants