Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

FEAT: Lora multi support #1273

Merged
merged 10 commits into from
Apr 17, 2024
Merged

Conversation

hainaweiben
Copy link
Contributor

No description provided.

@XprobeBot XprobeBot added this to the v0.10.1 milestone Apr 10, 2024
xinference/api/restful_api.py Outdated Show resolved Hide resolved
xinference/api/restful_api.py Outdated Show resolved Hide resolved
xinference/utils.py Outdated Show resolved Hide resolved
xinference/client/restful/restful_client.py Outdated Show resolved Hide resolved
xinference/client/restful/restful_client.py Outdated Show resolved Hide resolved
xinference/deploy/cmdline.py Outdated Show resolved Hide resolved
xinference/model/core.py Outdated Show resolved Hide resolved
xinference/model/image/core.py Outdated Show resolved Hide resolved
xinference/model/image/stable_diffusion/core.py Outdated Show resolved Hide resolved
xinference/model/llm/core.py Outdated Show resolved Hide resolved
@ChengjieLi28
Copy link
Contributor

Please add some screenshots and comments to explain what model and which lora(s) you used and what effect you achieved.

@ChengjieLi28
Copy link
Contributor

Also, you need to modify the releated documents: https://inference.readthedocs.io/zh-cn/latest/models/lora.html

@XprobeBot XprobeBot modified the milestones: v0.10.1, v0.10.2 Apr 12, 2024
@hainaweiben
Copy link
Contributor Author

Test for multi lora

image
The model gemma-it-2B, using both 1rsh/gemma-2b-it-fr and alfredplpl/gemma-2b-it-ja-poc, yields the following results:
image

…nference into lora-multi-support

Conflicts:
	xinference/client/restful/restful_client.py
	xinference/model/core.py
@hainaweiben hainaweiben reopened this Apr 17, 2024
@ChengjieLi28 ChengjieLi28 merged commit 9245831 into xorbitsai:main Apr 17, 2024
10 of 14 checks passed
@hainaweiben hainaweiben deleted the lora-multi-support branch April 17, 2024 07:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants