Skip to content

Commit

Permalink
feat(model): support new models (#457)
Browse files Browse the repository at this point in the history
Because

- in model-backend 0.12.0, we are going to utilize 4 A100 40G GPU 

This commit

- update model config based on the following strategy

GPU1:
yolov7
mobilenetv2
yolov7-stomata
llava 13b (needs 26G VRAM)

GPU 2:
stable-diffusion-xl  (needs 16G VRAM)
controlnet-canny  (needs 16G VRAM)

GPU 3:
llama2-7b  (needs 16G VRAM)
llama2-7b-chat  (needs 16G VRAM)

GPU 4:
llamacode 7b   (needs 16G VRAM)
zephyr-7b    (needs 16G VRAM)
  • Loading branch information
tonywang10101 committed Mar 28, 2024
1 parent 5cb7342 commit 7e311d5
Showing 1 changed file with 36 additions and 26 deletions.
62 changes: 36 additions & 26 deletions model-hub/model_hub_gpu.json
Expand Up @@ -29,6 +29,36 @@
"tag": "v0.7.0-yolov7-mask-ray-gpu"
}
},
{
"id": "llava-1-6-13b",
"description": "LLaVa-13b, from liuhaotian, is trained to generate text based on your prompts with miltimodal input.",
"task": "TASK_VISUAL_QUESTION_ANSWERING",
"model_definition": "model-definitions/github",
"configuration": {
"repository": "instill-ai/model-llava-13b-dvc",
"tag": "f16-gpuAuto-transformer-ray-v0.11.0"
}
},
{
"id": "stable-diffusion-xl",
"description": "Stable-Diffusion-XL, from StabilityAI, is trained to generate image based on your prompts.",
"task": "TASK_TEXT_TO_IMAGE",
"model_definition": "model-definitions/github",
"configuration": {
"repository": "instill-ai/model-diffusion-xl-dvc",
"tag": "f16-gpuAuto-diffusers-ray-v0.8.0"
}
},
{
"id": "controlnet-canny",
"description": "ControlNet-Canny Version, from Lvmin, is trained to generate image based on your prompts and images.",
"task": "TASK_IMAGE_TO_IMAGE",
"model_definition": "model-definitions/github",
"configuration": {
"repository": "instill-ai/model-controlnet-dvc",
"tag": "f16-gpuAuto-diffusers-ray-v0.8.0"
}
},
{
"id": "llama2-7b",
"description": "Llama2-7b, from meta, is trained to generate text based on your prompts.",
Expand All @@ -49,44 +79,24 @@
"tag": "f16-gpuAuto-transformer-ray-v0.8.0"
}
},
{
"id": "llava-1-6-13b",
"description": "LLaVa-13b, from liuhaotian, is trained to generate text based on your prompts with miltimodal input.",
"task": "TASK_VISUAL_QUESTION_ANSWERING",
"model_definition": "model-definitions/github",
"configuration": {
"repository": "instill-ai/model-llava-13b-dvc",
"tag": "f16-gpuAuto-transformer-ray-v0.11.0"
}
},
{
"id": "zephyr-7b",
"description": "Zephyr-7b, from Huggingface, is trained to generate text based on your prompts.",
"task": "TASK_TEXT_GENERATION_CHAT",
"model_definition": "model-definitions/github",
"configuration": {
"repository": "instill-ai/model-zephyr-7b-dvc",
"tag": "f32-cpu-transformer-ray-v0.8.0"
}
},
{
"id": "stable-diffusion-xl",
"description": "Stable-Diffusion-XL, from StabilityAI, is trained to generate image based on your prompts.",
"task": "TASK_TEXT_TO_IMAGE",
"model_definition": "model-definitions/github",
"configuration": {
"repository": "instill-ai/model-diffusion-xl-dvc",
"tag": "f16-gpuAuto-diffusers-ray-v0.8.0"
"tag": "f16-1gpu-transformer-ray-v0.11.0"
}
},
{
"id": "controlnet-canny",
"description": "ControlNet-Canny Version, from Lvmin, is trained to generate image based on your prompts and images.",
"task": "TASK_IMAGE_TO_IMAGE",
"id": "llamacode-7b",
"description": "Llamacode-7b, from Huggingface, is trained to generate text based on your prompts.",
"task": "TASK_TEXT_GENERATION_CHAT",
"model_definition": "model-definitions/github",
"configuration": {
"repository": "instill-ai/model-controlnet-dvc",
"tag": "f16-gpuAuto-diffusers-ray-v0.8.0"
"repository": "instill-ai/model-codellama-7b-dvc",
"tag": "f16-1gpu-transformer-ray-v0.11.0"
}
}
]

0 comments on commit 7e311d5

Please sign in to comment.