Skip to content

Conversation

@drbh
Copy link
Collaborator

@drbh drbh commented Sep 11, 2024

This PR correctly includes the missing adapter.revision param in load_module_map which should resolve a bug when loading multiple lora adapters and at least on has a revision

@Narsil Narsil merged commit 628334d into main Sep 12, 2024
@Narsil Narsil deleted the fix-missing-lora-adapter-revision branch September 12, 2024 15:04
Narsil pushed a commit that referenced this pull request Sep 14, 2024
#2510)

fix: pass missing revision arg for lora adapter when loading multiple adapters
@nbroad1881
Copy link
Contributor

nbroad1881 commented Sep 17, 2024

A customer is still reporting that this doesn't work.

Here is what I tried on Inference Endpoints and the associated error (TGI image taken from 1 hour ago):

from huggingface_hub import create_inference_endpoint

endpoint = create_inference_endpoint(
    "phi-3-lora-revision-test",
    repository="microsoft/Phi-3-mini-4k-instruct",
    framework="pytorch",
    task="text-generation",
    accelerator="gpu",
    vendor="aws",
    region="us-east-1",
    type="protected",
    instance_size="x1",
    instance_type="nvidia-a10g",
    custom_image={
        "health_route": "/health",
        "env": {
            "MAX_BATCH_PREFILL_TOKENS": "2048",
            "MAX_INPUT_LENGTH": "1024",
            "MAX_TOTAL_TOKENS": "1512",
            "MODEL_ID": "/repository",
            "LORA_ADAPTERS": "grounded-ai/phi3-hallucination-judge@5f5f8c0483200db2ceb4db66adfae9ce77273bde"
        },
        "url": "ghcr.io/huggingface/text-generation-inference:sha-ce85efa",
    },
    token="token"
)

error

Exit code: 1. Reason:                                          │
│ /opt/conda/lib/python3.11/site-packages/huggingface_hub/utils/_validators.py │
│ :160 in validate_repo_id                                                     │
│                                                                              │
│   157 │   │   )                                                              │
│   158 │                                                                      │
│   159 │   if not REPO_ID_REGEX.match(repo_id):                               │
│ ❱ 160 │   │   raise HFValidationError(                                       │
│   161 │   │   │   "Repo id must use alphanumeric chars or '-', '_', '.', '-- │
│   162 │   │   │   " forbidden, '-' and '.' cannot start or end the name, max │
│   163 │   │   │   f" '{repo_id}'."                                           │
│                                                                              │
│ ╭───────────────────────────────── locals ─────────────────────────────────╮ │
│ │ repo_id = 'grounded-ai/phi3-hallucination-judge@5f5f8c0483200db2ceb4db6… │ │
│ ╰──────────────────────────────────────────────────────────────────────────╯ │
╰──────────────────────────────────────────────────────────────────────────────╯
HFValidationError: Repo id must use alphanumeric chars or '-', '_', '.', '--' 
and '..' are forbidden, '-' and '.' cannot start or end the name, max length is 
96: 
'grounded-ai/phi3-hallucination-judge@5f5f8c0483200db2ceb4db66adfae9ce77273bde'.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants