fix: dispatch LLMLingua-2 models by loosened name markers (#168)#248
Open
ousamabenyounes wants to merge 1 commit intomicrosoft:mainfrom
Open
fix: dispatch LLMLingua-2 models by loosened name markers (#168)#248ousamabenyounes wants to merge 1 commit intomicrosoft:mainfrom
ousamabenyounes wants to merge 1 commit intomicrosoft:mainfrom
Conversation
) is_begin_of_new_word and get_pure_token in llmlingua/utils.py used to dispatch on literal substrings — "bert-base-multilingual-cased" for the BERT family and "xlm-roberta-large" for the XLM-RoBERTa family. Users who download LLMLingua-2 weights and load them from a renamed local directory (for example "/data/models/llmlingua-2-xlm" as in the reporter's setup) fall straight through to `raise NotImplementedError` deep inside `compress_prompt_llmlingua2`, with no hint of what went wrong. Factor the dispatch into `_model_family` and broaden the marker lists to include the shorter common prefixes (`xlm-roberta`, `bert-base-multilingual`), plus explicit `llmlingua-2-xlm` and `llmlingua-2-bert` local-path variants. The canonical names, tinybert/mobilebert, slingua and securitylingua all keep working unchanged, and the fallback NotImplementedError now lists the supported markers so diagnosing a bad model_name no longer requires reading the source. Generated by Claude Code Vibe coded by ousamabenyounes Co-Authored-By: Claude <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Fixes #168
`is_begin_of_new_word` and `get_pure_token` in `llmlingua/utils.py` dispatch on literal substrings — `"bert-base-multilingual-cased"` for the BERT family and `"xlm-roberta-large"` for the XLM-RoBERTa family. Users who download LLMLingua-2 weights and load them from a renamed local directory (for example the reporter's `/home/webservice/llm/compressFromNet/llmlingua-2-xlm`) fall straight through to `raise NotImplementedError` deep inside `compress_prompt_llmlingua2`, with no hint of what went wrong.
Fix
Factor the dispatch into `_model_family(model_name)` and broaden the marker lists:
`None`/empty `model_name` now returns `None` cleanly instead of crashing on the old `"..." in model_name` check, and the fallback `NotImplementedError` now lists the supported markers so diagnosing a bad model_name no longer requires reading the source.
The canonical model names, `tinybert`, `mobilebert`, `slingua` and `securitylingua` all keep working unchanged.
Before submitting
Verification
Generated by Claude Code
Vibe coded by ousamabenyounes