Hello again, I noticed a strange behavior while developing using llmlingua, for small prompts such as "hello" or "who", compress function throws an error for index 0 out of range.
` """
Token compression using llmlingua that uses gpt-2 small llm.
"""
llm_lingua = PromptCompressor(model_name=optimizer_model, device_map=device_map)
try:
print(user_prompt)
print("Compressing")
compressed_prompt = llm_lingua.compress_prompt(
context=[user_prompt],
ratio=0.2,
)
`
here is initialization to trigger error, thanks and I am looking forward to updates
Hello again, I noticed a strange behavior while developing using llmlingua, for small prompts such as "hello" or "who", compress function throws an error for index 0 out of range.
` """
Token compression using llmlingua that uses gpt-2 small llm.
"""
`
here is initialization to trigger error, thanks and I am looking forward to updates