support starcoder2

[grimoire]

Support starcoder2 https://huggingface.co/bigcode/starcoder2-7b

[AllentDan]

What is the expected version of transformers to run the model? I got

ERROR - ValueError: The checkpoint you are trying to load has model type `starcoder2` but Transformers does not recognize this architecture. This could be because of an issue with the checkpoint, or because your version of Transformers is out of date.
2024-04-24 12:48:26,763 - lmdeploy - ERROR - <transformers> test failed!
Load model config with transformers==4.38.2 failed. Please make sure model can be loaded with transformers API.

[grimoire]

4.40 I guess.

[AllentDan]

Got

double enter to end input >>> hi

hi _1_1_i_d_manager.html#a00000000000000000000000000000000", null ],
    [ "GetID", "class_hi_1_1_i_d_manager.html#a00000000000000000000000000000000", null ],
    [ "GetID", "class_hi_1_1_i_d_manager.html#a00000000000000000000000000000000", null ],
    [ "GetID", "class_hi_1_1_i_d_manager.html#a00000000000000000000000000000000", null ],
    [ "GetID", "class_hi_1_1_i_d_manager.html#a00000000000000000000000000000000", null ],
    [ "GetID", "class_hi_1_1_i_d_manager.html#a00000000000000000000000000000000", null ],
    [ "GetID", "class_hi_1_1_i_d_manager.html#a00000000000000000000000000000000", null ],
    [ "GetID", "class_hi_1_1_i_d_manager.html#a00000000000000000000000000000000", null ],
    [ "GetID", "class_hi_1_1_i_d_manager.html#a00000000000000000000000000

for the command lmdeploy chat torch /path/of/starcoder2/startcoder2-7b.

transformers version 4.40.1

[AllentDan]

May resolve the conflicts

[grimoire]

@AllentDan
This is not a chat model, this output might be expected.

official demo:

inputs = tokenizer.encode("def print_hello_world():", return_tensors="pt").to(device)
outputs = model.generate(inputs)

[AllentDan]

LGTM