Support Longchat

[LiuXiaoxuanPKU]

Add LlamaLinearScalingRotaryEmbedding, LlamaDynamicNTKScalingRotaryEmbedding. Attempt to fix #333, #464, #479

[zhuohan123]

CC @DachengLi1

[DachengLi1]

Wonderful! Thanks a lot Lily and Zhuohan! Will this be merged soon?

[zhuohan123]

Wonderful! Thanks a lot Lily and Zhuohan! Will this be merged soon?

I believe so! @LiuXiaoxuanPKU is working on some correctness tests and making sure everything works for a long context. Feel free to try out this PR if you would like to start immediately!

[winglian]

@LiuXiaoxuanPKU You also need to add rope_scaling as an argument to argparse (https://github.com/LiuXiaoxuanPKU/vllm/blob/longchat/vllm/engine/arg_utils.py#L40) , otherwise this call on line 141 failse
engine_args = cls(**{attr: getattr(args, attr) for attr in attrs})

[lucasjinreal]

@LiuXiaoxuanPKU Will also support Baichuan model?

https://github.com/keezen/ntk_alibi like using AlibiNTK for scaling in alibi

[alanxmay]

🤔 Is there any reason to prevent this PR from being merged

[LiuXiaoxuanPKU]

🤔 Is there any reason to prevent this PR from being merged

We think vLLM might have some correctness issues. It might or might not be caused by this PR. To be more concrete, take a look at tests/test_longprompt.py in this PR. The four test cases should pass (HF passes all four cases), but currently, vLLM fails on them. We are still debugging it.

[WoosukKwon]

@LiuXiaoxuanPKU Awesome! Many thanks for the great work and sorry again for the very late review.

I've made several changes (mostly on the code style):

1. I temporarily removed the tests for faster integration. I will submit another PR to add test for RoPE scaling.
2. As we discussed offline, I removed rope_scaling from ModelConfig and EngineArgs. Now rope_scaling is always read from the model's config.json.
3. I refactored rotary_embedding.py and added DynamicNTKScalingRotaryEmbedding.