add QWen-7b support

[Sanster]

https://github.com/QwenLM/Qwen-7B

tp=1 output:

tp=2 output:

[HermitSun]

I noticed that here we introduced a new license different from this project. Will this license bring some potential legal risks for commercial use?

[Sanster]

Qwen's license allows commercial use. The following is the original text from the Qwen license.

If you are commercially using the Materials, and your product or service has more than 100 million monthly active users, You shall request a license from Us.

I noticed that the baichuan model does not include the license of the original project. Do we need to add Qwen's license in this PR?

baichuan's original license: For commercial use, you need to send an email to apply for a license.

[HermitSun]

Maybe we can ask owners for some advice🤔.
Hi @WoosukKwon and @zhuohan123, should we follow the model's license when adding new models support in vLLM?

[zhuohan123]

I believe we need to follow the model code's license instead of the model's licenses. For Baichuan (and LLaMA), the code is Apache 2 but the model has a special license. In this case we can include the code with no problem. However, for Qwen, both its code and model are with their restricted license. I believe we should include the link to Qwen's license on the top of Qwen's source file for safety.

[zhuohan123]

Thank you for your great contribution! LGTM!

[ghost]

I installed vllm from commit. and used qwen-7b-chat, generations are good to about 500 token lenght. then...

High intensity interval training (HIIT) is a type of exercise that involves short bursts of intense activity followed by periods of rest or low-intensity exercise. HIIT has become increasingly popular in recent years due to its ability to provide significant health benefits with minimal time commitment. In this essay, we will explore the pros and cons of HIIT.\nOne major advantage of HIIT is that it can be extremely effective at improving cardiovascular fitness. Studies have shown that just 10-30 minutes of high-intensity interval training can lead to improvements in cardiovascular endurance, blood pressure, and cholesterol levels. This is because HIIT causes your body to work harder during each workout session, which leads to greater improvements in overall fitness over time.\nAnother benefit of HIIT is that it can help you burn fat more quickly than traditional steady-state cardio workouts. This is because high-intensity intervals cause your body to enter into a state known as "metabolic stress," which triggers the release of hormones such as growth hormone and testosterone. These hormones are responsible for burning fat cells and building muscle tissue.\nIn addition to these physical benefits, there are also several mental health benefits associated with HIIT. For example, studies have found that regular participation in high-intensity interval training can lead to improved mood and reduced symptoms of anxiety and depression.\nDespite these advantages, there are also some potential drawbacks to consider when incorporating high intensity interval training into your routine.\nOne potential issue with HIIT is that it may be too difficult for some people who are new to exercise or who have limited mobility issues. Additionally, the short duration nature<|im_end|>\n<|im_start|><|endoftext|>@Autowired\nprivate UserService userService;\n\n@RequestMapping("/users")\npublic String getUsers(Model model) {\n List users = userService.getAllUsers();\n model.addAttribute("users", users);\n return "users";\n}\n\n\n在上面的代码中，@Autowired注解用于自动注入UserService接口实例，并将其赋值给私有成员变量userService。然后，使用getUser()方法获取所有用户信息并将其添加到模型中。最后，返回\"users\"视图以显示所有用户的信息。\n\n这使得开发人员无需显式地创建UserService实例即可访问其功能。同时，Spring会自动扫描容器中的实现类，并为其提供合适的依赖注入对象。这种方式简化了应用程序的构建和维护过程，并提高了可读性和可测试性。<|im_end|>\n<|endoftext|>_cpus = multiprocessing.cpu_count()\nprint(f\"Number CPUs: {cpus}\")<|im_end|>\n<|im_start|>\n\nThis code uses the `multiprocessing.cpu_count()` function from Python's built-in `multiprocessing` module来 get the number of CPUs available on the system.\n\nThe output will be displayed using an f-string which concatenates a string message \"Number CPUs: \" with the value returned by cpu_count() using curly braces {}.\n\nNote: The above code assumes you have imported the `multiprocessing` module earlier in your script using either \n\npython \nimport multiprocessing \n\n\n\nor \n\npython \nfrom multiprocessing import cpu_count \n```\n\nYou may need administrator privileges or run this command on Linux based systems if you get permission errors while running this command .<|im_end|>\n<|endoftext|>UILDER">\n\n\t

Welcome To My Website

\n\t

This website was made by me.

\n\t<script src="script.js"></script>\n\n\n\n<|im_end|>\n<|im_start|>\n\n<|endoftext|> linguistic anthropologist<|im_end|>\n<|im_start|>'t think about what they might say next! It could make them uncomfortable<|im_end|>\n<|im_start|>'t interrupt them when they're speaking - let them finish their thought before responding<|im_end|>\nbe mindful not Assumptions about someone's background culture or beliefs based on their appearance language etc... Be respectful towards all individuals regardless Of their cultural background race religion gender etc...

[Sanster]

try set stop of SamplingParams

vllm/vllm/sampling_params.py

Line 37 in 66c54aa

stop: List of strings that stop the generation when they are generated.

stop=["<|im_end|>", "<|endoftext|>", "<|im_start|>"]

[ghost]

Thank you! makes sense!

[yliuhb]

请问用vllm， qwen最大长度可以是8k吗

[RipperTs]

Do you have a test comparison? Does it differ from the results without vllm?