Add new chat cli with auto backend feature

[RunningLeon]

Motivation

Support auto backend feature in chat cli

Modification

• Combine lmdeploy chat torch and lmdeploy chat turbomind into lmdeploy chat command. Note the old commands still work
• Support auto backend feature in cli: lmdeploy chat .

BC-breaking (Optional)

No BC

Use cases (Optional)

pytorch backend

lmdeploy chat internlm/internlm-chat-7b --backend pytorch

turbomind

lmdeploy chat internlm/internlm-chat-7b

Checklist

1. Pre-commit or other linting tools are used to fix the potential lint issues.
2. The modification is covered by complete unit tests. If not, please add more unit tests to ensure the correctness.
3. If the modification has a dependency on downstream projects of a newer version, this PR should be tested with all supported versions of downstream projects.
4. The documentation has been modified accordingly, like docstring or example tutorials.

[zhyncs]

Hi @RunningLeon After the recent auto backend feature, there have been a few mishaps. For example, I compiled TurboMind using Python 3.9, so the turbomind.so is for Python 3.9. If I try to import lmdeploy with a version other than Python 3.9, TurboMind won't work. Previously, without auto backend, it would directly throw an error. Now it doesn't show an error but falls back to PyTorch instead. In most cases, this isn't a problem. However, if a user actually wants to use TurboMind and is unaware of the automatic fallback, unexpected results may occur. Therefore, I am thinking that making auto backend an option might be more appropriate so users can explicitly force it off.

[RunningLeon]

Hi @RunningLeon After the recent auto backend feature, there have been a few mishaps. For example, I compiled TurboMind using Python 3.9, so the turbomind.so is for Python 3.9. If I try to import lmdeploy with a version other than Python 3.9, TurboMind won't work. Previously, without auto backend, it would directly throw an error. Now it doesn't show an error but falls back to PyTorch instead. In most cases, this isn't a problem. However, if a user actually wants to use TurboMind and is unaware of the automatic fallback, unexpected results may occur. Therefore, I am thinking that making auto backend an option might be more appropriate so users can explicitly force it off.

@zhyncs hi, thanks for your feedback. There is a warning for this case:

lmdeploy/lmdeploy/archs.py

Lines 42 to 50 in 8a2fed8

	try:
	from lmdeploy.turbomind.supported_models import \
	is_supported as is_supported_turbomind
	turbomind_has = is_supported_turbomind(model_path)
	except ImportError:
	logger.warning(
	'Lmdeploy with turbomind engine is not installed correctly. '
	'You may need to install lmdeploy from pypi or build from source '
	'for turbomind engine.')

[zhyncs]

Hi @RunningLeon Yes, I did see this warning in source code, but in the actual usage process, seeing the warning did not attract attention. Should we consider raising the log level to error or as mentioned above, adding an option to force it off?

[lvhan028]

Hi, @zhyncs
Seamlessly falling back to the PyTorch engine when a model isn't supported by TurboMind is a key feature of LMDeploy.
Our goal is to offer LMDeploy as a comprehensive suite rather than just an individual engine, ensuring users a hassle-free experience when deploying models without having to worry about engine-specific issues.
We appreciate your suggestion regarding the log level and the potential for an additional optional argument. However, after careful consideration, we have decided to maintain our current approach.
Could you try to set log_level='warning' when you use lmdeploy?

[lvhan028]

@RunningLeon please add logs when fallback to pytorch engine

try: 
     from lmdeploy.turbomind.supported_models import \ 
         is_supported as is_supported_turbomind 
     turbomind_has = is_supported_turbomind(model_path) 
 except ImportError: 
     logger.warning( 
         'Lmdeploy with turbomind engine is not installed correctly. ' 
         'You may need to install lmdeploy from pypi or build from source ' 
         'for turbomind engine.') 

If turbomind is installed, but the model is not supported, there is no log to show the engine falllbacks to pytorch engine

[lvhan028]

This case failed

CUDA_VISIBLE_DEVICES=1 lmdeploy chat /workspace/models-140/Qwen/Qwen-7B-Chat/ --backend pytorch

[grimoire]

I think these warning should only appear when user intent to use turbomind (for example, sending a turbomindconfig)

[RunningLeon]

The default is using turbomind. Autobackend works only when backendconfig is None or TurbomindConfig.