[Feature Request] Add support for CogVLM2

[samaritan1998]

Following up on Cogvlm, CogVlm2 is here: https://github.com/THUDM/CogVLM2
Easily one of the best open-source multimodal model, that is competitive to GPT-4 and Gemini.
https://github.com/THUDM/CogVLM2?tab=readme-ov-file#benchmark
The community would be grateful for having a RTP-LLM engine for such a good model.

[ySingularity]

Thanks for suggestions.
CogVLM has a completely different process due to the vision experts. It seems difficult to support CogVLM in the short term.

[samaritan1998]

It is indeed very difficult, but I see that tensorrt-llm already supports cogvlm. Can we copy it over? Would this be helpful for you?

[samaritan1998]

对应的PR> NVIDIA/TensorRT-LLM#1554

[ySingularity]

Thanks for information. We are working on it.