Do we support vllm based custom model endpoint deployment for Kong AI Proxy plugin ?

[Colstuwjx]

Hi team,

I've found a similar issue in stackoverflow and it seems that we didn't support yet, could you help to confirm this ?

[Water-Melon]

Hi @Colstuwjx , although I'm not familiar with the detailed functionality of AI-proxy, based on your error, I think setting your config.route_type to llm/v1/chat should resolve the issue. Could you give it a try?

[subnetmarco]

It seems that this issue has been solved in StackOverflow, closing it now but feel free to re-open if the issue persists.