[Tracking] Create a uniform schema for conversation template and embed into mlc-chat-config.json

[rickzx]

Overview

Currently, with every new model architecture supported, we need to manually add a conversation template object in c++, i.e. here. This is not very scalable, and we also need to make sure web-llm is catching up with every conversation template added, i.e. here.

We should instead create a uniform json schema for all conversation templates (we already have one here for MLC serve: https://github.com/mlc-ai/mlc-llm/blob/main/python/mlc_chat/protocol/conversation_protocol.py) and embed it into mlc-chat-config.json. This way both mlc-llm and web-llm can read the conversation template directly from mlc-chat-config.json, without needing to add a c++ template every time with a new model support.

Action Items

• Create a uniform json schema for all conversation templates and embed it into mlc-chat-config.json
• Update the current HF prebuilt weights with the new mlc-chat-config.json
• Clean-up the old C++ conv_template.cc
• Update docs on adding new conversation template
• Future support for Jinja chat template: example

Links to Related Issues and PRs

[tqchen]

this now landed