Instruct Templates & Chat Mode

Instruct Templates

Many modern LLMs are "instruction-tuned" and require specific formatting to understand the difference between a system prompt, user input, and its own response.

• Function: Instruct Templates automate this formatting. Mikupad comes with presets for some popular models (e.g., Mistral, Llama 3, ChatML).
• Selection: Choose the template that matches your model from the dropdown in the Parameters section.
• Usage:
  • Use the System Prompt (>_) and Instruct (i) buttons next to the dropdown to wrap selected text with the correct template strings.
  • Enable Chat Mode for automatic handling.
• Customization: Click the settings icon next to the dropdown to open the Template Editor, where you can modify existing templates or create your own.

Chat Mode

• Function: When Chat Mode is enabled, mikupad automatically manages the conversational structure for you.
• How it Works:
  1. When you press Predict, mikupad appends the template's "end of instruction" string (e.g., [/INST]).
  2. After the model finishes generating its response, mikupad automatically appends the "start of instruction" string (e.g., [INST]), ready for your next message.
• Use Case: This mode is perfect for question-answering, roleplaying, or any back-and-forth conversation with the model.

Chat Completions API

For OpenAI Compatible backends, you can enable the Chat Completions API. This option converts your prompt into a structured JSON format based on the selected Instruct Template, which is essential for backends that only support chat completion.