RKLLAMA 0.0.3

🚀 Rkllama Version: 0.0.3

Key Updates

🌐 Extended Compatibility

• Full Model Support: RKLLAMA now fully supports a wide range of models, including DeepSeek, Qwen, Llama, and many others. This ensures broader applicability and flexibility for users. 🤝

⚡ Enhanced Performance

• Tokenized Inputs: Inputs are now tokenized before being sent to the model, replacing raw prompts. This optimization significantly boosts response speed and efficiency. 🏃‍♂️

📂 Modelfile System

• Automated Initialization: Inspired by Ollama, the new Modelfile system allows automatic initialization of both the tokenizer and chattemplate by simply providing the HuggingFace path. 🔧
• Customizable Parameters: Easily adjust model parameters such as temperature, location, and system prompt for tailored performance. 🎛️

🗂️ Simplified Organization

• Automatic Folder Creation: Models are now organized into dedicated folders, automatically created when you run the rkllama list command. This streamlines model management. 📁
• Effortless Model Launch: Launch models using just the model name, as .rkllm files are directly referenced in the Modelfile. 🛫

🔄 Automatic Modelfile Creation

• Seamless Integration: When using the pull command, the Modelfile is generated automatically. For previously downloaded models, run a one-time command (e.g., rkllama run modelname file.rkllm huggingface_path) to create the Modelfile. 🔄

🌟 Future Enhancements

• Customization and Optimization: Upcoming updates will introduce further customization options for the chattemplate and additional hyperparameters (e.g., top_k) to enhance user experience. 🎨

📚 Upgrade Guide

• Rebuild Architecture: If you have already downloaded models and wish to avoid reinstalling everything, follow our guide: Rebuild Architecture. 📖