Skip to content

llmexport v0.0.3

Choose a tag to compare

View all tags
@wangzhaode wangzhaode released this 03 Sep 07:03
· 4 commits to master since this release
v0.0.3
1202b93

Release Notes - v0.0.3

🎉 Major Updates

This release represents a significant milestone with comprehensive architecture improvements and extensive new model support. The codebase has been completely restructured and synchronized with the latest MNN framework.

🚀 New Features

Model Support

  • ✅ SmolLM Series: Added support for SmolLM models with optimized configurations
  • ✅ MobileLLM Series: Enhanced support for mobile-optimized language models
  • ✅ BGE Models: Added support for bge-small embedding models
  • ✅ OpenELM: Support for Apple's OpenELM model series

Quantization Enhancements

  • 🔥 AWQ Quantization: Full implementation of AWQ (Activation-aware Weight Quantization)
  • 🔥 Symmetric Quantization: Added symmetric quantization support for improved performance
  • 🔥 Mixed Quantization: New mixed quantization strategies for optimal model compression
  • 🔥 HQQ Quantization: Half-Quadratic Quantization support added

Architecture Improvements

  • 📁 Modular Utils: Complete reorganization with dedicated utility modules:
    • Audio processing utilities (audio.py)
    • Vision model handling (vision.py)
    • GGUF file support (gguf/)
    • Advanced quantization modules
    • MNN conversion utilities
    • ONNX optimization tools

Enhanced Capabilities

  • 🎵 Audio Models: Added support for audio-enabled models (Qwen2-Audio, etc.)
  • 👁️ Vision Models: Enhanced vision model support with specialized processing
  • 🔧 LoRA Integration: Improved LoRA weight handling and merging
  • 🎯 Model Mapping: Advanced model architecture mapping system

🐛 Bug Fixes

  • Embedding Loading: Fixed critical embedding loading issues
  • ONNX Dynamic Axis: Resolved dynamic axis configuration problems
  • Linear Layer Bias: Fixed duplicate naming issues in ONNX export for Linear and bias operations
  • Model Compatibility: Enhanced compatibility across different model architectures

📚 Documentation Updates

  • README Optimization: Completely restructured README with professional badges, clear installation guides, and comprehensive feature documentation
  • Model Downloads: Added extensive model download links for both ModelScope and Hugging Face
  • Popular Models: Updated with latest high-demand models including:
    • DeepSeek-R1-1.5B-Qwen
    • Qwen2.5 series (0.5B, 1.5B)
    • GPT-OSS-20B
    • Qwen3-4B-Instruct-2507

🔧 Technical Improvements

  • Code Restructuring: Major refactoring with 10,297 lines added and modular architecture
  • Performance Optimization: Enhanced inference speed and memory efficiency
  • Cross-platform Support: Improved compatibility across different deployment platforms
  • Error Handling: Better error reporting and debugging capabilities

📦 Installation & Usage

# Install latest version
pip install llmexport==0.0.3

# Quick export example
llmexport --path Qwen2.5-1.5B-Instruct --export mnn --quant_bit 4

🔗 Related Projects

⚠️ Breaking Changes

This version includes significant architectural changes. Please review the updated documentation and examples when upgrading from previous versions.

🙏 Acknowledgments

Special thanks to all contributors and the MNN team for their continuous support and collaboration in making this release possible.

Full Changelog: v0.0.2...v0.0.3

Assets 2
Loading