Skip to content

QAI AppBuilder(v3.0.0) for QAIRT SDK 2.48.40.260702

Latest
Latest

Choose a tag to compare

View all tags
@tim202503 tim202503 released this 09 Jul 10:59
v2.48.40
2beb443

QAI AppBuilder(v3.0.0) for QAIRT SDK 2.48.40.260702 Release Notes

OVERVIEW

QAI AppBuilder is an extension of Qualcomm AI Runtime SDK that simplifies
on-device AI deployment on Snapdragon processors. Build and run AI applications
with natural language prompts — no complex coding required.

KEY FEATURES

CORE CAPABILITIES

  • NPU/GPU/CPU Inference: Load and run QNN context binaries (.bin), model
    libraries (.dll), and SNPE DLC (.dlc) files on Snapdragon NPU
  • Dual-Language APIs: Full C++ and Python bindings for flexible integration
  • Cross-Platform Support: Windows on Snapdragon (ARM64) and Linux environments
    with unified API
  • AI-Driven Model Conversion: Convert PyTorch/ONNX models to QNN format
    (FP16/W8A16/W8A8/W4A8) via natural language chat
  • Pre-built Models: Automatically download and run optimized models from
    Qualcomm AI Hub

ADVANCED FEATURES

  • Large Language Models (Genie): Run Llama 3, Qwen 3, Gemma 2 fully offline
    via GenieAPIService (OpenAI-compatible API)
  • Multimodal Support: Vision, speech, text processing (Whisper ASR, MeloTTS,
    PP-OCR, super-resolution, object detection)
  • WebUI Applications: StableDiffusionApp, ImageRepairApp, GenieWebUI — all
    running on-device, no internet required
  • Skill Plugin System: Extend with hot-reloadable plugins via SKILL.md
  • Native & Float I/O: Native (quantized) and float I/O for maximum throughput;
    multi-graph sessions; LoRA adapter support

QAI MODELBUILDER

QAI ModelBuilder is a local-first AI workspace for Snapdragon X-Series PCs,
providing an integrated environment for model conversion, optimization, and
inference on the Snapdragon NPU.

MODELBUILDER FEATURES

  • App Builder Workbench: Run ready-to-use Model Packs on local NPU with one
    click; supports multi-variant precision switching, benchmarking, and
    side-by-side comparison
  • Model Builder Skill: Convert PyTorch and ONNX models to QNN format via
    natural language chat; supports multiple precision levels (FP16, INT8,
    W8A16, etc.); targets small-to-medium models under ~2 GB
  • Multi-Agent Chat Interface: Real-time WebSocket streaming with automatic
    SSE fallback; full markdown rendering; discussion mode with multiple agent
    personas
  • Built-in Models: On-device ASR (Whisper Base, Zipformer ZH) and TTS
    (MeloTTS ZH) models
  • NPU Inference Daemon: GenieAPIService management for local model serving
  • Secure API Key Storage: OS keyring integration for credential management
  • Download Center: Access optimized NPU models from Qualcomm AI Hub

MODELBUILDER ARCHITECTURE

  • Clean architecture with bounded contexts (chat, app_builder, model_builder)
  • Separation of concerns: application layer, protocol adapters, frontend
  • Frontend: Vue 3 + Vite for responsive UI
  • Backend: Python-based inference and model conversion pipeline

SAMPLE APPLICATIONS & COVERAGE

The repository includes comprehensive samples across six major categories:

AUDIO

  • Text-to-Speech (TTS) with MeloTTS
  • Automatic Speech Recognition (ASR) with Whisper and Zipformer
  • Audio classification

COMPUTER VISION

  • Image classification (Inception V3, ResNet, etc.)
  • Object detection (YOLOv8, YOLO-NAS)
  • Image segmentation
  • Pose estimation

GENERATIVE AI

  • Stable Diffusion v1.5, v2.1, v3, v3.5
  • Image generation and manipulation

MULTIMODAL

  • Optical Character Recognition (OCR) with PP-OCR
  • Image embeddings and CLIP models
  • Vision-Language Models (VLM) like Qwen2.5-VL

WEBUI APPLICATIONS

  • ImageRepairApp: Image inpainting and restoration
  • StableDiffusionApp: Text-to-image generation UI
  • GenieWebUI: Chat interface for LLM interaction
  • All running on-device with Gradio framework

GENIE LLM SERVICE

  • OpenAI-compatible API for local LLM serving
  • Support for Llama 3, Qwen 3, Phi, Gemma 2
  • Fully offline inference on Snapdragon NPU

RECENT IMPROVEMENTS

  • Fixed MODEL_HELP_URL in Python samples
  • Refactored WebUI apps with Gradio 403 proxy error fixes
  • Improved Linux QNN library loading
  • Enhanced cross-platform compatibility
  • Optimized sample apps and launcher scripts
  • Added async model load and parallel inference for cross-process mode
  • Reorganized samples by AI task category
  • Support for overriding Hexagon DSP architectures via QAI_DSP_ARCHES
  • Cross-compilation support via QAI_CMAKE_TOOLCHAIN_FILE
  • Fixed RPATH settings on appbuilder and geniebuilder pybind modules
  • Added OnnxRuntimeContext for direct ONNX inference on HTP and CPU
  • Improved lifecycle management with deterministic release and fork safety

SYSTEM REQUIREMENTS

DEVELOPMENT ENVIRONMENT

  • Python: 3.10+ (x64)
  • Visual Studio 2022 Community (for model conversion)
  • QAIRT SDK: 2.45+

HARDWARE TARGETS

  • Snapdragon X Elite
  • Snapdragon X2 Elite
  • Snapdragon X Plus 8-Core

SUPPORTED PLATFORMS

  • Windows on Snapdragon (ARM64)
  • Linux (QCS8550, QCM6490, Ubuntu)

RUNTIME REQUIREMENTS

  • No internet connection required during inference (fully on-device)
  • NPU/GPU/CPU available for inference acceleration
  • Sufficient storage for model files (varies by model size)

INSTALLATION

PYTHON PACKAGE (QAI AppBuilder Core)
pip install qai-appbuilder

C++ PACKAGE
Download precompiled packages from GitHub Releases

QAI MODELBUILDER (Visual Interface)
Windows on Snapdragon:
1. Download qaimodelbuilder.zip from GitHub Releases
2. Extract and run: Setup.bat
3. Start the application: Start.bat

AUTOMATED SETUP
Use QAI AppBuilder Launcher batch file for streamlined configuration

TWO CORE SKILLS

SKILL 1: AI HUB MODEL RUN
Use this skill when the model you need already exists on Qualcomm AI Hub.

Features:

  • Supported Formats: QNN_CONTEXT_BINARY (.bin), QNN_DLC (.dlc), ONNX,
    VOICE_AI, TFLITE
  • Inference Engine: QNNContext on NPU/HTP; onnxruntime for CPU baseline
  • Target Devices: Snapdragon X Elite, X2 Elite, X Plus 8-Core
  • Typical Workflow: Auto-detect chipset → fetch from AI Hub → extract →
    run inference
  • No Conversion Needed: Pre-compiled .bin/.dlc load directly; first-run
    graph compilation takes 5-60s
  • CPU Baseline: Optional accuracy comparison (cosine similarity > 0.999 for
    float models)

Example Prompts:
"Download Inception V3 from AI Hub and run inference on my photo"
"Run YOLOv8 object detection on this image using the NPU"
"Use the pre-built Whisper model to transcribe my audio file"

SKILL 2: MODEL BUILDER
Use this skill when you have a custom PyTorch or ONNX model NOT on AI Hub.

Features:

  • Input Formats: PyTorch (.pt, .pth) → ONNX export, or existing ONNX (.onnx)
  • Output Formats: QNN context binary (.bin), QNN model library (.dll),
    SNPE DLC (.dlc)
  • Supported Precisions: FP16, FP32, W8A16, W8A8, W8A8B8, W4A16, W4A8
  • Auto Pipeline: Export → ONNX inspection → operator patching → QNN
    conversion → context binary generation → inference → accuracy validation
  • Operator Patching: Automatically detects and patches unsupported operators
    (Einsum, GridSample, ScatterND, Mod, Floor, etc.) with QNN-compatible
    equivalents
  • Accuracy Validation: Cosine similarity ≥0.99 (FP16/FP32) or ≥0.95 (INT8)
  • Model Scope: Best suited for small-to-medium models (recommended < 2 GB);
    LLM conversion not yet supported

Requirements:

  • QAIRT SDK 2.45+
  • Visual Studio 2022 Community
  • Python 3.10 x64

SUPPORTED FORMATS & MODELS

INPUT FORMATS

  • PyTorch (.pt, .pth)
  • ONNX (.onnx)
  • QNN Context Binary (.bin)
  • QNN DLC (.dlc)
  • TensorFlow Lite (.tflite)

QUANTIZATION PRECISIONS

  • FP16, FP32
  • W8A16, W8A8, W8A8B8
  • W4A16, W4A8

SUPPORTED MODELS

  • Large Language Models: Llama 3, Qwen 3, Gemma 2, Phi
  • Vision: Inception V3, YOLOv8, YOLO-NAS, ResNet, Qwen2.5-VL
  • Speech: Whisper (ASR), MeloTTS (TTS), Zipformer
  • OCR: PP-OCR
  • Generative: Stable Diffusion v1.5, v2.1, v3, v3.5
  • Other: Super-resolution, Object Detection, Pose Estimation, Image
    Segmentation, CLIP embeddings

LICENSE

BSD 3-Clause License

REPOSITORY & COMMUNITY

GitHub: https://github.com/qualcomm/qai-appbuilder
License: BSD 3-Clause License

DOCUMENTATION & RESOURCES

GETTING STARTED

  • Quick Start: pip install qai-appbuilder
  • Sample Applications: Available in repository samples/ directory
  • Tutorials: Refer to documentation and sample code
Assets 22
Loading