QAI AppBuilder for QAIRT SDK 2.47.0.260601

QAI AppBuilder for QAIRT SDK 2.47.0.260601:

What's Changed

• Genieapiservice arm64 linux by @jingweif in #94
• Linux dev by @biduan in #95
• Fix Qwen3-VL preprocess issue by @biduan in #96
• Lock sg_model_map to make QnnSampleApp pool access thread-safe by @Francis235 in #103

Full Changelog: 2.46.0...v2.47.0

QAI AppBuilder for QAIRT SDK 2.46.0.260424

QAI AppBuilder for QAIRT SDK 2.46.0.260424

QAI AppBuilder for QAIRT SDK 2.45.0.260326

QAI AppBuilder Release for QAIRT SDK 2.45.0.260326

QAI AppBuilder for QAIRT SDK 2.44.0.260225

This is a temporary version and has not undergone complete testing and verification.

Start Service:

GenieService_v2.3.7\GenieAPIService.exe -c models\Qwen2.0-7B-SSD\config.json -l -n -1

QAI AppBuilder for QAIRT SDK 2.42.0.251225

Note

• The meaning of some keywords in Python extension files.
  • "arm64" is for ARM64 Python, "amd64" is for X64 Python.
  • "v73" is for Snapdragon X Elite platform, "v81" is for Snapdragon X2 Elite platform.
  • "2.42.0" is for QAIRT SDK "2.42.0.251225".
  • "cp312" is for Python 3.12.x.

For Snapdragon X Elite platform:

ARM64 Python: qai_appbuilder-2.42.0.73-cp312-cp312-win_arm64.whl

X64 Python: qai_appbuilder-2.42.0.73-cp312-cp312-win_amd64.whl

For Snapdragon X2 Elite platform:

ARM64 Python: qai_appbuilder-2.42.0.81-cp312-cp312-win_arm64.whl

X64 Python: qai_appbuilder-2.42.0.81-cp312-cp312-win_amd64.whl

QAI AppBuilder for QAIRT SDK 2.41.0.251128

QAI AppBuilder for QAIRT SDK 2.41.0.251128

QAI AppBuilder for QAIRT SDK 2.40.0.251030

QAI AppBuilder for QAIRT SDK 2.40.0.251030

QAI AppBuilder for QAIRT SDK 2.38.0.250901

QAI AppBuilder

README
BUILD

GenieAPIService:

Note: Please note that the input length must not exceed the maximum number of tokens reserved for the input, which means it cannot exceed the model's maximum context length minus the value set for '--min_output_num'. You can invoke the Text Splitter to send the input text to the server for segmentation, and then sequentially pass the split segments to the LLM to complete the question-answering process.
Note: It is recommended to disable thinking mode when using the tools call function.

Reference:

GenieAPIService
API
BUILDER
Model

QAI AppBuilder for QAIRT SDK 2.37.0.250724

QAI AppBuilder

README
BUILD

GenieAPIService:

Note: Please note that the input length must not exceed the maximum number of tokens reserved for the input, which means it cannot exceed the model's maximum context length minus the value set for '--min_output_num'. You can invoke the Text Splitter to send the input text to the server for segmentation, and then sequentially pass the split segments to the LLM to complete the question-answering process.
Note: It is recommended to disable thinking mode when using the tools call function.

Reference:

GenieAPIService
API
BUILDER
Model

QAI AppBuilder for QAIRT SDK 2.36.0.250627

Update: We support Genie now. Here is sample code:

import os
from qai_appbuilder import (GenieContext)

lib_path = "qai_libs"  # The QAIRT runtime libraries path.
if not lib_path in os.getenv('PATH'):
    lib_path = os.getenv('PATH') + ";" + lib_path + ";"
    os.environ['PATH'] = lib_path

def response(text):
    # Print model generated text.
    print(text, end='', flush=True)
    return True

# Initialize the model.
config = "genie\\python\\models\\IBM-Granite-v3.1-8B\\config.json"
dialog = GenieContext(config)

# Ask question.
prompt = "<|start_of_role|>system<|end_of_role|>You are a helpful assistant.<|end_of_text|> <|start_of_role|>user<|end_of_role|>How to fish?<|end_of_text|> <|start_of_role|>assistant<|end_of_role|>"
dialog.Query(prompt, response)

For Python(ARM64) projects, install Python extension for Python 3.12.8(ARM64) with command below:

pip install qai_appbuilder-2.34.0-cp312-cp312-win_arm64.whl

Here is ARM64 Python download links:
https://www.python.org/ftp/python/3.12.8/python-3.12.8-arm64.exe

For Python(x64) projects, the below wheels are ARM64EC architecture which are compatible with the x64 version of Python. It can work on WoS device well. You need to run with the QNN libraries in 'C:\Qualcomm\AIStack\QAIRT\2.34.0.250424\lib\arm64x-windows-msvc':

qai_appbuilder-2.34.0-cp312-cp312-win_amd64.whl

Here are x64 Python download links:
https://www.python.org/ftp/python/3.12.8/python-3.12.8-amd64.exe

For C++ projects, use the files in 'QAI_AppBuilder-win_arm64-QNN2.34.0-Release.zip':

libappbuilder.dll
libappbuilder.lib
QAIAppSvc.exe
include\LibAppBuilder.hpp