QAI AppBuilder for QAIRT SDK 2.36.0.250627

Update: We support Genie now. Here is sample code:

import os
from qai_appbuilder import (GenieContext)

lib_path = "qai_libs"  # The QAIRT runtime libraries path.
if not lib_path in os.getenv('PATH'):
    lib_path = os.getenv('PATH') + ";" + lib_path + ";"
    os.environ['PATH'] = lib_path

def response(text):
    # Print model generated text.
    print(text, end='', flush=True)
    return True

# Initialize the model.
config = "genie\\python\\models\\IBM-Granite-v3.1-8B\\config.json"
dialog = GenieContext(config)

# Ask question.
prompt = "<|start_of_role|>system<|end_of_role|>You are a helpful assistant.<|end_of_text|> <|start_of_role|>user<|end_of_role|>How to fish?<|end_of_text|> <|start_of_role|>assistant<|end_of_role|>"
dialog.Query(prompt, response)

For Python(ARM64) projects, install Python extension for Python 3.12.8(ARM64) with command below:

pip install qai_appbuilder-2.34.0-cp312-cp312-win_arm64.whl

Here is ARM64 Python download links:
https://www.python.org/ftp/python/3.12.8/python-3.12.8-arm64.exe

For Python(x64) projects, the below wheels are ARM64EC architecture which are compatible with the x64 version of Python. It can work on WoS device well. You need to run with the QNN libraries in 'C:\Qualcomm\AIStack\QAIRT\2.34.0.250424\lib\arm64x-windows-msvc':

qai_appbuilder-2.34.0-cp312-cp312-win_amd64.whl

Here are x64 Python download links:
https://www.python.org/ftp/python/3.12.8/python-3.12.8-amd64.exe

For C++ projects, use the files in 'QAI_AppBuilder-win_arm64-QNN2.34.0-Release.zip':

libappbuilder.dll
libappbuilder.lib
QAIAppSvc.exe
include\LibAppBuilder.hpp