[Feature Request] Verbose Logging support for onnxruntime-qnn Python package

[mvijayakquic]

Describe the feature request

onnxruntime-qnn python package do not have option for verbose logging.

onnxruntime-qnn C++ package we do have option for verbose logging, and we can get the debug logs similar to how we get from Native QNN.

Could you please enable the verbose logging feature for onnxruntime-qnn python as which help Debugging QNN EP better.

Describe scenario use case

Without the verbose log it's hard to know what is happening at QNN level.
When QNN EP fails for some reason with the python package we do not have any clue on what is the failure is and how we can map it with QNN EP issues.

[edgchen1]

Does this API allow you to set it?
https://onnxruntime.ai/docs/api/python/api_summary.html#onnxruntime.set_default_logger_severity

[mvijayakquic]

@edgchen1 Thanks for your reply, We are using it already it works for onnxruntime, and giving few details on onnxruntime-qnn but not in detail for onnxruntime-qnn, which is needed for debugging issue with qnn ep.

[edgchen1]

Can you elaborate on what debug logs you are looking for that are not showing up?

onnxruntime-qnn C++ package we do have option for verbose logging, and we can get the debug logs similar to how we get from Native QNN.

How are you enabling verbose logging from C++?

[CorbinFoucart]

+1 interest in documenting how to set verbose logging for qnn-ep

Ort::Session create_qnn_session(
    Ort::Env& env,
    const std::string& model_path,
    const std::string& htp_backend_path,
    bool allow_fallback,
    const std::unordered_map<std::string, std::string>& extra_options = {}) {
    Ort::SessionOptions session_options;
    session_options.SetIntraOpNumThreads(1);
    session_options.SetGraphOptimizationLevel(GraphOptimizationLevel::ORT_DISABLE_ALL);
    session_options.SetLogSeverityLevel(OrtLoggingLevel::ORT_LOGGING_LEVEL_VERBOSE);

    std::unordered_map<std::string, std::string> qnn_options = {
        {"backend_path", htp_backend_path},
        {"htp_performance_mode", "high_performance"},
        {"qnn_context_priority", "high"},
    };
    qnn_options.insert(extra_options.begin(), extra_options.end());

    session_options.AddConfigEntry(kOrtSessionOptionsDisableCPUEPFallback, allow_fallback ? "0" : "1");
    session_options.AppendExecutionProvider("QNN", qnn_options);
    auto session = Ort::Session(env, model_path.c_str(), session_options);
    logger.info("QNN Ort::Session created for model:", model_path);
    return session;
}

which doesn't seem to trigger any verbose logging from the QNN-EP.

[CorbinFoucart]

I figured out that this may be an android specific issue. I can see that verbose output is written with adb logcat but it is not sent to stdout by default. So as a fix, I've written a wrapper that sends the onnxruntime output to std::cout and I am able to generate verbose output.

void stdout_callback(void* logger_param, OrtLoggingLevel level, const char* category,
                         const char* logid, const char* code_location, const char* message) {
    std::cout << "[ORT " << category << "] " << message << std::endl;

...
 Ort::Env env(ORT_LOGGING_LEVEL_VERBOSE, "pipeline-test", stdout_callback, /*logger_param=*/nullptr);