[Performance] createSession() slow on release 1.15 and 1.17.1 as compare to 1.14

[sarojlo]

Describe the issue

Loading model using createSession is slower than v 1.14,, It took 779 ms to createSession in 1.14 while 1.15 need 1319ms

To reproduce

    m_ort = OrtGetApiBase()->GetApi(ORT_API_VERSION);
    m_ort->CreateEnv(ORT_LOGGING_LEVEL_WARNING, "XXXXX", &m_env);
    m_ort->CreateSessionOptions(&m_sessionOptions);
    m_ort->SetIntraOpNumThreads(m_sessionOptions, 1);
    m_ort->SetSessionGraphOptimizationLevel(m_sessionOptions, ORT_ENABLE_ALL);
    m_ort->SetSessionLogSeverityLevel(m_sessionOptions, 3);
   m_ort->CreateSession(m_env, model_file, m_sessionOptions, &m_session);

Urgency

No response

Platform

Windows

OS Version

WIndows 11

ONNX Runtime Installation

Built from Source

ONNX Runtime Version or Commit ID

1.15.1

ONNX Runtime API

C

Architecture

X64

Execution Provider

Default CPU

Execution Provider Library Version

No response

Model File

No response

Is this a quantized model?

Yes

[hariharans29]

Can you please define "slow" with numbers before and after ?

[pranavsharma]

The easiest thing to do would be to disable all optimizers and compare. This would rule out the optimizers at least.

[For ORT team] As an aside I think it'll be really great if ORT printed which optimizers ran during session initialization. Then one can easily compare between different vers if more optimizers ran for the same model as a data point or which didn't.

[hariharans29]

The easiest thing to do would be to disable all optimizers and compare. This would rule out the optimizers at least.

[For ORT team] As an aside I think it'll be really great if ORT printed which optimizers ran during session initialization. Then one can easily compare between different vers if more optimizers ran for the same model as a data point or which didn't.

Isn't that info already available:

onnxruntime/onnxruntime/core/optimizer/graph_transformer.cc

Line 15 in 9443366

LOGS(logger, INFO) << "GraphTransformer " << Name() << " modified: " << modified << " with status: " << status;

?

[sarojlo]

Do you want me to run experiment with m_ort->SetSessionGraphOptimizationLevel(m_sessionOptions, ORT_DISABLE_ALL)? and share the performance profile json?

[hariharans29]

Do you want me to run experiment with m_ort->SetSessionGraphOptimizationLevel(m_sessionOptions, ORT_DISABLE_ALL)? and share the performance profile json?

You can try doing that. You can also try enabling INFO level logs to enable the logs I pasted above to see the diff.

Btw - what is the impact of this ? I know session creation time has nearly doubled, but unless you have a scenario where you need to create many sessions, the impact can't be significant ?

[cbourjau]

I didn't create this issue, but session creation time is important to us, too. We use onnxruntime for on-the-fly constant folding in Spox and higher-level in-house solutions. We want to push our tooling to a point were we can deliver a similar interactive UX as NumPy and Pandas.

[github-actions]

This issue has been automatically marked as stale due to inactivity and will be closed in 30 days if no further activity occurs. If further support is needed, please provide an update and/or more details.