Pinning the process to specific cores when inferencing using onnx runtime

[anandvishes]

Hi,

I am trying to pin the process to specific cores using 'taskset' when inferencing using onnx runtime.
When pinned to specific cores, the minimum inference time and median of all inference time are not near to each other(the inference time varies).
But when no cores are pinned, the minimum and median are near to each other.
What is the right way to pin the cores in onnx runtime? Even when the cores are pinned those particular cores alone are not used.
TIA.

PFB the inference timing table

<style> </style>

Core Pinning	Min(ms)	Max(ms)	Median(ms)	Mean(ms)
Cores pinned	11.27	35.40	23.31	22.56
No pinning	10.81	12.01	10.92	10.94

[snnn]

ONNX Runtime doesn't work well with "taskset" if you didn't explicitly set the number of threads which onnxruntime should create. So, did you set the option?

  /** \brief Sets the number of threads used to parallelize the execution within nodes
  *
  * When running a single node operation, ex. add, this sets the maximum number of threads to use.
  *
  * \note If built with OpenMP, this has no effect on the number of threads used. In this case
  *       use the OpenMP env variables to configure the number of intra op num threads.
  *
  * \param[in] options
  * \param[in] intra_op_num_threads Number of threads to use<br>
  *   A value of 0 will use the default number of threads<br>
  *
  * \snippet{doc} snippets.dox OrtStatus Return Value
  */
  ORT_API2_STATUS(SetIntraOpNumThreads, _Inout_ OrtSessionOptions* options, int intra_op_num_threads);

[anandvishes]

Thank you.
I am able to get the desired result when intra_op_num_threads = number of cores pinned.