Pass SessionOptions to XnnpackProviderFactoryCreator. #13318

wejoncy · 2022-10-14T05:37:30Z

Description

To pass session_options to Xnnpack EP via XnnpackProviderFactoryCreator for Initializing xnnpack's threadpool.

If you want to use different threadpool size or even disable xnnpack's threadpool, just setting intra_threadpool to 1 by xnnpack EP's provider_options.

Motivation and Context

skottmckay · 2022-10-17T21:29:52Z

include/onnxruntime/core/framework/execution_provider.h

@@ -299,6 +300,14 @@ class IExecutionProvider {
  /** Does the EP support concurrent calls to InferenceSession::Run to execute the model.
   */
  virtual bool ConcurrentRunSupported() const { return true; }
+  /** Some session option values (default or user provided) may not work with some EPs.
+   * The EP should alter the SessionOptions object fit its implementation to avoid potential crash.


This probably needs a design discussion. Not a huge fan of letting all EPs alter the session options in arbitrary ways. What if two conflict in their requirements? Whichever runs second is going to win, and one will always be broken.

I can understand EPs being able to validate whether they can run with the session options and report an issue. I also prefer returning an error instead of silently 'fixing' an invalid option as that way the user knows we're not running in the way they might think we are.

This method also seems to be serving two purposes. One is to allow the EP to validate the options, the other is to pass them through. The naming does not call out the second usage, and the method being non-const would I think be unexpected given that.

@pranavsharma thoughts?

I think it would be better to let the EP copy a new one and legalize for that one only instead of the shared one. Then the behavior will not be shared between EPs. Originally in #11138 , there is only CUDA and DML to be specially handled, so it would not be a big problem at that time.

But what if session options of ep specific has side-effect for another EP?

@wejoncy Then it will be codesign scenario, the dev need to maintain both EPs.

Let's take this offline in an email.

onnxruntime/core/framework/config_options.h

…to EP impl." This reverts commit 873fae8.

wejoncy · 2022-10-18T08:41:47Z

Hi @pengwa Could you please take a look if this renaming for CudaProviderCreating has any effect on training?

This reverts commit dc429d3.

…c logic to EP impl."" This reverts commit 6332740.

skottmckay · 2022-10-19T07:15:55Z

include/onnxruntime/core/framework/execution_provider.h

+   * @param session_options The SessionOptions object to be altered.
+   * @param logger The EP write the reason of the altering to it.
+   */
+  virtual common::Status CheckSessionOptions(const SessionOptions& /*session_options*/) {


nit: add an empty line between the previous function and the start of the comment for this one.

The comment needs updating as well as the EP should not alter the SessionOptions object.
I would suggest adding an @return saying the EP should return an error if it's incompatible with the SessionOptions that specifies which option is problematic so the user knows what needs to be fixed.

'Check' doesn't really convey that the EP might update its internals based on the session options, which is why this method is non-const. Maybe ProcessSessionOptions would be slightly better.

We do have process EP itself in this function, so we can't mark this method as const.
This function should works with two purpose: 1 check if EP could work SessionOptions, 2. Adjust EP-self to make compatiable to run normally. The first one is compulsory, the second is not. How about combine them together? say CheckSessionOptionsAndProcess

'Process' is fairly generic. 'checking' is a form of processing, as is updating say threadpool sizes. I don't think anything is 'compulsory' - it's up to the EP to decide what it wants to do, so I'm not sure I see value in the longer name. If it was compulsory I would have expected every single EP was required to implement it.

Can we rename it to ValidateSessionOptions? Any reason we can't or shouldn't make it const? Do we want the EP to do something when this function is called?

I guess we don't need this API any more since we agreed to go with the factory based solution, right?

Not quite sure if this condition based on EP-in-factory solution works as expected.

For example, we can create factory and EP with python-binding

onnxruntime/onnxruntime/python/onnxruntime_pybind_schema.cc

Line 65 in aefc82b

onnxruntime::DMLProviderFactoryCreator::Create(0),

However, it didn't touch any sessionOptions here, which means we have to pass nullptr to xxFactory::create, Then we create EP successfully, But it will lead to troubles or even crash.

Any thoughs on this question? Thanks

skottmckay · 2022-10-19T07:16:51Z

onnxruntime/core/providers/dml/DmlExecutionProvider/src/ExecutionProvider.cpp

+        // DML's memory is not byte addressable and hence mem pattern doesn't work.
+        if (so.enable_mem_pattern) {
+            return ORT_MAKE_STATUS(ONNXRUNTIME,
+                                FAIL,


nit: indentation is off. please use clang-format

skottmckay · 2022-10-19T07:18:08Z

onnxruntime/core/providers/xnnpack/xnnpack_execution_provider.cc

+  if (so.execution_mode != ExecutionMode::ORT_SEQUENTIAL) {
+    return ORT_MAKE_STATUS(ONNXRUNTIME, FAIL,
+                           "Parallel execution mode does not support the Xnnpack Execution Provider. "
+                           "Please set the execution mode as sequential for this "
+                           "session since it uses the Xnnpack Execution Provider.");
+  }


What's the reason for this? For CUDA it's due to how the CUDA streams work. What about the xnnpack kernel execution would not work with the parallel executor?

For all xnn_setup_xxxx like functions, it will modify xnn_operator_t.

We prevent concurrent calls to Run to avoid an individual node being executed concurrently for that reason. However, the parallel executor is not going to execute an individual node concurrently - it will potentially execute different nodes concurrently which would all be separate instances of the kernel.

skottmckay · 2022-10-19T07:20:14Z

onnxruntime/core/providers/xnnpack/xnnpack_execution_provider.cc

+  }
+  // 0 means user didn't set the value.


nit: leave a space between completely separate blocks so it's easier to skim between them. when they're joined together it takes time to process whether it's a new 'if' or a continuation of the previous one (like an 'else if' or 'if').

skottmckay · 2022-10-19T07:27:43Z

onnxruntime/core/providers/xnnpack/xnnpack_execution_provider.cc

+  }
+  if (so.intra_op_param.allow_spinning && so.intra_op_param.thread_pool_size > 1) {
+    LOGS_DEFAULT(WARNING)
+        << "XNNPACK EP utilize pthreadpool for multi-threading. So, if allow_spinning on ORT's"


suggestion:
The XNNPACK EP utilizes an internal pthread-based thread pool for multi-threading.
If ORT's thread pool size is > 1 and spinning is enabled there will be contention between the two thread pools, and performance will suffer.
Please set either intra_op_param.allow_spinning to 0 in the SessionOption config params, or the ORT intra-op threadpool size to 1 and try again.

skottmckay · 2022-10-19T07:28:22Z

onnxruntime/core/providers/xnnpack/xnnpack_execution_provider.cc

+  }
+  // 0 means user didn't set the value.
+  if (so.intra_op_param.thread_pool_size > 1 && info_.xnn_thread_pool_size == 0) {
+    LOGS_DEFAULT(INFO) << "XNNPACK pool size is not set. Using ORT's thread-pool size as default:";


nit: include the size in the message

skottmckay · 2022-10-19T07:30:45Z

onnxruntime/test/framework/inference_session_test.cc

-  // execution mode is sequential since we have registered the CUDA EP
-  // (which isn't supported by the parallel execution mode)
-  ASSERT_TRUE(so_queried.execution_mode == ExecutionMode::ORT_SEQUENTIAL);
+  ASSERT_FALSE(status.IsOK());


Is there any point to keeping this test?

you are right, this test will be covered with checksessionoption test.

skottmckay · 2022-10-19T07:31:40Z

onnxruntime/core/providers/xnnpack/xnnpack_execution_provider.cc

  if (info.xnn_thread_pool_size > 1) {
    // pthreadpool is independent of ort-threadpoool, so we have to disable cpu spinning for ort-threadpool.
    // otherwise, the pthreadpool will be starved and harm performance a lot.
    xnnpack_thread_pool_ = pthreadpool_create(static_cast<size_t>(info.xnn_thread_pool_size));
  }
 }

+common::Status XnnpackExecutionProvider::CheckSessionOptions(const SessionOptions& so) {


Would be good to add a unit test that validates this new code.

Added unit-test for Xnnpack/DML/CUDA with different configs settings on execution_mode and enable_mem_pattern

pranavsharma · 2022-12-09T03:18:14Z

include/onnxruntime/core/session/onnxruntime_c_api.h

@@ -3500,6 +3500,7 @@ struct OrtApi {
   *
   * XNNPACK supported keys:
   *   "intra_op_num_threads": number of thread-pool size to use for XNNPACK execution provider.
+   *      default value is 0, which means to use the global thread-pool size.


Global has a different meaning. Did you mean "session" here?

I see. ORT can have multiple session-configs

It's because we've the concept of a global threadpool and a session threadpool. Here you mean session since you're passing session options.

Got it. No wonder the web side writes it only takes global threadpool.

pranavsharma · 2022-12-09T03:35:28Z

onnxruntime/core/providers/xnnpack/xnnpack_execution_provider.cc

+  int xnn_thread_pool_size = info.xnn_thread_pool_size;
+  int ort_thread_pool_size = info.session_options ? info.session_options->intra_op_param.thread_pool_size : 1;
+  bool allow_intra_op_spinning =
+      info.session_options && info.session_options->config_options.GetConfigOrDefault(


Is it expected to turn off spinning if session options is null? In the regular inferencing workflow spinning is turned on by default.

Thanks. It's reasonable to keep alignment with the default value.

pranavsharma · 2022-12-09T06:31:44Z

Please update PR description to reflect the latest changes as it seems out of date.

wejoncy · 2022-12-09T09:19:26Z

Please update PR description to reflect the latest changes as it seems out of date.

sure

### Description To pass session_options to Xnnpack EP via `XnnpackProviderFactoryCreator` for Initializing xnnpack's threadpool. If you want to use different threadpool size or even disable xnnpack's threadpool, just setting intra_threadpool to 1 by xnnpack EP's provider_options. ### Motivation and Context Co-authored-by: Guangyun Han <guangyunhan@microsoft.com> Co-authored-by: Jicheng Wen <jicwen@microsoft.com>

* Update libclang version for building C/C++ API docs * Temporarily add a workflow dispatch * Fix libclang-cpp too * Fix libclang-cpp again * Upgrade doxygen * Upgrade doxygen again * Upgrade doxygen yet again * Update Doxygen config * Temporarily disable warning as error to test remainder of workflow * Add path to upload-pages-artifact * Remove doxygen binaries after using * Pass SessionOptions to XnnpackProviderFactoryCreator. (microsoft#13318) ### Description To pass session_options to Xnnpack EP via `XnnpackProviderFactoryCreator` for Initializing xnnpack's threadpool. If you want to use different threadpool size or even disable xnnpack's threadpool, just setting intra_threadpool to 1 by xnnpack EP's provider_options. ### Motivation and Context Co-authored-by: Guangyun Han <guangyunhan@microsoft.com> Co-authored-by: Jicheng Wen <jicwen@microsoft.com> * Allow using separate GPT2 decoder subgraphs for the initial run and the subsequent runs in BeamSearch/GreedySearch (microsoft#13914) * Add two daily build jobs to validate some extra build configs (microsoft#13921) ### Description Add two daily build jobs to validate some extra build configs ### Motivation and Context To catch issues like: microsoft#13893 * Output cache stats (microsoft#13937) ### Description Output cache stats * Add support for other data types to Split CPU kernel. (microsoft#13900) Split copies data - we can add support for all data types without too much binary size impact by using data type size-based implementations. The DispatchStridedCopy() function used here does this. * Build docs with jekyll * [DML EP] Disable DML Graph Fusion for lower graph optimization level OR setOptimizedFilePath true (microsoft#13913) ### Description DML EP won't fuse the ONNX Graph if ORT Graph optimization level is <= 1 or `SessionOption::SetOptimizedFilePath` is passed. This is the successor of microsoft#11346. ### Motivation and Context - **Why is this change required? What problem does it solve?** Requested by few a users (issues below) and also helps in debugging. - **If it fixes an open issue, please link to the issue here:** - microsoft#13535 - microsoft#8440 * Debug build * Debug build 2 * Cjian/pad ops bug (microsoft#13930) * Debug build 3 * Enabling thread pool to be numa-aware (microsoft#13778) The PR enables ort thread pool to be numa-aware, so that threads could be evenly created and distributed among numa nodes. In addition, to facilitate performance tuning, the PR opens a new API allowing customers to attach threads to certain logical processors. Please check the API [definition](https://github.com/microsoft/onnxruntime/pull/13778/files#diff-5845a5c76fb64abdc8f0cffe21b37f8da1712674eb3abc4cd87190891be1bd48) for details. Co-authored-by: Randy Shuai <rashuai@microsoft.com> * Move C API docs into the right output area and clean up unused artifacts * Permission to remove old C API docs * Permission to move new api docs * More permissions * Debug * Use onnxruntime_fetchcontent_makeavailable cmake function for TRT (microsoft#13918) ### Description Use onnxruntime_fetchcontent_makeavailable cmake function for TRT. See the comment for the reason. ### Motivation and Context To support a newer TRT version. Previously they have a "BUILD_EXE" build option to allow us to exclude such things from build. But in onnx/onnx-tensorrt#879 they deleted the build option. It wouldn't be a problem if we continue to use git submodules as before, because cmake's add_subdirectories function has an "EXCLUDE_FROM_ALL" keyword. However, cmake's FetchContent module doesn't. That's why I needed to create our own version of the macro. * Bump decode-uri-component from 0.2.0 to 0.2.2 in /js/react_native (microsoft#13846) Bumps [decode-uri-component](https://github.com/SamVerschueren/decode-uri-component) from 0.2.0 to 0.2.2. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/SamVerschueren/decode-uri-component/releases">decode-uri-component's releases</a>.</em></p> <blockquote> <h2>v0.2.2</h2> <ul> <li>Prevent overwriting previously decoded tokens 980e0bf</li> </ul> <p><a href="https://github.com/SamVerschueren/decode-uri-component/compare/v0.2.1...v0.2.2">https://github.com/SamVerschueren/decode-uri-component/compare/v0.2.1...v0.2.2</a></p> <h2>v0.2.1</h2> <ul> <li>Switch to GitHub workflows 76abc93</li> <li>Fix issue where decode throws - fixes <a href="https://github-redirect.dependabot.com/SamVerschueren/decode-uri-component/issues/6">#6</a> 746ca5d</li> <li>Update license (<a href="https://github-redirect.dependabot.com/SamVerschueren/decode-uri-component/issues/1">#1</a>) 486d7e2</li> <li>Tidelift tasks a650457</li> <li>Meta tweaks 66e1c28</li> </ul> <p><a href="https://github.com/SamVerschueren/decode-uri-component/compare/v0.2.0...v0.2.1">https://github.com/SamVerschueren/decode-uri-component/compare/v0.2.0...v0.2.1</a></p> </blockquote> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/SamVerschueren/decode-uri-component/commit/a0eea469d26eb0df668b081672cdb9581feb78eb"><code>a0eea46</code></a> 0.2.2</li> <li><a href="https://github.com/SamVerschueren/decode-uri-component/commit/980e0bf09b64d94f1aa79012f895816c30ffd152"><code>980e0bf</code></a> Prevent overwriting previously decoded tokens</li> <li><a href="https://github.com/SamVerschueren/decode-uri-component/commit/3c8a373dd4837e89b3f970e01295dd03e1405a33"><code>3c8a373</code></a> 0.2.1</li> <li><a href="https://github.com/SamVerschueren/decode-uri-component/commit/76abc939783fe3900fadb7d384a74d324d5557f3"><code>76abc93</code></a> Switch to GitHub workflows</li> <li><a href="https://github.com/SamVerschueren/decode-uri-component/commit/746ca5dcb6667c5d364e782d53c542830e4c10b9"><code>746ca5d</code></a> Fix issue where decode throws - fixes <a href="https://github-redirect.dependabot.com/SamVerschueren/decode-uri-component/issues/6">#6</a></li> <li><a href="https://github.com/SamVerschueren/decode-uri-component/commit/486d7e26d3a8c0fbe860fb651fe1bc98c2f2be30"><code>486d7e2</code></a> Update license (<a href="https://github-redirect.dependabot.com/SamVerschueren/decode-uri-component/issues/1">#1</a>)</li> <li><a href="https://github.com/SamVerschueren/decode-uri-component/commit/a65045724e6234acef87f31da499d4807b20b134"><code>a650457</code></a> Tidelift tasks</li> <li><a href="https://github.com/SamVerschueren/decode-uri-component/commit/66e1c2834c0e189201cb65196ec3101372459b02"><code>66e1c28</code></a> Meta tweaks</li> <li>See full diff in <a href="https://github.com/SamVerschueren/decode-uri-component/compare/v0.2.0...v0.2.2">compare view</a></li> </ul> </details> <br /> [![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=decode-uri-component&package-manager=npm_and_yarn&previous-version=0.2.0&new-version=0.2.2)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) - `@dependabot use these labels` will set the current labels as the default for future PRs for this repo and language - `@dependabot use these reviewers` will set the current reviewers as the default for future PRs for this repo and language - `@dependabot use these assignees` will set the current assignees as the default for future PRs for this repo and language - `@dependabot use this milestone` will set the current milestone as the default for future PRs for this repo and language You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/microsoft/onnxruntime/network/alerts). </details> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * More permissions * Update site dir * Update upload pages version * Remove quotes from artifact upload path * Re order jekyll generation * Debug * Clean repo after gh-pages checkout * Create API docs dir * Update protobuf version to 3.18.3 in tools/ci_build/github/linux/docker/scripts/requirements.txt. (microsoft#13922) ### Description  Update protobuf version to 3.18.3 in tools/ci_build/github/linux/docker/scripts/requirements.txt. ### Motivation and Context Address component governance alert CVE-2022-1941 * Cjian/where python operator (microsoft#12795) **Description**: This PR will enable the python tool to run QWhere and QDQWhere operation **Limitation**: s8s8 Where is still not supported. * Upload API artifact before checking out gh-pages * Name upload and download artifact actions * Add params to upload and download artifact * Add deploy as a separate step * Debug after download artifact * Correct attribute of download artifact * Remove debug and add an explicit pages.yml file * Re set WARN_AS_ERROR to true * Restrict running workflow to files containing C/C++ API sources * fix(cuda): install missing python3-packaging in Dockerfile Signed-off-by: Jian Zeng <anonymousknight96@gmail.com> * Labeler: Add ROCm ep issue labeling (microsoft#13923) ### Description Add ROCm EP to auto labeler * Auto add docs issues to project (microsoft#13897) * Correct logic for GPU backend detection (microsoft#13944) Currently these checks yield the opposite of the desired logic. * Bump engine.io and socket.io in /js/web (microsoft#13723) Bumps [engine.io](https://github.com/socketio/engine.io) and [socket.io](https://github.com/socketio/socket.io). These dependencies needed to be updated together. Updates `engine.io` from 6.1.3 to 6.2.1 <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/socketio/engine.io/releases">engine.io's releases</a>.</em></p> <blockquote> <h2>6.2.1</h2> <p>:warning: This release contains an important security fix :warning:</p> <p>A malicious client could send a specially crafted HTTP request, triggering an uncaught exception and killing the Node.js process:</p> <pre><code>Error: read ECONNRESET at TCP.onStreamRead (internal/stream_base_commons.js:209:20) Emitted 'error' event on Socket instance at: at emitErrorNT (internal/streams/destroy.js:106:8) at emitErrorCloseNT (internal/streams/destroy.js:74:3) at processTicksAndRejections (internal/process/task_queues.js:80:21) { errno: -104, code: 'ECONNRESET', syscall: 'read' } </code></pre> <p>Please upgrade as soon as possible.</p> <h3>Bug Fixes</h3> <ul> <li>catch errors when destroying invalid upgrades (<a href="https://github-redirect.dependabot.com/socketio/engine.io/issues/658">#658</a>) (<a href="https://github.com/socketio/engine.io/commit/425e833ab13373edf1dd5a0706f07100db14e3c6">425e833</a>)</li> </ul> <h2>6.2.0</h2> <h2>Features</h2> <ul> <li>add the "maxPayload" field in the handshake details (<a href="https://github.com/socketio/engine.io/commit/088dcb4dff60df39785df13d0a33d3ceaa1dff38">088dcb4</a>)</li> </ul> <p>So that clients in HTTP long-polling can decide how many packets they have to send to stay under the maxHttpBufferSize value.</p> <p>This is a backward compatible change which should not mandate a new major revision of the protocol (we stay in v4), as we only add a field in the JSON-encoded handshake data:</p> <pre><code>0{"sid":"lv_VI97HAXpY6yYWAAAC","upgrades":["websocket"],"pingInterval":25000,"pingTimeout":5000,"maxPayload":1000000} </code></pre> <h4>Links</h4> <ul> <li>Diff: <a href="https://github.com/socketio/engine.io/compare/6.1.3...6.2.0">https://github.com/socketio/engine.io/compare/6.1.3...6.2.0</a></li> <li>Client release: <a href="https://github.com/socketio/engine.io-client/releases/tag/6.2.0">6.2.0</a></li> <li>ws version: <a href="https://github.com/websockets/ws/releases/tag/8.2.3">~8.2.3</a></li> </ul> </blockquote> </details> <details> <summary>Changelog</summary> <p><em>Sourced from <a href="https://github.com/socketio/engine.io/blob/main/CHANGELOG.md">engine.io's changelog</a>.</em></p> <blockquote> <h2><a href="https://github.com/socketio/engine.io/compare/6.2.0...6.2.1">6.2.1</a> (2022-11-20)</h2> <p>:warning: This release contains an important security fix :warning:</p> <p>A malicious client could send a specially crafted HTTP request, triggering an uncaught exception and killing the Node.js process:</p> <pre><code>Error: read ECONNRESET at TCP.onStreamRead (internal/stream_base_commons.js:209:20) Emitted 'error' event on Socket instance at: at emitErrorNT (internal/streams/destroy.js:106:8) at emitErrorCloseNT (internal/streams/destroy.js:74:3) at processTicksAndRejections (internal/process/task_queues.js:80:21) { errno: -104, code: 'ECONNRESET', syscall: 'read' } </code></pre> <p>Please upgrade as soon as possible.</p> <h3>Bug Fixes</h3> <ul> <li>catch errors when destroying invalid upgrades (<a href="https://github-redirect.dependabot.com/socketio/engine.io/issues/658">#658</a>) (<a href="https://github.com/socketio/engine.io/commit/425e833ab13373edf1dd5a0706f07100db14e3c6">425e833</a>)</li> </ul> <h1><a href="https://github.com/socketio/engine.io/compare/3.5.0...3.6.0">3.6.0</a> (2022-06-06)</h1> <h3>Bug Fixes</h3> <ul> <li>add extension in the package.json main entry (<a href="https://github-redirect.dependabot.com/socketio/engine.io/issues/608">#608</a>) (<a href="https://github.com/socketio/engine.io/commit/3ad0567dbd57cfb7c2ff4e8b7488d80f37022b4a">3ad0567</a>)</li> <li>do not reset the ping timer after upgrade (<a href="https://github.com/socketio/engine.io/commit/1f5d4699862afee1e410fcb0e1f5e751ebcd2f9f">1f5d469</a>), closes <a href="https://github-redirect.dependabot.com//github-redirect.dependabot.com/socketio/socket.io-client-swift/pull/1309/issues/issuecomment-768475704">socketio/socket.io-client-swift#1309</a></li> </ul> <h3>Features</h3> <ul> <li>decrease the default value of maxHttpBufferSize (<a href="https://github.com/socketio/engine.io/commit/58e274c437e9cbcf69fd913c813aad8fbd253703">58e274c</a>)</li> </ul> <p>This change reduces the default value from 100 mb to a more sane 1 mb.</p> <p>This helps protect the server against denial of service attacks by malicious clients sending huge amounts of data.</p> <p>See also: <a href="https://github.com/advisories/GHSA-j4f2-536g-r55m">https://github.com/advisories/GHSA-j4f2-536g-r55m</a></p> <ul> <li>increase the default value of pingTimeout (<a href="https://github.com/socketio/engine.io/commit/f55a79a28a5fbc6c9edae876dd11308b89cc979e">f55a79a</a>)</li> </ul>  </blockquote> <p>... (truncated)</p> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/socketio/engine.io/commit/24b847be6a61b64efc8c8c4d058a69259ad67693"><code>24b847b</code></a> chore(release): 6.2.1</li> <li><a href="https://github.com/socketio/engine.io/commit/425e833ab13373edf1dd5a0706f07100db14e3c6"><code>425e833</code></a> fix: catch errors when destroying invalid upgrades (<a href="https://github-redirect.dependabot.com/socketio/engine.io/issues/658">#658</a>)</li> <li><a href="https://github.com/socketio/engine.io/commit/99adb00ba11d80ab27a4a2f4afd0eebd8aa406c5"><code>99adb00</code></a> chore(deps): bump xmlhttprequest-ssl and engine.io-client in /examples/latenc...</li> <li><a href="https://github.com/socketio/engine.io/commit/d196f6a6b746b5e362b131a1a16901a3db12cb21"><code>d196f6a</code></a> chore(deps): bump minimatch from 3.0.4 to 3.1.2 (<a href="https://github-redirect.dependabot.com/socketio/engine.io/issues/660">#660</a>)</li> <li><a href="https://github.com/socketio/engine.io/commit/7c1270f98c51e51dfae1237492a56276070fd10e"><code>7c1270f</code></a> chore(deps): bump nanoid from 3.1.25 to 3.3.1 (<a href="https://github-redirect.dependabot.com/socketio/engine.io/issues/659">#659</a>)</li> <li><a href="https://github.com/socketio/engine.io/commit/535a01d8898a5cc858c9d6031fc5ecda96ea4579"><code>535a01d</code></a> ci: add Node.js 18 in the test matrix</li> <li><a href="https://github.com/socketio/engine.io/commit/1b71a6f5cb868c934696ae3cc1a92d1168ec8505"><code>1b71a6f</code></a> docs: remove "Vanilla JS" highlight from README (<a href="https://github-redirect.dependabot.com/socketio/engine.io/issues/656">#656</a>)</li> <li><a href="https://github.com/socketio/engine.io/commit/917d1d29e13f2e8f523c3738f6413f67b587aebe"><code>917d1d2</code></a> refactor: replace deprecated <code>String.prototype.substr()</code> (<a href="https://github-redirect.dependabot.com/socketio/engine.io/issues/646">#646</a>)</li> <li><a href="https://github.com/socketio/engine.io/commit/020801ab8ce2d4cba517fe04df89b39d403123a5"><code>020801a</code></a> chore: add changelog for version 3.6.0</li> <li><a href="https://github.com/socketio/engine.io/commit/ed1d6f912ce61b13e2ae7ce7a1027b8c5fae2f15"><code>ed1d6f9</code></a> test: make test script work on Windows (<a href="https://github-redirect.dependabot.com/socketio/engine.io/issues/643">#643</a>)</li> <li>Additional commits viewable in <a href="https://github.com/socketio/engine.io/compare/6.1.3...6.2.1">compare view</a></li> </ul> </details> <br /> Updates `socket.io` from 4.4.1 to 4.5.3 <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/socketio/socket.io/releases">socket.io's releases</a>.</em></p> <blockquote> <h2>4.5.3</h2> <h3>Bug Fixes</h3> <ul> <li><strong>typings:</strong> accept an HTTP2 server in the constructor (<a href="https://github.com/socketio/socket.io/commit/d3d0a2d5beaff51fd145f810bcaf6914213f8a06">d3d0a2d</a>)</li> <li><strong>typings:</strong> apply types to "io.timeout(...).emit()" calls (<a href="https://github.com/socketio/socket.io/commit/e357daf5858560bc84e7e50cd36f0278d6721ea1">e357daf</a>)</li> </ul> <h4>Links:</h4> <ul> <li>Diff: <a href="https://github.com/socketio/socket.io/compare/4.5.2...4.5.3">https://github.com/socketio/socket.io/compare/4.5.2...4.5.3</a></li> <li>Client release: <a href="https://github.com/socketio/socket.io-client/releases/tag/4.5.3">4.5.3</a></li> <li>engine.io version: <code>~6.2.0</code></li> <li>ws version: <code>~8.2.3</code></li> </ul> <h2>4.5.2</h2> <h3>Bug Fixes</h3> <ul> <li>prevent the socket from joining a room after disconnection (<a href="https://github.com/socketio/socket.io/commit/18f3fdab12947a9fee3e9c37cfc1da97027d1473">18f3fda</a>)</li> <li><strong>uws:</strong> prevent the server from crashing after upgrade (<a href="https://github.com/socketio/socket.io/commit/ba497ee3eb52c4abf1464380d015d8c788714364">ba497ee</a>)</li> </ul> <h4>Links:</h4> <ul> <li>Diff: <a href="https://github.com/socketio/socket.io/compare/4.5.1...4.5.2">https://github.com/socketio/socket.io/compare/4.5.1...4.5.2</a></li> <li>Client release: <a href="https://github.com/socketio/socket.io-client/releases/tag/4.5.2">4.5.2</a></li> <li>engine.io version: <code>~6.2.0</code></li> <li>ws version: <code>~8.2.3</code></li> </ul> <h2>4.5.1</h2> <h3>Bug Fixes</h3> <ul> <li>forward the local flag to the adapter when using fetchSockets() (<a href="https://github.com/socketio/socket.io/commit/30430f0985f8e7c49394543d4c84913b6a15df60">30430f0</a>)</li> <li><strong>typings:</strong> add HTTPS server to accepted types (<a href="https://github-redirect.dependabot.com/socketio/socket.io/issues/4351">#4351</a>) (<a href="https://github.com/socketio/socket.io/commit/9b43c9167cff817c60fa29dbda2ef7cd938aff51">9b43c91</a>)</li> </ul> <h4>Links:</h4> <ul> <li>Diff: <a href="https://github.com/socketio/socket.io/compare/4.5.0...4.5.1">https://github.com/socketio/socket.io/compare/4.5.0...4.5.1</a></li> <li>Client release: <a href="https://github.com/socketio/socket.io-client/releases/tag/4.5.1">4.5.1</a></li> <li>engine.io version: <code>~6.2.0</code></li> <li>ws version: <code>~8.2.3</code></li> </ul> <h2>4.5.0</h2> <h3>Bug Fixes</h3> <ul> <li><strong>typings:</strong> ensure compatibility with TypeScript 3.x (<a href="https://github-redirect.dependabot.com/socketio/socket.io/issues/4259">#4259</a>) (<a href="https://github.com/socketio/socket.io/commit/02c87a85614e217b8e7b93753f315790ae9d99f6">02c87a8</a>)</li> </ul> <h3>Features</h3> <ul> <li>add support for catch-all listeners for outgoing packets (<a href="https://github.com/socketio/socket.io/commit/531104d332690138b7aab84d5583d6204132c8b4">531104d</a>)</li> </ul> <p>This is similar to <code>onAny()</code>, but for outgoing packets.</p>  </blockquote> <p>... (truncated)</p> </details> <details> <summary>Changelog</summary> <p><em>Sourced from <a href="https://github.com/socketio/socket.io/blob/main/CHANGELOG.md">socket.io's changelog</a>.</em></p> <blockquote> <h2><a href="https://github.com/socketio/socket.io/compare/4.5.2...4.5.3">4.5.3</a> (2022-10-15)</h2> <h3>Bug Fixes</h3> <ul> <li><strong>typings:</strong> accept an HTTP2 server in the constructor (<a href="https://github.com/socketio/socket.io/commit/d3d0a2d5beaff51fd145f810bcaf6914213f8a06">d3d0a2d</a>)</li> <li><strong>typings:</strong> apply types to "io.timeout(...).emit()" calls (<a href="https://github.com/socketio/socket.io/commit/e357daf5858560bc84e7e50cd36f0278d6721ea1">e357daf</a>)</li> </ul> <h2><a href="https://github.com/socketio/socket.io/compare/4.5.1...4.5.2">4.5.2</a> (2022-09-02)</h2> <h3>Bug Fixes</h3> <ul> <li>prevent the socket from joining a room after disconnection (<a href="https://github.com/socketio/socket.io/commit/18f3fdab12947a9fee3e9c37cfc1da97027d1473">18f3fda</a>)</li> <li><strong>uws:</strong> prevent the server from crashing after upgrade (<a href="https://github.com/socketio/socket.io/commit/ba497ee3eb52c4abf1464380d015d8c788714364">ba497ee</a>)</li> </ul> <h1><a href="https://github.com/socketio/socket.io/compare/2.4.1...2.5.0">2.5.0</a> (2022-06-26)</h1> <h3>Bug Fixes</h3> <ul> <li>fix race condition in dynamic namespaces (<a href="https://github.com/socketio/socket.io/commit/05e1278cfa99f3ecf3f8f0531ffe57d850e9a05b">05e1278</a>)</li> <li>ignore packet received after disconnection (<a href="https://github.com/socketio/socket.io/commit/22d4bdf00d1a03885dc0171125faddfaef730066">22d4bdf</a>)</li> <li>only set 'connected' to true after middleware execution (<a href="https://github.com/socketio/socket.io/commit/226cc16165f9fe60f16ff4d295fb91c8971cde35">226cc16</a>)</li> <li>prevent the socket from joining a room after disconnection (<a href="https://github.com/socketio/socket.io/commit/f223178eb655a7713303b21a78f9ef9e161d6458">f223178</a>)</li> </ul> <h2><a href="https://github.com/socketio/socket.io/compare/4.5.0...4.5.1">4.5.1</a> (2022-05-17)</h2> <h3>Bug Fixes</h3> <ul> <li>forward the local flag to the adapter when using fetchSockets() (<a href="https://github.com/socketio/socket.io/commit/30430f0985f8e7c49394543d4c84913b6a15df60">30430f0</a>)</li> <li><strong>typings:</strong> add HTTPS server to accepted types (<a href="https://github-redirect.dependabot.com/socketio/socket.io/issues/4351">#4351</a>) (<a href="https://github.com/socketio/socket.io/commit/9b43c9167cff817c60fa29dbda2ef7cd938aff51">9b43c91</a>)</li> </ul> <h1><a href="https://github.com/socketio/socket.io/compare/4.4.1...4.5.0">4.5.0</a> (2022-04-23)</h1> <h3>Bug Fixes</h3> <ul> <li><strong>typings:</strong> ensure compatibility with TypeScript 3.x (<a href="https://github-redirect.dependabot.com/socketio/socket.io/issues/4259">#4259</a>) (<a href="https://github.com/socketio/socket.io/commit/02c87a85614e217b8e7b93753f315790ae9d99f6">02c87a8</a>)</li> </ul>  </blockquote> <p>... (truncated)</p> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/socketio/socket.io/commit/945c84be47d2923a9132786c9fd11dd90fa2c6db"><code>945c84b</code></a> chore(release): 4.5.3</li> <li><a href="https://github.com/socketio/socket.io/commit/d3d0a2d5beaff51fd145f810bcaf6914213f8a06"><code>d3d0a2d</code></a> fix(typings): accept an HTTP2 server in the constructor</li> <li><a href="https://github.com/socketio/socket.io/commit/19b225b0c8a093d7f54ccf1b9d3765bc8f463a65"><code>19b225b</code></a> docs(examples): update dependencies of the basic CRUD example</li> <li><a href="https://github.com/socketio/socket.io/commit/8fae95dd182ee1fdd033f7646eacc6beca6f456a"><code>8fae95d</code></a> docs: add jsdoc for each public method</li> <li><a href="https://github.com/socketio/socket.io/commit/e6f6b906db8209996b1adb564332cb443df38fc6"><code>e6f6b90</code></a> docs: add deprecation notice for the allSockets() method</li> <li><a href="https://github.com/socketio/socket.io/commit/596eb88af7fcd41e9d7c0abca4d1305a7e2c2fea"><code>596eb88</code></a> ci: upgrade to actions/checkout@3 and actions/setup-node@3</li> <li><a href="https://github.com/socketio/socket.io/commit/e357daf5858560bc84e7e50cd36f0278d6721ea1"><code>e357daf</code></a> fix(typings): apply types to "io.timeout(...).emit()" calls</li> <li><a href="https://github.com/socketio/socket.io/commit/10fa4a2690fafcf9415e49aad507394e0b9a9ab0"><code>10fa4a2</code></a> refactor: add list of possible disconnection reasons</li> <li><a href="https://github.com/socketio/socket.io/commit/8be95b3bd323f83b9bc5d7b0292abc2dbea9ce56"><code>8be95b3</code></a> chore(release): 4.5.2</li> <li><a href="https://github.com/socketio/socket.io/commit/ba497ee3eb52c4abf1464380d015d8c788714364"><code>ba497ee</code></a> fix(uws): prevent the server from crashing after upgrade</li> <li>Additional commits viewable in <a href="https://github.com/socketio/socket.io/compare/4.4.1...4.5.3">compare view</a></li> </ul> </details> <br /> Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) - `@dependabot use these labels` will set the current labels as the default for future PRs for this repo and language - `@dependabot use these reviewers` will set the current reviewers as the default for future PRs for this repo and language - `@dependabot use these assignees` will set the current assignees as the default for future PRs for this repo and language - `@dependabot use this milestone` will set the current milestone as the default for future PRs for this repo and language You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/microsoft/onnxruntime/network/alerts). </details> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Fix react native ci (microsoft#13948) Find build error in react native ci pipeline by adding the common header. Co-authored-by: Randy Shuai <rashuai@microsoft.com> * Bug Fix - ORT Web build script (microsoft#13925) Copying the right files according to the build documentation. The bug originated to address a run break under some machines (needed threaded SIMD instead of only threaded), analysis ongoing. Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: Jian Zeng <anonymousknight96@gmail.com> Co-authored-by: JiCheng <wejoncy@163.com> Co-authored-by: Guangyun Han <guangyunhan@microsoft.com> Co-authored-by: Jicheng Wen <jicwen@microsoft.com> Co-authored-by: Hariharan Seshadri <shariharan91@gmail.com> Co-authored-by: Changming Sun <chasun@microsoft.com> Co-authored-by: Yi Zhang <zhanyi@microsoft.com> Co-authored-by: Edward Chen <18449977+edgchen1@users.noreply.github.com> Co-authored-by: Sumit Agarwal <sumitagarwal330@gmail.com> Co-authored-by: Jian Chen <cjian@microsoft.com> Co-authored-by: RandySheriffH <48490400+RandySheriffH@users.noreply.github.com> Co-authored-by: Randy Shuai <rashuai@microsoft.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Jian Zeng <anonymousknight96@gmail.com> Co-authored-by: Faith Xu <faxu@microsoft.com> Co-authored-by: Joseph Groenenboom <joseph.groenenboom@amd.com> Co-authored-by: shalvamist <94086448+shalvamist@users.noreply.github.com>

### Description To pass session_options to Xnnpack EP via `XnnpackProviderFactoryCreator` for Initializing xnnpack's threadpool. If you want to use different threadpool size or even disable xnnpack's threadpool, just setting intra_threadpool to 1 by xnnpack EP's provider_options. ### Motivation and Context Co-authored-by: Guangyun Han <guangyunhan@microsoft.com> Co-authored-by: Jicheng Wen <jicwen@microsoft.com>

Add LegalizeSessionOptions interface. Move EP specific logic to EP impl.

873fae8

wejoncy requested review from pranavsharma and skottmckay October 14, 2022 05:41

logger not defined

265eca3

wejoncy changed the title ~~Add LegalizeSessionOptions interface. Move EP specific logic to EP impl.~~ Add LegalizeSessionOptions interface Oct 14, 2022

Fix various redefinition error caused by including provider_api.h

08f7f27

cloudhan mentioned this pull request Oct 14, 2022

Add LegalizeSessionOptions interface. Move EP specific logic to EP impl. #11138

Closed

fix

48a49b4

skottmckay reviewed Oct 17, 2022

View reviewed changes

wejoncy added 4 commits October 18, 2022 15:37

Revert "Add LegalizeSessionOptions interface. Move EP specific logic …

6332740

…to EP impl." This reverts commit 873fae8.

refactor CreateProvider with SessionOptions as pa

dc429d3

rename createprovider

ce6335f

move ep specific log into createprovider

d8e2ab0

wejoncy changed the title ~~Add LegalizeSessionOptions interface~~ Create execution provider with new API CreateProviderWithSessionOption Oct 18, 2022

wejoncy added 3 commits October 18, 2022 19:07

Revert "refactor CreateProvider with SessionOptions as pa"

aefc82b

This reverts commit dc429d3.

Revert "Revert "Add LegalizeSessionOptions interface. Move EP specifi…

833d992

…c logic to EP impl."" This reverts commit 6332740.

CheckSessionOptions works well

b4092b1

wejoncy changed the title ~~Create execution provider with new API CreateProviderWithSessionOption~~ Add CheckSessionOptions interface to verify sessionOption for EP Oct 18, 2022

wejoncy added 5 commits October 18, 2022 19:43

amend

449340a

fix test

f2a6575

-

3ef3f28

dml test

82b7515

-

2ede93d

skottmckay mentioned this pull request Oct 18, 2022

Document generation CI is broken #13308

Merged

wejoncy added 3 commits October 19, 2022 11:58

-

af622ab

-

c8def25

-

a4ac767

skottmckay reviewed Oct 19, 2022

View reviewed changes

-

f61f934

wejoncy and others added 11 commits October 21, 2022 13:50

-

72341f4

-

680bd0c

-

ab61920

-

55dea0d

-

b7ea333

-

77c8d7e

snake_case naming

76211e1

-

c3f1e7e

-

746d160

add back DML resetting default options

8066382

-

96d5c15

wejoncy requested a review from pranavsharma October 28, 2022 09:30

wejoncy added 7 commits December 2, 2022 21:13

Merge branch 'main' into jicwen/refactor_eps

0b71e18

remove api validatesessionOptiosn

c1f5c64

Delete ttttt.py

638cd7a

amend

cf6a032

amend

938af17

Merge branch 'main' into jicwen/refactor_eps

0d3f792

improve

6ea2416

pranavsharma reviewed Dec 9, 2022

View reviewed changes

fix commemts

109256f

wejoncy changed the title ~~Add CheckSessionOptions interface to verify sessionOption for EP~~ Pass SessionOptions to XnnpackProviderFactoryCreator. Dec 9, 2022

pranavsharma approved these changes Dec 9, 2022

View reviewed changes

wejoncy merged commit 22fa621 into main Dec 10, 2022

wejoncy deleted the jicwen/refactor_eps branch December 10, 2022 06:23

Pass SessionOptions to XnnpackProviderFactoryCreator. #13318

Pass SessionOptions to XnnpackProviderFactoryCreator. #13318

Conversation

wejoncy commented Oct 14, 2022 • edited

Description

Motivation and Context

Choose a reason for hiding this comment

cloudhan Oct 18, 2022 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

wejoncy commented Oct 18, 2022

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

pranavsharma Oct 20, 2022 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

wejoncy Oct 19, 2022 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

wejoncy Oct 19, 2022 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

pranavsharma commented Dec 9, 2022

wejoncy commented Dec 9, 2022

wejoncy commented Oct 14, 2022 •

edited

cloudhan Oct 18, 2022 •

edited

pranavsharma Oct 20, 2022 •

edited

wejoncy Oct 19, 2022 •

edited

wejoncy Oct 19, 2022 •

edited