v2.0.0-rc.2に追従する (#4)

## 内容

[v2.0.0-rc.2](https://github.com/pykeio/ort/releases/tag/v2.0.0-rc.2)

## 関連 Issue

## スクリーンショット・動画など

## その他

.github/workflows/code-quality.yml

@@ -56,7 +56,7 @@ jobs:
       - name: Checkout sources
         uses: actions/checkout@v4
       - name: Install Rust
-        uses: dtolnay/rust-toolchain@stable
+        uses: dtolnay/rust-toolchain@nightly
       - name: Get Rust version
         id: rust-version
         run: echo "::set-output name=version::$(cargo --version | cut -d ' ' -f 2)"
@@ -65,15 +65,15 @@ jobs:
         id: tarpaulin-cache
         with:
           path: ~/.cargo/bin/cargo-tarpaulin
-          key: ${{ runner.os }}-cargo-${{ steps.rustc-version.outputs.version }}
+          key: ${{ runner.os }}-cargo-${{ steps.rust-version.outputs.version }}-0.28.0
       - name: Install tarpaulin
         if: steps.tarpaulin-cache.outputs.cache-hit != 'true'
-        run: cargo install cargo-tarpaulin
+        run: cargo install cargo-tarpaulin --version 0.28.0
       - name: Generate code coverage
         run: |
-          cargo tarpaulin -p ort --features fetch-models --verbose --timeout 120 --out xml
+          cargo tarpaulin -p ort --features fetch-models --verbose --timeout 120 --out xml --engine llvm --doc --lib --tests
       - name: Upload to codecov.io
-        uses: codecov/codecov-action@v3
+        uses: codecov/codecov-action@v4
         with:
           token: ${{ secrets.CODECOV_TOKEN }}
           fail_ci_if_error: false

.github/workflows/test.yml

@@ -47,6 +47,21 @@ jobs:
       - name: Run tests
         run: |
           cargo test -p ort --verbose --features fetch-models -- --test-threads 1
+          # Test examples that use in-tree graphs (do NOT run any of the examples that download ~700 MB graphs from pyke parcel...)
+          cargo run --example custom-ops
+  test-wasm:
+    name: Test WebAssembly
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - name: Install nightly Rust toolchain
+        uses: dtolnay/rust-toolchain@nightly
+      - name: Install wasm-pack
+        run: cargo install wasm-pack
+      - name: Run tests
+        working-directory: examples/webassembly
+        run: |
+          wasm-pack test --node
   # Disable cross-compile until cross updates aarch64-unknown-linux-gnu to Ubuntu 22.04
   # ref https://github.com/cross-rs/cross/pull/973
   #cross-compile:

.gitignore

@@ -186,6 +186,7 @@ WixTools/
 # ONNX Runtime downloaded models
 **/*.onnx
 **/*.ort
+!examples/webassembly/**/*.ort
 !tests/data/*.onnx
 !tests/data/*.ort
 

Cargo.toml

@@ -6,7 +6,8 @@ members = [
 	'examples/gpt2',
 	'examples/model-info',
 	'examples/yolov8',
-	'examples/modnet'
+	'examples/modnet',
+	'examples/webassembly'
 ]
 default-members = [
 	'.',
@@ -17,11 +18,12 @@ default-members = [
 	'examples/yolov8',
 	'examples/modnet'
 ]
+exclude = [ 'examples/cudarc' ]
 
 [package]
 name = "ort"
 description = "A safe Rust wrapper for ONNX Runtime 1.17 - Optimize and Accelerate Machine Learning Inferencing"
-version = "2.0.0-rc.1"
+version = "2.0.0-rc.2"
 edition = "2021"
 rust-version = "1.70"
 license = "MIT OR Apache-2.0"
@@ -44,6 +46,7 @@ codegen-units = 1
 
 [package.metadata.docs.rs]
 features = [ "ndarray", "half", "operator-libraries", "fetch-models", "load-dynamic", "copy-dylibs" ]
+targets = ["x86_64-unknown-linux-gnu", "wasm32-unknown-unknown"]
 rustdoc-args = [ "--cfg", "docsrs" ]
 
 [features]
@@ -77,7 +80,7 @@ qnn = [ "ort-sys/qnn" ]
 [dependencies]
 ndarray = { version = "0.15", optional = true }
 thiserror = "1.0"
-ort-sys = { version = "2.0.0-rc.1", path = "ort-sys" }
+ort-sys = { version = "2.0.0-rc.2", path = "ort-sys" }
 libloading = { version = "0.8", optional = true }
 
 ureq = { version = "2.1", optional = true, default-features = false, features = [ "tls" ] }
@@ -90,6 +93,10 @@ libc = { version = "0.2", optional = true }
 [target.'cfg(windows)'.dependencies]
 winapi = { version = "0.3", optional = true, features = [ "std", "libloaderapi" ] }
 
+[target.'cfg(target_family = "wasm")'.dependencies]
+js-sys = "0.3"
+web-sys = "0.3"
+
 [dev-dependencies]
 anyhow = "1.0"
 ureq = "2.1"
@@ -99,6 +106,7 @@ tracing-subscriber = { version = "0.3", default-features = false, features = [ "
 glassbench = "0.4"
 tokio = { version = "1.36", features = [ "test-util" ] }
 tokio-test = "0.4.3"
+wasm-bindgen-test = "0.3"
 
 [[bench]]
 name = "squeezenet"

README.md

@@ -1,9 +1,9 @@
 <div align=center>
-    <img src="https://raw.githubusercontent.com/pykeio/ort/v2/docs/banner.png" width="350px">
+    <img src="https://parcel.pyke.io/v2/cdn/assetdelivery/ortrsv2/docs/trend-banner.png" width="350px">
     <hr />
     <a href="https://app.codecov.io/gh/pykeio/ort" target="_blank"><img alt="Coverage Results" src="https://img.shields.io/codecov/c/gh/pykeio/ort?style=for-the-badge"></a> <a href="https://crates.io/crates/ort" target="_blank"><img alt="Crates.io" src="https://img.shields.io/crates/d/ort?style=for-the-badge"></a> <a href="https://opencollective.com/pyke-osai" target="_blank"><img alt="Open Collective backers and sponsors" src="https://img.shields.io/opencollective/all/pyke-osai?style=for-the-badge&label=sponsors"></a>
     <br />
-    <a href="https://crates.io/crates/ort" target="_blank"><img alt="Crates.io" src="https://img.shields.io/crates/v/ort?style=for-the-badge&label=ort&logo=rust"></a> <img alt="ONNX Runtime" src="https://img.shields.io/badge/onnxruntime-v1.17.0-blue?style=for-the-badge&logo=cplusplus">
+    <a href="https://crates.io/crates/ort" target="_blank"><img alt="Crates.io" src="https://img.shields.io/crates/v/ort?style=for-the-badge&label=ort&logo=rust"></a> <img alt="ONNX Runtime" src="https://img.shields.io/badge/onnxruntime-v1.17.3-blue?style=for-the-badge&logo=cplusplus">
 </div>
 
 `ort` is an (unofficial) [ONNX Runtime](https://onnxruntime.ai/) 1.17 wrapper for Rust based on the now inactive [`onnxruntime-rs`](https://github.com/nbigaouette/onnxruntime-rs). ONNX Runtime accelerates ML inference on both CPU & GPU.
@@ -27,6 +27,8 @@
 - **[pyke Diffusers](https://github.com/pykeio/diffusers)** uses `ort` for efficient Stable Diffusion image generation on both CPUs & GPUs.
 - **[edge-transformers](https://github.com/npc-engine/edge-transformers)** uses `ort` for accelerated transformer model inference at the edge.
 - **[Ortex](https://github.com/relaypro-open/ortex)** uses `ort` for safe ONNX Runtime bindings in Elixir.
+- **[Supabase](https://supabase.com/)** uses `ort` to remove cold starts for their edge functions.
+- **[Lantern](https://github.com/lanterndata/lantern_extras)** uses `ort` to provide embedding model inference inside Postgres.
 
 ## 🌠 Sponsor `ort`
 <a href="https://opencollective.com/pyke-osai">

docs/introduction.mdx

@@ -2,58 +2,46 @@
 title: Introduction
 ---
 
-<p style={{ fontSize: '1.5rem', textAlign: 'center' }}>
-    `ort` is an open-source Rust binding for [ONNX Runtime](https://onnxruntime.ai/).
-</p>
-
-`ort` makes it easy to deploy your machine learning models to production via [ONNX Runtime](https://onnxruntime.ai/), a hardware-accelerated inference engine. With `ort` + ONNX Runtime, you can run almost any ML model (including ResNet, YOLOv8, BERT, LLaMA) on almost any hardware, often far faster than PyTorch, and with the added bonus of Rust's efficiency.
+<div style={{ display: 'flex', flexDirection: 'column', alignItems: 'center' }}>
+    <img src="/assets/trend-banner.png" style={{ maxHeight: '360px' }} />
+    <p style={{ fontSize: '1.5rem', textAlign: 'center', marginTop: 0 }}><code>ort</code> is an open-source Rust binding for <a href="https://onnxruntime.ai/">ONNX Runtime</a>.</p>
+</div>
 
 <Warning>
-    These docs are for the latest alpha version of `ort`, `2.0.0-rc.1`. This version is production-ready (just not API stable) and we recommend new & existing projects use it.
+    These docs are for the latest alpha version of `ort`, `2.0.0-rc.2`. This version is production-ready (just not API stable) and we recommend new & existing projects use it.
 </Warning>
 
-# Why `ort`?
-There are a few other ONNX Runtime crates out there, so why use `ort`?
+`ort` makes it easy to deploy your machine learning models to production via [ONNX Runtime](https://onnxruntime.ai/), a hardware-accelerated inference engine. With `ort` + ONNX Runtime, you can run almost any ML model (including ResNet, YOLOv8, BERT, LLaMA) on almost any hardware, often far faster than PyTorch, and with the added bonus of Rust's efficiency.
 
-For one, `ort` simply supports more features:
+[ONNX](https://onnx.ai/) is an interoperable neural network specification. Your ML framework of choice -- PyTorch, TensorFlow, Keras, PaddlePaddle, etc. -- turns your model into an ONNX graph comprised of basic operations like `MatMul` or `Add`. This graph can then be converted into a model in another framework, or **inferenced directly with ONNX Runtime**.
 
-| Feature comparison        | **📕 ort** | **📗 [ors](https://github.com/HaoboGu/ors)** | **🪟 [onnxruntime-rs](https://github.com/microsoft/onnxruntime/tree/main/rust)** |
-|---------------------------|-----------|-----------|----------------------|
-| Upstream version          | **v1.17.1** | v1.12.0 | v1.8               |
-| `dlopen()`?               | ✅         | ✅         | ❌                    |
-| Execution providers?      | ✅         | ❌         | ❌                    |
-| I/O Binding?              | ✅         | ❌         | ❌                    |
-| String tensors?           | ✅         | ❌         | ⚠️ input only         |
-| Multiple output types?    | ✅         | ✅         | ❌                    |
-| Multiple input types?     | ✅         | ✅         | ❌                    |
-| In-memory session?        | ✅         | ✅         | ✅                    |
-| WebAssembly?              | ✅         | ❌         | ❌                    |
-| Provides static binaries? | ✅         | ❌         | ❌                    |
-| Sequence & map types?     | ✅         | ❌         | ❌                    |
+<img width="100%" src="/assets/sample-onnx-graph.png" alt="An example visual representation of an ONNX graph, showing how an input tensor flows through layers of convolution nodes." />
 
-Users of `ort` appreciate its ease of use and ergonomic API. `ort` is also battle tested in some pretty serious production scenarios.
-- [**Twitter**](https://twitter.com/) uses `ort` in part of their recommendations system, serving hundreds of millions of requests a day.
+Converting a neural network to a graph representation like ONNX opens the door to more optimizations and broader acceleration hardware support. ONNX Runtime can significantly improve the inference speed/latency of most models and enable acceleration with NVIDIA CUDA & TensorRT, Intel OpenVINO, Qualcomm QNN, Huawei CANN, and [much more](/perf/execution-providers).
+
+`ort` is the Rust gateway to ONNX Runtime, allowing you to infer your ONNX models via an easy-to-use and ergonomic API. Many commercial, open-source, & research projects use `ort` in some pretty serious production scenarios to boost inference performance:
+- [**Twitter**](https://twitter.com/) uses `ort` in [part of their recommendations system](https://github.com/twitter/the-algorithm/tree/main/navi), serving hundreds of millions of requests a day.
 - [**Bloop**](https://bloop.ai/)'s semantic code search feature is powered by `ort`.
-- [**SurrealDB**](https://surrealdb.com/) uses `ort` in their [`surrealml`](https://github.com/surrealdb/surrealml) package.
-- [**Numerical Elixir**](https://github.com/elixir-nx) uses `ort` to create ONNX Runtime bindings for the Elixir language.
+- [**SurrealDB**](https://surrealdb.com/)'s powerful SurrealQL query language supports calling ML models, including ONNX graphs through `ort`.
+- [**Google's Magika**](https://github.com/google/magika) file type detection library is powered by `ort`.
+- [**Wasmtime**](https://github.com/bytecodealliance/wasmtime), an open-source WebAssembly runtime, supports ONNX inference for the [WASI-NN standard](https://github.com/WebAssembly/wasi-nn) via `ort`.
 - [**`rust-bert`**](https://github.com/guillaume-be/rust-bert) implements many ready-to-use NLP pipelines in Rust à la Hugging Face Transformers with both [`tch`](https://crates.io/crates/tch) & `ort` backends.
-- [**`edge-transformers`**](https://github.com/npc-engine/edge-transformers) also implements Hugging Face Transformers pipelines in Rust using `ort`.
 
 # Getting started
 <Steps>
     <Step title="Add ort to your Cargo.toml">
         If you have a [supported platform](/setup/platforms) (and you probably do), installing `ort` couldn't be any simpler! Just add it to your Cargo dependencies:
         ```toml
         [dependencies]
-        ort = "2.0.0-rc.1"
+        ort = "2.0.0-rc.2"
         ```
     </Step>
     <Step title="Convert your model">
-        Your model will need to be converted to the [ONNX](https://onnx.ai/) format before you can use it.
+        Your model will need to be converted to an ONNX graph before you can use it.
         - The awesome folks at Hugging Face have [a guide](https://huggingface.co/docs/transformers/serialization) to export 🤗 Transformers models to ONNX with 🤗 Optimum.
-        - For other PyTorch models: [`torch.onnx`](https://pytorch.org/docs/stable/onnx.html)
-        - For `scikit-learn`: [`sklearn-onnx`](https://onnx.ai/sklearn-onnx/)
-        - For TensorFlow, Keras, TFlite, TensorFlow.js: [`tf2onnx`](https://github.com/onnx/tensorflow-onnx)
+        - For any PyTorch model: [`torch.onnx`](https://pytorch.org/docs/stable/onnx.html)
+        - For `scikit-learn` models: [`sklearn-onnx`](https://onnx.ai/sklearn-onnx/)
+        - For TensorFlow, Keras, TFlite, & TensorFlow.js: [`tf2onnx`](https://github.com/onnx/tensorflow-onnx)
         - For PaddlePaddle: [`Paddle2ONNX`](https://github.com/PaddlePaddle/Paddle2ONNX)
     </Step>
     <Step title="Load your model">
@@ -73,13 +61,7 @@ Users of `ort` appreciate its ease of use and ergonomic API. `ort` is also battl
 
         ```rust
         let outputs = model.run(ort::inputs!["image" => image]?)?;
-
-        // Postprocessing
-        let output = outputs["output0"]
-            .extract_tensor::<f32>()?
-            .t()
-            .slice(s![.., .., 0])
-            .into_owned();
+        let predictions = outputs["output0"].try_extract_tensor::<f32>()?;
         ...
         ```
 

docs/migrating/opsets.mdx

@@ -0,0 +1,12 @@
+---
+title: ONNX opsets
+description: Supported opsets by `ort` version
+---
+
+| **ort**  | **ONNX opset version** | **ONNX ML opset version** |
+| -------- |:----------------------:|:-------------------------:|
+| v2.0.0+ | 20 | 4 |
+| v1.16.0-v1.16.2 | 19 | 3 |
+| v1.15.0-v1.15.5 | 19 | 3 |
+| v1.14.0-v1.14.8 | 18 | 3 |
+| v1.13.0-v1.13.3 | 17 | 3 |