Add `mindir` format support. #1227

Ethan-Chen-plus · 2024-02-18T04:06:26Z

🤗👋Hi! We have added our .mindir format. Please review our commit.

lutzroeder · 2024-02-18T17:07:22Z

@Ethan-Chen-plus can you provide some context:

What is mindir, what problem is it solving and for how many users?
Why is a new format needed given there are many similar formats?
Create a script in ./tools following the patterns used for other frameworks to recreate the -proto file.
Add a test to ./test/models.json and share an index of 50+ existing files in this format.
Run npm run validate

Ethan-Chen-plus · 2024-03-29T11:30:30Z

PS I:\netron> npm run validate
eslint
pylint
test
gguf/phi-2.Q2_K.gguf
onnx/candy.onnx
keras/1151.4.keras
coreml/Exermote.mlmodel
pytorch/alexnet.ptl
pytorch/DCGAN2.pt
tf/conv-layers.pb.zip
tflite/mobilenet_v1_0.75_160_quantized.tflite
tflite/squeezenet.tflite

I have updated my code, please review.
@lutzroeder

lutzroeder · 2024-03-30T02:26:16Z

@Ethan-Chen-plus can you provide some context:

What is mindir, what problem is it solving and for how many users?
Why is a new format needed given there are many similar formats?
Please rebase to latest main.
The ./tools scripts should generate regenerate to files from the original .proto. When changing the format or generator for -proto.js or -metadata.json these files should automatically get regenerated from the original source when running ./tools/mindir sync schema metadata.

Ethan-Chen-plus · 2024-04-24T08:42:55Z

@lutzroeder Thank you again and we have updated our code. And let me introduce some background infomation:

What is mindir, what problem is it solving and for how many users?
MindIR (MindSpore IR) is a graph-based functional Intermediate Representation (IR) format developed for the MindSpore framework by Huawei. Its primary purpose is to define the logical structure of networks and the attributes of operators, facilitating the decoupling of model files from hardware platforms through a unified IR. This supports a train-once, deploy-multiple-times paradigm. A key advantage of the MindIR format is its ability to store network structures and weight parameter values, enabling deployment for inference tasks on cloud-based Serving and edge-oriented Lite platforms. The same MindIR file can be deployed across various hardware types, including Ascend AI processors, GPUs, and CPUs.

The user base for the MindIR format primarily comprises researchers, developers, and enterprises needing to deploy AI models across multiple hardware platforms. Given its compatibility with diverse hardware including Ascend AI processors, GPUs, and CPUs, MindIR is particularly well-suited for applications that require deployment both in the cloud and on edge devices. For instance, MindIR can be utilized in Huawei's Atlas 200/300/500 inference products, which are specifically designed for edge AI processing scenarios.

-Why is a new format needed given there are many similar formats?

Compared to other similar IR formats such as ONNX and AIR, MindIR presents unique advantages and application scenarios. ONNX (Open Neural Network Exchange) is a universal format designed for the expression of machine learning models, primarily used for model transfer between different frameworks or on inference engines like TensorRT. AIR (Ascend Intermediate Representation), defined by Huawei, is an open file format tailored for machine learning to better accommodate Huawei AI processors, typically used for inference tasks on Ascend 310. MindIR, on the other hand, is a functional IR based on graph representation. It not only defines an extensible graph structure and operator IR representations but also eliminates model discrepancies across different backends. It is generally used for cross-platform inference tasks, such as executing inference on Ascend 310, GPUs, and MindSpore Lite for models trained on Ascend 910.

Ethan-Chen-plus · 2024-04-29T02:18:49Z

@lutzroeder Hello.👋 I noticed that eslint failed. Could you please tell me which config file eslint is using? I'll make the necessary fixes based on that config file. Thank you!

Ethan-Chen-plus · 2024-05-05T05:01:10Z

Workflow requires approval, thank you🤗

Ethan-Chen-plus · 2024-05-24T02:44:08Z

Workflow requires approval, thank you again🤗

lutzroeder

Compared to other similar IR formats such as ...

This format looks very similar to ONNX. Instead of defining a new format, can this be accomplished by using metadata extensions in ONNX to augment with additional information.

There is also an existing MindSpore Lite format and a dozen other formats that look similar in principle. Consider a path that leverages existing formats or standards instead of creating another duplicate format that needs to be maintained.

lutzroeder · 2024-05-27T14:49:45Z

source/base.js

@@ -734,7 +734,8 @@ base.Metadata = class {
            'ptl', 't7',
            'dlc', 'uff', 'armnn',
            'mnn', 'ms', 'ncnn', 'om', 'tm', 'mge', 'tmfile', 'tnnproto', 'xmodel', 'kmodel', 'rknn',
-            'tar', 'zip'
+            'tar', 'zip',
+            'mindir', 'mind_ir'


Remove for now for testing.

lutzroeder · 2024-05-27T14:50:29Z

source/mind_ir-proto.js

@@ -0,0 +1,713 @@
+export const protobuf = {};


Rename to mindir-proto.js. Sync to latest and regenerate this file using ./tools/mindir sync schema.

lutzroeder · 2024-05-27T14:52:10Z

tools/mind_ir.proto

@@ -0,0 +1,252 @@
+syntax = "proto2";


This file should not be added to the repro and instead downloaded to a location in ./third_party/source/mindspore/mindspore/core/proto/mind_ir.proto.

lutzroeder · 2024-05-27T15:04:23Z

tools/mindir

+sync() {
+    echo "mind_ir sync"
+    mkdir -p "./third_party/source/mindspore/mindspore/mindir/schema/"
+    curl --silent --location --output "./third_party/source/mindspore/mindspore/mindir/schema/mind_ir-metadata.json" "https://raw.githubusercontent.com/Ethan-Chen-plus/mindir/main/mind_ir-metadata.json"


The metadata file should not be downloaded from an external location and instead be updated in place as part of ./tools/mindir metadata.

lutzroeder · 2024-05-27T15:12:05Z

source/view.js

@@ -5455,6 +5455,34 @@ view.ModelFactoryService = class {
        this.register('./modular', ['.maxviz']);
        this.register('./cambricon', ['.cambricon']);
        this.register('./weka', ['.model']);
+        this.register('./mslite', ['.ms']);


Resolve incorrect merge.

lutzroeder · 2024-05-27T15:15:00Z

source/mind_ir.js

+ * The Model class represents a mind_ir model and contains information about the model's format, name, and graphs.
+ * The Graph class represents a graph within a mind_ir model and contains information about the graph's inputs, outputs, and nodes.
+ */
+export const protobuf = {};


Remove, this should not be needed.

lutzroeder · 2024-05-27T15:15:33Z

source/mind_ir.js

+        const identifier = context.identifier;
+        const extension = identifier.split('.').pop().toLowerCase();
+        if (extension === "mindir") {
+            return 'mind_ir';


This has changed in more recent versions. Sync to latest.

lutzroeder · 2024-05-27T15:16:52Z

source/mind_ir.js

+    }
+};
+
+mind_ir.Parameter = class {


Renamed in latest. Parameter to Argument and Argument to Value.

lutzroeder · 2024-05-27T15:17:25Z

source/mind_ir.js

+
+    constructor(message) {
+        super(message);
+        this.name = 'SnapML Model Load Error.';


Fix error message.

lutzroeder · 2024-05-27T15:18:53Z

source/mind_ir.js

+
+    constructor(context, attribute) {
+
+        this._name = attribute.name;


Attribute has been removed in latest and replaced with using Argument to represent attribtues.

lutzroeder force-pushed the main branch 3 times, most recently from 9a4785e to 1a817ad Compare March 3, 2024 20:36

lutzroeder force-pushed the main branch 3 times, most recently from 1f388bc to 0cfad16 Compare March 17, 2024 17:17

lutzroeder force-pushed the main branch 3 times, most recently from 994aca4 to 0de981a Compare March 26, 2024 03:00

Ethan-Chen-plus added 5 commits March 30, 2024 11:05

ok!

de44015

ok!

2403dae

edit

ec21668

0329

53632b3

2024

98a3a82

Ethan-Chen-plus force-pushed the main branch from 57eb1d0 to 98a3a82 Compare March 30, 2024 03:07

lutzroeder force-pushed the main branch 2 times, most recently from 66d40fe to 19e4123 Compare March 31, 2024 17:34

lutzroeder force-pushed the main branch 10 times, most recently from 8abf19b to dc7a947 Compare April 13, 2024 20:39

0424

756db3d

Ethan-Chen-plus force-pushed the main branch from 61e6f44 to 756db3d Compare April 24, 2024 08:25

0424

15c6dc1

0424

a1af601

lutzroeder force-pushed the main branch 2 times, most recently from 514284f to 0250d3e Compare April 25, 2024 02:16

lutzroeder force-pushed the main branch 4 times, most recently from 952b6ed to 2781faa Compare May 5, 2024 03:43

Ethan-Chen-plus added 2 commits May 5, 2024 12:54

0505

0f53013

0505

3f9ac2d

lutzroeder force-pushed the main branch 3 times, most recently from b448189 to 1d342d8 Compare May 5, 2024 17:41

lutzroeder force-pushed the main branch from 4bae21b to 1860e18 Compare May 14, 2024 04:28

0514

c8d25e8

lutzroeder force-pushed the main branch 3 times, most recently from d9a23ec to 5028ef6 Compare May 20, 2024 17:26

lutzroeder force-pushed the main branch 4 times, most recently from 1a3b633 to dc51e07 Compare May 26, 2024 14:49

lutzroeder requested changes May 27, 2024

View reviewed changes

lutzroeder force-pushed the main branch from 382cebc to 4aa2a96 Compare May 31, 2024 03:25

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add `mindir` format support. #1227

Add `mindir` format support. #1227

Ethan-Chen-plus commented Feb 18, 2024

lutzroeder commented Feb 18, 2024 •

edited

Ethan-Chen-plus commented Mar 29, 2024

lutzroeder commented Mar 30, 2024

Ethan-Chen-plus commented Apr 24, 2024

Ethan-Chen-plus commented Apr 29, 2024

Ethan-Chen-plus commented May 5, 2024 •

edited

Ethan-Chen-plus commented May 24, 2024

lutzroeder left a comment

lutzroeder May 27, 2024

lutzroeder May 27, 2024 •

edited

lutzroeder May 27, 2024

lutzroeder May 27, 2024

lutzroeder May 27, 2024 •

edited

lutzroeder May 27, 2024 •

edited

lutzroeder May 27, 2024 •

edited

lutzroeder May 27, 2024

lutzroeder May 27, 2024

lutzroeder May 27, 2024


		constructor(context, attribute) {

		this._name = attribute.name;

Add mindir format support. #1227

Are you sure you want to change the base?

Add mindir format support. #1227

Conversation

Ethan-Chen-plus commented Feb 18, 2024

lutzroeder commented Feb 18, 2024 • edited

Ethan-Chen-plus commented Mar 29, 2024

lutzroeder commented Mar 30, 2024

Ethan-Chen-plus commented Apr 24, 2024

Ethan-Chen-plus commented Apr 29, 2024

Ethan-Chen-plus commented May 5, 2024 • edited

Ethan-Chen-plus commented May 24, 2024

lutzroeder left a comment

Choose a reason for hiding this comment

lutzroeder May 27, 2024

Choose a reason for hiding this comment

lutzroeder May 27, 2024 • edited

Choose a reason for hiding this comment

lutzroeder May 27, 2024

Choose a reason for hiding this comment

lutzroeder May 27, 2024

Choose a reason for hiding this comment

lutzroeder May 27, 2024 • edited

Choose a reason for hiding this comment

lutzroeder May 27, 2024 • edited

Choose a reason for hiding this comment

lutzroeder May 27, 2024 • edited

Choose a reason for hiding this comment

lutzroeder May 27, 2024

Choose a reason for hiding this comment

lutzroeder May 27, 2024

Choose a reason for hiding this comment

lutzroeder May 27, 2024

Choose a reason for hiding this comment

Add `mindir` format support. #1227

Add `mindir` format support. #1227

lutzroeder commented Feb 18, 2024 •

edited

Ethan-Chen-plus commented May 5, 2024 •

edited

lutzroeder May 27, 2024 •

edited

lutzroeder May 27, 2024 •

edited

lutzroeder May 27, 2024 •

edited

lutzroeder May 27, 2024 •

edited