Is it possible to make onnx-go run distributedly?

[BruceWangNo1]

Is your feature request related to a problem? Please describe.
I am a graduate student who needs a graduation project. I am thinking about whether it is necessary and feasible to make the program run distributedly.

Describe the solution you'd like
Possibly with the help of github.com/chrislusf/gleam ?

[owulveryck]

Hello,

Thanks for your interest in onnx-go.
This is a good subject and an excellent idea!

IMHO, onnx-go by itself would not benefit from a distributed execution (it's "only" a parsing mechanism that reconstructs a graph). But, the execution backend would.

By now, the most advanced backend is Gorgonia, and Gorgonia would benefit from a distributed computation system.

brief sidenote: Gorgonia's architecture is extensible. It uses an Expression Graph that is computed by a VM. The VM is an interface.

Last week I created a distributed VM based on go-routines and channels. I think it can be a good starting point for distributed computing.
My idea is to eventually end with something similar to the pregel algo used in spark.

More info here:

https://github.com/gorgonia/gorgonia/blob/1969ec3b19e8c3f9a3ffe9a854571223d4e16054/x/vm/vm_gomachine.go#L14-L25

I am also present on the #gorgonia channel on the #gopherslack. Feel free to join and/or to send me PM there. I would be glad to help you.

[BruceWangNo1]

Hi Olivier, thank you so much for your quick, insightful and helpful reply. I am gonna take a look.

…

------------------ Original ------------------ From: Olivier Wulveryck <notifications@github.com&gt; Date: Mon,Oct 7,2019 5:18 PM To: owulveryck/onnx-go <onnx-go@noreply.github.com&gt; Cc: Bruce Wang <brucewangno1@qq.com&gt;, Author <author@noreply.github.com&gt; Subject: Re: [owulveryck/onnx-go] Is it possible to make onnx-go run distributedly? (#147) Hello, Thanks for your interest in onnx-go. This is a good subject and an excellent idea! IMHO, onnx-go by itself would not benefit from a distributed execution (it's "only" a parsing mechanism that reconstructs a graph). But, the execution backend would. By now, the most advanced backend is Gorgonia, and Gorgonia would benefit from a distributed computation system. brief sidenote: Gorgonia's architecture is extensible. It uses an Expression Graph that is computed by a VM. The VM is an interface. Last week I created a distributed VM based on go-routines and channels. I think it can be a good starting point for distributed computing. My idea is to eventually end with something similar to the pregel algo used in spark. More info here: https://github.com/gorgonia/gorgonia/blob/1969ec3b19e8c3f9a3ffe9a854571223d4e16054/x/vm/vm_gomachine.go#L14-L25 I am also present on the #gorgonia channel on the #gopherslack. Feel free to join and/or to send me PM there. I would be glad to help you. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

[BruceWangNo1]

Hi Olivier, what is "#gorgonia channel on the #gopherslack?"

[owulveryck]

It's a channel on gophers.slack.com. You don't need any invitation to join this.

This URL should be direct access to the channel https://app.slack.com/client/T029RQSE6/CP1GMBUP2

[BruceWangNo1]

Thanks for the clarification. Slack prompted "Contact the workspace administrator for an invitation," though.

[owulveryck]

An invitation is now required for gophers.slack.com?

Besides, can we close this issue?

[BruceWangNo1]

Yes, sure.

…

------------------ Original ------------------ From: Olivier Wulveryck <notifications@github.com&gt; Date: Mon,Oct 7,2019 10:39 PM To: owulveryck/onnx-go <onnx-go@noreply.github.com&gt; Cc: Bruce Wang <brucewangno1@qq.com&gt;, Author <author@noreply.github.com&gt; Subject: Re: [owulveryck/onnx-go] Is it possible to make onnx-go run distributedly? (#147) Can we close this issue? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

[BruceWangNo1]

If it is possible, could you please invite me to gophers.slack.com? @owulveryck

[owulveryck]

Don't know how to do that TBH, but the internet suggests this link: https://invite.slack.golangbridge.org/ Hope this helps.

…

[BruceWangNo1]

Thanks