Just another question

[botev]

Hi guys,

I was always very happy about Nervana and their work. Really kudos to you, that you could manage to make such an amazing breaktrough in low level stuff for DL.

Now, this is not really an issue, but only place to write to the devs directly. I was wondering why you guys started this in python and not C++/C? And let me elaborate why I think that is a better idea. With it you can easily frontend any other language. Additionally, when you get to optimizing properly a graph you would actually care about speed, as the task is equivalent to a full blown compiler? I do understand the user base need for python, but I do think a more optimized language is needed for the graph traversal.

Since you guys did this, I just was hoping if you would take a look at a project I started quite recently, which is to provide a Graph IR, used mainly for the autodiff part in c++. I was hoping this somehow to bring more people on an LLVM style arch as you mention, where we optimize stuff in this representation. One of the key benefits of my project, is that it provides "compiler" time guarantees that if your inputs are valid the whole graph can execute, which is done by clever meta data propagation. Obviously however, I can not drive this myself only and I well if any of you find that interesting please let me know.

Here is a link

[diyessi]

Thanks. We used Python because neon was Python, but tried to keep the implementation such that we were not forever tied to Python, for the reason you mentioned.

[binarybana]

Thanks for the note @botev. We indeed are considering C API's for language interoperability and language agnostic graph serialization formats (eg protobuf to net something similar to LLVM bitcode) which align with that goal.

We also are interested in more static verification of computational graphs (I've personally been playing with some approaches in Rust on the side) and I'll take a closer look at GIR.

[jennifermyers]

Thanks for the feedback @botev! I'm going to close this ticket now that we have #3 to track the C API feature and this ticket has been linked to that.

[botev]

@binarybana Oh I was also trying this in Rust. The reason I chose C++ rather than Rust is that specifically for a Graph structure, where often you need to modify the Graph which owns the nodes, trough the nodes themselves makes it very hard when you have borrow checking. Thus I thought that this might possibly put off a lot of people who want to develop this, while there are many great people in Compiler Engineering who know well C++. I've also spoked with some people partially related to AMD and they seemed keen more on C++ interface. As for the serialization, currently for the graph I have JSON and was thinking to add protobuf as well for the same reasons you mentioned.

Note that currently the GIR has no backends implemented, thus you can only construct and diff graphs. I'm atm working on adding simple backend for at least testing the autodiff bit via numerical diff. When you get a look at it if you have any problems please ping me on this, and if it looks interesting to you guys I will be very happy.

[diyessi]

ngraph.util.utils.numeric_derivative is a numeric derivative used to check autodiff in a number of the tests.