pytorch 1.3 graph

[lanpa]

This patch fixes the problem that the rendered graph is not meaningful.

Since pytorch 1.2, the representation of the traced graph has changed alot, which includes:

1. Many operators were added.
2. The parameter node is moved to graph.nodes(), rather than graph.inputs()
3. The graph is been traced in detail. Take convolution for example, the traced result looks like
   aten::_convolution(%input.5, %weight.2, %7, %66, %69, %72, %73, %76, %77, %78, %79, %80)
   Besides input, weight, and bias, other nodes are parameters such as stride, kernel size, etc. This information are encoded as "attributes" before.

cc @orionr @sanekmelnikov

[orionr]

@sanekmelnikov and @lanpa, can we try and land this before the PyTorch 1.4 release cut on 11/19?

[orionr]

Also, @lanpa, can you include a before and after visualization for this so we can see what changed/improved?

[orionr]

Also, @lanpa, can you include a before and after visualization for this so we can see what changed/improved? cc @J0Nreynolds @natalialunova

[lanpa]

@orionr @sanekmelnikov Here are two side by side comparision on whether applying c934d7a or not. And the zip file contains the event files that can be visualized in TB.
Archive.zip

fig.1 The pink arrow indicates the output node of the graph.

*fig.2 The figure shows the inside of a convolutional layer. The right one contains all of the nodes I mentioned in the PR description, while the left is the simplified graph (point #3) *

[lanpa]

Close since v1.3 is legacy.