Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Import netlistsvg ELK json #26

Open
nobodywasishere opened this issue Oct 4, 2021 · 18 comments
Open

Import netlistsvg ELK json #26

nobodywasishere opened this issue Oct 4, 2021 · 18 comments

Comments

@nobodywasishere
Copy link

Internally, netlistsvg generates a JSON object very similar to the ELK JSON used by this project. For example, using one of the Fomu blink examples, this is generated by netlistsvg:

image

And this is the corresponding ELK JSON before being handed off to ELK for rendering into an SVG:

{
    "id": "Fomu_Blink",
    "children": [
        {
            "id": "clk_generator",
            "width": 30,
            "height": 20,
            "ports": [
                {
                    "id": "clk_generator.clk",
                    "width": 1,
                    "height": 1,
                    "x": 0,
                    "y": 10,
                    "labels": [
                        {
                            "id": "clk_generator.clk.label",
                            "text": "clk",
                            "x": -13,
                            "y": -10,
                            "width": 18,
                            "height": 11
                        }
                    ]
                },
                {
                    "id": "clk_generator.cnt",
                    "width": 1,
                    "height": 1,
                    "x": 30,
                    "y": 10,
                    "labels": [
                        {
                            "id": "clk_generator.cnt.label",
                            "text": "cnt",
                            "x": -5,
                            "y": -10,
                            "width": 18,
                            "height": 11
                        }
                    ]
                }
            ],
            "layoutOptions": {
                "org.eclipse.elk.portConstraints": "FIXED_POS"
            },
            "labels": [
                {
                    "id": "clk_generator.label.ref",
                    "text": "clk_generator",
                    "x": "15",
                    "y": -10,
                    "height": 11,
                    "width": 78
                }
            ]
        },
        {
            "id": "rgba_driver",
            "width": 30,
            "height": 100,
            "ports": [
                {
                    "id": "rgba_driver.CURREN",
                    "width": 1,
                    "height": 1,
                    "x": 0,
                    "y": 10,
                    "labels": [
                        {
                            "id": "rgba_driver.CURREN.label",
                            "text": "CURREN",
                            "x": -13,
                            "y": -10,
                            "width": 36,
                            "height": 11
                        }
                    ]
                },
                {
                    "id": "rgba_driver.RGB0PWM",
                    "width": 1,
                    "height": 1,
                    "x": 0,
                    "y": 30,
                    "labels": [
                        {
                            "id": "rgba_driver.RGB0PWM.label",
                            "text": "RGB0PWM",
                            "x": -13,
                            "y": -10,
                            "width": 42,
                            "height": 11
                        }
                    ]
                },
                {
                    "id": "rgba_driver.RGB1PWM",
                    "width": 1,
                    "height": 1,
                    "x": 0,
                    "y": 50,
                    "labels": [
                        {
                            "id": "rgba_driver.RGB1PWM.label",
                            "text": "RGB1PWM",
                            "x": -13,
                            "y": -10,
                            "width": 42,
                            "height": 11
                        }
                    ]
                },
                {
                    "id": "rgba_driver.RGB2PWM",
                    "width": 1,
                    "height": 1,
                    "x": 0,
                    "y": 70,
                    "labels": [
                        {
                            "id": "rgba_driver.RGB2PWM.label",
                            "text": "RGB2PWM",
                            "x": -13,
                            "y": -10,
                            "width": 42,
                            "height": 11
                        }
                    ]
                },
                {
                    "id": "rgba_driver.RGBLEDEN",
                    "width": 1,
                    "height": 1,
                    "x": 0,
                    "y": 90,
                    "labels": [
                        {
                            "id": "rgba_driver.RGBLEDEN.label",
                            "text": "RGBLEDEN",
                            "x": -13,
                            "y": -10,
                            "width": 48,
                            "height": 11
                        }
                    ]
                },
                {
                    "id": "rgba_driver.RGB0",
                    "width": 1,
                    "height": 1,
                    "x": 30,
                    "y": 10,
                    "labels": [
                        {
                            "id": "rgba_driver.RGB0.label",
                            "text": "RGB0",
                            "x": -5,
                            "y": -10,
                            "width": 24,
                            "height": 11
                        }
                    ]
                },
                {
                    "id": "rgba_driver.RGB1",
                    "width": 1,
                    "height": 1,
                    "x": 30,
                    "y": 30,
                    "labels": [
                        {
                            "id": "rgba_driver.RGB1.label",
                            "text": "RGB1",
                            "x": -5,
                            "y": -10,
                            "width": 24,
                            "height": 11
                        }
                    ]
                },
                {
                    "id": "rgba_driver.RGB2",
                    "width": 1,
                    "height": 1,
                    "x": 30,
                    "y": 50,
                    "labels": [
                        {
                            "id": "rgba_driver.RGB2.label",
                            "text": "RGB2",
                            "x": -5,
                            "y": -10,
                            "width": 24,
                            "height": 11
                        }
                    ]
                }
            ],
            "layoutOptions": {
                "org.eclipse.elk.portConstraints": "FIXED_POS"
            },
            "labels": [
                {
                    "id": "rgba_driver.label.ref",
                    "text": "rgba_driver",
                    "x": "15",
                    "y": -10,
                    "height": 11,
                    "width": 66
                }
            ]
        },
        {
            "id": "clki",
            "width": 30,
            "height": 20,
            "ports": [
                {
                    "id": "clki.Y",
                    "width": 0,
                    "height": 0,
                    "x": 30,
                    "y": 10
                }
            ],
            "layoutOptions": {
                "org.eclipse.elk.portConstraints": "FIXED_POS"
            },
            "labels": [
                {
                    "id": "clki.label.ref",
                    "text": "clki",
                    "x": "15",
                    "y": -10,
                    "height": 11,
                    "width": 24
                }
            ]
        },
        {
            "id": "rgb0",
            "width": 30,
            "height": 20,
            "ports": [
                {
                    "id": "rgb0.A",
                    "width": 0,
                    "height": 0,
                    "x": 0,
                    "y": 10
                }
            ],
            "layoutOptions": {
                "org.eclipse.elk.portConstraints": "FIXED_POS"
            },
            "labels": [
                {
                    "id": "rgb0.label.ref",
                    "text": "rgb0",
                    "x": "15",
                    "y": -10,
                    "height": 11,
                    "width": 24
                }
            ]
        },
        {
            "id": "rgb1",
            "width": 30,
            "height": 20,
            "ports": [
                {
                    "id": "rgb1.A",
                    "width": 0,
                    "height": 0,
                    "x": 0,
                    "y": 10
                }
            ],
            "layoutOptions": {
                "org.eclipse.elk.portConstraints": "FIXED_POS"
            },
            "labels": [
                {
                    "id": "rgb1.label.ref",
                    "text": "rgb1",
                    "x": "15",
                    "y": -10,
                    "height": 11,
                    "width": 24
                }
            ]
        },
        {
            "id": "rgb2",
            "width": 30,
            "height": 20,
            "ports": [
                {
                    "id": "rgb2.A",
                    "width": 0,
                    "height": 0,
                    "x": 0,
                    "y": 10
                }
            ],
            "layoutOptions": {
                "org.eclipse.elk.portConstraints": "FIXED_POS"
            },
            "labels": [
                {
                    "id": "rgb2.label.ref",
                    "text": "rgb2",
                    "x": "15",
                    "y": -10,
                    "height": 11,
                    "width": 24
                }
            ]
        },
        {
            "id": "usb_dp",
            "width": 30,
            "height": 20,
            "ports": [
                {
                    "id": "usb_dp.A",
                    "width": 0,
                    "height": 0,
                    "x": 0,
                    "y": 10
                }
            ],
            "layoutOptions": {
                "org.eclipse.elk.portConstraints": "FIXED_POS"
            },
            "labels": [
                {
                    "id": "usb_dp.label.ref",
                    "text": "usb_dp",
                    "x": "15",
                    "y": -10,
                    "height": 11,
                    "width": 36
                }
            ]
        },
        {
            "id": "usb_dn",
            "width": 30,
            "height": 20,
            "ports": [
                {
                    "id": "usb_dn.A",
                    "width": 0,
                    "height": 0,
                    "x": 0,
                    "y": 10
                }
            ],
            "layoutOptions": {
                "org.eclipse.elk.portConstraints": "FIXED_POS"
            },
            "labels": [
                {
                    "id": "usb_dn.label.ref",
                    "text": "usb_dn",
                    "x": "15",
                    "y": -10,
                    "height": 11,
                    "width": 36
                }
            ]
        },
        {
            "id": "usb_dp_pu",
            "width": 30,
            "height": 20,
            "ports": [
                {
                    "id": "usb_dp_pu.A",
                    "width": 0,
                    "height": 0,
                    "x": 0,
                    "y": 10
                }
            ],
            "layoutOptions": {
                "org.eclipse.elk.portConstraints": "FIXED_POS"
            },
            "labels": [
                {
                    "id": "usb_dp_pu.label.ref",
                    "text": "usb_dp_pu",
                    "x": "15",
                    "y": -10,
                    "height": 11,
                    "width": 54
                }
            ]
        },
        {
            "id": "1",
            "width": 30,
            "height": 20,
            "ports": [
                {
                    "id": "1.Y",
                    "width": 0,
                    "height": 0,
                    "x": 31,
                    "y": 10
                }
            ],
            "layoutOptions": {
                "org.eclipse.elk.portConstraints": "FIXED_POS"
            },
            "labels": [
                {
                    "id": "1.label.ref",
                    "text": "1",
                    "x": "15",
                    "y": -10,
                    "height": 11,
                    "width": 6
                }
            ]
        },
        {
            "id": "0",
            "width": 30,
            "height": 20,
            "ports": [
                {
                    "id": "0.Y",
                    "width": 0,
                    "height": 0,
                    "x": 31,
                    "y": 10
                }
            ],
            "layoutOptions": {
                "org.eclipse.elk.portConstraints": "FIXED_POS"
            },
            "labels": [
                {
                    "id": "0.label.ref",
                    "text": "0",
                    "x": "15",
                    "y": -10,
                    "height": 11,
                    "width": 6
                }
            ]
        },
        {
            "id": "$split$,6,7,8,",
            "width": 5,
            "height": 60,
            "ports": [
                {
                    "id": "$split$,6,7,8,.A",
                    "width": 1,
                    "height": 1,
                    "x": 0,
                    "y": 20
                },
                {
                    "id": "$split$,6,7,8,.2",
                    "width": 1,
                    "height": 1,
                    "x": 4,
                    "y": 10,
                    "labels": [
                        {
                            "id": "$split$,6,7,8,.2.label",
                            "text": "2",
                            "x": -5,
                            "y": -10,
                            "width": 6,
                            "height": 11
                        }
                    ]
                },
                {
                    "id": "$split$,6,7,8,.1",
                    "width": 1,
                    "height": 1,
                    "x": 4,
                    "y": 30
                },
                {
                    "id": "$split$,6,7,8,.0",
                    "width": 1,
                    "height": 1,
                    "x": 4,
                    "y": 50
                }
            ],
            "layoutOptions": {
                "org.eclipse.elk.portConstraints": "FIXED_POS"
            },
            "labels": []
        }
    ],
    "edges": [
        {
            "id": "e0",
            "sources": [
                "clki.Y"
            ],
            "targets": [
                "clk_generator.clk"
            ],
            "layoutOptions": {
                "org.eclipse.elk.layered.priority.direction": 10,
                "org.eclipse.elk.edge.thickness": 1
            }
        },
        {
            "id": "e1",
            "sources": [
                "1.Y"
            ],
            "targets": [
                "rgba_driver.CURREN"
            ],
            "layoutOptions": {
                "org.eclipse.elk.layered.priority.direction": 10,
                "org.eclipse.elk.edge.thickness": 1
            }
        },
        {
            "id": "e2",
            "sources": [
                "1.Y"
            ],
            "targets": [
                "rgba_driver.RGBLEDEN"
            ],
            "layoutOptions": {
                "org.eclipse.elk.layered.priority.direction": 10,
                "org.eclipse.elk.edge.thickness": 1
            }
        },
        {
            "id": "e3",
            "sources": [
                "$split$,6,7,8,.2"
            ],
            "targets": [
                "rgba_driver.RGB0PWM"
            ],
            "layoutOptions": {
                "org.eclipse.elk.layered.priority.direction": 10,
                "org.eclipse.elk.edge.thickness": 1
            }
        },
        {
            "id": "e4",
            "sources": [
                "$split$,6,7,8,.1"
            ],
            "targets": [
                "rgba_driver.RGB1PWM"
            ],
            "layoutOptions": {
                "org.eclipse.elk.layered.priority.direction": 10,
                "org.eclipse.elk.edge.thickness": 1
            }
        },
        {
            "id": "e5",
            "sources": [
                "$split$,6,7,8,.0"
            ],
            "targets": [
                "rgba_driver.RGB2PWM"
            ],
            "layoutOptions": {
                "org.eclipse.elk.layered.priority.direction": 10,
                "org.eclipse.elk.edge.thickness": 1
            }
        },
        {
            "id": "e6",
            "sources": [
                "rgba_driver.RGB0"
            ],
            "targets": [
                "rgb0.A"
            ],
            "layoutOptions": {
                "org.eclipse.elk.layered.priority.direction": 10,
                "org.eclipse.elk.edge.thickness": 1
            }
        },
        {
            "id": "e7",
            "sources": [
                "rgba_driver.RGB1"
            ],
            "targets": [
                "rgb1.A"
            ],
            "layoutOptions": {
                "org.eclipse.elk.layered.priority.direction": 10,
                "org.eclipse.elk.edge.thickness": 1
            }
        },
        {
            "id": "e8",
            "sources": [
                "rgba_driver.RGB2"
            ],
            "targets": [
                "rgb2.A"
            ],
            "layoutOptions": {
                "org.eclipse.elk.layered.priority.direction": 10,
                "org.eclipse.elk.edge.thickness": 1
            }
        },
        {
            "id": "e9",
            "sources": [
                "0.Y"
            ],
            "targets": [
                "usb_dp.A"
            ],
            "layoutOptions": {
                "org.eclipse.elk.layered.priority.direction": 10,
                "org.eclipse.elk.edge.thickness": 1
            }
        },
        {
            "id": "e10",
            "sources": [
                "0.Y"
            ],
            "targets": [
                "usb_dn.A"
            ],
            "layoutOptions": {
                "org.eclipse.elk.layered.priority.direction": 10,
                "org.eclipse.elk.edge.thickness": 1
            }
        },
        {
            "id": "e11",
            "sources": [
                "0.Y"
            ],
            "targets": [
                "usb_dp_pu.A"
            ],
            "layoutOptions": {
                "org.eclipse.elk.layered.priority.direction": 10,
                "org.eclipse.elk.edge.thickness": 1
            }
        },
        {
            "id": "e12",
            "labels": [
                {
                    "id": "",
                    "text": "3",
                    "width": 4,
                    "height": 6,
                    "x": 0,
                    "y": 0,
                    "layoutOptions": {
                        "org.eclipse.elk.edgeLabels.inline": true
                    }
                }
            ],
            "sources": [
                "clk_generator.cnt"
            ],
            "targets": [
                "$split$,6,7,8,.A"
            ],
            "layoutOptions": {
                "org.eclipse.elk.layered.priority.direction": 10,
                "org.eclipse.elk.edge.thickness": 2
            }
        }
    ]
}

Trying to import this directly, I had to change layoutOptions to properties and add empty hwMeta for the different nodes, but still came up with errors. I'm also not that familiar with this format so you may be able to see something I can't.

What I'm thinking about is if it'd be possible to add an option to netlistsvg to export this ELK JSON, and then have d3-hwschematic import it, maybe with minor modifications. The variable kgraph here is where this JSON/object is generated/stored. This way, d3-hwschematic could generate diagrams for VHDL/Verilog, and netlistsvg could have a new "frontend".

@Nic30
Copy link
Owner

Nic30 commented Oct 4, 2021

Hello,
I made small working example for this

image

It is far from complete (I did not translate the shapes, label possitions, etc) but at least something.

@Nic30
Copy link
Owner

Nic30 commented Oct 4, 2021

@Nic30
Copy link
Owner

Nic30 commented Oct 4, 2021

But I think it could be better to just load yosys input directly.

@nobodywasishere
Copy link
Author

Loading the Yosys json directly would also work. Just wanted to see how easy it'd be to import the netlistsvg elk json.

I wonder if it'd be possible to take an elk json generated for d3-hwsch and translate it to work with the netlistsvg renderer, so that netlistsvg could be used as a "front end" for this as well...

I think yosys json would be the best way forward with this idea.

@Nic30
Copy link
Owner

Nic30 commented Oct 5, 2021

import the netlistsvg elk json

Actually the format should be same (but there is a difference in elkjs version, the 0.7.1 and 0.7.2 had some issues that's why I did not update yet.)

  • The hwMeta property should be optional but because it was not tested it was not working properly.
  • The port and component ids in netlistsvg have special meaning which translates to shape.

netlistsvg could be used as a "front end"

From what I see:

  • component ports must be flattened (easy)
  • component shapes must be translated (easy)
  • component port position must be resolved exactly (easy)

Also I would be super happy if we can merge netlistsvg and d3-hwschematic somehow, but I do not have the authority to convince anyone to do it or at least agree with it.
It is also simple to implement "skins" like they are implemented in netlistsvg, d3-hwschematic currently relies on js functions which are generating the shapes, which is more generic and efficient. But svg with skins may be more user friendly.

@nobodywasishere
Copy link
Author

the 0.7.1 and 0.7.2 had some issues that's why I did not update yet.

Ah I did not know about this. What issues are there and what version do you recommend? To be honest, when I updated the version, I didn't change how the elk json was generated, just checked to make sure everything still operated correctly, so it should be the same as what it was for 0.3.1 (what I think it was before updating).

Also I would be super happy if we can merge netlistsvg and d3-hwschematic somehow, but I do not have the authority to convince anyone to do it or at least agree with it.

I don't have the authority to do that either. At a minimum though, I think just providing that interoperability between them, maybe with some compatibility scripts like the one you were writing earlier, would be good enough for now for anyone wanting that functionality.

/cc @nturley

@Nic30
Copy link
Owner

Nic30 commented Oct 7, 2021

What issues are there and what version do you recommend?

Currently I do not know latest version and in this case it means that the latest version is probably best.
I remember that 2 months back I was discussing some problems similar to this kieler/elkjs#98, but I can not find the correct issue link.

maybe with some compatibility scripts

Problem with compatibility scripts is that that someone has to maintain them. It does not seems like big deal but there is always some sort of problem and if you are not familiar with latest update in other project, it is hard.

That is why I was mentioning that project merge or that we need someone who updates them once some breaking change appears.

With potential merge there is one significant problem. We are using d3-hwschematic to visualize code on system/HLS level, but netlistsvg/yosys focuses on gate/operator level which means that they do not need the features this project have and thus we are some sort of outsiders for them.

@Nitcloud
Copy link

May I ask whether this project supports the direct import of the netlist JSON file output by yosys at present? According to the previous conversation, I did not get a clear answer. If so, do you have relevant operation instructions?

@Nic30
Copy link
Owner

Nic30 commented May 13, 2022

Sadly no, this project does not accept yosys JSON format, it would be simple to support it however I do not have the luxury of time to do it, but I can help if someone decides to implement it.

@Nitcloud
Copy link

I will try this work and will contact you if there is any progress or difficulty. Finally, thank you for your contribution.

@Nic30
Copy link
Owner

Nic30 commented Jul 23, 2022

@Bestduan I hired some student to work on this, https://github.com/sv12359/d3-hwschematic, It is nearly complete we now working on split/concatenation nodes which require a special care.
Are you an user of yosys?

We are currently using

synth -run coarse
  • Do you know about any better target which we should use?
  • Do you think that it is better to create our own target?

@Nitcloud
Copy link

First of all, thank you for your team's contribution. I am indeed a user of yosys. My previous work was to render the netlist generated by yosys in vscode, which will have more practical engineering significance.This is a preview of the relevant features I work on:
netlist

After the synthesis of yosys, there are some relatively fixed cells. I think it is better to use the corresponding icon for fixed cells. The svg below is a little work I made on top of this:
digital

You can use it directly, or make a better icon with it. In the end, the work I have done is not complete. For more detailed information, please refer to the doc of yosys. You can find a detailed description of the fixed cells included with yosys in Chapter 5 of the documentation.

@Nic30
Copy link
Owner

Nic30 commented Jul 23, 2022

doc of yosys

Sadly for every thing we need, there is a FIXME in this doc. For example $alu, $macc, $fa, and $lcu.
@sv12359

@Bestduan Do you know how to force yosys to include $slice and $concat cells in output json?

@Nitcloud
Copy link

I think you can try splice [options] [selection] This command will adds $slice and $concat cells to the design to make the splicing of multi-bit signals explicit. This for example is useful for coarse grain synthesis, where dedicated hardware is needed to splice signals. For more details you can find in P179 C.177 of this doc.

@Nitcloud
Copy link

Nitcloud commented Jul 24, 2022

for example

module test(a, b, y);

input [15:0] a, b;
output [15:0] y;

wire [7:0] ah = a[15:8], al = a[7:0];
wire [7:0] bh = b[15:8], bl = b[7:0];

wire [7:0] th = ah + bh, tl = al + bl;
wire [15:0] t = {th, tl}, k = t ^ 16'hcd;

assign y = { k[7:0], k[15:8] };

endmodule

and yosys@0.19 script

read_verilog splice.v
hierarchy -check; opt
copy test gold

cd test
splice
# show

cd ..
write_json out.json

you can find $slice and $concat at modules[test]

@qarlosalberto
Copy link

Hi! Do you plant to continue with this? I would like to use it in TerosHDL :)

@Nic30
Copy link
Owner

Nic30 commented Mar 23, 2023

@qarlosalberto yes, 2 years at least and most likely even after if there is no better alternative. I am currently building VSCode extension for it.

@qarlosalberto
Copy link

qarlosalberto commented Mar 23, 2023 via email

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants