variables: move code from Makefile to scripts/defaults.py

[oharboe]

Also, ported platforms/nangate45/config.mk to config.py.

Next up is porting all the rest of the platforms/**/config.mk files to config.py, after which more stuff from Makefile can be moved into scripts/config.py

The flow.sh and synth.sh scripts and interface are work in progress and they won't be ready before all platforms/**/config.mk are ported to config.py.

The interface will probably be something like:

$ make DESIGN_CONFIG=designs/nangate45/config.mk clean_synth floorplan
$ DESIGN_NAME=gcd CORE_UTILIZATION=55 PLACE_DENSITY_LB_ADDON=0.20 PLATFORM=nangate45 PLATFORM_DIR=platforms/nangate45 scripts/flow.sh 2_1_floorplan floorplan
Running floorplan.tcl, stage 2_1_floorplan
[deleted]
Design area 650 u^2 57% utilization.
Elapsed time: 0:00.40[h:]min:sec. CPU time: user 0.33 sys 0.09 (106%). Peak memory: 121724KB.
$

[oharboe]

@maliberty Ready for review. I think this approach is quite promising. The next step is to decide if we want to go this way, and if so, merge this PR.

The next steps are:

• convert one platform config.mk at the time to config.py
• once all platforms are converted, convert the rest of the variables in Makefile to defaults.py code
• finalize the flow.sh and synth.sh interface
• convert bazel-orfs to use flow.sh
• convert bazel-orfs to use synth.sh

[oharboe]

@maliberty Green 😌

[maliberty]

I'm going to need a bit of time to carefully study this as it is the first step of a big change.

[oharboe]

I'm going to need a bit of time to carefully study this as it is the first step of a big change.

Yes, not your garden variety rubber stamping review...

I will try to tackle asap7 config.mk to .py, but in a not ready for review follow up PR. I will try to leave this PR alone as much as possible.

[oharboe]

@maliberty After tinkering with asap7. I believe there are some bugs and subtle intentional and not so intentional default variable handling with early and late expansion we'll get to contend with... #2990

I don't know that there is a way around this if we want to move forward. At least the issue demonstrates and easy way to do A/B comparisons in a second without having to run flows.

[oharboe]

I studied platforms/sky130/config.mk platforms/gf180/config.mk and platforms/ihp-sg13g2/config.mk. asap7 seems like the most advanced of these config.mk files, so the rest should be fairly easy after asap7 is in the can.

[rovinski]

config.py as currently set up seems like a substantial downgrade in readability vs. mk. Why not JSON?
If the goal is to scrap the makefile-based system (which I disagree with) then why not move to an established flow management system like mflowgen or Hammer or SiliconCompiler... etc?

[oharboe]

mflowgen seems dead from commits visible in last year. fusesoc is somewhat domain related, but reimplements things like artifacts, didnt look at it further. bazel is supported by google and buck2 by facebook; similar concepts. siliconcompiler could probably do with being rearticulated on something like bazel/buck2 as one company can't hope to compete with the infrastructure of google/facebook to do distributed builds, etc.

for now, I am trying to think about the use-cases.

LIB_FILES and ADDITIONAL_LIBS are interesting. These variables define file dependencies, defaults and they demonstrate the value of lazy recursive evaluation in make. ADDITIONAL_LIBS is appended to by user, pdk and the makefile and LIB_FILES is defined by the PDK but ADDITIONAL_LIBS is added to it, but it is used by the makefile to invoke yosys and openroad.

[oharboe]

This needs more pondering of usecases.

[rovinski]

mflowgen seems dead from commits visible in last year.

It's not dead, it's just stable. It does the job that it needs to do AFAICT.

My main concern is really just how much work does it take for a new designer to understand how to use the flow? The flow IMO should either steer towards

1. being intuitive and have a shallow enough learning curve to start modifying it quickly, or
2. being automated to a degree where the user doesn't need to handle complicated details

Right now, it's kind of trending towards neither. It's becoming more complicated and the users have to understand that complication in order to try new things.

Maybe ask is this the kind of flow that you from 5 years ago would want? Would this be easier or harder to understand than it was before? I try to teach this to a new batch of grad students every year and its become harder and harder to explain to fresh students how the flow internals work.

[oharboe]

mflowgen seems dead from commits visible in last year.

It's not dead, it's just stable. It does the job that it needs to do AFAICT.

My main concern is really just how much work does it take for a new designer to understand how to use the flow? The flow IMO should either steer towards

1. being intuitive and have a shallow enough learning curve to start modifying it quickly, or
2. being automated to a degree where the user doesn't need to handle complicated details

Right now, it's kind of trending towards neither. It's becoming more complicated and the users have to understand that complication in order to try new things.

Maybe ask is this the kind of flow that you from 5 years ago would want? Would this be easier or harder to understand than it was before? I try to teach this to a new batch of grad students every year and its become harder and harder to explain to fresh students how the flow internals work.

As simple as possible, but no simpler.

I think bazel-orfs is simpler to set up and use than ORFS provided we accept that there are some modern concepts that the student must learn to work on non trivial projects: git, build systems more complex than make. I think bazel is simpler than cmake(fewer barnacles) and a lot more powerful.

Students no longer fight learning git and I dont think anyone at unis fight teaching it.

I predict Bazel/buck2 is representative of a new generation build tools that will be taught at unis, but like git it will be fought by teachers and students for a while

Did you consider looking at bazel-orfs?

fusesoc is another build framework that reimplements infrastructure that exists in Bazel/Buck2:

olofk/serv#146

[oharboe]

@rovinski I read quickly about mflowgen. It lists advantages that are provided as part of the bazel infrastructure, but it is unclear to me what it offers beyond that. Reimplementing something that Bazel offers is a liability to me. I find that Bazel is hard to learn because it introduces new fundamental concepts like a label. Once I understand these concepts, I find that it is necessary complexity that "easier to learn" systems simply lack and as you scale up.usage you find yourself suffering pain for the lack of the necessary complexity.

I would give mflowgen a wide berth unless it can be rearticulated on top of a stronger more sustainable and maintained infrastructure line Buck2 or Bazel.

As for students at uni: isn't uni about learning hard valuable concepts rather than easy to pick up bad habbits?

[oharboe]

@rovinski My plan is to slowly replace make and other ad hoc build systems for this use case https://github.com/The-OpenROAD-Project/OpenROAD-flow-scripts/blob/master/flow%2Fdesigns%2Fasap7%2Fmock-array%2Fconfig.mk#L57

I want to explore relative power consumption using activations from a simulation .vcd file for RTL design paramaters as well as ORFS backend parameters.

This is a multidiscilinary build infrastructure challenge.

Bits of pieces are coming together in Bazel from Chisel, to Verilaror, to OpenROAD to bazel-orfs.

Bazel is replacing a ton of ad hoc script, python, docker and whatnot.

Someone who is somewhat versed in Bazel will be able to discover and dive into areas of the flow of interest.

[rovinski]

Students no longer fight learning git and I dont think anyone at unis fight teaching it.

I would love if universities forced students to learn git or any version control at all. But many (most?) students heading into physical design are ECE students who start from engineering physics and math, not from software development. I encounter many students who have problems performing basic tasks in the terminal. Ideally these knowledge issues should be fixed upstream in another course, but most ECE programs are simply not set up to force students to learn software. I think they should, but I can't bend the academic world to my whim.

I would give mflowgen a wide berth

That's fine, I'm not married to it. It was developed by a colleague of mine but I don't use it.

As for students at uni: isn't uni about learning hard valuable concepts rather than easy to pick up bad habbits?

Aspirationally, yes, practically no. We cannot take half of the time of a VLSI course to introduce students to complicated software topics. We also can't force them to learn these things on their own because we are already moving at full speed just to try and cover the VLSI topics in the syllabus. The net result would be that students simply drop the class or discontinue taking the VLSI sequence if they get overloaded.

My plan is to slowly replace make and other ad hoc build systems for this use case

Replace? Or add as substitute? Again, a couple years ago I could have people jump into the flow and modify it because they could see each step in order and understand what it was doing. The syntax was simple. I would not be able to hand a VLSI student the config.py in this example and have them understand it and modify it.

I want to explore relative power consumption using activations from a simulation .vcd file for RTL design paramaters as well as ORFS backend parameters.

This is a complicated example. This is not the kind of thing new students would be doing (although I also don't see a fundamental need to change build systems for this because I have done things like this in Make before...)

If this is the direction ORFS is headed in, I may just need to swap over to a different flow or perhaps jump back to teaching commercial flows.

[oharboe]

Thank you for shedding light on the many competing concerns for teaching and learning ASIC and EDA tooling. Makes sense.

My goal would be to untangle the concerns in ORFS such that it can be used with whatever build system anyone sees fit. bazel-orfs and make would be the two that make sense to me.

I think mflowgen was ahead of its time and doesnt have a future unless it is rearticulated: it is solving bazel type problems but without the momentum of bazel.

I would advice keeping it very simple and not introduce any build concepts or if things get more complicated, take the hit of learning bazel instead of adding complexity piecemeal to a system. My experience is that it doesnt take much complexity to makefiles before Bazel is easier. mock-array is an example in point.

as for .vcd power flows: we have a 320gbyte .vcd file so make is a non starter for the issues that arise. bazel allows us to handle such hetereogenous(multidomain) flows easily.

[maliberty]

@rovinski I think it is more important to teach the steps of the flow and the contents of scripts/*tcl then the details of Makefile or any replacement. How jobs are launched will vary across companies. Most designers are not flow developers and flows themselves vary a lot in style (eg OL/ORFS/SiC/mflowgen/Hammer/fusesoc/etc without even getting into proprietary foundation flows).

[rovinski]

@maliberty but students also want to understand how things fit together. Understanding a flow step is like understanding a puzzle piece. And I can tell them how the puzzle pieces fit together, but it's a different thing for them to see how the pieces are put together themselves.

I wish you the best of luck on this Bazel project and I hope it works out.

we have a 320gbyte .vcd file

That sounds like a methodology issue. It's unfortunate that major FOSS simulators don't support dumping SAIF which would remedy the issue, but you should look into dumping VCD only for the critical or hot loops of the program you're simulating.

[oharboe]

@rovinski We have a business, not engineering requirement to process one full .vcd file. better .vcd to .saif converters are in flight(from anrmicro) but .saif is a bit too fringe for verilator to implement directly.

bazel handles this all easily, so it is not a technical problem, even though it is an eyesore.

for engineering purposes we do sampling of course.

[oharboe]

@rovinski @maliberty I had a realisation and I have a plan for a new PR. The problem that needs to be solved is framework inversion, the dependency on the make binary is unproblematic if it is an implementation detail.

The user should be able to use ORFS with any build system he chooses. Stay tuned for a new PR.

make is good for a simple lab exercise to get a feel for .tcl scripts and major variables like placement density with a caveat to students that real flows can be more complicated and need more powerful concepts to include things such as RTL generation&simulation as well as backend exploration flows and power simulations. bazel-orfs is an example of more advanced build systems.

[oharboe]

@rovinski BTW, as far as simplicity for a lab exercise and focusing on the domain issues, I'm thinking Bazel can be less complex when it comes to dependencies. No sudo access required, no learning and debugging how to build things.

1. install bazelisk. Nothing else.
2. Run bazelisk run foo_grt /tmp/foo gui_grt

Now everything is built and the GUI is running on the machine, no need to run setup scripts that mangle machines, no sudo access required.

Also, it will run quickly: ORFS should come with a read only artifact repository set up.

Example of a 1 hour lab exercise(provided adeqaute background knowledge about domain):

1. Install Bazelisk on laptops/workstations without root access (10 minutes)
2. Run bazelisk run :gcd_route /tmp/route gui_route and wander through the GUI
3. Look at a more complex example, talk about macros, examine timing histograph, discuss reg2reg vs. reg2out, for macros, etc.: bazelisk run //designs/asap7/mock-array:MockArray_route /tmp/route gui_route

[rovinski]

For class purposes we pre-install OR in a shared directory on a server, so the build process isn't necessarily important, it's more about the flow.

[oharboe]

For class purposes we pre-install OR in a shared directory on a server, so the build process isn't necessarily important, it's more about the flow.

Makes sense. Flow artifacts are cached w bazel-orfs.