Proposal: a unified set_config / get_config / reset_config / report_config

[maliberty]

Problem

Tool settings are currently spread across per-tool commands: set_opt_config, set_cts_config, set_dft_config, set_global_sizing_config, and soon set_place_config (#11610). Each has its own matching reset_* and report_* command, though reset_dft_config is missing. Today:

• No persistence. Settings aren't saved in the odb, so a flow restarted from a checkpoint has to re-issue every set_*_config.
• No overall view. You can't see the full configuration state in one place, or list which knobs exist.
• No get_*. Settings can be printed but not read back into a script.
• Drift. The commands are inconsistent: missing resets, flags named like commands (-set_early_sizing_cap_ratio), and negated booleans (-disable_buffer_pruning).

Proposal

A single set of commands with hierarchical names:

set_config    opt.sizing_area_limit 100
get_config    opt.sizing_area_limit          ;# returns 100
reset_config  opt.sizing_area_limit
report_config opt.*                          ;# globs supported

At startup, each tool registers its settings with a type, constraints (e.g. non-negative integer), a default and a help string. set_config checks values when they're set, so errors report the same way from Tcl and Python:

[ERROR RSZ-xxxx] opt.sizing_area_limit -5 not >= 0

report_config shows every setting and marks the ones changed from their defaults. All settings are written to the odb and restored by read_db.

Today	Proposed
set_opt_config -limit_sizing_area 100	set_config opt.sizing_area_limit 100
set_opt_config -disable_buffer_pruning true	set_config opt.buffer_pruning false
set_cts_config -sink_clustering_size 20	set_config cts.sink_clustering.size 20
set_dft_config -max_chains 4	set_config dft.max_chains 4
set_global_sizing_config -max_iterations 50	set_config opt.global_sizing.max_iterations 50

Scope

Config holds sticky state that affects multiple commands or steps. Something that only affects one run of a command stays a flag on that command.

Proposed naming rules

• Segments are [a-z0-9_], separated by ., and can nest: place.global.density, place.detail.max_displacement.
• The top level is the flow step (place, cts, opt, route, dft), not the code module (gpl, rsz). Users think in steps, and modules get split or rewritten.
• One name per setting, no aliases. Old flags are accepted only through deprecation wrappers.
• Booleans are phrased positively (buffer_pruning false, not disable_buffer_pruning true), and names contain no verbs.
• Units are part of the name or a declared unit field, never left implicit.
• A top-level namespace (e.g. flow.*) may be reserved for ORFS and other flow scripts.

Questions

1. Is this a good idea? 😄
2. Migration. The set_*_config commands would eventually be removed. Should they first become deprecated wrappers that warn and call set_config?
3. Settings that vary by object, mode, corner or clock. Are there real cases where a setting must vary by corner/scenario (derates, cap ratios), clock (per-tree CTS settings), or module/region? Should those use an object qualifier on set_config, or stay as object commands like set_dont_use and set_dont_touch? Related: settings that name design objects (cts -buf_list, -root_buf, -skip_nets) — store them in the odb by name or by object, and what happens when the object is renamed or removed?
4. Naming. Any objections or additions to the rules above? Is global sizing its own top level or opt.global_sizing?
5. read_db. Does it overwrite values already set in the session, merge with them, or warn on conflict?
6. Effort levels. Should there be a global effort level with per-tool overrides?

   set_config effort low
   set_config cts.effort high    ;# cts uses high; every other tool inherits low

7. Unset vs. default. Do we need to tell "the user never set this" apart from "the user set the default value"? Q6 needs it for inheritance, and some defaults are computed at runtime (e.g. CTS picks buffers from the library).
8. Settings that depend on each other. Check them in set_config (which makes the order of set_config calls matter), or when the tool runs?

@QuantamHD @gadfort @oharboe @rovinski @donn @povik @mikesinouye

[povik]

Clarification

No persistence. Settings aren't saved in the odb, so a flow restarted from a checkpoint has to re-issue every set_*_config.

Not true of set_opt_config

[maliberty]

set_place_config is also going to save which just proves the need for a common approach as you only get a partial restore today.

[gadfort]

1. yes, good idea. If the values are stored in the DB, how are we registering them? and how is renames / read_db compatibility handled?
2. N/A seems fine
3. CTS certainly yes, per mode / scenario. Opt by module and region also seems like a possibility.
4. naming seems reasonable to me
5. How would you read_db with a session that is already has values set?
6. there should definitely be per category overrides if we have global states (and we would need real documentation about that low/medium/high/etc mean).
7. not sure
8. when the tool runs (it would painful to have order matter).

[maliberty]

Dumb but possible

set_config opt.sizing_area_limit 100
read_db foo.db [internally opt.sizing_area_limit =50]

[oharboe]

Thanks for writing this up. Before answering the numbered questions, I did a Chesterton's-fence check on why these commands exist at all.

Why they exist today

1. Tools call rsz internally. Timing-driven global_placement (findResizeSlacks), CTS and rmp all call into the resizer. The user has no command line to pass options on for those inner calls, so they read ambient config.
2. Persistence across processes. set_opt_config and set_global_sizing_config already store their values as odb block properties. set_cts_config and set_dft_config don't.
3. Long flag lists. set_cts_config was split out of clock_tree_synthesis to shorten its arguments. The same flags on clock_tree_synthesis call the same setters, so they're sticky as well.
4. Multi-command sequences. The DFT commands share one plan.

What I'd rather do

Separate design state from tool options, and keep the second out of global state entirely:

• Design state (netlist, placement, routing, constraints, dont_use/dont_touch) belongs in the odb and SDC. Arguably sizing restrictions such as an area or VT limit belong here too, as design constraints.
• Tool options are explicit arguments to the command. When an outer command calls rsz internally, it forwards an explicit options value rather than reading globals.
• Anything else that must cross processes goes in an explicit file that the flow writes and reads.

This is how ORFS already works:

• every stage does load_design X.odb Y.sdc and writes its SDC;
• side artifacts like mem.json and memories_inferred.json are explicit files that Make/Bazel can see;
• tool options are passed again from the flow script at each stage.

The current odb persistence breaks this. For example, EARLY_SIZING_CAP_RATIO set in resize.tcl silently persists into every later checkpoint, and nothing in the file list shows it.

Other domains with complex flows draw the same line:

• Climate and weather models: restart files carry state; namelists carry config and are re-read on every run.
• ML training: checkpoints carry weights and optimizer state; hyperparameters come from the current run's config. JAX made RNG keys explicit arguments for the same reason.
• Pipeline tools: each step declares its inputs, outputs and params.

On POLA: sticky persisted options are familiar from commercial tools, so they're unsurprising for interactive use. For checkpointed, scripted flows, though, a stage that behaves differently because of something stored by an earlier stage is the most surprising outcome.

Answers to the questions

1. Good idea? Consistent naming, get_*, validation and a report command are good. I'd apply them to command arguments rather than to a global store.
2. Migration: deprecate set_*_config with warnings, then remove them, rather than turning them into wrappers around a new global store.
3. Per-object/corner/clock settings: these are design constraints. Keep them as object commands like set_dont_use.
4. read_db: it shouldn't apply tool options. If they're recorded at all, record them as provenance: write_db notes the options in effect, report_config can show them, and read_db at most warns when the current explicit options differ.
5. Global effort with inheritance: I'd avoid it. It adds another layer of action at a distance.
6. Dependent settings: if settings are arguments, the tool checks them when it runs, and the order of separate set calls stops mattering.

[maliberty]

#2745 also talked about passing options through to subtools but it is terribly cumbersome in tcl. global_place would need to take all the options for resizer, global router, and pin placement. Anytime any of those adds an option all related commands would also need updating which is a maintenance burden. You can report the settings at anytime so they aren't really hidden state.

[oharboe]

One more angle: a primary user of OpenROAD going forward is AI agents driving flows, and I want to avoid sending them squirrel chasing. An agent reasons from the command lines and files it can see. If behavior depends on config set earlier in the session, or restored from an odb, the agent chases squirrels: timing constraints, placement density, the library, anything except the hidden setting that actually caused it. Copying a command from a log into a fresh session won't reproduce the result either, which sends it after yet another squirrel. If each command's behavior is fully determined by its inputs (odb, SDC, arguments), the log line is the reproducer and there are no squirrels to chase. That's a strong reason to keep tool options as explicit arguments.

[maliberty]

You can just teach your AI about report_config or use reset_config.

There is the opposite POLA argument - other tools work this way and people will be surprised OR doesn't.

[oharboe]

I dont recognize this problem from elsewhere.

Is it a Tcl problem?

Would it be OpenROAD's job to solve it if the flow was in Python or C++?

I know Tcl gas classes and methods in some package, but I dont consider anything but very simple Tcl idomatic Tcl.

Perhaps some worked A/B examples?

Perhaps an idiomatic Python solution would cast light on it?

[donn]

I will try to remember to give a more comprehensive reply in the morning but the general idea is I welcome this because this is quite close to how I do configuration.

I think persisting the configuration in the database at all is confusing though, but a read_db flag would take care of that.

@mole99 Opinion also appreciated here

[mole99]

I agree, organizing the configuration variables by category through a common interface is a nice improvement over the current state.

And for LibreLane, any persisting configuration in the odb will be cleared anyway, since we need to load all configuration variables when a step starts (a step could be replayed with a different configuration).

[rovinski]

I may be in the minority, but I never really liked the idea of configurations at all. Configurations lead to hidden state and there can be confusion about which configurations affect which tools. Yes, there are reporting of configs, but a user has to know how to do this and also think about checking the configs in the first place. When it's passed by argument, it's clear prima facie what settings are being applied to the tool.

Persisting the configurations in the db can also be frustrating because if there was an unintentional saving of an option into a db, the user can't e.g. exit OR, reload the design, and have the problematic option reset.

Maybe this is moot in the age of AI but that was my experience with a certain proprietary tool that likes to use configs somewhat haphazardly.

[oharboe]

I am in minority too!

Wonderfully articulated!

How can I know which config affects which cmd bug, by bug?

I want log_cmd to tell me everything.

I am surprised by the enthusiasm for this config feature, I think I am missing something important about the use case for it to make sense.

[maliberty]

I hear either dislike for db stored options or irrelevance due to overriding. It sounds like that isn't a win.

I'm trying to reconcile this with #2745 where I argued for explicit arguments in Python. I still prefer that but didn't see a good parallel for TCL. Claude suggests using a dict as in this elaborate example:

set gp_opts {
  density 0.72
  initial {
    max_iter 30
    max_fanout 150
  }
  timing {
    enabled 1
    repair_timing 1
    nets_percentage 15
    net_weight_max 4
    rsz {
      sizing_area_limit 50
      buffer_pruning 0
      early_sizing_cap_ratio 1.2
    }
  }
  routability {
    enabled 1
    use_rudy 0
    max_density 0.95
    target_rc_metric 1.05
    inflation { ratio_coef 2.5 max_ratio 3 max_total 0.25 }
    grt {
      layer_adjustments { metal2 0.5 metal3 0.3 }
      congestion_iterations 30
    }
  }
}

global_placement -options $gp_opts

You have the usual TCL tricks to assemble the dictionary and use variables. It does push validation more heavily on the c++ code as dictionaries have no schema. Opinions? I'm guessing the 'minority' will like this more.

[oharboe]

Very nice explanation of the use-case.

However, I want to see filtering per stage so I don't have to re-run in bazel-orfs from the top when e.g. grt options change for a 5 our parent(XiangShan would be an example...).

What do you have in mind for for log_cmd?

[oharboe]

I wonder if what we can do is to use variables.yaml to generate something like the above.

[oharboe]

Bear with me, this is a long post. A lot of work has already gone into this problem, and it touches far more code than one would think. The length is needed to gather the threads in one place; the code already working on it is listed at the end.

I wonder if what we can do is to use variables.yaml to generate something like the above.

Here is what that looks like if OpenROAD doesn't change at all. This isn't an argument against changing OpenROAD; it's a baseline to compare against.

A sidecar next to each .odb, managed with variables.yaml

A design's settings start in a design.json that config.mk points to. From there, every stage result gets a plain JSON file next to it: 1_synth.odb + 1_synth.vars.json, 2_floorplan.odb + 2_floorplan.vars.json, and so on. The .odb is never written to, so read_db keeps its current meaning and no state is persisted in the database.

variables.yaml already lists, for each variable, which stages read it (stages:). The stage script uses that list to manage the sidecar. For floorplan, floorplan.tcl does this:

1. Read 1_synth.vars.json.
2. Inject the variables whose first stage in stages: is floorplan, taking their values from design.json, with defaults from variables.yaml. CORE_UTILIZATION ([floorplan]) enters here. A grt-only variable like GRT_SEED ([grt]) doesn't, so floorplan's result can't depend on it.
3. Run using only what is in the sidecar.
4. Prune the variables whose last stage is floorplan, so CORE_UTILIZATION doesn't travel on to place. PLACE_DENSITY ([floorplan, place]) stays until place has run.
5. Write 2_floorplan.vars.json.

A stage can compute values for a later stage. The synth stage script can read the area it just produced and write CORE_UTILIZATION and PLACE_DENSITY into 1_synth.vars.json for floorplan to read. It's the same mechanism as a value from design.json.

Every option spelled out. The sidecar holds the resolved value of every variable that is still live, defaults included. It is the full record of what each stage ran with, and you can diff two runs with diff, without scraping the log or calling report_config.

Whether a later stage may override a value an earlier stage injected, for example replacing the flow's default PLACE_DENSITY, is a decision for later.

Why JSON. Python scripts can read it without loading OpenROAD. People and AI can inspect it directly. And it is tiny: a stage's live variables fit easily in any AI context, every time.

Why this stops the rebuild from the top

A stage's inputs are the previous .odb and .vars.json, plus the variables injected at that stage. Changing GRT_SEED changes only grt's inputs, so a build system that keys on those inputs (bazel-orfs does) keeps synth through cts cached. Plain make needs no new rule: you rerun from grt, and every earlier result is still valid.

bazel-orfs already does all of this, in Starlark rather than in the stage scripts (see the list at the end). It is not a constraint here: as its maintainer, I'll rework it onto whichever solution ORFS and OpenROAD settle on.

What it would take in ORFS, OpenROAD untouched

• A Tcl helper in load.tcl, called at the top and bottom of each stage script, that does read / inject / prune / write as above. It reads variables.yaml directly.
• Nothing else is required. A variable with no stages:, or with "All stages", is injected at synth and travels to the end, so changing it reruns from the top, as it does today. Narrowing a variable's stages: list is optional and can be done one variable at a time; each one narrowed means a change to it reruns less of the flow.

Aside, and scope creep: two sets of names

Today a user has to learn two sets of names: the ORFS variables in variables.yaml, and the OpenROAD commands and options they map to.

log_cmd already bridges the two well. It prints each command with the options that command takes, and only those. That matters: if every command received every option, the user would have to work out for themselves which options were dead, meaning they had no effect on that command.

So in a way this comes full circle: explicit command lines, as log_cmd prints them today. The new part is the sidecar, which records the value of every option a stage still needs and nothing else.

Aside, and scope creep: what else should be a sidecar?

This raises the question of whether anything else in the .odb today should live in a sidecar, or the other way round. Size decides. The .sdc is a sidecar today, and #11260 moves it into the .odb: restoring an ID-based record is faster than parsing text, and a large design's .sdc is not something AI should read raw anyway; ask STA instead. The options sidecar is the opposite case: tiny, and its whole value is that people, Python and AI read it directly. The settings set_opt_config and its siblings persist in the .odb today are the obvious candidates to move out, since that persistence is the hidden state several people in this thread raised.

Code already working on this problem

ORFS:

• flow/scripts/variables.yaml, stages:: which stages read each variable. 199 of 249 variables have a stage list; the other 50 are unscoped or "All stages".
• flow/scripts/util.tcl, log_cmd: prints each command with the options it actually got.
• flow/scripts/util.tcl, append_env_var and repair_timing_helper: the variable-to-option mapping, e.g. SETUP_SLACK_MARGIN → repair_timing -setup_margin. Floorplan, cts and grt all call the helper.
• flow/scripts/global_route.tcl: maps SETUP_SLACK_MARGIN to -setup_margin a second time by hand.
• flow/scripts/util.tcl, the PLACE_DENSITY_LB_ADDON handling: PLACE_DENSITY is changed before it reaches global_placement -density.

OpenROAD:

• set_opt_config and its siblings persist some settings in the .odb (@povik's point above).
• dbSta: store the timing constraints in the .odb (write_db -sdc) #11260 (write_db -sdc, open): the .sdc goes into the .odb.

bazel-orfs:

• private/stages.bzl, the "two time domains" block: per-stage filtering from variables.yaml stages:, decided once at analysis time.
• private/merge_arguments.py: applies that filter as a denylist when it writes each stage's config.mk.
• OrfsInfo.arguments: a set of JSON files that travels with the .odb from stage to stage. This is the sidecar.
• orfs_arguments(script = ...) and compute_floorplan_shape.tcl: a Tcl script reads 1_synth.odb and adds CORE_UTILIZATION and CORE_MARGIN for floorplan.
• later_stage_arguments (private/attrs.bzl): carries later stages' literal values in a deployed tree.

[maliberty]

A bit orthogonal to the OR proposal. In ORFS we could implement a side car to carry the options that are going to be passed but the existing infrastructure already does that. How does the extra file benefit above the existing mechanism. It will mostly be generated from it.

[oharboe]

I think bundling options into .odb will cause a full rebuild if I change e.g. a grt option. Certainly I need to find a way to filter and inject at the right time.

Do you plan to work out the ramifications for bazel-orfs?

[maliberty]

#11755 shows a prototype for the global_placement command of the idea of explicit options in tcl, pyton, and c++. (AI written but I did review it).

[oharboe]

I asked Claude to analyze #11755 against bazel-orfs's use cases. I haven't read the analysis in detail, but I like that log_cmd still works, that options are explicit and that the commands no longer reach into global state.

Where I'd like this to go: whittle global state down until commands take only explicitly passed options, so that the log_cmd line tells the whole story. Copying it into a fresh session on the same .odb would then reproduce the result.

Claude's analysis:

Global state left in the converted commands, the list to whittle down:

• the std::optional fields (max_utilization, critical_nets_percentage, res_aware_nets_percentage, use_cugr) keep whatever an earlier command set when they are left unset;
• max_wire_length = 0 falls back to the resizer's stored value;
• set_opt_config settings (sizing area limit, buffer pruning, early sizing cap ratio) are not in the structs, so timing-driven placement still reads them from the resizer and the .odb.

bazel-orfs use cases:

• Rebuild only from the first stage that reads a changed option. Works: no options are written to the .odb, so my concern above about a grt option rebuilding from the top does not apply. Options recorded in the .odb even as provenance would break this. An option change that doesn't change the result would still change the .odb's bytes, and every later stage would miss the cache.
• Branching from a shared prefix (sweeps, a stage rerun on a cached .odb). Works for the converted commands. set_opt_config persistence in the .odb still leaks into branches; ORFS's EARLY_SIZING_CAP_RATIO is the case.
• Values computed from an earlier stage (a script reads 1_synth.odb and writes CORE_UTILIZATION for floorplan). Works as long as ORFS builds the -options dict in the stage that calls the command. default_options lets such a script read a default instead of copying it.
• Per-stage filtering through variables.yaml stages:. place now also takes grt and repair_design options, with different defaults (routability.global_route.congestion_iterations is 0 in gpl, 50 in global_route). One ORFS variable per knob, put into the dict by the stage script, keeps today's granularity. One dict-valued variable per command would rerun from the first stage that reads it whenever any key changes.