Add support for multicore in global routing, especially focusing on speeding up congestion cleanup iterations by ebrevdo · Pull Request #9598 · The-OpenROAD-Project/OpenROAD

ebrevdo · 2026-03-02T22:56:19Z

Summary

Make snapshot-batched FastRoute the default semantic mode for global_route by setting the default -snapshot_batched_width to 16. Users can still force the legacy non-batched path with -snapshot_batched_width 0, while execution width continues to follow set_thread_count.

The public GRT surface is updated so the new default is visible and testable through Tcl, SWIG, and GlobalRouter, instead of being hidden behind internal-only knobs. The same range also wires OpenROAD thread-count changes into GRT so router-side reporting and post-route accounting use the requested thread count.

src/grt/src/GlobalRouter.tcl adds and defaults -snapshot_batched_width
src/grt/README.md documents the new default and the 0 escape hatch
src/grt/src/GlobalRouter.i exposes get_snapshot_batched_width() and get_snapshot_batch_count()
src/grt/include/grt/GlobalRouter.h and src/grt/src/GlobalRouter.cpp carry the new width/thread plumbing
src/OpenRoad.cc now forwards setThreadCount() into GlobalRouter::setNumThreads()

Router Changes

The FastRoute core now batches contiguous nets against frozen planar congestion snapshots, routes each batch on worker copies, and then commits the results back in deterministic order. The implementation keeps bounded serial cleanup for late overflow-reduction iterations and skips snapshot batching in cases that are unlikely to benefit, such as debug flows, tiny net sets, or non-soft NDR nets.

The same range also adds better internal timing and accounting around the router phases, and parallelizes some read-only reporting work so multi-threaded runs scale past the maze kernel itself. That includes wirelength, per-layer resource totals, and congestion summaries.

src/grt/src/fastroute/src/maze.cpp splits FastRouteCore::mazeRouteMSMD() from mazeRouteMSMDSequential() and adds snapshot-wave execution
src/grt/src/fastroute/src/FastRoute.cpp adds snapshot worker construction, batch heuristics, cleanup gates, and per-phase metrics
src/grt/src/fastroute/include/Graph2D.h and src/grt/src/fastroute/src/graph2d.cpp add Graph2D::copyRoutingStateFrom()
src/grt/src/GlobalRouter.cpp parallelizes computeWirelength() and propagates snapshot width into configFastRoute()
src/grt/src/fastroute/src/FastRoute.cpp parallelizes getOriginalResources() and computeCongestionInformation()

Tests And Compatibility

The regression coverage is retargeted to a real congestion-heavy stress case instead of relying on earlier goldens that no longer guaranteed batching. The new smoke, incremental-state, bus-route, and thread-report tests all exercise the current width-16 default, and the shared helper now asserts that snapshot batching actually executed.

maliberty · 2026-03-02T22:58:11Z

Please note you need to sign your commits to pass DCO (ie git commit -s). See the DCO link for info on how to fix existing commits.

maliberty · 2026-03-02T22:59:43Z

 | `-critical_nets_percentage` | Set the percentage of nets with the worst slack value that are considered timing critical, having preference over other nets during congestion iterations (e.g. `-critical_nets_percentage 30`). The default value is `0`, and the allowed values are integers `[0, MAX_INT]`. |
 | `-skip_large_fanout_nets` | Skips routing for nets with a fanout higher than the specified limit. Nets above this pin count threshold are ignored by the global router and will not have routing guides, meaning they will also be skipped during detailed routing. This option is useful in debugging or estimation flows where high-fanout nets (such as pre-CTS clock nets) can be ignored. The default value is 0, indicating no fanout limit. The default value is `MAX_INT`. The allowed values are integers `[0, MAX_INT]`. |
 | `-allow_congestion` | Allow global routing results to be generated with remaining congestion. The default is false. |
+| `-multicore` | Enable the optional multicore Track B routing path. |


What does "Track B routing path" mean?

gemini-code-assist

Code Review

This pull request introduces multicore support for the global router, which is a significant performance improvement. The changes are extensive, including a snapshot-based parallel routing approach for maze routing and parallelization of reporting functions. The implementation of the parallel logic seems sound.

My review focuses on a few areas for improvement:

Refactoring a large manual state-copying function, though this is not considered a high priority for maintainability.
Removing redundant helper functions and string conversions that are now handled by a new generic logging mechanism introduced in this PR.

Overall, this is a great enhancement to the tool.

gemini-code-assist · 2026-03-02T23:00:16Z

                              "port \"{}\" will be pushed back the queue.",
                              ex.what(),
-                              worker_address,
+                              worker_address.to_string(),


With the new logArg helper in utl/Logger.h, you should be able to pass worker_address directly to the logger without calling .to_string(). The logger will use fmt::streamed for types with operator<<.

Suggested change

worker_address.to_string(),

worker_address,

gemini-code-assist · 2026-03-02T23:00:16Z

      auto worker = copy.top();
      logger_->report("Worker {}/{} handled {} jobs",
-                      worker.ip,
+                      worker.ip.to_string(),


With the new logArg helper in utl/Logger.h, you should be able to pass worker.ip directly to the logger without calling .to_string(). The logger will use fmt::streamed for types with operator<<.

Suggested change

worker.ip.to_string(),

worker.ip,

gemini-code-assist · 2026-03-02T23:00:16Z

+std::string rectToString(const Rect& rect)
+{
+  std::ostringstream stream;
+  stream << rect;
+  return stream.str();
+}


With the new logArg helper in utl/Logger.h, this rectToString helper function and the inclusion of <sstream> are no longer necessary. The logger can now automatically handle types that have an operator<<, like odb::Rect, by using fmt::streamed. You can remove this helper and call the logger with the odb::Rect object directly.

gemini-code-assist · 2026-03-02T23:00:16Z

             guide->getOID(),
             layer->getName(),
-             box);
+             rectToString(box));


You can remove rectToString and pass box directly to the logger. The new logArg helper will handle it.

box);

gemini-code-assist · 2026-03-02T23:00:16Z

             guide->getId(),
             guide->getLayer()->getName(),
-             guide->getBox());
+             rectToString(guide->getBox()));


You can remove rectToString and pass guide->getBox() directly to the logger. The new logArg helper will handle it.

guide->getBox());

gemini-code-assist · 2026-03-02T23:00:16Z

+std::string rectToString(const Rect& rect)
+{
+  std::ostringstream stream;
+  stream << rect;
+  return stream.str();
+}


With the new logArg helper in utl/Logger.h, this rectToString helper function and the inclusion of <sstream> are no longer necessary. The logger can now automatically handle types that have an operator<<, like odb::Rect, by using fmt::streamed. You can remove this helper and call the logger with the odb::Rect object directly.

gemini-code-assist · 2026-03-02T23:00:16Z

                 guide_id,
                 guide->getLayer()->getName(),
-                 guide->getBox());
+                 rectToString(guide->getBox()));


You can remove rectToString and pass guide->getBox() directly to the logger. The new logArg helper will handle it.

guide->getBox());

gemini-code-assist · 2026-03-02T23:00:16Z

                 guide->getId(),
                 layer->getName(),
-                 guide->getBox());
+                 rectToString(guide->getBox()));


You can remove rectToString and pass guide->getBox() directly to the logger. The new logArg helper will handle it.

guide->getBox());

github-actions

clang-tidy made some suggestions

github-actions · 2026-03-02T23:10:00Z

-    total_wirelength += computeNetWirelength(net_route.first);
+#pragma omp parallel for num_threads(num_threads_) reduction(+ : total_wirelength)
+  for (int i = 0; i < static_cast<int>(routed_nets.size()); i++) {
+    total_wirelength += computeNetWirelength(routed_nets[i]);


warning: use range-based for loop instead [modernize-loop-convert]

Suggested change

total_wirelength += computeNetWirelength(routed_nets[i]);

(auto & routed_net : routed_nets)routed_net

github-actions · 2026-03-02T23:10:00Z

+void FastRouteCore::applySnapshotBatchRoute(const int net_id, StTree&& sttree)
+{
+  FrNet* net = nets_[net_id];
+  const int edge_cost = net->getEdgeCost();


warning: 'signed char' to 'const int' conversion; consider casting to 'unsigned char' first. [bugprone-signed-char-misuse]

const int edge_cost = net->getEdgeCost(); ^

github-actions · 2026-03-02T23:10:00Z

 #include <cmath>
 #include <cstdint>
+#include <limits>
 #include <map>


warning: included header limits is not used directly [misc-include-cleaner]

Suggested change

#include <map>

#include <map>

github-actions · 2026-03-02T23:10:00Z

 namespace grt {

 using utl::GRT;
+using utl::DebugScopedTimer;


warning: using decl 'DebugScopedTimer' is unused [misc-unused-using-decls]

using utl::DebugScopedTimer; ^

Additional context

src/grt/src/fastroute/src/maze.cpp:24: remove the using

using utl::DebugScopedTimer; ^

github-actions · 2026-03-02T23:10:00Z

+  {
+    const int wave_size
+        = std::min((int) batch_net_ids.size(), std::max(1, num_threads_));
+    std::vector<std::unique_ptr<FastRouteCore>> workers;


warning: no header providing "std::unique_ptr" is directly included [misc-include-cleaner]

src/grt/src/fastroute/src/maze.cpp:8:

- #include <set> + #include <memory> + #include <set>

github-actions · 2026-03-02T23:10:00Z

-  std::vector<bool> pop_heap2(y_grid_ * x_range_, false);
+  src_heap.clear();
+  dest_heap.clear();
+  std::fill(pop_heap2.begin(), pop_heap2.end(), false);


warning: use a ranges version of this algorithm [modernize-use-ranges]

Suggested change

std::fill(pop_heap2.begin(), pop_heap2.end(), false);

std::ranges::fill(pop_heap2,, false);

github-actions · 2026-03-02T23:10:00Z

 #include <iosfwd>
 #include <numbers>
+#include <string>
 #include <tuple>


warning: included header string is not used directly [misc-include-cleaner]

Suggested change

#include <tuple>

#include <tuple>

github-actions · 2026-03-02T23:10:00Z

+
+namespace {
+
+std::string rectToString(const Rect& rect)


warning: no header providing "std::string" is directly included [misc-include-cleaner]

src/odb/src/db/dbGuide.cpp:15:

+ #include <string>

github-actions · 2026-03-02T23:10:01Z


+namespace {
+
+std::string rectToString(const Rect& rect)


warning: no header providing "odb::Rect" is directly included [misc-include-cleaner]

src/odb/src/db/dbJournal.cpp:23:

- #include "utl/Logger.h" + #include "odb/geom.h" + #include "utl/Logger.h"

github-actions · 2026-03-02T23:10:01Z


+namespace {
+
+std::string rectToString(const Rect& rect)


warning: no header providing "std::string" is directly included [misc-include-cleaner]

src/odb/src/db/dbJournal.cpp:9:

+ #include <string>

Convert streamable custom types before logging or formatting in a few cleanup paths, and teach utl::Logger to fall back to streamed formatting for non-formattable arguments. Signed-off-by: Eugene Brevdo <ebrevdo@users.noreply.github.com> Co-authored-by: Codex <noreply@openai.com>

Reuse 2D maze scratch storage across iterations, retain capacity in hot route containers, replace the linear-search 2D decrease-key path with an indexed heap, and plumb OpenRoad thread count into safe read-only GRT/FastRoute reductions. Signed-off-by: Eugene Brevdo <ebrevdo@users.noreply.github.com> Co-authored-by: Codex <noreply@openai.com>

Complete the remaining exact-track work from optimize_global_route.md by adding phase timing instrumentation in FastRouteCore::run and a focused threaded grt regression. The FastRoute change wraps the major routing phases with DebugScopedTimer scopes and emits phase metrics for the exact-preserving path without changing existing routing behavior or the normal verbose report surface. The new timings cover the full run, initial RSMT, initial routeLAll, congestion-driven RSMT, via-guided routeLAll, spiral routing, initial routeZAll, monotonic routing, overflow iterations, and finalization. The test change adds src/grt/test/thread_count_reports.tcl, which runs global_route -verbose at thread counts 1 and 2 and compares only the reporting paths that were newly parallelized: the routing resources analysis block, the final congestion report block, and the total wirelength line. CMake wires it in as a pass/fail regression, and Bazel uses an explicit regression_test target with check_passfail=True instead of the default log-diff flow. The earlier indexed 2D decrease-key heap experiment was not a win on the stress profile, so this final exact-track state drops A1 and keeps the scratch reuse, capacity retention, timing, and reporting-threading work that held up under profiling. Validation: - git -c core.fsmonitor=false diff --check - dev-cpu-boron rebuild of build-linux-tests after reconfiguring against /root/openroad-deps completed successfully - ctest --output-on-failure -R '^grt\.thread_count_reports\.tcl$|^grt\.report_wire_length1\.tcl$' passed (2/2) - ctest -N confirmed grt.thread_count_reports.tcl registration - stress-case gprof on shuffle_stress showed the A1 rollback recovered the prior regression Signed-off-by: Eugene Brevdo <ebrevdo@users.noreply.github.com> Co-authored-by: Codex <noreply@openai.com>

Add the optional Track B multicore global routing path to FastRoute. The new path batches contiguous nets against frozen planar congestion snapshots, routes each batch on worker copies, commits results deterministically, and falls back to bounded serial cleanup later in overflow reduction. The default path remains the existing serial route flow unless `global_route -multicore` is enabled. Expose the Track B surface through the Tcl, SWIG, and GlobalRouter plumbing, and add the internal FastRoute helpers needed to build snapshot workers, copy planar routing state, and apply routed batches back into the main overflow loop. Add Graph2D routing-state copy support and Track B timing metrics so the new path can be measured without changing the default router behavior. Keep the measured multicore taper behavior baked into the implementation instead of exposing runtime tuning knobs. The final tree uses the validated 25/50/75 taper trajectory and always uses the built-in thread-count heuristic for snapshot batch sizing inside Track B. Local benchmark, regression, profiling, and initiative artifacts are intentionally kept out of the git tree. This commit carries only the product code and user-facing command documentation that back the optional Track B path. - add the `-multicore` opt-in routing control - implement snapshot-batch workers and deterministic batch commit order in `FastRouteCore` - copy planar routing state into per-wave workers with `Graph2D::copyRoutingStateFrom` - bound late Track B cleanup with best-route snapshots and patience tracking - export snapshot batch sync, route, and apply metrics for profiling - route the new control through Tcl, SWIG, and `GlobalRouter` - keep the taper schedule fixed at the validated default operating point - always use the built-in batch-sizing heuristic inside Track B - document the new grt command option in the README Signed-off-by: Eugene Brevdo <ebrevdo@users.noreply.github.com> Co-authored-by: Codex <noreply@openai.com>

Fix Track B state management for incremental reroute and snapshot workers, tighten batching behavior for tiny designs, and add multicore regression coverage including a fixed-thread congestion7 variant. Validated on dev-cpu-fluorine with the focused grt multicore slice. Signed-off-by: Eugene Brevdo <ebrevdo@users.noreply.github.com> Co-authored-by: Codex <noreply@openai.com>

Reset the public snapshot-batched-width default to 0, keep explicit width-16 coverage in dedicated snapshot-batched regressions, and rename the old multicore-specific tests to the snapshot-batched terminology. Add congestion1/2 snapshot-batched variants and refresh the congestion7 snapshot-batched goldens from one-thread runs, then verify the focused fluorine slice still passes with the checked-in tests pinned at 16 threads. Signed-off-by: Eugene Brevdo <ebrevdo@users.noreply.github.com> Co-authored-by: Codex <noreply@openai.com>

Reset the third-party/abc gitlink to the commit recorded on origin/master so the parent repo no longer points at the divergent submodule hash. Signed-off-by: Eugene Brevdo <ebrevdo@users.noreply.github.com> Co-authored-by: Codex <noreply@openai.com>

Update the parent-repo gitlink for third-party/abc so it matches the current origin/master submodule commit exactly. Signed-off-by: Eugene Brevdo <ebrevdo@users.noreply.github.com> Co-authored-by: Codex <noreply@openai.com>

Make snapshot-batched width 16 the default surface for global_route while keeping the current internal fallback gates that protect legacy regressions. Retarget explicit snapshot-batched coverage to a real high-overflow stress case, remove the old gcd-based snapshot_batched goldens that no longer batch, and add the tracked shuffle_stress DEF needed by the new smoke tests. Validation: - dev-cpu-fluorine rebuild of /root/openroad-builds/build-linux-tests - focused width16-default slice: 11/11 passed - full grt suite on fluorine: 119/119 passed Signed-off-by: Eugene Brevdo <ebrevdo@users.noreply.github.com> Co-authored-by: Codex <noreply@openai.com>

maliberty reviewed Mar 2, 2026

View reviewed changes

ebrevdo changed the title ~~Multicore congestion~~ Add support for multicore in global routing, especially focusing on speeding up congestion cleanup iterations Mar 2, 2026

gemini-code-assist Bot reviewed Mar 2, 2026

View reviewed changes

github-actions Bot reviewed Mar 2, 2026

View reviewed changes

eder-matheus self-requested a review March 4, 2026 19:51

openroad-ci mentioned this pull request Apr 22, 2026

grt: multicore congestion handling #10229

Merged

4 tasks

ebrevdo and others added 11 commits April 28, 2026 16:32

Merge remote-tracking branch 'ebrevdo/master'

f990388

Restore abc submodule gitlink

c8e8f50

Reset the third-party/abc gitlink to the commit recorded on origin/master so the parent repo no longer points at the divergent submodule hash. Signed-off-by: Eugene Brevdo <ebrevdo@users.noreply.github.com> Co-authored-by: Codex <noreply@openai.com>

Merge remote-tracking branch 'origin/master' into multicore_congestion

8bccee6

Align abc submodule with origin/master

c7ac47f

Update the parent-repo gitlink for third-party/abc so it matches the current origin/master submodule commit exactly. Signed-off-by: Eugene Brevdo <ebrevdo@users.noreply.github.com> Co-authored-by: Codex <noreply@openai.com>

ebrevdo force-pushed the multicore_congestion branch from 8c4fca7 to f2e1f9b Compare April 28, 2026 23:33

github-actions Bot added the size/XL label Apr 28, 2026

eder-matheus merged commit 4a53f21 into The-OpenROAD-Project:master Apr 29, 2026
3 of 4 checks passed

	total_wirelength += computeNetWirelength(routed_nets[i]);
	(auto & routed_net : routed_nets)routed_net

	std::fill(pop_heap2.begin(), pop_heap2.end(), false);
	std::ranges::fill(pop_heap2,, false);

Uh oh!

Conversation

ebrevdo commented Mar 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Router Changes

Tests And Compatibility

Uh oh!

maliberty commented Mar 2, 2026

Uh oh!

maliberty Mar 2, 2026

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

gemini-code-assist Bot Mar 2, 2026

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist Bot Mar 2, 2026

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist Bot Mar 2, 2026

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist Bot Mar 2, 2026

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist Bot Mar 2, 2026

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist Bot Mar 2, 2026

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist Bot Mar 2, 2026

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist Bot Mar 2, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions Bot left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions Bot Mar 2, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions Bot Mar 2, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions Bot Mar 2, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions Bot Mar 2, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions Bot Mar 2, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions Bot Mar 2, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions Bot Mar 2, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions Bot Mar 2, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions Bot Mar 2, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions Bot Mar 2, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

ebrevdo commented Mar 2, 2026 •

edited

Loading