v0.3.3 — Graph-layer architecture review: model/renderer split, correctness + perf fixes, issue #1

Large architecture-review pass over the alphaDeesp graph layer: correctness
fixes, performance work, two deep revisions, the interactive-viewer
decomposition, and the fix for the multigraph Dijkstra-weight bug (#1).

Backward-compatible: all public import paths are unchanged
(from alphaDeesp.core.graphsAndPaths import ... still works). 418 unit tests
pass without grid2op; changes were cross-checked by adversarial multi-agent
verification. The grid2op integration suites and the new capacity-weighted
routing mode still want validation on reference grids in CI.

Bug fixes

• Multigraph Dijkstra weight in the null-flow path search (#1). On the overflow MultiDiGraph, networkx passes a callable weight the {key: attr} parallel-edge view, so _compute_sssp_paths' attr.get("capacity", 0) read 0 — routing was hop-cost-only. The weight is now precomputed as an edge attribute (string-weight Dijkstra) with a capacity_weighted switch: False (default) reproduces the historical behaviour bit-identical ("bless"); True enables capacity-weighted routing with correct min-parallel capacity and (u,v)/(u,v,key) promoted matching ("fix"). Threaded through add_relevant_null_flow_lines[_all_paths] so downstream callers can opt in without patching.
• shortest_paths.py: removed a dead branch; made promoted-edge matching multigraph-correct (was silently weighting every parallel edge as 0).
• OverFlowGraph.__init__ no longer mutates the caller's DataFrame (copies it); rename_nodes clarified (behaviour unchanged).
• Structured_Overload_Distribution_Graph.get_dispatch_edges_nodes guards the empty-loops case (no more TypeError on a grid with no loop paths).
• to_DiGraph defaults a missing edge capacity to 0.0 (was 1.0), which no longer skews rank_red_loops' min-cut.

Deep revisions

• OverFlowGraph model/renderer split: new OverflowGraphRenderer owns all Graphviz presentation; OverFlowGraph keeps the semantic model. New edge_roles.py (edge_role_of, EDGE_ROLE_*, OverFlowGraph.edge_role) is the single authority mapping an edge's base colour to a role — no consumer parses colour strings; highlight_significant_line_loading records an authoritative base_color.
• AlphaDeesp explicit pipeline: ranking moved to run(); AlphaDeesp(..., auto_run=True) (default) preserves previous behaviour, auto_run=False builds without side effects.
• Lazy Structured_Overload_Distribution_Graph: colour views, red_loops and hubs are functools.cached_property computed from a construction-time snapshot — behaviour-identical and order-independent.

Performance

• delete_color_edges accepts a colour or an iterable of colours (single graph copy); derived views built in single passes.
• sort_hubs and the initial-inflow lookup vectorised (were per-hub / per-edge iterrows scans).
• simulation.create_df vectorised (400-case differential fuzz pins equivalence).
• Null-flow Dijkstra uses a precomputed string weight instead of a per-edge Python callable.

Maintainability & docs

• core/interactive_html.py (976 LOC) split into the core/interactive_html/ package (8 focused modules) with CSS/JS/HTML externalised under assets/ and reassembled byte-exactly at runtime (shipped via package_data).
• find_loops / consolidation path enumeration gained opt-in cutoffs (default None = unbounded = original behaviour).
• New Architecture guide + autodoc API reference (docs/ARCHITECTURE.rst, docs/API.rst); docs/CODE_REVIEW.md (full review + downstream-impact analysis on Expert_op4grid_recommender); CLAUDE.md refreshed.

Tests

New suites: test_edge_roles, test_overflow_renderer, test_simulation_create_df, test_null_flow_weighting, test_graph_consolidation, test_elements, test_simulation_helpers, test_alphadeesp_combinations; expanded test_graphs_package, test_overflow_graph, test_alphadeesp_unit, test_shortest_paths, test_interactive_html, test_topo_applicator.

Full changelog: v0.3.2.post4...v0.3.3

v0.3.2.post4 — Readable node names in the interactive overflow viewer

Highlights

The interactive overflow-graph viewer now shows human-readable node names
(e.g. Saucats 400kV) instead of only opaque internal IDs, while keeping node
identity stable so all existing interactions (selection, adjacency highlight,
double-click SLD resolution) keep working unchanged.

New Features

• Readable node names in the interactive viewer (#78): the viewer uses each
  node's display label (the Graphviz label node attribute, set upstream by
  the recommender) as the readable name.
  • Search matches on both the readable display name and the stable id.
  • The hover tooltip and selection panel show the readable name as the
    header, with the id underneath only when it differs.
  • Node identity (SVG <title> / data-name) is unchanged — selection,
    adjacency highlight and double-click SLD resolution still key off the id.

Bug Fixes

• No more \N placeholder leak: label-less nodes carry Graphviz's \N
  ("use node name") placeholder in their label attribute. nodeDisplayName now
  ignores any backslash escape and falls back to the id, so tooltips and search
  never surface a literal \N.

Tests

• Readable label surfaces as data-attr-label without losing the id.
• Search consults both the id and the resolved display name.
• Tooltip/selection skip the duplicated label attribute.
• Label-less nodes fall back to the id (no \N leak).

Full changelog: see CHANGELOG.md ([0.3.2.post4]).

v0.3.2.post3

Changes since v0.3.2.post1

• feat(overflow_graph): OverFlowGraph now accepts an extra_lines_to_cut argument naming a subset of lines_to_cut that the operator chose to disconnect to prevent flow increase elsewhere (ExpertAgent's additionalLinesToCut semantic). Those edges keep their natural flow colour (coral/blue) instead of black/yellow, are stamped is_extra_cut=True (alongside constrained=True) so downstream consumers can find them by flag, and are excluded from is_overload / is_monitored tagging in highlight_significant_line_loading. The before% → after% loading annotation still fires on extras so the operator sees how their cut materialises.
• feat(interactive_html): new "Extra lines to prevent flow increase" semantic layer (semantic:is_extra_cut) in the overflow-graph viewer's Individual entities properties sidebar section, with a dashed-blue swatch.
• tests: TestExtraLinesToCut (6 cases) covers default empty extras, set storage, natural coral/blue colour, is_extra_cut/constrained flagging, exclusion from overload/monitored layers while still annotating the label, and the legacy "no extras" path. test_layer_index_emits_extra_cut_layer_with_endpoints verifies the new viewer layer (endpoint nodes, swatch, label, section).

This is a PEP 440 post-release of 0.3.2; install metadata, public API back-compat, and dependency set are otherwise identical.

Shipped via PR #76.

v0.3.2.post2

Changes since v0.3.2.post1

• feat(overflow_graph): OverFlowGraph now accepts an extra_lines_to_cut argument naming a subset of lines_to_cut that the operator chose to disconnect to prevent flow increase elsewhere (ExpertAgent's additionalLinesToCut semantic). Those edges keep their natural flow colour (coral/blue) instead of black/yellow, are stamped is_extra_cut=True (alongside constrained=True) so downstream consumers can find them by flag, and are excluded from is_overload / is_monitored tagging in highlight_significant_line_loading. The before% → after% loading annotation still fires on extras so the operator sees how their cut materialises.
• feat(interactive_html): new "Extra lines to prevent flow increase" semantic layer (semantic:is_extra_cut) in the overflow-graph viewer's Individual entities properties sidebar section, with a dashed-blue swatch.
• tests: TestExtraLinesToCut (6 cases) covers default empty extras, set storage, natural coral/blue colour, is_extra_cut/constrained flagging, exclusion from overload/monitored layers while still annotating the label, and the legacy "no extras" path. test_layer_index_emits_extra_cut_layer_with_endpoints verifies the new viewer layer (endpoint nodes, swatch, label, section).

This is a PEP 440 post-release of 0.3.2; install metadata, public API back-compat, and dependency set are otherwise identical.

Shipped via PR #76.

v0.3.2.post1

Changes since v0.3.2

• feat(interactive_html): add Production / Consumption node filter layers (node:prod / node:load) to the overflow-graph viewer's Individual entities properties sidebar section. Built from the upstream prod_or_load attribute build_nodes writes on every node, with a 1 MW absolute-value floor (_PROD_LOAD_VALUE_FLOOR_MW) so the conventional zero-balance load placeholders don't flood the layer. Coral / lightblue circle swatches mirror the actual node fillcolors.

This is a PEP 440 post-release of 0.3.2; install metadata, public API, and dependency set are otherwise identical.

ExpertOp4Grid v0.3.2 — Semantic overflow-graph layers

Highlights

This release elevates the interactive overflow-graph viewer from
colour / shape heuristics to semantic, source-of-truth layer
toggles. The recommender now propagates explicit boolean flags on
graph nodes and edges, and the viewer surfaces them through three
labelled checkbox sections with a dim-instead-of-hide model.

• Semantic node / edge tags — is_hub, in_red_loop,
  on_constrained_path on nodes; is_overload, is_monitored,
  in_red_loop, on_constrained_path on edges. is_overload is a
  strict subset of is_monitored (every overloaded contingency line
  is also a low-margin line).
• New public taggers on OverflowGraph —
  tag_red_loops(lines, nodes) and
  tag_constrained_path(lines, nodes) consume the recommender's
  pre-computed lists. set_hubs_shape now also marks hubs with
  in_red_loop AND on_constrained_path so they always appear
  under those layers.
• highlight_significant_line_loading reworked — now tags every
  monitored line with is_monitored=True and the strict
  contingency-overloaded subset with is_overload=True.
• Grouped layer sidebar in the interactive viewer — three
  labelled sections (Structural Paths / Individual entities
  properties / Flow redispatch values) ordered via
  _LAYER_SECTIONS and _SECTION_ORDER.
• Membership-based dim model — opacity drops to 0.12 for
  unchecked elements; no display:none, so click / hover stay
  live. An element is visible iff at least one of its claiming
  layers is currently checked.
• Endpoint-node inclusion — edge-only layers also dim /
  highlight the bbox-endpoint nodes of their claimed edges, so
  toggling "Overloads" alone keeps the substations the overload
  connects visible (no floating edges in dimmed space).
• Select-all / Unselect-all link row above the layer list,
  dispatching the same change event individual checkboxes do.
• _align_edge_ids_with_svg() regression guard — graphviz's
  SVG and JSON edge orderings drift independently; we now re-key
  the JSON edge model from the SVG <title> endpoints so
  data-source / data-target always match the title content.

What's Changed

PR #75 — Add semantic source-of-truth tagging + grouped layer toggles

• OverflowGraph.set_hubs_shape adds is_hub=True (plus
  in_red_loop and on_constrained_path) on every hub.
• New OverflowGraph.tag_red_loops(lines, nodes) and
  OverflowGraph.tag_constrained_path(lines, nodes) helpers.
• OverflowGraph.highlight_significant_line_loading reworked to
  tag is_monitored ⊃ is_overload semantics.
• interactive_html._build_layer_index scans data-attr-* on
  both nodes and edges and emits five new semantic layers.
• Layers are grouped into three sections via _LAYER_SECTIONS
  rendered in _SECTION_ORDER with section-header markers.
• Membership-based dim model + endpoint-node inclusion + Select-
  all / Unselect-all controls.
• _align_edge_ids_with_svg() regression guard for graphviz
  SVG-vs-JSON edge-id misalignment.
• Comprehensive coverage in tests/test_overflow_graph.py and
  tests/test_interactive_html.py.

Companion releases

• expert_op4grid_recommender v0.2.1
  wires the constrained-path / red-loop / overloads lists into the
  new taggers.
• The Co-Study4Grid v0.7+ overflow-analysis tab embeds the
  resulting interactive HTML viewer with an action-pin overlay.

Compatibility

Backwards compatible. Existing callers see no behaviour change —
the new tags are additive, and the new public taggers are no-ops
when not invoked. Any existing serialised overflow graphs continue
to render correctly with the upgraded viewer.

🤖 Generated with Claude Code

ExpertOp4Grid v0.3.1 — Code Quality & Maintainability Overhaul

Highlights

This release is a major code quality and maintainability milestone. Six pull requests across a focused sprint bring ExpertOp4Grid's core codebase from a prototype-grade state to a well-structured, statically-analysable library:

• Structured logging replaces all 101 print() calls — embedders now control verbosity
• Type annotations across ~220 function signatures — mypy and radon added to CI
• Architecture cleanup — the 2295-line graphsAndPaths.py monolith split into 8 focused modules with a backward-compatible shim
• Complexity reduction — four critical high-complexity functions (CC up to 68) decomposed into composable, testable helpers
• Safer encoding — twin-node string-prefix hack ("666" + str(id)) replaced by an offset-based scheme that fixes silent collisions
• Packaging refresh — correct python_requires, updated dependency floors, duplicate license removed

---

What's Changed

PR #67 — Code quality cleanup: pyflakes, bare excepts, mutable defaults

• Replaced all 9 bare except: clauses with specific exception types
• Removed 15+ unused imports and dead local variables across alphadeesp.py, graphsAndPaths.py, printer.py, Grid2opSimulation.py, PypownetSimulation.py
• Replaced from alphaDeesp.core.elements import * with explicit imports in three modules (star-import cleanup)
• Fixed two mutable default arguments in AlphaDeesp.__init__ and rank_current_topo_at_node_x
• Added pyflakes lint step to CircleCI — 0 findings on the full CI scope
• Gated print(df) in Simulation.create_df behind a debug flag

PR #68 — Short-term code quality improvements: logging, docstrings, packaging

• Logging migration: all 101 print() calls converted to logger.info/warning/debug across 10 modules; main.py installs basicConfig only for CLI entry point, leaving embedders in control
• Abstract method documentation: replaced all 9 """TODO""" stubs in Simulation with full Sphinx-style docstrings including contracts and backend-specific notes
• CI re-enablement: replaced --ignore flags with pytest.importorskip guards — optional test suites now run automatically when optional dependencies are present
• Packaging cleanup: added python_requires=">=3.9", bumped Grid2Op>=1.12.1 and lightsim2grid>=0.10.3, removed stale pathlib backport and dead commented-out dependencies, deleted duplicate LICENSE.md

PR #69 — Refactor four high-complexity functions into composable helpers

• AlphaDeesp.rank_current_topo_at_node_x: CC 68 → 7, extracted six scoring helpers
• AlphaDeesp.apply_new_topo_to_graph: CC 36 → 7, extracted five topology-pipeline helpers
• OverFlowGraph.detect_edges_to_keep: CC 46 → 2, fully orchestrating main function with four path-classification helpers
• OverFlowGraph.add_relevant_null_flow_lines: CC 44 → 7, extracted five null-flow helpers
• New module core/twin_nodes.py: offset-based twin-node scheme (TWIN_NODE_OFFSET = 10_000_000) replacing fragile "666" + str(id) encoding — fixes silent collisions for substation ids ≥ 1000
• 21 new unit tests in test_alphadeesp_unit.py; 30+ new tests in test_graphs_and_paths_unit.py; total: 130 passing

PR #70 — Add type hints to core base classes and fix mutable defaults

• Fully annotated core/elements.py (all four element dataclasses: Production, Consumption, OriginLine, ExtremityLine)
• Fully annotated core/simulation.py abstract base class — all 10 abstract methods, all helper methods, factory method
• Exported SubstationElement type alias (Union[Production, Consumption, OriginLine, ExtremityLine])
• Fixed mutable defaults ltc=[9] / other_ltc=[] in Grid2opSimulation.__init__ and PypownetSimulation.__init__

PR #71 — Add type hints propagation and mypy/radon to CI

• Signature-level type annotations added to six previously untyped modules: core/alphadeesp.py (~38 signatures), core/graphsAndPaths.py (~76 signatures), core/network.py, core/printer.py, core/grid2op/Grid2opSimulation.py (~33 signatures), core/pypownet/PypownetSimulation.py (~27 signatures)
• Type-annotated functions: 36 → ~220
• Mypy added to CircleCI: strict mode on seed modules (simulation.py, elements.py) blocks the build on failure; permissive mode on remaining modules is informational
• Radon added to CircleCI: cyclomatic complexity, maintainability index, and raw metrics reported on every build

PR #72 — Refactor: Split graphsAndPaths monolith into focused graphs package

• alphaDeesp/core/graphsAndPaths.py (2295 lines) split into alphaDeesp/core/graphs/ package with 8 focused modules: power_flow_graph.py, overflow_graph.py, structured_overload_graph.py, constrained_path.py, graph_utils.py, shortest_paths.py, null_flow.py, constants.py
• Zero breaking changes: graphsAndPaths.py reduced to a 49-line backward-compatible shim re-exporting all 16 public symbols
• Explicit __all__ defines the committed public API
• 443 lines of new tests in test_graphs_package.py covering structural invariants, import acyclicity, and algorithmic correctness
• Architecture documentation added in docs/refactor-graphs-package.md

---

Upgrade Notes

• No breaking changes to the public API. All existing from alphaDeesp.core.graphsAndPaths import ... imports continue to work unchanged.
• Embedders using Grid2opSimulation(ltc=[...]) or PypownetSimulation(ltc=[...]) are unaffected — keyword-argument callers remain compatible.
• Console output is now suppressed by default. To restore verbose output, configure logging before calling the library: logging.basicConfig(level=logging.INFO).
• Minimum Python version is now officially 3.9 (up from the stale 3.6/3.7 classifiers in setup.py).

---

Full Changelog: v0.3.0.post1...v0.3.1

ExpertOp4Grid v0.3.0.post1

Changelog

[0.3.0.post1] - 2026-03-10

New Features

• Standardized overflow graph arrow sizes (PR #66): Implemented linear scaling for penwidth in OverFlowGraph based on absolute delta flow. This ensures arrow sizes are reasonable and readable even for very high delta flows, with a standardized maximum penwidth.
• Improved arrow scaling in PowerFlowGraph: Also applied linear scaling to PowerFlowGraph for consistent visualization across different graph types.

Bug Fixes

• Fixed collapse_red_loops logic: Optimized the node collapse condition to ensure ALL connected edges are coral before collapsing a node into a "point" shape.

Tests

• Added unit tests for arrow scaling and collapse_red_loops in test_graphs_and_paths_unit.py.
• Adapted test_visualization_filtering.py in the recommender repo to verify collapse_red_loops call.

[0.3.0] - 2026-02-10

New Features

• Filter graph components without overloads (PR #64): Added keep_overloads_components() method to OverFlowGraph that filters out connected components not containing any overloaded (black) edges by recolouring them to grey. This ensures only components with actual overloads are considered significant in further analysis.

• Collapse coral-only loop nodes (PR #65): Added collapse_red_loops() method to OverFlowGraph that collapses nodes purely part of coral-only loops into "point" shapes for cleaner visualization. A node is collapsed when all its edges are coral, its shape is "oval", it has no peripheries attribute, and none of its edges are dashed or dotted.

Performance Improvements

• Optimized null flow detection (PR #63): Major refactoring and performance optimization of the null flow line detection pipeline:
  • Extracted _setup_null_flow_styles() to avoid redundant style setup across target path iterations
  • Pre-compute structural info (red loop nodes, constrained path endpoints) once and reuse across all target paths
  • Replace delete_color_edges graph copy operations with direct edge color computation for connex analysis
  • Build gray component graph in a single edge pass instead of 3 sequential copy+remove operations
  • Use single-source Dijkstra per source node instead of per source-target pair, reducing from O(S*T) to O(S) Dijkstra calls
  • Pre-filter edges of interest to component membership for early exits
  • Cache BFS results and incident edge lookups to eliminate redundant computation
  • Flip negative capacities once globally instead of per source-target pair
  • Optimized remove_unused_added_double_edge to avoid subgraph creation
  • Optimized add_double_edges_null_redispatch from 4 nx.get_edge_attributes calls to a single edge pass
  • Increased max_null_flow_path_length default from 5 to 7
  • Properly handle double edges when finding relevant null flow paths

Tests

• Added 78 unit tests for graphsAndPaths helper functions and detect_edges_to_keep (PR #63), covering: delete_color_edges, nodepath_to_edgepath, incident_edges, from_edges_get_nodes, find_multidigraph_edges_by_name, shortest_path_with_promoted_edges, all_simple_edge_paths_multi, add_double_edges_null_redispatch, remove_unused_added_double_edge, detect_edges_to_keep, ConstrainedPath methods, and shortest_path_min_weight_then_hops
• Added 13 unit tests for keep_overloads_components (PR #64), covering: component with/without overloads, multiple mixed components, already-gray edges, empty graph, single edges, parallel edges, gray bridge not merging components, three components, and idempotency

[0.2.8]

Performance Improvements

• Speed up path findings with smarter graph algorithms and rustworkx (PR #61)
• Speed up AlphaDeesp representation creation: optimize create_and_fill_internal_structures, accelerate get_model_obj_from_or and get_model_obj_from_ext (PR #62)
• Added rustworkx as a dependency for faster graph operations

Features

• Added ability to consider undirected edges in loop paths (PR #59)
• Extended topology scoring to reduced specified action space with warm-start support (PR #58)
• Exposed depth and breadth search parameters for reconnectable lines in upper functions

[0.2.7]

• Highlight non-connected paths with additional blue edge path info (PR #57)
• Ambiguous path handling improvements (PR #55, #56)
• Highlight possible overloads (PR #53, #54)
• Display reconnectable lines (PR #52)
• Improved graphviz constrained path visualization (PR #51)
• Getting started guide and examples (PR #50)
• Refactored plotting (PR #47, #48)

Release 0.3.0

[0.3.0] - 2026-02-10
New Features
Filter graph components without overloads (PR #64): Added keep_overloads_components() method to OverFlowGraph that filters out connected components not containing any overloaded (black) edges by recolouring them to grey. This ensures only components with actual overloads are considered significant in further analysis.

Collapse coral-only loop nodes (PR #65): Added collapse_red_loops() method to OverFlowGraph that collapses nodes purely part of coral-only loops into "point" shapes for cleaner visualization. A node is collapsed when all its edges are coral, its shape is "oval", it has no peripheries attribute, and none of its edges are dashed or dotted.

Performance Improvements
Optimized null flow detection (PR #63): Major refactoring and performance optimization of the null flow line detection pipeline:
Extracted _setup_null_flow_styles() to avoid redundant style setup across target path iterations
Pre-compute structural info (red loop nodes, constrained path endpoints) once and reuse across all target paths
Replace delete_color_edges graph copy operations with direct edge color computation for connex analysis
Build gray component graph in a single edge pass instead of 3 sequential copy+remove operations
Use single-source Dijkstra per source node instead of per source-target pair, reducing from O(S*T) to O(S) Dijkstra calls
Pre-filter edges of interest to component membership for early exits
Cache BFS results and incident edge lookups to eliminate redundant computation
Flip negative capacities once globally instead of per source-target pair
Optimized remove_unused_added_double_edge to avoid subgraph creation
Optimized add_double_edges_null_redispatch from 4 nx.get_edge_attributes calls to a single edge pass
Increased max_null_flow_path_length default from 5 to 7
Properly handle double edges when finding relevant null flow paths
Tests
Added 78 unit tests for graphsAndPaths helper functions and detect_edges_to_keep (PR #63), covering: delete_color_edges, nodepath_to_edgepath, incident_edges, from_edges_get_nodes, find_multidigraph_edges_by_name, shortest_path_with_promoted_edges, all_simple_edge_paths_multi, add_double_edges_null_redispatch, remove_unused_added_double_edge, detect_edges_to_keep, ConstrainedPath methods, and shortest_path_min_weight_then_hops
Added 13 unit tests for keep_overloads_components (PR #64), covering: component with/without overloads, multiple mixed components, already-gray edges, empty graph, single edges, parallel edges, gray bridge not merging components, three components, and idempotency

Speedup bottlenecks for larger grids

Speeding up Alphadeesp representation creation (#62) as well as path finding algorithms that use networkx (#61). Those 2 were the main bottlenecks when working with larger grids.