feat: SSA dataflow analysis with Graphviz DOT export#2
Conversation
Design for analyzing LLO VLIW Bundles ↔ Pallas source code correspondence. Adds new bundle_parser, bundle_domain, and bundle_exporter modules with analyze-bundles CLI subcommand. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…tion Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Preserve the SSA boundary (inputs/outputs) computed by group_boundary_ssa() as attributes on every scheduled OpEvent, enabling downstream dataflow analysis to build dependency graphs. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Introduces DFNode, DFEdge, and DataFlowGraph dataclasses along with extract_dataflow() which walks the simulated OpEvent tree, creates nodes for LEAF/BLOCK/STALL events, tracks LOOP containers, and builds deduplicated SSA dependency edges including cross-stream (DMA<->VPU). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…raph Renders DataFlowGraph as DOT format with hardware clusters (VPU/DMA), loop subgraph clusters, time-bucket rank constraints, cross-stream edge highlighting (red dashed), and node styling by stream/kind. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Wire the dataflow extraction and DOT exporter into the CLI so users can generate Graphviz dataflow graphs with a single flag. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Exercises the full pipeline (parse LLO -> simulate -> extract dataflow -> export DOT) using real post-finalize-llo files from ir_dumps/mosaic/. Tests skip gracefully when simulation requires arg overrides. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
There was a problem hiding this comment.
Code Review
This pull request implements SSA dataflow analysis and Graphviz DOT visualization. It adds a data model for dataflow graphs, extraction logic from simulated events, and a DOT exporter featuring hardware/loop clustering and time-bucketed alignment. The simulator now persists SSA metadata, and a --dataflow-output CLI flag is introduced. Review feedback highlights a bug in edge construction when variable names are reused, the omission of control-flow nodes in the graph extraction, and an escaping issue in the DOT exporter that breaks multi-line labels.
| # Map variable name -> producer node id (last writer wins) | ||
| producer: Dict[str, int] = {} | ||
| for node in nodes: | ||
| for var in node.ssa_outputs: | ||
| producer[var] = node.id | ||
|
|
||
| seen: Set[Tuple[int, int, str]] = set() | ||
| edges: List[DFEdge] = [] | ||
|
|
||
| for node in nodes: | ||
| for var in node.ssa_inputs: | ||
| src_id = producer.get(var) | ||
| if src_id is None: | ||
| continue # external input (e.g. %arg0) | ||
| key = (src_id, node.id, var) | ||
| if key in seen: | ||
| continue | ||
| seen.add(key) | ||
| edges.append(DFEdge(src=src_id, dst=node.id, variable=var)) |
There was a problem hiding this comment.
The current implementation of _build_edges builds a global map of producers (producer) before iterating to create edges. This logic is incorrect for programs where variable names are reused, such as in expanded loops where the same SSA name appears in multiple iterations. In such cases, every consumer of a variable will incorrectly point to the last definition in the entire program rather than the most recent one in execution order.
To fix this, you should build the producer map while iterating through the nodes, ensuring that each use links to the definition that was most recently encountered.
| # Map variable name -> producer node id (last writer wins) | |
| producer: Dict[str, int] = {} | |
| for node in nodes: | |
| for var in node.ssa_outputs: | |
| producer[var] = node.id | |
| seen: Set[Tuple[int, int, str]] = set() | |
| edges: List[DFEdge] = [] | |
| for node in nodes: | |
| for var in node.ssa_inputs: | |
| src_id = producer.get(var) | |
| if src_id is None: | |
| continue # external input (e.g. %arg0) | |
| key = (src_id, node.id, var) | |
| if key in seen: | |
| continue | |
| seen.add(key) | |
| edges.append(DFEdge(src=src_id, dst=node.id, variable=var)) | |
| producer: Dict[str, int] = {} | |
| seen: Set[Tuple[int, int, str]] = set() | |
| edges: List[DFEdge] = [] | |
| for node in nodes: | |
| # 1. Find producers for inputs using the current state of the map | |
| for var in node.ssa_inputs: | |
| src_id = producer.get(var) | |
| if src_id is not None: | |
| key = (src_id, node.id, var) | |
| if key not in seen: | |
| seen.add(key) | |
| edges.append(DFEdge(src=src_id, dst=node.id, variable=var)) | |
| # 2. Update the map with this node's outputs for subsequent consumers | |
| for var in node.ssa_outputs: | |
| producer[var] = node.id | |
| return edges |
There was a problem hiding this comment.
Good catch — accepted. The incremental producer map correctly handles expanded loops where SSA names are reused across iterations. Fixed in the next commit.
| def _walk( | ||
| ev: OpEvent, | ||
| loop_depth: int, | ||
| parent_loop_id: Optional[int], | ||
| ) -> None: | ||
| nonlocal next_id, next_loop_id | ||
|
|
||
| if ev.kind in _NODE_KINDS: | ||
| node_id = next_id | ||
| next_id += 1 | ||
| node = DFNode( | ||
| id=node_id, | ||
| name=ev.name, | ||
| stream=ev.stream, | ||
| kind=ev.kind, | ||
| start_ns=ev.start_time_ns, | ||
| end_ns=ev.end_time_ns, | ||
| flops=ev.flops, | ||
| bytes=ev.bytes, | ||
| ssa_outputs=list(ev.attributes.get("ssa_outputs", [])), | ||
| ssa_inputs=list(ev.attributes.get("ssa_inputs", [])), | ||
| attributes=dict(ev.attributes), | ||
| loop_depth=loop_depth, | ||
| parent_loop_id=parent_loop_id, | ||
| ) | ||
| nodes.append(node) | ||
| if parent_loop_id is not None: | ||
| loops.setdefault(parent_loop_id, []).append(node_id) | ||
| return | ||
|
|
||
| if ev.kind == OpKind.LOOP: | ||
| loop_id = next_loop_id | ||
| next_loop_id += 1 | ||
| loops.setdefault(loop_id, []) | ||
| for child in ev.children: | ||
| _walk(child, loop_depth + 1, loop_id) | ||
| return | ||
|
|
||
| # ROOT, IF -- just recurse | ||
| for child in ev.children: | ||
| _walk(child, loop_depth, parent_loop_id) |
There was a problem hiding this comment.
The _walk function currently treats LOOP and IF events as containers but not as nodes in the dataflow graph. However, the design document specifies that control-flow operations should be represented as nodes (e.g., as diamonds) to capture dependencies like loop bounds or branch conditions.
Additionally, the current logic returns early when an event kind is in _NODE_KINDS, which would prevent recursion into loop or branch bodies if those kinds were added to the set. The function should be restructured to both create a node for the event and recurse into its children when appropriate.
There was a problem hiding this comment.
Intentional design decision: LOOP/IF events are pure containers in the OpEvent tree — they don't define or use SSA variables themselves. Adding them as DFNodes would create orphan diamond nodes with no data-dependency edges, adding visual noise without information. The diamond styling for `OpStream.CONTROL` in the exporter targets leaf/block control operations (e.g. `scf.yield`) that do carry SSA semantics, not structural containers. Loop structure is already represented via `subgraph cluster_loop_N` clusters.
| def _escape_dot(text: str) -> str: | ||
| """Escape characters that are special inside DOT label strings.""" | ||
| return text.replace("\\", "\\\\").replace('"', '\\"') |
There was a problem hiding this comment.
The _escape_dot function escapes backslashes, which transforms the \n newline sequences used in _node_label into \\n. This causes Graphviz to render a literal backslash and 'n' instead of a newline in the node labels.
Since DOT labels allow literal newlines and the \n sequence is specifically intended for Graphviz, you should avoid escaping backslashes here, or only escape them if they are not part of a known escape sequence.
| def _escape_dot(text: str) -> str: | |
| """Escape characters that are special inside DOT label strings.""" | |
| return text.replace("\\", "\\\\").replace('"', '\\"') | |
| def _escape_dot(text: str) -> str: | |
| """Escape characters that are special inside DOT label strings.""" | |
| return text.replace('"', '\\"') |
There was a problem hiding this comment.
Good catch — the backslash escaping indeed breaks the `\n` DOT newline sequences produced by `_node_label`. Fixed by removing the backslash replacement; only double-quote escaping is needed for our labels.
- Build SSA producer map incrementally during edge construction so expanded loops with reused variable names link to the most recent producer, not the global last writer. - Remove backslash escaping in _escape_dot that was breaking \n newline sequences in multi-line DOT node labels. Addresses review comments on PR #2. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…tion fix - Add GCS upload/download path formula contract test (reviewer item #2) - Add AC2 download trigger chain test: kubectl wait → gcloud cp → tar (item #3) - Fix tautological assertion in test_kernel_is_positional_before_all_flags (item #5) - Add shell script download dir convention test Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Summary
--dataflow-outputCLI flag to generate Graphviz DOT files showing SSA def-use dependency graphsDesign
See
docs/plans/2026-05-01-dataflow-analysis-design.mdfor the full design document.RFC: primatrix/wiki#143
Changes
dataflow.py— DFNode/DFEdge/DataFlowGraph data model +extract_dataflow()extractiondataflow_exporter.py—DataFlowDotExporterrenders DOT with time buckets, hardware clusters, loop nestingsimulator.py— persist SSA boundary info on scheduled eventscli.py—--dataflow-outputflagUsage
Test plan
🤖 Generated with Claude Code