refactor: dataflow scheduler + stripped PipelineGraph API#8
Merged
Conversation
…eline-ir # Conflicts: # CLAUDE.md
git-subtree-dir: dsls/harmont-py git-subtree-split: 2d47999cfbd5bffd1f7fd7894302791117ef1edb
markovejnovic
changed the title
markovejnovic
changed the title
Delete Pipeline, Step, WaitStep types and the PipelineGraph::build() constructor. The JSON wire format is now the petgraph-serde graph directly — scheduler receives a pre-deserialized PipelineGraph instead of building one from a flat step list. CommandStep loses its builds_in field (edges carry that information). The anyhow dependency is removed from hm-pipeline-ir since nothing uses it anymore. Test files in hm-pipeline-ir (graph_build, parse_pipeline, schema_snapshot) will not compile until their fixtures are rewritten to the new graph format in the next task.
…rmat Replace _lower_to_dicts with _lower_to_graph in pipeline.py to emit the new graph-based wire format (nodes/edges/edge_property/node_holes) instead of a flat steps list. Update keygen.py to accept a graph dict and derive parent relationships from builds_in edges. Update all call sites (json_emit.py, _envelope.py, dev/_registry_dump.py).
Rewrite assertions across 26 test files to match the new graph-based IR: p["steps"] -> p["graph"]["nodes"], step["type"]/step["builds_in"] removed, parent relationships read from edges, wait barriers become depends_on edges, env merged into node dicts.
markovejnovic
changed the title
lib.rs now only re-exports from the private graph module.
All downstream imports use hm_pipeline_ir::{PipelineGraph, ...} directly.
…r, rewrite chains() functionally - NodeWeight → Transition (public), accessed via PipelineGraph::get_transition() - Replaces separate command_step()/step_env() accessors - chains() rewritten using std::iter::successors — no mutable placed map - Removed unused node_indices() method
Walk the DAG in topological order, spawning a Shared<BoxFuture> per node. Each future awaits its predecessors, acquires a parallelism permit, then dispatches to the executor plugin. Snapshot lineage flows through future results — no cross-chain map needed. Deletes run_chain(); the entire scheduling model is now ~50 lines in the dataflow loop.
markovejnovic
changed the title
…eline-ir # Conflicts: # Cargo.lock # crates/hm-plugin-protocol/src/ir.rs
Import hm_pipeline_ir types directly instead of going through a one-line re-export module.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.