Skip to content

pikaci: staged run plans#485

Merged
justinmoon merged 2 commits intomasterfrom
pika-ci-merge
Mar 7, 2026
Merged

pikaci: staged run plans#485
justinmoon merged 2 commits intomasterfrom
pika-ci-merge

Conversation

@justinmoon
Copy link
Copy Markdown
Collaborator

@justinmoon justinmoon commented Mar 7, 2026

Summary

  • Add staged plan skeleton for pikaci run orchestration
  • Tighten persisted run plans with structured model types and improved executor logic

Test plan

  • CI pre-merge checks pass

🤖 Generated with Claude Code


Open with Devin

Summary by CodeRabbit

  • New Features

    • Plan-based execution: jobs are scheduled into persisted plans (plan.json) with per-job plan IDs; status now optionally shows plan path.
  • Refactor

    • Runner provisioning and build preparation centralized and simplified; public API surface expanded to support planning and execution workflows.
  • Tests

    • Added tests and scaffolding for plan generation, execution wiring, and persistence.

@coderabbitai
Copy link
Copy Markdown

coderabbitai bot commented Mar 7, 2026

📝 Walkthrough

Walkthrough

Jobs are now planned into a RunPlanRecord with Prepare and Execute nodes; run flow builds and persists a plan, then executes planned jobs using per-job HostContext. VFKit runner materialization was refactored into dedicated installable-producing helpers and API surface expanded.

Changes

Cohort / File(s) Summary
Executor refactor
crates/pikaci/src/executor.rs
Extracted VFKit runner materialization into materialize_vfkit_runner_flake() and vfkit_runner_installable(), made compiled_guest_command() pub(crate), and switched nix build to use the installable string. Result path resolution now uses ctx.job_dir.join("artifacts/result.json").
Plan model additions
crates/pikaci/src/model.rs
Added plan-related public types: PlanExecutorKind, PrepareNode, ExecuteNode, PlanNodeRecord, RunPlanRecord, PlanScope. Added optional plan_node_id to JobRecord and plan_path to RunRecord with serde defaults.
Plan-based execution flow
crates/pikaci/src/run.rs
Introduced build_run_plan() and internal RunPlan/PlannedJob types, persist plan.json via write_run_plan_record(), changed run_jobs_against_snapshot() to iterate planned jobs, and updated run_one_job() to accept plan_node_id and HostContext. Added per-job HostContext creation and prepare-node wiring for VFKit.
Public re-exports
crates/pikaci/src/lib.rs
Re-exported plan-related types and new run functions for external use (ExecuteNode, PlanExecutorKind, PlanNodeRecord, PlanScope, PrepareNode, RunPlanRecord, record_skipped_run, rerun_jobs_with_metadata, run_job, run_jobs, run_jobs_with_metadata).
CLI / records
crates/pikaci/src/main.rs
Added optional plan_path to RunRecord and plan_node_id to JobRecord; updated status printing and tests to account for new optional fields.
Tests & scaffolding
crates/pikaci/src/run.rs (tests)
Updated and added tests to validate plan generation, node wiring, plan.json persistence, and sample scaffolding for prepared runs and snapshots.

Sequence Diagram

sequenceDiagram
    participant Submit as Job Submitter
    participant Builder as Plan Builder
    participant Store as Plan Store
    participant Executor as Per-Job Executor
    participant Host as Host Environment

    Submit->>Builder: run_jobs_against_snapshot(jobs)
    Builder->>Builder: build_run_plan(jobs) 
    Builder->>Builder: create PrepareNode (vfkit runner)
    Builder->>Builder: create ExecuteNode (VmCommand per job)
    Builder->>Store: write_run_plan_record(plan.json)
    Store-->>Builder: plan_path
    Builder->>Executor: for each PlannedJob (plan_node_id, ctx)
    Executor->>Host: run_one_job(plan_node_id, ctx)
    Host-->>Executor: execution result
    Executor-->>Submit: RunRecord (includes plan_path)
Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~60 minutes

🐰
A plan is drawn under moonlit log,
Nodes lined up like hops through fog,
Prepare the runner, build the scene,
Execute each job, neat and clean,
Snapshots kept where carrots once were logged.

🚥 Pre-merge checks | ✅ 2 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 10.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (2 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title 'pikaci: staged run plans' clearly and specifically summarizes the main changes: introducing staged execution plans for pikaci's run orchestration system.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
  • 📝 Generate docstrings (stacked PR)
  • 📝 Generate docstrings (commit on current branch)
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch pika-ci-merge

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Copy Markdown

@devin-ai-integration devin-ai-integration bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Devin Review: No Issues Found

Devin Review analyzed this PR and found no potential bugs to report.

View in Devin Review to see 4 additional findings.

Open in Devin Review

Copy link
Copy Markdown

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (1)
crates/pikaci/src/run.rs (1)

436-450: Keep plan construction side-effect free.

Line 438 materializes files on disk while just building metadata, and crates/pikaci/src/executor.rs Line 62 does the same work again during execution. That makes planning mutate job state and couples build_run_plan() to executor internals more tightly than needed.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@crates/pikaci/src/run.rs` around lines 436 - 450, The plan currently calls
materialize_vfkit_runner_flake during planning (in the build_run_plan path where
job.runner_kind() == RunnerKind::VfkitLocal) which mutates disk state; instead
have build_run_plan record only the installable descriptor/flake reference
(e.g., a serializable Installable/flake identifier) in the
PlanNodeRecord::Prepare / PrepareNode::NixBuild without calling
materialize_vfkit_runner_flake, and move the actual materialization call into
the executor path (the code around the materialization in
crates/pikaci/src/executor.rs) so materialize_vfkit_runner_flake is invoked at
execution time; update any types/serde for the PrepareNode::NixBuild payload so
it can carry the flake/installable metadata rather than a materialized path.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@crates/pikaci/src/run.rs`:
- Around line 136-141: The loop over plan.jobs presently propagates any Err from
run_one_job() and exits without persisting a terminal run state; update the loop
to catch errors from run_one_job(&planned_job.job, &planned_job.execute_node_id,
&planned_job.ctx) and, on error, set the run status to a terminal state (e.g.,
Failed), set finished_at to now, and write the updated status.json / run.json to
disk before returning the error; ensure the write/directory creation logic used
elsewhere for persisting terminal status is invoked here (or factored into a
helper) and that write failures are handled/logged or propagated so the run
cannot remain permanently marked as running.

---

Nitpick comments:
In `@crates/pikaci/src/run.rs`:
- Around line 436-450: The plan currently calls materialize_vfkit_runner_flake
during planning (in the build_run_plan path where job.runner_kind() ==
RunnerKind::VfkitLocal) which mutates disk state; instead have build_run_plan
record only the installable descriptor/flake reference (e.g., a serializable
Installable/flake identifier) in the PlanNodeRecord::Prepare /
PrepareNode::NixBuild without calling materialize_vfkit_runner_flake, and move
the actual materialization call into the executor path (the code around the
materialization in crates/pikaci/src/executor.rs) so
materialize_vfkit_runner_flake is invoked at execution time; update any
types/serde for the PrepareNode::NixBuild payload so it can carry the
flake/installable metadata rather than a materialized path.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 3d204ea6-d10a-4dd7-a20c-ce2d16e859bc

📥 Commits

Reviewing files that changed from the base of the PR and between 1e07ca7 and cbda437.

📒 Files selected for processing (5)
  • crates/pikaci/src/executor.rs
  • crates/pikaci/src/lib.rs
  • crates/pikaci/src/main.rs
  • crates/pikaci/src/model.rs
  • crates/pikaci/src/run.rs
🚧 Files skipped from review as they are similar to previous changes (1)
  • crates/pikaci/src/main.rs

Comment on lines +136 to 141
for planned_job in &plan.jobs {
let job_record = run_one_job(
job,
&snapshot.snapshot_dir,
&prepared.jobs_dir,
&prepared.shared_cargo_home_dir,
&prepared.run_target_dir,
&planned_job.job,
&planned_job.execute_node_id,
&planned_job.ctx,
)?;
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

Persist a terminal run state when job execution setup fails.

At Line 137, any Err from run_one_job() exits this function with run.json still marked as running, because the terminal status and finished_at are only written later. A failed status.json write or directory creation will leave the run permanently dangling.

💡 Suggested fix
     let mut run_failed = false;
     for planned_job in &plan.jobs {
-        let job_record = run_one_job(
-            &planned_job.job,
-            &planned_job.execute_node_id,
-            &planned_job.ctx,
-        )?;
+        let job_record = match run_one_job(
+            &planned_job.job,
+            &planned_job.execute_node_id,
+            &planned_job.ctx,
+        ) {
+            Ok(job_record) => job_record,
+            Err(err) => {
+                run_record.status = RunStatus::Failed;
+                run_record.finished_at = Some(Utc::now().to_rfc3339());
+                run_record.message = Some(format!("{err:#}"));
+                write_run_record(&prepared.run_dir, &run_record)?;
+                return Err(err);
+            }
+        };
         if job_record.status == RunStatus::Failed {
             run_failed = true;
         }
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@crates/pikaci/src/run.rs` around lines 136 - 141, The loop over plan.jobs
presently propagates any Err from run_one_job() and exits without persisting a
terminal run state; update the loop to catch errors from
run_one_job(&planned_job.job, &planned_job.execute_node_id, &planned_job.ctx)
and, on error, set the run status to a terminal state (e.g., Failed), set
finished_at to now, and write the updated status.json / run.json to disk before
returning the error; ensure the write/directory creation logic used elsewhere
for persisting terminal status is invoked here (or factored into a helper) and
that write failures are handled/logged or propagated so the run cannot remain
permanently marked as running.

@justinmoon justinmoon merged commit 93933e3 into master Mar 7, 2026
18 checks passed
@justinmoon justinmoon deleted the pika-ci-merge branch March 20, 2026 14:52
@justinmoon justinmoon restored the pika-ci-merge branch March 20, 2026 21:19
@justinmoon justinmoon deleted the pika-ci-merge branch March 20, 2026 21:21
@justinmoon justinmoon restored the pika-ci-merge branch March 20, 2026 21:49
@justinmoon justinmoon deleted the pika-ci-merge branch March 20, 2026 21:53
@justinmoon justinmoon restored the pika-ci-merge branch March 20, 2026 21:57
@justinmoon justinmoon deleted the pika-ci-merge branch March 20, 2026 21:59
@justinmoon justinmoon restored the pika-ci-merge branch March 20, 2026 22:03
@justinmoon justinmoon deleted the pika-ci-merge branch March 20, 2026 22:04
@justinmoon justinmoon restored the pika-ci-merge branch March 20, 2026 22:09
@justinmoon justinmoon deleted the pika-ci-merge branch March 20, 2026 22:10
@justinmoon justinmoon restored the pika-ci-merge branch March 20, 2026 22:15
@justinmoon justinmoon deleted the pika-ci-merge branch March 20, 2026 22:20
@justinmoon justinmoon restored the pika-ci-merge branch March 20, 2026 22:21
@justinmoon justinmoon deleted the pika-ci-merge branch March 20, 2026 22:26
@justinmoon justinmoon restored the pika-ci-merge branch March 20, 2026 22:26
@justinmoon justinmoon deleted the pika-ci-merge branch March 20, 2026 22:31
@justinmoon justinmoon restored the pika-ci-merge branch March 21, 2026 18:35
@justinmoon justinmoon deleted the pika-ci-merge branch March 21, 2026 18:36
@justinmoon justinmoon restored the pika-ci-merge branch March 21, 2026 18:41
@justinmoon justinmoon deleted the pika-ci-merge branch March 21, 2026 18:46
@justinmoon justinmoon restored the pika-ci-merge branch March 21, 2026 18:47
@justinmoon justinmoon deleted the pika-ci-merge branch March 21, 2026 18:52
@justinmoon justinmoon restored the pika-ci-merge branch March 21, 2026 18:53
@justinmoon justinmoon deleted the pika-ci-merge branch March 21, 2026 18:57
@justinmoon justinmoon restored the pika-ci-merge branch March 21, 2026 18:59
@justinmoon justinmoon deleted the pika-ci-merge branch March 21, 2026 19:02
@justinmoon justinmoon restored the pika-ci-merge branch March 21, 2026 19:04
@justinmoon justinmoon deleted the pika-ci-merge branch March 21, 2026 19:07
@justinmoon justinmoon restored the pika-ci-merge branch March 21, 2026 19:10
@justinmoon justinmoon deleted the pika-ci-merge branch March 21, 2026 19:13
@justinmoon justinmoon restored the pika-ci-merge branch March 21, 2026 19:16
@justinmoon justinmoon deleted the pika-ci-merge branch March 21, 2026 19:18
@justinmoon justinmoon restored the pika-ci-merge branch March 21, 2026 19:21
@justinmoon justinmoon deleted the pika-ci-merge branch March 21, 2026 19:23
@justinmoon justinmoon restored the pika-ci-merge branch March 21, 2026 19:27
@justinmoon justinmoon deleted the pika-ci-merge branch March 21, 2026 19:29
@justinmoon justinmoon restored the pika-ci-merge branch March 21, 2026 19:33
@justinmoon justinmoon deleted the pika-ci-merge branch March 21, 2026 19:34
@justinmoon justinmoon restored the pika-ci-merge branch March 21, 2026 19:38
@justinmoon justinmoon deleted the pika-ci-merge branch March 21, 2026 19:39
@justinmoon justinmoon restored the pika-ci-merge branch March 21, 2026 19:44
@justinmoon justinmoon deleted the pika-ci-merge branch March 21, 2026 19:45
@justinmoon justinmoon restored the pika-ci-merge branch March 21, 2026 19:50
@justinmoon justinmoon deleted the pika-ci-merge branch March 21, 2026 19:55
@justinmoon justinmoon restored the pika-ci-merge branch March 21, 2026 19:59
@justinmoon justinmoon deleted the pika-ci-merge branch March 21, 2026 20:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant