worker_plan: represent plan_raw.json as InitialPlanRawTask#666
Merged
Conversation
`plan_raw.json` is supplied by the caller before the pipeline runs.
Three tasks needed to read it:
- `SetupTask` (always, to produce `plan.txt`)
- `PromptAdherenceTask` (wants the bare prompt)
- `ScreenPlanningPromptTask` (wants the bare prompt)
Each was reaching directly into the run directory:
raw_path = self.run_id_dir / FilenameEnum.INITIAL_PLAN_RAW.value
with `SetupTask` additionally hand-rolling its own existence
check. Both `PromptAdherence` and `ScreenPlanningPrompt` listed
`SetupTask` in `requires()` purely for ordering, even though
neither actually consumed `plan.txt`.
Introduce `InitialPlanRawTask` (`luigi.ExternalTask`) that
exposes `plan_raw.json` as a Luigi target. Consumers now go
through the DAG:
def requires(self):
return self.clone(InitialPlanRawTask)
def run_with_llm(self, llm):
plan_file = PlanFile.load(self.input().path)
plan_prompt = plan_file.plan_prompt
`SetupTask`'s manual `FileNotFoundError` check is removed —
Luigi's `ExternalTask.complete()` already gates the pipeline on
the file's existence. `PromptAdherenceTask` swaps its `'setup'`
requires entry for `'plan_raw'`, dropping the unused dependency
on the formatted `plan.txt`.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
plan_raw.jsonis supplied by the caller before the pipeline starts. Three tasks needed to read it (SetupTask,PromptAdherenceTask,ScreenPlanningPromptTask), and each was reaching directly into the run dir withself.run_id_dir / FilenameEnum.INITIAL_PLAN_RAW.value.SetupTaskadditionally hand-rolled an existence check; the two diagnostic tasks listedSetupTaskinrequires()purely for ordering even though neither readplan.txt.This PR introduces
InitialPlanRawTask— aluigi.ExternalTaskthat exposesplan_raw.jsonas a Luigi target — and routes all three consumers through it.SetupTasknowrequires(InitialPlanRawTask); the manualFileNotFoundErrorcheck is gone (Luigi'sExternalTask.complete()already gates on file existence).ScreenPlanningPromptTaskswaps itsSetupTaskdependency forInitialPlanRawTask— it never usedplan.txt.PromptAdherenceTaskswaps the'setup'requires entry for'plan_raw'— same reason.Consumer code becomes
PlanFile.load(self.input().path)instead of building the path manually.Test plan
plan.txt,screen_planning_prompt.json, andprompt_adherence_raw.jsonare still produced.plan_raw.jsonfrom a fresh run dir and confirm Luigi reports the external dependency as not satisfied (instead of the oldFileNotFoundError).🤖 Generated with Claude Code