feat: allow task prompt to be loaded from file (#157)#200
Merged
Conversation
spboyer
requested review from
chlowell,
richardpark-msft and
wbreza
as code owners
Contributor
There was a problem hiding this comment.
Pull request overview
Adds support for specifying a task prompt via an external file (inputs.prompt_file) in task YAML, resolving the file path relative to the task YAML directory.
Changes:
- Extend task stimulus model to support
prompt_fileand load file contents into the prompt duringLoadTestCase(). - Add tests covering prompt file loading behavior and validation scenarios.
- Update
schemas/task.schema.jsonto allow eitherpromptorprompt_fileviaoneOf.
Show a summary per file
| File | Description |
|---|---|
schemas/task.schema.json |
Allows inputs.prompt_file as an alternative to inline inputs.prompt via oneOf. |
internal/models/testcase.go |
Adds prompt_file field and resolves it into Stimulus.Message at load time. |
internal/models/testcase_test.go |
Adds unit tests for prompt file loading and related parsing/validation behavior. |
Copilot's findings
- Files reviewed: 3/3 changed files
- Comments generated: 5
Add prompt_file field to TestStimulus as an alternative to inline prompt. When prompt_file is set, the file content is read and used as the prompt message. The path is resolved relative to the task YAML file's directory. Validation: - Error if both prompt and prompt_file are set - Error if prompt_file doesn't exist Includes 7 test cases covering file load, subdirectory paths, mutual exclusivity, missing file, inline fallback, empty inputs, and multiline. Updates task.schema.json with prompt_file property and oneOf constraint. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…dPath filepath.IsAbs does not recognize /etc/evil.yaml as absolute on Windows since it lacks a drive letter. Add explicit check for leading / to catch cross-platform absolute path injection. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Reject absolute paths and path traversal (security, per runner pattern) - Clear MessageFile after resolve to avoid leaking paths in serialized output - Add minLength: 1 to prompt_file in JSON schema - Add 3 new tests: absolute path, path traversal, MessageFile clearing Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
spboyer
force-pushed
the
squad/157-prompt-from-file
branch
from
65ede22 to
296628d
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Closes #157
Adds
prompt_fileas an alternative to inlinepromptin task YAML. Users can now specify:The file path is resolved relative to the task YAML file's directory.
Changes
internal/models/testcase.go— AddedMessageFilefield toTestStimulus(yaml:"prompt_file"). AddedresolvePromptFile()method called fromLoadTestCase()that reads file content intoMessage.internal/models/testcase_test.go— 7 new test cases: file load, subdirectory paths, mutual exclusivity error, missing file error, inline fallback, empty inputs, multiline content.schemas/task.schema.json— Addedprompt_fileproperty andoneOfconstraint replacing the hardrequired: ["prompt"].Validation Rules
promptandprompt_fileare setprompt_filepath doesn't existWorking as Linus (Backend Developer)