Skip to content

Stream recipe output in real time#49

Merged
lhparker1 merged 1 commit into
mainfrom
feat/streaming-output
Mar 25, 2026
Merged

Stream recipe output in real time#49
lhparker1 merged 1 commit into
mainfrom
feat/streaming-output

Conversation

@lhparker1
Copy link
Copy Markdown
Member

@lhparker1 lhparker1 commented Mar 23, 2026

Summary

  • Replace subprocess.run(capture_output=True) with a streaming Popen wrapper in all recipe execution backends (container, local, venv)
  • Output lines are printed to stdout/stderr as they arrive, so users can see progress during long-running fits
  • Sets PYTHONUNBUFFERED=1 in subprocess environment automatically
  • Still retains the last 2000 chars of each stream in ExecutionResult metadata

Test plan

  • Run prism run locally and verify output streams line-by-line
  • Run with venv backend and verify streaming works
  • Verify ExecutionResult.metadata still contains stdout/stderr tails
  • Run a failing recipe and verify stderr is streamed and captured

🤖 Generated with Claude Code

Signed-off-by: lhparker1 liamholdenparker@gmail.com

Recipe execution previously captured all output and only stored the
last 2000 chars in metadata. This made it impossible to see progress
during long-running fits. Replace subprocess.run(capture_output=True)
with a streaming Popen wrapper that prints lines as they arrive while
still retaining the tail for ExecutionResult metadata. Also sets
PYTHONUNBUFFERED=1 in the subprocess environment.

Applies to all execution backends: container, local, and venv.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@lhparker1 lhparker1 requested a review from EiffL March 23, 2026 15:05
Copy link
Copy Markdown
Member

@EiffL EiffL left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions
Copy link
Copy Markdown

github-actions Bot commented Mar 25, 2026

Developer Certificate of Origin

All contributors have signed the DCO. This PR is ready to merge.

Status

@lhparker1
Copy link
Copy Markdown
Member Author

I have read the Developer Certificate of Origin and I hereby sign the DCO for this PR

@lhparker1 lhparker1 merged commit fd67eda into main Mar 25, 2026
6 of 7 checks passed
@aboucaud aboucaud deleted the feat/streaming-output branch May 8, 2026 10:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants