Skip to content

Add parallel execution, result capture, and volumes support#1

Merged
vdemeester merged 3 commits intomainfrom
feature/parallel-execution
Feb 3, 2026
Merged

Add parallel execution, result capture, and volumes support#1
vdemeester merged 3 commits intomainfrom
feature/parallel-execution

Conversation

@vdemeester
Copy link
Copy Markdown
Owner

Summary

  • Parallel task execution: DAG-based scheduler executes independent tasks concurrently based on runAfter dependencies
  • Result capture: Read results from /tekton/results/ directory and enable $(tasks.foo.results.bar) substitution
  • Volume support: EmptyDir, ConfigMap, and Secret volumes with step-level volumeMounts

Changes

Parallel Execution

  • New DAG type organizes tasks into execution levels
  • Tasks at the same level run concurrently via goroutines
  • Respects runAfter dependencies between levels

Result Capture

  • Reads result files from /tekton/results/<name> after step execution
  • Stores results for substitution in downstream tasks
  • Trims whitespace from result values

Volumes

  • Added Volume, VolumeSource, VolumeMount types
  • Supports EmptyDir (cache volumes), ConfigMap, Secret
  • Step-level volumeMounts for mounting into containers

New Files

pkg/executor/scheduler.go      - DAG builder and parallel executor
pkg/executor/results.go        - Result capture logic
pkg/executor/volumes.go        - Volume parsing
examples/simple/results-pipelinerun.yaml
examples/simple/volumes-pipelinerun.yaml

Test plan

  • 9 tests for DAG building and parallel execution
  • 5 tests for result capture and substitution
  • 5 tests for volume parsing
  • All 26 tests passing
  • golangci-lint passes with 0 issues
  • Manual test with Dagger running

🤖 Generated with Claude Code

vdemeester and others added 3 commits February 3, 2026 16:27
- Add DAG type that organizes tasks into execution levels based on runAfter
- Tasks at the same level execute concurrently via goroutines
- Tasks wait for all dependencies before starting (level-based scheduling)
- Includes comprehensive tests for DAG building and parallel execution

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add captureResults function to read from /tekton/results/ directory
- Results are trimmed of whitespace before storage
- Enables $(tasks.taskname.results.resultname) substitution
- Add results-pipelinerun.yaml example demonstrating result passing

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add Volume, VolumeSource, VolumeMount types to pkg/types
- Support EmptyDir, ConfigMap, and Secret volume types
- Add VolumeMounts field to Step for per-step mounting
- Integrate volume mounting into executor
- Add volumes-pipelinerun.yaml example

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
@vdemeester vdemeester merged commit df47690 into main Feb 3, 2026
@vdemeester vdemeester deleted the feature/parallel-execution branch February 3, 2026 15:36
vdemeester added a commit that referenced this pull request Feb 5, 2026
Successfully completed all 5 tasks of Phase 1:

✅ Task #1: Backend interface created (326 LOC, 7 tests)
✅ Task #2: Orchestrator package extracted (600 LOC, 40+ tests)
✅ Task #3: Dagger backend refactored (550 LOC, 15 tests)
✅ Task #4: CLI updated to use new backend
✅ Task #5: All tests passing, chisel builds and runs

Architecture:
- pkg/backend/backend.go - Interface for pluggable backends
- pkg/orchestrator/ - Generic orchestration logic (reusable)
- pkg/backend/dagger/ - Dagger implementation
- cmd/chisel - Uses backend abstraction

Results:
- Zero regressions in chisel functionality
- 80% code reuse achieved (types, parser, UI, orchestrator)
- Clean separation of concerns
- Ready for Phase 2 (Mallet CLI stub)

Binary: 29MB, all examples work, dry-run validated
vdemeester added a commit that referenced this pull request Feb 5, 2026
Successfully completed all 5 tasks of Phase 1:

✅ Task #1: Backend interface created (326 LOC, 7 tests)
✅ Task #2: Orchestrator package extracted (600 LOC, 40+ tests)
✅ Task #3: Dagger backend refactored (550 LOC, 15 tests)
✅ Task #4: CLI updated to use new backend
✅ Task #5: All tests passing, chisel builds and runs

Architecture:
- pkg/backend/backend.go - Interface for pluggable backends
- pkg/orchestrator/ - Generic orchestration logic (reusable)
- pkg/backend/dagger/ - Dagger implementation
- cmd/chisel - Uses backend abstraction

Results:
- Zero regressions in chisel functionality
- 80% code reuse achieved (types, parser, UI, orchestrator)
- Clean separation of concerns
- Ready for Phase 2 (Mallet CLI stub)

Binary: 29MB, all examples work, dry-run validated
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant