Skip to content

v0.0.53

Choose a tag to compare

@tigerroll tigerroll released this 03 Oct 14:16
· 34 commits to main since this release
5321013

Release

Overview

  • This release completes the implementation of Step-Level Concurrency Control for PartitionStep, formalizes Partition Execution Semantics, expands automated Failure Matrix integration tests, and introduces OpenTelemetry observability across worker executions.

What's Changed

1. Step-Level Concurrency Control (PartitionStep)

  • concurrency Parameter Support: Added partition.concurrency configuration in JSL step definitions, allowing developers to set maximum concurrent partition worker bounds per step.
  • Safe Defaults: Defaults to concurrency: 1 (enforcing "Serial-First" execution safety) when unspecified.
  • Semaphore Throttling: Integrated golang.org/x/sync/semaphore inside PartitionStep.Execute to throttle active workers and prevent resource exhaustion (e.g., DB connection starvation, memory spikes).

2. Execution Semantics & Architecture Specifications (docs/strategy/partition_execution.md)

  • Worker Lifecycle State Machine: Formalized partition worker state transitions (STARTING $\rightarrow$ STARTED $\rightarrow$ COMPLETED / FAILED / STOPPED).
  • Cancellation Propagation: Standardized context cancellation and errgroup error teardown rules between Controller Step and active Partition Workers.

3. Failure Matrix Integration Tests & Observability

  • Failure Matrix Coverage (partition_step_test.go): Added extensive integration test cases covering edge-case scenarios:
    • Partial worker failure state aggregation
    • Context cancellation propagation
    • Multi-worker failure aggregation
    • Worker execution timeouts
    • Job repository persistence failure handling
  • OpenTelemetry Instrumentation: Added worker-level OpenTelemetry span tracking for worker start, success, and error events with safe NoOp fallbacks when unconfigured.

⚠️ Developer Guidance & Migration Notes

  1. JSL Configuration: When defining type: partition steps in JSL, configure concurrency based on target workload requirements (e.g., concurrency: 8). Unconfigured steps default to 1.
  2. Observability: Partition worker traces will now automatically export through configured OpenTelemetry collectors.