Enhance error handling with Timeout variant and per-attempt timeout#13
Enhance error handling with Timeout variant and per-attempt timeout#13
Conversation
…n TaskConfig - Added `Timeout` variant to `CanoError` for bounded operation deadline exceedance. - Introduced `attempt_timeout` in `TaskConfig` to wrap attempts in `tokio::time::timeout`. - Updated documentation to reflect new timeout features and their integration with retries. - Improved panic safety in workflows by catching panics and returning `CanoError::TaskExecution`. - Added tests for timeout behavior and bulkhead concurrency limits.
There was a problem hiding this comment.
Pull request overview
Enhances workflow resilience and control by introducing per-attempt task timeouts, adding a Timeout error variant, improving panic handling for single-task execution, and introducing a bulkhead mechanism to cap split-task concurrency; also bumps crate/docs version to 0.10.3.
Changes:
- Added
CanoError::TimeoutplusTaskConfig::attempt_timeoutintegrated intorun_with_retriesviatokio::time::timeout. - Added panic-to-
CanoError::TaskExecutionconversion for single-task execution and introducedJoinConfig::bulkheadto bound split concurrency. - Updated docs/CSS and added tests covering attempt timeouts, panic capture, and bulkhead behavior; bumped version strings to 0.10.3.
Reviewed changes
Copilot reviewed 10 out of 11 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| docs/static/styles.css | Widens content layout and adjusts padding for ultrawide displays. |
| docs/content/workflows.md | Documents new Timeout error, panic safety behavior, and bulkhead concurrency limits. |
| docs/content/task.md | Documents per-attempt timeout configuration and how it composes with retries. |
| docs/content/resources.md | Adjusts TOC wording for resource derive macros. |
| docs/content/nodes.md | Adjusts TOC wording and documents per-attempt timeout for nodes. |
| docs/config.toml | Updates docs version string to 0.10.3. |
| cano/src/workflow.rs | Adds bulkhead support for split tasks and catches panics for single-task execution; adds tests. |
| cano/src/task.rs | Adds attempt_timeout to TaskConfig and enforces it in run_with_retries; adds tests. |
| cano/src/error.rs | Adds CanoError::Timeout variant, constructor, and related Display/category/eq handling + tests. |
| Cargo.toml | Bumps workspace version to 0.10.3. |
| Cargo.lock | Updates locked crate versions to 0.10.3. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
|
|
||
| [workspace.package] | ||
| version = "0.10.2" | ||
| version = "0.10.3" |
There was a problem hiding this comment.
The workspace version is bumped as a patch (0.10.2 → 0.10.3), but this PR introduces breaking public API changes (e.g., adding a new CanoError variant and new public fields on TaskConfig/JoinConfig breaks exhaustive matches and struct literals in downstream crates). Consider bumping the minor version (e.g., 0.11.0) or otherwise documenting/mitigating the breaking surface (e.g., #[non_exhaustive] + constructor-only patterns going forward).
| version = "0.10.3" | |
| version = "0.11.0" |
| /// This error is automatically generated when a node fails and | ||
| /// all configured retry attempts have been used up. | ||
| RetryExhausted(String), | ||
|
|
||
| /// A bounded operation exceeded its deadline. | ||
| /// | ||
| /// Emitted by per-attempt task timeouts (see | ||
| /// [`crate::task::TaskConfig::with_attempt_timeout`]). The retry loop treats | ||
| /// this as a recoverable failure: each attempt that times out is retried | ||
| /// (subject to the configured `RetryMode`), and if all attempts time out | ||
| /// the loop wraps the final timeout error in | ||
| /// [`CanoError::RetryExhausted`]. | ||
| Timeout(String), | ||
|
|
There was a problem hiding this comment.
Adding a new public Timeout variant to CanoError is a breaking change for downstream code that exhaustively matches on CanoError. If the crate intends to add variants over time without breaking consumers, consider marking CanoError as #[non_exhaustive] (and bumping the version accordingly) and/or updating public guidance to include a wildcard arm when matching.
| pub struct TaskConfig { | ||
| /// Retry strategy for failed executions | ||
| pub retry_mode: RetryMode, | ||
| /// Optional per-attempt timeout. When set, each attempt inside | ||
| /// [`run_with_retries`] is wrapped with [`tokio::time::timeout`]; an | ||
| /// expired attempt produces a [`CanoError::Timeout`] and the retry loop | ||
| /// continues per `retry_mode`. `None` (the default) preserves the | ||
| /// previous behavior of letting attempts run unbounded. | ||
| pub attempt_timeout: Option<Duration>, | ||
| } |
There was a problem hiding this comment.
TaskConfig is a public struct with public fields; adding attempt_timeout is a breaking change for any downstream code constructing TaskConfig via struct literal syntax. If backwards compatibility is important, consider moving toward a non-exhaustive / builder-style API (e.g., #[non_exhaustive] + constructors) and bumping the crate version to reflect the break.
| pub timeout: Option<Duration>, | ||
| /// State to transition to after join condition is met | ||
| pub join_state: TState, | ||
| /// Optional bulkhead: maximum number of split tasks allowed to run | ||
| /// concurrently. When `None` (default) all tasks run as soon as the | ||
| /// runtime can schedule them. When `Some(n)`, a `tokio::sync::Semaphore` | ||
| /// with `n` permits gates each task body, so excess tasks queue until | ||
| /// a permit is free. `Some(0)` is rejected at execution time with | ||
| /// [`CanoError::Configuration`]. | ||
| pub bulkhead: Option<usize>, |
There was a problem hiding this comment.
JoinConfig is a public struct with public fields; adding the new bulkhead field is a breaking change for downstream users constructing JoinConfig with a struct literal. Consider a non-exhaustive / constructor-only approach for public config structs (and bump the version accordingly) to avoid future breakage when adding fields.
Timeoutvariant toCanoErrorfor bounded operation deadline exceedance.attempt_timeoutinTaskConfigto wrap attempts intokio::time::timeout.CanoError::TaskExecution.Update the lib to 0.10.3