Repository navigation
v1.10.0
Notable Changes
Added
- Added experimental
versioning_override:toWorkflow.start_child_workflowandWorkflow.execute_child_workflow, includingVersioningOverride::OneTime, to route children independently of their parent on Temporal Server 1.32.0 or later. Rejected overrides raiseError::ChildWorkflowErrorwith anError::InvalidVersioningOverrideErrorcause. - Core: The OTLP metrics exporter can export histograms as OpenTelemetry exponential histograms, which Prometheus-compatible backends store as native histograms, when an SDK enables the option, including through the C bridge. It applies to every histogram, including custom ones, and names each series with a suffix,
_nativeby default, so it never shares a name with the fixed-bucket series it replaces. It cannot be combined with histogram bucket overrides. Explicit buckets remain the default. - Core: The
temporal_activity_execution_failedandtemporal_local_activity_execution_failedworker metrics now carry afailure_reasonattribute. Each is now split into one time series per reason, which may affect existing dashboards. - Core: External workflow signal and cancellation resolution activations now include the typed server failure cause alongside the existing failure.
- Core: Workflow completion-as-cancelled commands can now carry details for recording on the terminal history event.
- Core: Workflow task completions larger than the gRPC request size limit are now paginated automatically when the namespace supports it. Paginated workflow task completions require Temporal Server 1.32.0 or later.
Stabilized
- Priority is no longer marked as experimental.
- The
dev_server_andtest_server_options ofTesting::WorkflowEnvironment.start_localandstart_time_skippingare no longer described as unstable.
💥 Breaking Changes
Schedule::Policy#catchup_windownow defaults tonil. If unspecified the server configured default is used.- Core: Replay detects changes to the service or operation of scheduled Nexus operations, reporting nondeterminism while retaining compatibility with histories recorded before these checks.
Fixed
- The test
ActivityEnvironmentno longer addsdetail_hintsto the details it passes toon_heartbeat. - Require
json < 3to prevent JSON payload decoding failures with JSON 3. - Core: Workers now replay workflows paused before a scheduled task starts, allowing those workflows to make progress after unpause.
- Core: Task-poll targets no longer decrease after cancelled or timed-out polls. Affected pollers still retain their slot during backoff, while resource-exhaustion errors still reduce the target.
- Core: Replay now preserves which local activity results were delivered together for newly recorded histories. This prevents local activity scheduling which was conditional on local activity resolution order from assigning recorded results to different handles. Older markers keep their existing behavior.
- Core: Workflow poll balancing now lets non-sticky pollers use capacity after sticky pollers reach their configured or autoscaled polling limit.
- Core: Every path that fails a workflow task now only reports the failure to server on the task's first attempt, and later attempts are left to time out. Previously
PayloadsTooLargefailures and history fetch failures were re-reported on every attempt. - Core: The
workflow_task_execution_failedmetric is now recorded for every failed workflow task attempt, including attempts whose failure was not sent to the server, and itsfailure_reasontag distinguishesGrpcMessageTooLarge,PayloadsTooLarge, andRequestTooLargeon every path. - Core: Worker shutdown now drains activity completions that are still flushing their result to the server before finishing. Previously such a completion — typically one whose final heartbeat RPC was still in flight — could be permanently stranded by shutdown: the activity's result was never reported (the server had to time the attempt out before retrying it), and workers missed shutdown's slot-permit release deadline, panicking in debug builds.
- Core: The Prometheus exporter now appends
_totalto counter metric names when an SDK enables the counter suffix option. - Core: Update-with-start
ExecuteMultiOperationcalls now use Core's long-poll timeout instead of the normal RPC timeout, avoiding premature failures while waiting for an update to reach its requested stage. - Core: An activity failure caused by oversized final heartbeat details is now counted in the
temporal_activity_execution_failedmetric asfailure_reason="PayloadsTooLarge". Previously it was counted under the reason for the failure the activity itself reported, and was not counted at all when that failure was benign, even though a payload-limit failure was reported instead. - Core: Workers now warn when autoscaling task polling encounters errors continuously for one minute. Repeated warnings use exponential backoff up to 15-minute intervals and stop after polling recovers.
- Core: Workers no longer send worker heartbeats or appear in centralized heartbeat reports before they begin polling.
- Core: Nexus tasks are now timed out locally even when the server sends a
request-timeoutheader that falls outside the Nexus duration grammar, such as a negative value for a task whose deadline has already elapsed, a sub-millisecond unit, or a multi-unit value like1m30s. Previously such a header was ignored entirely, so the handler was never told the task had timed out, and a task left unanswered could block worker shutdown indefinitely. - Core: Workers with a small workflow cache no longer briefly stop accepting new workflows. Sticky workflow-task pollers could consume every workflow-cache permit and starve the non-sticky poller, so the worker would stop picking up new workflows until a poll timed out (up to ~60s). The poll balancer now reserves a non-sticky slot against the workflow cache size rather than the slot supplier size.
- Core: Worker heartbeats now report correct task-slot and poller counts when using buffered or custom metrics backends, including the Python SDK's
MetricBuffer. Previously these counts could remain0because heartbeat accounting depended on reading labels from backend-owned attributes. - Core: Local activities still running when their workflow is evicted or the worker shuts down now always receive a cancellation. Local activities no longer retry ordinary failures or start-to-close timeouts after cancellation has been requested, matching non-local activities. Workflow code no longer observes a spurious local activity cancellation when its run is evicted or completes while the local activity is backing off between retries.
- Core: Workers no longer panic when evicting a workflow with buffered local activity results. Completed activations' output is recorded before eviction, without recording eviction-induced cancellations.
- Core: A workflow task that arrives while a workflow waits on a local activity, such as a legacy query, now waits until the current workflow task is reported. Previously it replaced that workflow task ("Trying to send a new WFT for a run which already has one!"), which then timed out on the server and could leave the workflow stuck on a nondeterminism error.
Security
- Core: Ephemeral server downloads now use
zip9.0 for ZIP extraction, replacing the 8.x dependency yanked upstream for security issues.
New Contributors
- @yuandrew made their first contribution in #588
- @sdk-sentinel-bot made their first contribution in #569
- @RaphaelFakhri made their first contribution in #626
Full Changelog: v1.9.0...v1.10.0