Fix legacy entity protocol: propagate call_entity failures and stop double-encoding results

[andystaples]

Two related fixes to OrchestrationContext.call_entity under the legacy entity protocol — the protocol the Azure Functions Durable (WebJobs) extension uses when talking to out-of-proc Python workers. The current entity protocol (entityOperationCompleted / entityOperationFailed, used by durabletask.azuremanaged / DTS) already behaved correctly; only the legacy eventRaised-based path was affected.

1. Entity operation failures were not propagated

A failed entity operation was silently completing (typically returning None) instead of raising, so orchestrations could never observe or handle the failure.

Legacy-protocol entity results arrive as an eventRaised handled by _handle_entity_event_raised, which unconditionally called entity_task.complete(...) and never inspected the response for a failure indicator. The wire contract is DurableTask.Core's ResponseMessage, where IsErrorResult => exceptionType != null || failureDetails != null (exceptionType carries the error message; failureDetails carries a structured FailureDetails).

Fix: _handle_entity_event_raised now detects a failed response and fails the task with an EntityOperationFailedException, so an awaiting call_entity raises TaskFailedError — matching the current entity protocol and the .NET SDK (CallEntityAsync throws EntityOperationFailedException when the result IsError).

2. Successful results were double-encoded

The ResponseMessage.result field holds a serialized JSON string of the return value (just like the new protocol's entityOperationCompleted.output). The success path used coerce, which applies a type to an already-parsed value and does not parse JSON strings. As a result, string/dict/list returns arrived double-encoded (e.g. a returned "done" surfaced as "\"done\""), and only numeric returns with an explicit return_type happened to work.

Fix: deserialize the result string (type-directed) instead of coercing it, mirroring the current-protocol handler.

Also addressed

• Cleaned up the legacy eventRaised dispatch to pop() _entity_task_id_map / _entity_lock_task_id_map entries once handled (they were looked up with get() and never removed), preventing unbounded map growth in long-running orchestrations — consistent with the new-protocol handlers. (From PR review.)

Changes

• durabletask/worker.py: failure detection + propagation in _handle_entity_event_raised; type-directed deserialization of the result; pop() map cleanup.
• durabletask/internal/helpers.py: entity_response_failure_details(...) plus a recursive FailureDetails converter.
• tests/durabletask/test_orchestration_executor.py: legacy-protocol tests for failure via failureDetails, failure via exceptionType, and a parametrized success round-trip across str / int / float / bool / dict / list (no return_type required).
• CHANGELOG.md: FIXED entries for both bugs.

Validation

• New legacy-protocol tests pass; entity + orchestration-executor suites pass.
• flake8 and Pylance clean on changed files.
• durabletask.azuremanaged uses the current protocol and is unaffected.

[Copilot]

Pull request overview

This PR fixes a correctness gap in the core durabletask SDK where entity operation failures were not propagated to orchestrations when using the legacy entity protocol (used by the Azure Functions Durable/WebJobs extension). It aligns legacy-protocol behavior with the current protocol (and .NET) by detecting failure indicators in legacy ResponseMessage payloads and failing the awaiting call_entity task accordingly.

Changes:

• Added legacy ResponseMessage failure detection/conversion helpers to produce TaskFailureDetails when failureDetails or exceptionType indicate an entity operation failure.
• Updated legacy-protocol eventRaised handling to fail call_entity tasks with EntityOperationFailedException when failures are detected.
• Added orchestration executor tests covering legacy-protocol entity call failure/success cases, and documented the fix in CHANGELOG.md.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.

File	Description
tests/durabletask/test_orchestration_executor.py	Adds regression tests for legacy-protocol entity call failure propagation (both failureDetails and exceptionType) and a success round-trip.
durabletask/worker.py	Detects legacy-protocol entity call failures in eventRaised handling and propagates them as task failures to orchestrations.
durabletask/internal/helpers.py	Adds helpers to extract/convert legacy ResponseMessage failure data into TaskFailureDetails.
CHANGELOG.md	Documents the user-visible fix under ## Unreleased → FIXED.

[berndverst]

Reviewed this and it's in good shape overall — the double-encoding fix and the map cleanup look correct, and propagating failures instead of silently returning None is clearly the right direction. One substantive concern before merge, on the failure message extraction for the legacy path.

The exceptionType-only branch uses the field's value as the error message:

error_message = response.get("exceptionType")
if error_message is not None:
    return pb.TaskFailureDetails(errorType="", errorMessage=str(error_message))

But _handle_entity_event_raised handles the WebJobs extension "old protocol" eventRaised (as the code comment notes). In that producer — Microsoft.Azure.WebJobs.Extensions.DurableTask/EntityScheduler/ResponseMessage.cs — the message is serialized into result, and exceptionType is the exception's type name, not the message:

public void SetExceptionResult(Exception exception, ...) {
    this.ExceptionType = exception.GetType().AssemblyQualifiedName;  // a TYPE NAME
    this.Result = errorDataConverter.Serialize(exception);          // the actual message/content
}

The sibling Functions SDKs corroborate that exceptionType is only a presence-marker and the message comes from result:

• azure-functions-durable-python ResponseMessage.from_dict: is_error = "exceptionType" in d.keys(); content taken from d["result"].
• azure-functions-durable-python TaskOrchestrationExecutor: new_value = df_loads(event_payload.result, ...); on is_exception it wraps that result value in Exception(...) — the value of exceptionType is never used.
• Its test harness emits error events as {"result": json.dumps(msg), "exceptionType": "True"}.
• azure-functions-durable-js: if (exceptionType !== undefined) taskResult = Error(result_string).

So for a real WebJobs failure the current code would surface an assembly-qualified type name (or a literal marker like "Error"/"True") as the message and discard the real message that lives in result. Failures do propagate now (strictly better than before), but the message would be wrong.

Worth noting: the failureDetails branch and the "exceptionType == message" model actually match a different type — DurableTask.Core.Entities.EventFormat.ResponseMessage (JSON exceptionType → ErrorMessage, plus failureDetails), which belongs to the current protocol, not the WebJobs eventRaised shape. So the two ResponseMessage shapes may be getting conflated.

Two suggestions before merge:

1. Confirm the actual wire shape by capturing a real failed-entity eventRaised payload from the WebJobs extension. If it's the WebJobs shape, derive the failure message from the deserialized result (falling back to failureDetails/exceptionType), matching azure-functions-durable-python.
2. The failure tests use {"result": None, "exceptionType": "Something went wrong!"}, which encodes that assumption, so they pass without exercising the real format — worth updating to the confirmed shape once verified.

[andystaples]

Thanks @berndverst — you're right, and I dug in to confirm before fixing. Pushed a correction in 864f1dd.

Verification. I checked both the producer and the canonical consumers:

• Producer (WebJobs extension): TaskEntityShim.ProcessOperationRequestAsync → ResponseMessage.SetExceptionResult, which sets ExceptionType = exception.GetType().AssemblyQualifiedName (a type name) and Result = errorDataConverter.Serialize(exception) (the content/message).
• Consumer azure-functions-durable-python: ResponseMessage.from_dict → is_error = "exceptionType" in d.keys() (presence only); TaskOrchestrationExecutor → new_value = df_loads(event_payload.result, ...), and on is_exception wraps that result value in Exception(...). The exceptionType value is never used.
• Consumer azure-functions-durable-js: TaskOrchestrationExecutor.ts → if (eventPayload.exceptionType !== undefined) ... Error(result) — presence check; message from result.
• Reference test harness (durable-python ContextBuilder): emits error events as { "result": json.dumps(msg), "exceptionType": "True" } — exceptionType is a literal marker; message is in result.

So the two ResponseMessage shapes were indeed being conflated: the eventRaised old protocol is the WebJobs shape, not DurableTask.Core.Entities.EventFormat.ResponseMessage.

The fix (864f1dd):

• Failure detection is now presence-based: "exceptionType" in response (with a structured failureDetails object still accepted defensively as a superset).
• The failure message is taken from the deserialized result payload; exceptionType is treated only as a marker / type name and is no longer surfaced as the message.
• Updated the failure test to the real wire shape — message in result, an assembly-qualified type name in exceptionType — and added an assertion guarding against surfacing the type name instead of the message.

On capturing a live payload: agreed it's the gold standard, but standing up the Functions host is heavy for this PR, and the reference SDKs (which shipped as the consumers of exactly this out-of-proc-Python eventRaised) plus the WebJobs producer source pin the contract unambiguously. Happy to add a captured-payload fixture in a follow-up if you'd like belt-and-suspenders coverage.