[copilot-finds] Bug: Entity execution not tracked in pendingWorkItems, causing silent data loss on shutdown #172
Description
Problem
In packages/durabletask-js/src/worker/task-hub-grpc-worker.ts, the entity execution methods _executeEntity() (line 837) and _executeEntityV2() (line 921) are async methods whose returned promises are never tracked in the _pendingWorkItems set.
In contrast, _executeOrchestrator() (line 558) and _executeActivity() (line 746) both follow a wrapper pattern: they are synchronous methods that call an async *Internal() method, add the returned promise to _pendingWorkItems, and use .finally() to clean up.
Since entity promises are not tracked, the stop() method (line 447) which waits for Promise.all(this._pendingWorkItems) does not wait for in-flight entity operations.
Root Cause
When entity execution support was added, the methods were written as direct async methods rather than following the synchronous wrapper + tracking pattern established by orchestrator and activity execution. The returned promises are silently discarded by the stream "data" event handler.
Proposed Fix
Refactor _executeEntity and _executeEntityV2 to follow the same pattern as _executeOrchestrator and _executeActivity:
- Rename the async methods to
_executeEntityInternaland_executeEntityV2Internal - Create synchronous wrapper methods that track the work promises in
_pendingWorkItems - Update
_executeEntityV2Internalto call_executeEntityInternaldirectly (avoiding double-tracking)
Impact
Severity: High — During worker shutdown (graceful or crash recovery), in-flight entity operations may be:
- Interrupted mid-execution, leaving entity state partially mutated
- Never send their results to the sidecar, leaving it waiting indefinitely
- Not participate in the shutdown timeout mechanism, bypassing the graceful shutdown guarantee
This affects any deployment using entities where the worker shuts down while entity operations are in progress.