Measured on GKE 3-node, 1.35.6-gke.1710000, gVisor, demos/counter. 3 workers on 3 nodes with one ActorTemplate, found that on a node that never ran a golden, static-files is absent entirely. As a result, resumes actor on workers of other 2 nodes cause livelock. Tested this hypothesis by moving all workers to the warm node gives HTTP 200 in ~1 s; then cordoning it so a new template's golden lands on a cold node flips that node to warm and it resumes instantly — same node, cache alone.
e2e misses it because every suite creates its own template, hence its own golden and a warm node, before resuming anything — and single-node kind shares one cache across all workers.
The actor is also unreapable. After minutes in ACTOR_STATE_RESUMING, suspend returns MarkSuspending prerequisite not met (got: ACTOR_STATE_RESUMING, want ACTOR_STATE_RUNNING or ACTOR_STATE_PAUSED) and delete returns not in a deletable state. The only move left is deleting the worker pod, which makes the actor CRASHED.
Measured on GKE 3-node,
1.35.6-gke.1710000, gVisor, demos/counter. 3 workers on 3 nodes with oneActorTemplate, found that on a node that never ran a golden,static-filesis absent entirely. As a result, resumes actor on workers of other 2 nodes cause livelock. Tested this hypothesis by moving all workers to the warm node gives HTTP 200 in ~1 s; then cordoning it so a new template's golden lands on a cold node flips that node to warm and it resumes instantly — same node, cache alone.e2e misses it because every suite creates its own template, hence its own golden and a warm node, before resuming anything — and single-node kind shares one cache across all workers.
The actor is also unreapable. After minutes in
ACTOR_STATE_RESUMING,suspendreturnsMarkSuspending prerequisite not met (got: ACTOR_STATE_RESUMING, want ACTOR_STATE_RUNNING or ACTOR_STATE_PAUSED)anddeletereturnsnot in a deletable state. The only move left is deleting the worker pod, which makes the actorCRASHED.