Open
Description
I've encountered following errors when building using Bazel with RBE and Remote Cache enabled:
Error log:
(14:26:00) ERROR: /home/dkostyrev/.cache/bazel/_bazel_dkostyrev/b31a559a55463066c351c3c03000b506/external/src/infra/e2etest/harness/e2etesting/BUILD.bazel:21:11: GoCompilePkg external/src/infra/e2etest/harness/e2etesting/e2etesting.a failed: (Exit 34): NOT_FOUND: status: NotFound, message: "Could not find AwaitedAction for the given operation id OperationIdToAwaitedAction(String(\"{\\\"Uuid\\\":\\\"2233180b-dd3d-406f-84c6-8755dab4d492\\\"}\")) : In OperationSubscriber::borrow : In SimpleSchedulerStateManager::filter_operations : In SimpleScheduler::find_by_client_operation_id getting filter result : Error running find_existing_action in ExecutionServer::wait_execution", details: [], metadata: MetadataMap { headers: {} } : Failed on wait_execution() command
java.io.IOException: io.grpc.StatusRuntimeException: NOT_FOUND: status: NotFound, message: "Could not find AwaitedAction for the given operation id OperationIdToAwaitedAction(String(\"{\\\"Uuid\\\":\\\"2233180b-dd3d-406f-84c6-8755dab4d492\\\"}\")) : In OperationSubscriber::borrow : In SimpleSchedulerStateManager::filter_operations : In SimpleScheduler::find_by_client_operation_id getting filter result : Error running find_existing_action in ExecutionServer::wait_execution", details: [], metadata: MetadataMap { headers: {} } : Failed on wait_execution() command
The problem lies in inconsistency between expected data format when writing and reading ClientOperationID to WorkerOperationID mapping in Redis Scheduler Store. Nativelink writes OperationID as a JSON structure for every cid_* key, but reads it back as a String. This results in Error 34 (Not found) errors when Bazel client calls Execution.WaitExecution GRPC method.
Metadata
Metadata
Assignees
Labels
No labels