[WIP] proto#62772
Conversation
There was a problem hiding this comment.
Code Review
This pull request introduces a Flight-based object store for Ray to handle Arrow tables out-of-band, utilizing Arrow Flight RPC for remote transfers and process_vm_readv for high-performance same-node transfers. The implementation includes a C++ core, Cython wrappers, and integration into Ray's task output and serialization paths. Feedback highlights several critical issues: the use of destructive read semantics in the Flight server which conflicts with Ray's immutable object model, a lack of platform guards for Linux-specific system calls, and a busy-wait loop in the worker output logic. Additionally, concerns were raised regarding memory efficiency due to eager serialization, security risks from binding to all network interfaces, and potential runtime errors in the Bazel repository rule.
…es/arrow-proto
f184f9f to
6888dd9
Compare
No description provided.