Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[C++][Flight] Flight benchmark doesn't work anymore #41780

Closed
pitrou opened this issue May 22, 2024 · 8 comments
Closed

[C++][Flight] Flight benchmark doesn't work anymore #41780

pitrou opened this issue May 22, 2024 · 8 comments

Comments

@pitrou
Copy link
Member

pitrou commented May 22, 2024

Describe the bug, including details regarding any error messages, version, and platform.

On my local build:

$ /build/build-release/relwithdebinfo/arrow-flight-benchmark 
Testing method: DoGet
Using spawned TCP server
Server running with pid 71195
Server host: localhost
Server port: 31337
Failed with error: << IOError: Flight returned unavailable error, with message: failed to connect to all addresses; last error: UNKNOWN: ipv4:127.0.0.1:31337: Failed to connect to remote host: Connection refused. Detail: Unavailable

Component(s)

Benchmarking, C++, FlightRPC

@pitrou
Copy link
Member Author

pitrou commented May 22, 2024

@lidavidm @kou

@kou
Copy link
Member

kou commented May 23, 2024

Hmm. It seems that

if (client->DoAction(call_options, action).ok()) {
returns OK even when arrow-flight-perf-server isn't listen a socket.

@lidavidm
Copy link
Member

That isn't surprising for gRPC; streams only give errors when finished. (Another reason not to wrap gRPC so heavily.)

@lidavidm
Copy link
Member

A while ago DoAction would eagerly consume all results up front (the wrong behavior!) meaning that it would give errors immediately. This was fixed by a contributor but had the side effect of breaking this kind of usage (which was wrong all along, but hidden by Flight being a heavy wrapper over another library that we don't properly understand).

@kou
Copy link
Member

kou commented May 23, 2024

Ah, I see. The following works. Is it OK? Or do we have more simpler way?

diff --git a/cpp/src/arrow/flight/flight_benchmark.cc b/cpp/src/arrow/flight/flight_benchmark.cc
index f53b1c6dce..057ef15c3c 100644
--- a/cpp/src/arrow/flight/flight_benchmark.cc
+++ b/cpp/src/arrow/flight/flight_benchmark.cc
@@ -131,7 +131,8 @@ struct PerformanceStats {
 Status WaitForReady(FlightClient* client, const FlightCallOptions& call_options) {
   Action action{"ping", nullptr};
   for (int attempt = 0; attempt < 10; attempt++) {
-    if (client->DoAction(call_options, action).ok()) {
+    auto result_stream_result = client->DoAction(call_options, action);
+    if (result_stream_result.ok() && (*result_stream_result)->Drain().ok()) {
       return Status::OK();
     }
     std::this_thread::sleep_for(std::chrono::milliseconds(1000));

@lidavidm
Copy link
Member

That should be good.

kou added a commit to kou/arrow that referenced this issue May 23, 2024
We should read from result stream to get an error of this RPC. If we
don't read from result stream, we can't detect an error of this RPC.
kou added a commit to kou/arrow that referenced this issue May 23, 2024
We should read from result stream to get an error of this RPC. If we
don't read from result stream, we can't detect an error of this RPC.
@kou
Copy link
Member

kou commented May 23, 2024

#41793

kou added a commit that referenced this issue May 23, 2024
### Rationale for this change

We should read from result stream to get an error of this RPC. If we don't read from result stream, we can't detect an error of this RPC.

### What changes are included in this PR?

Call `Drain()` to detect an error.

### Are these changes tested?

Yes.

### Are there any user-facing changes?

No.
* GitHub Issue: #41780

Authored-by: Sutou Kouhei <kou@clear-code.com>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
@kou kou added this to the 17.0.0 milestone May 23, 2024
@kou
Copy link
Member

kou commented May 23, 2024

Issue resolved by pull request 41793
#41793

@kou kou closed this as completed May 23, 2024
vibhatha pushed a commit to vibhatha/arrow that referenced this issue May 25, 2024
…pache#41793)

### Rationale for this change

We should read from result stream to get an error of this RPC. If we don't read from result stream, we can't detect an error of this RPC.

### What changes are included in this PR?

Call `Drain()` to detect an error.

### Are these changes tested?

Yes.

### Are there any user-facing changes?

No.
* GitHub Issue: apache#41780

Authored-by: Sutou Kouhei <kou@clear-code.com>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants