Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ARROW-14698: [Docs][FlightRPC] Add API docs for Flight SQL #12616

Closed
wants to merge 7 commits into from

Conversation

lidavidm
Copy link
Member

This improves the protocol docs and adds C++ API documentation. Java already has Javadocs so this PR doesn't touch anything there.

@github-actions
Copy link

@lidavidm
Copy link
Member Author

Rendered (new protocol docs section)

image

The API docs are a little too large to render.

@lidavidm
Copy link
Member Author

CC @jduo if you'd like to take a look

Copy link
Member

@pitrou pitrou left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is really nice, thank you!

@@ -33,113 +33,143 @@ namespace arrow {
namespace flight {
namespace sql {

/// \defgroup flight-sql-protocol-messages Flight SQL Protocol Messages
/// POD wrappers for various protocol messages, used to avoid exposing
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Spell out "POD" in full (or use an equivalent term)?


.. doxygentypedef:: arrow::flight::sql::SqlInfoResultMap

.. doxygenenum:: arrow::flight::sql::SqlInfoOptions::SqlInfo
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps you want to define a doxygen group encompassing these typedefs and enums?

@@ -0,0 +1,26 @@
%% Licensed to the Apache Software Foundation (ASF) under one
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For the record, how are the corresponding SVGs generated? (do you want to add a comment as to that?)

participant Server
Client->>Server: GetFlightInfo(CommandStatementQuery)
Server->>Client: FlightInfo{…Ticket…}
loop for each Ticket in FlightInfo
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this the case? I though all returned tickets were equivalent and only one of them needed to be fetched, am I misremembering?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah yes, this should be "for each FlightEndpoint in FlightInfo"

* The possible values are listed in
* `arrow.flight.protocol.sql.SqlSupportedCaseSensitivity`.
*/
/// Retrieves a uint32 value representing the enu uint32 ordinal
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

*enum int32 ordinal

/// - return 2 (0b10) => [SQL_UNION_ALL];
/// - return 3 (0b11) => [SQL_UNION, SQL_UNION_ALL].
///
/// Valid SQL positioned commands are described under
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Valid union operators

Client->>Server: GetFlightInfo(CommandGetTables)
Server->>Client: FlightInfo{…Ticket…}
Client->>Server: DoGet(Ticket)
Server->>Client: stream of FlightData
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: newline at end of file

@rafael-telles
Copy link
Contributor

Hey @lidavidm !
Commenting here as this may affect this docs: we just noticed that on FlightSql.proto there is some wrong documentation around enum SqlInfo values, saying they should retrieve uint32 values where it should be int64. Like here:

  // Retrieves a uint32 value representing the maximum number of hex characters allowed in an inline binary literal.
  SQL_MAX_BINARY_LITERAL_LENGTH = 541;

uint32 values are not returned by CommandGetSqlInfo, just int64 ones.

Is it better to address this in another PR or in this one?

Thanks!

@lidavidm
Copy link
Member Author

I think if it's just clarifications/fixes, we can address them here.

@lidavidm
Copy link
Member Author

Thank you all for the suggestions, I've cleaned up the reST markup a bit, tweaked the charts, and fixed up the types in the Protobuf and C++ header

@lidavidm
Copy link
Member Author

lidavidm commented Apr 5, 2022

Any final comments? We can open another issue to address anything else we see

@lidavidm lidavidm closed this in dd52b38 Apr 8, 2022
@ursabot
Copy link

ursabot commented Apr 9, 2022

Benchmark runs are scheduled for baseline = b285350 and contender = dd52b38. dd52b38 is a master commit associated with this PR. Results will be available as each benchmark for each run completes.
Conbench compare runs links:
[Finished ⬇️0.0% ⬆️0.0%] ec2-t3-xlarge-us-east-2
[Finished ⬇️0.29% ⬆️0.0%] test-mac-arm
[Failed ⬇️0.0% ⬆️0.0%] ursa-i9-9960x
[Finished ⬇️0.09% ⬆️0.13%] ursa-thinkcentre-m75q
Buildkite builds:
[Finished] <https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ec2-t3-xlarge-us-east-2/builds/476| dd52b384 ec2-t3-xlarge-us-east-2>
[Finished] <https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-test-mac-arm/builds/461| dd52b384 test-mac-arm>
[Failed] <https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ursa-i9-9960x/builds/462| dd52b384 ursa-i9-9960x>
[Finished] <https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ursa-thinkcentre-m75q/builds/471| dd52b384 ursa-thinkcentre-m75q>
[Finished] <https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ec2-t3-xlarge-us-east-2/builds/475| b2853507 ec2-t3-xlarge-us-east-2>
[Finished] <https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-test-mac-arm/builds/460| b2853507 test-mac-arm>
[Failed] <https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ursa-i9-9960x/builds/461| b2853507 ursa-i9-9960x>
[Finished] <https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ursa-thinkcentre-m75q/builds/470| b2853507 ursa-thinkcentre-m75q>
Supported benchmarks:
ec2-t3-xlarge-us-east-2: Supported benchmark langs: Python, R. Runs only benchmarks with cloud = True
test-mac-arm: Supported benchmark langs: C++, Python, R
ursa-i9-9960x: Supported benchmark langs: Python, R, JavaScript
ursa-thinkcentre-m75q: Supported benchmark langs: C++, Java

jcralmeida pushed a commit to rafael-telles/arrow that referenced this pull request Apr 19, 2022
This improves the protocol docs and adds C++ API documentation. Java already has Javadocs so this PR doesn't touch anything there.

Closes apache#12616 from lidavidm/arrow-14698

Authored-by: David Li <li.davidm96@gmail.com>
Signed-off-by: David Li <li.davidm96@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants