[Java][FlightRPC] Allow FlightInfo#Schema to be nullable for long-running queries #37553
Comments
tdcmeehan
added a commit
to tdcmeehan/arrow
that referenced
this issue
Sep 5, 2023
…g-running queries With apache#36155, implementations of Flight RPC may not return quickly via a newly added pollFlightInfo function. Sometimes, the system implementing this function may not know the output schema for some time--for example, after a lengthy queue time has elapsed, or after planning. In proto3, fields may not be present, and it's a coding convention to require them. To support upcoming client integration work for pollFlightInfo, the schema field can be made optional so that it's not a requirement to populate the FlightInfo's schema on the first pollFlightInfo request. We can modify our client code to allow this field to be optional. This is already the case for the Go code. This changes the Java client code to allow the Schema to be null. `getSchema` methods now return `Optional<Schema>`, which is a backwards incompatible change.
tdcmeehan
added a commit
to tdcmeehan/arrow
that referenced
this issue
Sep 6, 2023
…g-running queries With apache#36155, implementations of Flight RPC may not return quickly via a newly added pollFlightInfo function. Sometimes, the system implementing this function may not know the output schema for some time--for example, after a lengthy queue time has elapsed, or after planning. In proto3, fields may not be present, and it's a coding convention to require them. To support upcoming client integration work for pollFlightInfo, the schema field can be made optional so that it's not a requirement to populate the FlightInfo's schema on the first pollFlightInfo request. We can modify our client code to allow this field to be optional. This is already the case for the Go code. This changes the Java client code to allow the Schema to be null. `getSchema` methods now return `Optional<Schema>`, which is a backwards incompatible change.
tdcmeehan
added a commit
to tdcmeehan/arrow
that referenced
this issue
Sep 6, 2023
…g-running queries With apache#36155, implementations of Flight RPC may not return quickly via a newly added pollFlightInfo function. Sometimes, the system implementing this function may not know the output schema for some time--for example, after a lengthy queue time has elapsed, or after planning. In proto3, fields may not be present, and it's a coding convention to require them. To support upcoming client integration work for pollFlightInfo, the schema field can be made optional so that it's not a requirement to populate the FlightInfo's schema on the first pollFlightInfo request. We can modify our client code to allow this field to be optional. This is already the case for the Go code. This changes the Java client code to allow the Schema to be null. `getSchema` methods now return `Optional<Schema>`, which is a backwards incompatible change.
tdcmeehan
added a commit
to tdcmeehan/arrow
that referenced
this issue
Sep 6, 2023
…g-running queries With apache#36155, implementations of Flight RPC may not return quickly via a newly added pollFlightInfo function. Sometimes, the system implementing this function may not know the output schema for some time--for example, after a lengthy queue time has elapsed, or after planning. In proto3, fields may not be present, and it's a coding convention to require them. To support upcoming client integration work for pollFlightInfo, the schema field can be made optional so that it's not a requirement to populate the FlightInfo's schema on the first pollFlightInfo request. We can modify our client code to allow this field to be optional. This is already the case for the Go code. This changes the Java client code to allow the Schema to be null. `getSchema` methods now return `Optional<Schema>`, which is a backwards incompatible change.
tdcmeehan
added a commit
to tdcmeehan/arrow
that referenced
this issue
Sep 6, 2023
…g-running queries With apache#36155, implementations of Flight RPC may not return quickly via a newly added pollFlightInfo function. Sometimes, the system implementing this function may not know the output schema for some time--for example, after a lengthy queue time has elapsed, or after planning. In proto3, fields may not be present, and it's a coding convention to require them. To support upcoming client integration work for pollFlightInfo, the schema field can be made optional so that it's not a requirement to populate the FlightInfo's schema on the first pollFlightInfo request. We can modify our client code to allow this field to be optional. This is already the case for the Go code. This changes the Java client code to allow the Schema to be null. `getSchema` methods now return `Optional<Schema>`, which is a backwards incompatible change.
tdcmeehan
added a commit
to tdcmeehan/arrow
that referenced
this issue
Sep 6, 2023
…g-running queries With apache#36155, implementations of Flight RPC may not return quickly via a newly added pollFlightInfo function. Sometimes, the system implementing this function may not know the output schema for some time--for example, after a lengthy queue time has elapsed, or after planning. In proto3, fields may not be present, and it's a coding convention to require them. To support upcoming client integration work for pollFlightInfo, the schema field can be made optional so that it's not a requirement to populate the FlightInfo's schema on the first pollFlightInfo request. We can modify our client code to allow this field to be optional. This is already the case for the Go code. This changes the Java client code to allow the Schema to be null. `getSchema` methods now return `Optional<Schema>`, which is a backwards incompatible change.
tdcmeehan
added a commit
to tdcmeehan/arrow
that referenced
this issue
Sep 6, 2023
…g-running queries With apache#36155, implementations of Flight RPC may not return quickly via a newly added pollFlightInfo function. Sometimes, the system implementing this function may not know the output schema for some time--for example, after a lengthy queue time has elapsed, or after planning. In proto3, fields may not be present, and it's a coding convention to require them. To support upcoming client integration work for pollFlightInfo, the schema field can be made optional so that it's not a requirement to populate the FlightInfo's schema on the first pollFlightInfo request. We can modify our client code to allow this field to be optional. This is already the case for the Go code. This changes the Java client code to allow the Schema to be null. `getSchema` methods now return `Optional<Schema>`, which is a backwards incompatible change.
tdcmeehan
added a commit
to tdcmeehan/arrow
that referenced
this issue
Sep 7, 2023
…g-running queries With apache#36155, implementations of Flight RPC may not return quickly via a newly added pollFlightInfo function. Sometimes, the system implementing this function may not know the output schema for some time--for example, after a lengthy queue time has elapsed, or after planning. In proto3, fields may not be present, and it's a coding convention to require them. To support upcoming client integration work for pollFlightInfo, the schema field can be made optional so that it's not a requirement to populate the FlightInfo's schema on the first pollFlightInfo request. We can modify our client code to allow this field to be optional. This is already the case for the Go code. This changes the Java client code to allow the Schema to be null. `getSchema` methods now return `Optional<Schema>`, which is a backwards incompatible change.
lidavidm
pushed a commit
that referenced
this issue
Sep 7, 2023
…ing queries (#37528) With #36155, implementations of Flight RPC may not return quickly via a newly added pollFlightInfo function. Sometimes, the system implementing this function may not know the output schema for some time--for example, after a lengthy queue time as elapsed, or after planning. In proto3, fields may not be present, and it's a coding convention to require them 1. To support upcoming client integration work for pollFlightInfo, the schema field can be made optional so that it's not a requirement to populate the FlightInfo's schema on the first pollFlightInfo request. We can modify our client code to allow this field to be optional. This is already the case for the Go code. This changes the Java client code to allow the Schema to be null. A new `getSchemaOptional` method returns `Optional<Schema>`, which is a backwards compatible change. The existing method is deprecated, but will still return an empty schema if the schema is not present on wire (as it used to before). ### Rationale for this change With #36155, implementations of Flight RPC may not return quickly via a newly added pollFlightInfo function. Sometimes, the system implementing this function may not know the output schema for some time--for example, after a lengthy queue time as elapsed, or after planning. In proto3, fields may not be present, and it's a coding convention to require them 1. To support upcoming client integration work for pollFlightInfo, the schema field can be made optional so that it's not a requirement to populate the FlightInfo's schema on the first pollFlightInfo request. CC: `@ lidavidm` ### What changes are included in this PR? This changes the Java client code to allow the Schema to be null. `getSchema` is now deprecated and a new `getSchemaOptional` returns `Optional<Schema>`, which is a backwards compatible change. ### Are these changes tested? Existing tests ensure serialization and deserialization continue to work. ### Are there any user-facing changes? The `getSchema` methods are now deprecated in favor of `getSchemaOptional`. * Closes: #37553 Authored-by: Tim Meehan <tim@timdmeehan.com> Signed-off-by: David Li <li.davidm96@gmail.com>
loicalleyne
pushed a commit
to loicalleyne/arrow
that referenced
this issue
Nov 13, 2023
…g-running queries (apache#37528) With apache#36155, implementations of Flight RPC may not return quickly via a newly added pollFlightInfo function. Sometimes, the system implementing this function may not know the output schema for some time--for example, after a lengthy queue time as elapsed, or after planning. In proto3, fields may not be present, and it's a coding convention to require them 1. To support upcoming client integration work for pollFlightInfo, the schema field can be made optional so that it's not a requirement to populate the FlightInfo's schema on the first pollFlightInfo request. We can modify our client code to allow this field to be optional. This is already the case for the Go code. This changes the Java client code to allow the Schema to be null. A new `getSchemaOptional` method returns `Optional<Schema>`, which is a backwards compatible change. The existing method is deprecated, but will still return an empty schema if the schema is not present on wire (as it used to before). ### Rationale for this change With apache#36155, implementations of Flight RPC may not return quickly via a newly added pollFlightInfo function. Sometimes, the system implementing this function may not know the output schema for some time--for example, after a lengthy queue time as elapsed, or after planning. In proto3, fields may not be present, and it's a coding convention to require them 1. To support upcoming client integration work for pollFlightInfo, the schema field can be made optional so that it's not a requirement to populate the FlightInfo's schema on the first pollFlightInfo request. CC: `@ lidavidm` ### What changes are included in this PR? This changes the Java client code to allow the Schema to be null. `getSchema` is now deprecated and a new `getSchemaOptional` returns `Optional<Schema>`, which is a backwards compatible change. ### Are these changes tested? Existing tests ensure serialization and deserialization continue to work. ### Are there any user-facing changes? The `getSchema` methods are now deprecated in favor of `getSchemaOptional`. * Closes: apache#37553 Authored-by: Tim Meehan <tim@timdmeehan.com> Signed-off-by: David Li <li.davidm96@gmail.com>
dgreiss
pushed a commit
to dgreiss/arrow
that referenced
this issue
Feb 19, 2024
…g-running queries (apache#37528) With apache#36155, implementations of Flight RPC may not return quickly via a newly added pollFlightInfo function. Sometimes, the system implementing this function may not know the output schema for some time--for example, after a lengthy queue time as elapsed, or after planning. In proto3, fields may not be present, and it's a coding convention to require them 1. To support upcoming client integration work for pollFlightInfo, the schema field can be made optional so that it's not a requirement to populate the FlightInfo's schema on the first pollFlightInfo request. We can modify our client code to allow this field to be optional. This is already the case for the Go code. This changes the Java client code to allow the Schema to be null. A new `getSchemaOptional` method returns `Optional<Schema>`, which is a backwards compatible change. The existing method is deprecated, but will still return an empty schema if the schema is not present on wire (as it used to before). ### Rationale for this change With apache#36155, implementations of Flight RPC may not return quickly via a newly added pollFlightInfo function. Sometimes, the system implementing this function may not know the output schema for some time--for example, after a lengthy queue time as elapsed, or after planning. In proto3, fields may not be present, and it's a coding convention to require them 1. To support upcoming client integration work for pollFlightInfo, the schema field can be made optional so that it's not a requirement to populate the FlightInfo's schema on the first pollFlightInfo request. CC: `@ lidavidm` ### What changes are included in this PR? This changes the Java client code to allow the Schema to be null. `getSchema` is now deprecated and a new `getSchemaOptional` returns `Optional<Schema>`, which is a backwards compatible change. ### Are these changes tested? Existing tests ensure serialization and deserialization continue to work. ### Are there any user-facing changes? The `getSchema` methods are now deprecated in favor of `getSchemaOptional`. * Closes: apache#37553 Authored-by: Tim Meehan <tim@timdmeehan.com> Signed-off-by: David Li <li.davidm96@gmail.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Describe the enhancement requested
Part of #37527
Change our Java client code to allow nullable references to
Schema. This is to accomodate systems which may not know this information upfront (e.g. there is a queue, or it is an expensive operation, or it happens later in query processing). We can make references to this field optional, and deprecate non-optional existing usages.CC: @lidavidm, @kou
Component(s)
Java
The text was updated successfully, but these errors were encountered: