-
Notifications
You must be signed in to change notification settings - Fork 96
Open
Description
Summarize
Arrow JDBC Adapter throw exception when wrapping the mysql driver with unsigned interger data.
Example
create a field with INT UNSIGNED type in Mysql, one of the field value is 4294967293. when use JDBC Adapter, it will throw:
Caused by: java.lang.RuntimeException: Error occurred while consuming data.
at org.apache.arrow.adapter.jdbc.ArrowVectorIterator.consumeData(ArrowVectorIterator.java:117)
ArrowVectorIterator.java:117
at org.apache.arrow.adapter.jdbc.ArrowVectorIterator.load(ArrowVectorIterator.java:159)
at org.apache.arrow.adapter.jdbc.ArrowVectorIterator.next(ArrowVectorIterator.java:177)
ArrowVectorIterator.java:177
... 76 more
Caused by: java.sql.SQLDataException: Value '4294967293' is outside of valid range for type java.lang.Integer
at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:115)
SQLError.java:115
at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:98)
at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:90)
SQLError.java:90
at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:64)
at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:74)
SQLError.java:74
at com.mysql.cj.jdbc.exceptions.SQLExceptionsMapping.translateException(SQLExceptionsMapping.java:92)
SQLExceptionsMapping.java:92
at com.mysql.cj.jdbc.result.ResultSetImpl.getObject(ResultSetImpl.java:1431)
ResultSetImpl.java:1431
at com.mysql.cj.jdbc.result.ResultSetImpl.getInt(ResultSetImpl.java:830)
ResultSetImpl.java:830
at org.apache.arrow.adapter.jdbc.consumer.IntConsumer$NullableIntConsumer.consume(IntConsumer.java:56)
IntConsumer.java:56
at org.apache.arrow.adapter.jdbc.consumer.CompositeJdbcConsumer.consume(CompositeJdbcConsumer.java:46)
CompositeJdbcConsumer.java:46
at org.apache.arrow.adapter.jdbc.ArrowVectorIterator.consumeData(ArrowVectorIterator.java:106)
ArrowVectorIterator.java:106
... 78 more
Caused by: com.mysql.cj.exceptions.NumberOutOfRange: Value '4294967293' is outside of valid range for type java.lang.Integer
at com.mysql.cj.result.IntegerValueFactory.createFromLong(IntegerValueFactory.java:62)
at com.mysql.cj.result.IntegerValueFactory.createFromLong(IntegerValueFactory.java:44)
at com.mysql.cj.protocol.a.MysqlTextValueDecoder.decodeUInt4(MysqlTextValueDecoder.java:110)
which is thrown from https://github.com/apache/arrow/blob/c9fbc88eccd56fda27d6bc655732ed17388317ac/java/adapter/jdbc/src/main/java/org/apache/arrow/adapter/jdbc/consumer/IntConsumer.java#L56
Expected
make jdbc adpater support unsigned datatype.
possible datatype mapping:
| SQL Type | Java Type |
|---|---|
| TINYINT UNSIGNED | java.lang.Integer |
| SMALLINT UNSIGNED | java.lang.Integer |
| MEDIUMINT UNSIGNED | java.lang.Long |
| INT UNSIGNED | java.lang.Long |
| BIGINT UNSIGNED | java.math.BigInteger |
Component(s)
Java
Metadata
Metadata
Assignees
Labels
No labels