Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Java] Improve JdbcConsumer errors #39355

Closed
aiguofer opened this issue Dec 22, 2023 · 0 comments · Fixed by #39356
Closed

[Java] Improve JdbcConsumer errors #39355

aiguofer opened this issue Dec 22, 2023 · 0 comments · Fixed by #39356

Comments

@aiguofer
Copy link
Contributor

Describe the enhancement requested

Currently, when an error occurs while consuming a JDBC ResultSet, it's impossible to debug what might have happened. For example, here's a redacted stack trace:

java.lang.RuntimeException: Error occurred while getting next schema root.
	at org.apache.arrow.adapter.jdbc.ArrowVectorIterator.next(ArrowVectorIterator.java:181)
        ...
Caused by: java.lang.RuntimeException: Error occurred while consuming data.
	at org.apache.arrow.adapter.jdbc.ArrowVectorIterator.consumeData(ArrowVectorIterator.java:117)
	at org.apache.arrow.adapter.jdbc.ArrowVectorIterator.load(ArrowVectorIterator.java:159)
	at org.apache.arrow.adapter.jdbc.ArrowVectorIterator.next(ArrowVectorIterator.java:177)
	... 10 common frames omitted
Caused by: java.lang.UnsupportedOperationException: Decimal size greater than 16 bytes: 17
	at org.apache.arrow.vector.util.DecimalUtility.writeByteArrayToArrowBufHelper(DecimalUtility.java:175)
	at org.apache.arrow.vector.util.DecimalUtility.writeBigDecimalToArrowBuf(DecimalUtility.java:136)
	at org.apache.arrow.vector.DecimalVector.set(DecimalVector.java:365)
	at org.apache.arrow.adapter.jdbc.consumer.DecimalConsumer.set(DecimalConsumer.java:79)
	at org.apache.arrow.adapter.jdbc.consumer.DecimalConsumer$NullableDecimalConsumer.consume(DecimalConsumer.java:101)
	at org.apache.arrow.adapter.jdbc.consumer.CompositeJdbcConsumer.consume(CompositeJdbcConsumer.java:46)
	at org.apache.arrow.adapter.jdbc.ArrowVectorIterator.consumeData(ArrowVectorIterator.java:106)
	... 12 common frames omitted

These types of errors are likely due to an incorrect mapping from JdbcType to ArrowType, but at this point there's no insight into that mapping. My proposal is to introduce a JdbcConsumerException that stores a JdbcFieldInfo for the ResultSet column and ArrowType for the vector. The easiest way to do this would be in a try/catch inside the CompositeJdbcConsumer and then bubble it up in ArrowVectorIterator. Users would then be able to extract that information from the exception to make more informative logs.

It would be nice to also store the actual value that caused the exception, but that could potentially expose PII so maybe not the best idea.

I'll make a PR and can discuss further.

Component(s)

Java

lidavidm pushed a commit that referenced this issue Jan 8, 2024
### Rationale for this change

This helps debug Arrow conversion errors from JDBC by exposing the JdbcFieldInfo for the related column and the ArrowType for the corresponding vector.

### What changes are included in this PR?

A new JdbcConsumerException which is thrown by the CompositeJdbcConsumer while consuming data for a specific vector.

### Are these changes tested?

N/A

### Are there any user-facing changes?

Users can now catch `JdbcConsumerException`s and extract the related ArrowType and JdbcFieldInfo from it for debugging.

* Closes: #39355

Authored-by: Diego Fernandez <aiguo.fernandez@gmail.com>
Signed-off-by: David Li <li.davidm96@gmail.com>
@lidavidm lidavidm added this to the 15.0.0 milestone Jan 8, 2024
clayburn pushed a commit to clayburn/arrow that referenced this issue Jan 23, 2024
### Rationale for this change

This helps debug Arrow conversion errors from JDBC by exposing the JdbcFieldInfo for the related column and the ArrowType for the corresponding vector.

### What changes are included in this PR?

A new JdbcConsumerException which is thrown by the CompositeJdbcConsumer while consuming data for a specific vector.

### Are these changes tested?

N/A

### Are there any user-facing changes?

Users can now catch `JdbcConsumerException`s and extract the related ArrowType and JdbcFieldInfo from it for debugging.

* Closes: apache#39355

Authored-by: Diego Fernandez <aiguo.fernandez@gmail.com>
Signed-off-by: David Li <li.davidm96@gmail.com>
dgreiss pushed a commit to dgreiss/arrow that referenced this issue Feb 19, 2024
### Rationale for this change

This helps debug Arrow conversion errors from JDBC by exposing the JdbcFieldInfo for the related column and the ArrowType for the corresponding vector.

### What changes are included in this PR?

A new JdbcConsumerException which is thrown by the CompositeJdbcConsumer while consuming data for a specific vector.

### Are these changes tested?

N/A

### Are there any user-facing changes?

Users can now catch `JdbcConsumerException`s and extract the related ArrowType and JdbcFieldInfo from it for debugging.

* Closes: apache#39355

Authored-by: Diego Fernandez <aiguo.fernandez@gmail.com>
Signed-off-by: David Li <li.davidm96@gmail.com>
zanmato1984 pushed a commit to zanmato1984/arrow that referenced this issue Feb 28, 2024
### Rationale for this change

This helps debug Arrow conversion errors from JDBC by exposing the JdbcFieldInfo for the related column and the ArrowType for the corresponding vector.

### What changes are included in this PR?

A new JdbcConsumerException which is thrown by the CompositeJdbcConsumer while consuming data for a specific vector.

### Are these changes tested?

N/A

### Are there any user-facing changes?

Users can now catch `JdbcConsumerException`s and extract the related ArrowType and JdbcFieldInfo from it for debugging.

* Closes: apache#39355

Authored-by: Diego Fernandez <aiguo.fernandez@gmail.com>
Signed-off-by: David Li <li.davidm96@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants