ARROW-16600: [Java] Configurable RoundingMode to handle inconsistent scale in BigDecimals #13433
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
|
toddfarmer
commented
Jun 24, 2022
java/adapter/jdbc/src/main/java/org/apache/arrow/adapter/jdbc/JdbcToArrowConfig.java
Show resolved
Hide resolved
…ing inconsistent scale within a ResultSet
toddfarmer
force-pushed
the
toddfarmer/arrow-16600
branch
from
bbd5d75
to
e677295
Compare
lidavidm
approved these changes
Jun 27, 2022
djnavarro
pushed a commit
to djnavarro/arrow
that referenced
this pull request
Jun 28, 2022
…scale in BigDecimals (apache#13433) Under certain conditions, JDBC vendors may return ResultSets where the scale of BigDecimal values differ by row. Existing logic required exact matching of every row to the established scale for the column (target vector), and throws UnsupportedOperationException when there is a mismatch, aborting ResultSet processing. This change enables configuration of a java.math.RoundingMode to be applied in any required scale conversion, and should enable both full-fidelity truncation (trimming trailing zeros right of decimal) as well as lossy conversion (see [RoundingModes javadoc](https://docs.oracle.com/javase/8/docs/api/java/math/RoundingMode.html)). Note that this is implemented as a global configuration options - it will apply to all BigDecimal columns in a ResultSet being processed using the supplied config. It's possible to provide per-column control over this behavior, but I assessed that to complicate configuration for little benefit. Please indicate if this decision should be reevaluated. Authored-by: Todd Farmer <todd@fivefarmers.com> Signed-off-by: David Li <li.davidm96@gmail.com>
vibhatha
pushed a commit
to vibhatha/arrow
that referenced
this pull request
Jul 5, 2022
…scale in BigDecimals (apache#13433) Under certain conditions, JDBC vendors may return ResultSets where the scale of BigDecimal values differ by row. Existing logic required exact matching of every row to the established scale for the column (target vector), and throws UnsupportedOperationException when there is a mismatch, aborting ResultSet processing. This change enables configuration of a java.math.RoundingMode to be applied in any required scale conversion, and should enable both full-fidelity truncation (trimming trailing zeros right of decimal) as well as lossy conversion (see [RoundingModes javadoc](https://docs.oracle.com/javase/8/docs/api/java/math/RoundingMode.html)). Note that this is implemented as a global configuration options - it will apply to all BigDecimal columns in a ResultSet being processed using the supplied config. It's possible to provide per-column control over this behavior, but I assessed that to complicate configuration for little benefit. Please indicate if this decision should be reevaluated. Authored-by: Todd Farmer <todd@fivefarmers.com> Signed-off-by: David Li <li.davidm96@gmail.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Under certain conditions, JDBC vendors may return ResultSets where the scale of BigDecimal values differ by row. Existing logic required exact matching of every row to the established scale for the column (target vector), and throws UnsupportedOperationException when there is a mismatch, aborting ResultSet processing. This change enables configuration of a java.math.RoundingMode to be applied in any required scale conversion, and should enable both full-fidelity truncation (trimming trailing zeros right of decimal) as well as lossy conversion (see RoundingModes javadoc).
Note that this is implemented as a global configuration options - it will apply to all BigDecimal columns in a ResultSet being processed using the supplied config. It's possible to provide per-column control over this behavior, but I assessed that to complicate configuration for little benefit. Please indicate if this decision should be reevaluated.