[SPARK-30518][SQL] Precision and scale should be same for values between -1.0 and 1.0 in Decimal

[Ngone51]

What changes were proposed in this pull request?

For decimal values between -1.0 and 1.0, it should has same precision and scale in Decimal, in order to make it be consistent with DecimalType.

Why are the changes needed?

Currently, for values between -1.0 and 1.0, precision and scale is inconsistent between Decimal and DecimalType. For example, for numbers like 0.3, it will have (precision, scale) as (2, 1) in Decimal, but (1, 1) in DecimalType:

scala> Literal(new BigDecimal("0.3")).dataType.asInstanceOf[DecimalType].precision
res3: Int = 1

scala> Literal(new BigDecimal("0.3")).value.asInstanceOf[Decimal].precision
res4: Int = 2

We should make Decimal be consistent with DecimalType. And, here, we change it to only count precision digits after dot for values between -1.0 and 1.0 as other DBMS does, like hive:

hive> create table testrel as select 0.3;
hive> describe testrel;
OK
_c0                 	decimal(1,1)

This could bring larger scale for values between -1.0 and 1.0.

Does this PR introduce any user-facing change?

No.

How was this patch tested?

Updated existed tests.

[SparkQA]

Test build #116775 has finished for PR 27217 at commit 12e26c3.

• This patch fails Spark unit tests.
• This patch merges cleanly.
• This patch adds no public classes.

[SparkQA]

Test build #116788 has finished for PR 27217 at commit ff3cd01.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[maropu]

This fix changes the result?

[Ngone51]

No. IntegralDivide requires same data type for left and right. And the type of Literal(Decimal(0.6)) has changed from DecimalType(2, 1) to DecimalType(1, 1).

[maropu]

Ah, I see.

[maropu]

Thanks! Merged to master.

[Ngone51]

thank you!