optimize decimal: reduce validation when construct the decimal array or cast to the decimal array

[liukun4515]

Is your feature request related to a problem or challenge? Please describe what you are trying to do.

When we use the ballista/datafusion to execute some query( the table has may decimal data type), and perf the cpu.

The validation of decimal cost a lot of cpu about 5% from below picture.

Describe the solution you'd like

Describe alternatives you've considered

Additional context

• Combine DecimalArray validation #2447
• add no validation in the decimal array, optimize performance of the cast and other operation optimize: no validation for decimal array for cast/take #2551
• optimize cast from other numeric type to decimal without validation
• optimize the performance of reading data from parquet file

[liukun4515]

In the arrow-rs, there are many places to generate the decimal array.
If the precision/range of the target decimal is larger than the source decimal/data value, we don't need to validation the generated decimal array.

For example, Reading the decimal(n,0) from parquet int64 column and the n is greater equal to 18, we don't need to verify the result of the decimal array, because the value from int64 will not be overflow the target precision.

From above the method, we use less cpu for decimal data type.

[tustvold]

Also potentially related to this, the way decimal data is currently read from parquet is hopelessly inefficient #2318. I keep meaning to fix it, but I haven't got to it yet. Perhaps I can find some time this weekend... 🤔

[liukun4515]

Also potentially related to this, the way decimal data is currently read from parquet is hopelessly inefficient #2318. I keep meaning to fix it, but I haven't got to it yet. Perhaps I can find some time this weekend... 🤔

Your optimization will improve the performance of reading decimal data.
This issue may reduce unnecessary validation for generated decimal array.

[liukun4515]

Decimal also can be deserialized from INT32 or INT64 type of parquet

[alamb]

This PR #2383 may have improved performance as well

[tustvold]

We no longer perform validation except where explicitly opted in to