`Compute error: Overflow happened` in `tpcds_logical_q58`

[jackwener]

Describe the bug
after disable skip_failed_rules.

tpcds_logical_q58 failed in

'simplify_expressions' failed due to unexpected error: Arrow error: Compute error: Overflow happened on: 100 * 100000000000000000000000000000000000000"

To Reproduce
Steps to reproduce the behavior:

Expected behavior
A clear and concise description of what you expected to happen.

Additional context
Add any other context about the problem here.

[jackwener]

I investigated it, I found bug in ConstEvaluator.

It will failed in expr.rewrite(&mut const_evaluator)?

But I have not found a specific problem, if there are anyone interested in this bug, welcome !

[jackwener]

cc @HaoYang670 .
After we fix this problem, we can disable skip_failed_rules

[HaoYang670]

After we fix this problem, we can disable skip_failed_rules

This is really a great news! Great works @jackwener.

[jackwener]

failed in expression "CAST(Int64(100) AS Decimal128(38, 38))"

It will fail in let col_val = phys_expr.evaluate(&self.input_batch)?;

and throw "ArrowError(ComputeError("Overflow happened on: 100 * 100000000000000000000000000000000000000"))"

[viirya]

I think this is related to what we've dealt with in #3690.

[tustvold]

failed in expression "CAST(Int64(100) AS Decimal128(38, 38))"

This sounds like a type inference bug, as opposed to something related to the handling of decimal arithmetic. In particular where has this scale of 38 come from? Such a datatype cannot represent anything greater 1 and so the query is perfectly correct to bail out with an error.

The original schema has a precision of 2, as it represents a monetary quantity, and so ending up with a scale of 38 is extremely suspect...

[Jefffrey]

skip_failed_rules is now disabled by default by #6265

datafusion/datafusion/common/src/config.rs

Lines 546 to 549 in 19356b2

	/// When set to true, the logical plan optimizer will produce warning
	/// messages if any optimization rules produce errors and then proceed to the next
	/// rule. When set to false, any rules that produce errors will cause the query to fail
	pub skip_failed_rules: bool, default = false

And tpcds_logical_q58 is not ignored and runs successfully in test suite:

datafusion/datafusion/core/tests/tpcds_planning.rs

Lines 319 to 322 in 19356b2

	#[tokio::test]
	async fn tpcds_logical_q58() -> Result<()> {
	create_logical_plan(58).await
	}

Closing issue