Skip to content

Support 2-argument ceil(decimal, scale) for Spark #22560

@diegoQuinas

Description

@diegoQuinas

Is your feature request related to a problem or challenge?

Follow-up to #21560 / #21710.

#21710 added 2-argument ceil(value, scale) to datafusion-spark for integer and floating-point inputs. The decimal path was deliberately left out of scope: ceil(Decimal128(p, s), scale) currently errors at planning time with "2-argument ceil is not yet supported for decimal inputs".

Spark supports this form for decimal inputs and derives the result (precision, scale) from the literal scale argument, matching RoundCeil semantics. For example:

  • ceil(3.1411::decimal(5,4), 3)decimal(5, 3) value 3.142
  • ceil(3.1411::decimal(5,4), -3)decimal(4, 0) value 1000

Describe the solution you'd like

Implement the decimal path in SparkCeil:

  • Use return_field_from_args so the literal scale can drive the output (precision, scale).
  • When scale is not visible as a literal at plan time (column, non-literal cast), fall back to a conservative Decimal128(min(p + 1, 38), s), same approach as datafusion/functions/round.
  • Execution: operate on the unscaled integer, then rescale to the advertised output scale.
  • Follow Spark's wrapping behavior on overflow (no error), matching the existing 1-arg Decimal128 path.

Describe alternatives you've considered

No response

Additional context

No response

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type
No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions