Skip to content

[Bug] pmod and mod report Arrow's divide-by-zero message instead of Spark's REMAINDER_BY_ZERO text #23897

Description

@andygrove

Describe the bug

Under datafusion.execution.enable_ansi_mode = true, datafusion-spark's
pmod and mod correctly raise on a zero divisor, but the message is Arrow's:

DataFusion error: Arrow error: Divide by zero error

Spark 4.2.0 raises:

[REMAINDER_BY_ZERO] Remainder by zero. Use `try_mod` to tolerate divisor being 0
and return NULL instead. If necessary set "spark.sql.ansi.enabled" to "false" to
bypass this error. SQLSTATE: 22012

The condition detected is the same. Only the text differs. DataFusion does not
model Spark's error classes or SQLSTATE values, so an exact reproduction is not
possible, but the message body is expressible.

Cross-version note

The error class name is itself version dependent:

Spark Error class
3.5.8 DIVIDE_BY_ZERO
4.0.4 DIVIDE_BY_ZERO
4.1.3 REMAINDER_BY_ZERO
4.2.0 REMAINDER_BY_ZERO

Pmod.eval calls QueryExecutionErrors.divideByZeroError in 3.5.8 and 4.0.4
and QueryExecutionErrors.remainderByZeroError in 4.1.3 and later. DataFusion
has no mechanism for version-specific expectations, tracked by
#23887.

To Reproduce

set datafusion.execution.enable_ansi_mode = true;
SELECT pmod(10::int, 0::int);
-- DataFusion error: Arrow error: Divide by zero error

Expected behavior

A message matching Spark 4.2.0's REMAINDER_BY_ZERO text.

Scope

This is a repository-wide question rather than a pmod question. The in-repo
precedent, datafusion/spark/src/function/math/abs.rs and
datafusion/sqllogictest/test_files/spark/math/abs.slt, asserts Arrow's native
overflow message rather than Spark's ARITHMETIC_OVERFLOW text, so changing
pmod alone would make the crate less consistent. A decision on whether
datafusion-spark should reproduce Spark's ANSI error text in general would be
useful before individual functions are changed.

Relevant code

datafusion/spark/src/function/math/modulus.rs, spark_pmod and try_rem.

Surfaced by the audit-datafusion-spark-expression skill.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingspark

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions